Thank you very much.
Does your company do business with National Defence or the Canadian military to get contracts?
Evidence of meeting #38 for Industry and Technology in the 45th Parliament, 1st session. (The original version is on Parliament’s site, as are the minutes.) The winning word was quantum.
A recording is available from Parliament.
Bloc
Gabriel Ste-Marie Bloc Joliette—Manawan, QC
Thank you very much.
Does your company do business with National Defence or the Canadian military to get contracts?
Vice-President, AI Solution Engineering, Oracle
Oracle is obviously a multinational organization. We have had contracts in place with the Canadian federal government, and the provincial and territorial governments—as I mentioned in my opening statement—for many years. We've been in existence for 40-plus years. For a large majority of that, we've always considered Canada to be one of the first markets we invest in, outside of the U.S.
Bloc
Gabriel Ste-Marie Bloc Joliette—Manawan, QC
Okay. Thank you very much.
Your company, the parent company, is a super multinational, as you just mentioned. Your CEO is one of the richest people in the world. For a few years, he was even the richest. He is currently considering acquiring the American mass media network CNN.
What I'm concerned about and what I think my colleagues here are concerned about is making sure that large companies like yours respect Canadian law and protect personal data.
For example, yesterday we learned something in the media about ChatGPT, which was created by another big company, OpenAI. Privacy commissioners at the federal level and in some provinces have found that ChatGPT violated privacy laws without obtaining free and informed consent. OpenAI was not fined because the law was not adapted to the new reality.
In your opinion, should we adapt our laws to ensure better protection of privacy?
Might your company also have violated privacy laws, to your knowledge?
Vice-President, AI Solution Engineering, Oracle
I'll comment based on our experiences. In my role today, I work with many governmental organizations in the U.S. and Canada. I've worked with our commercial industries, as well.
Broadly speaking, it is always a good idea to put as much protection of personally identifiable information about individuals...and continuously see what additional protections and guardrails we can put in place to enable more data protection, fundamentally.
Forgetting the large language model for a second, it's a belief we've had for decades that data is the most important asset in this kind of an architecture. From a protection and privacy standpoint, we want to enable it technically. In my domain, I'm a technical practitioner of this, so I'll speak to it from a technical standpoint.
We're continuously looking at additional technologies and capabilities, whether it's encrypting the data; controlling who has access to what, where and how; or auditability, so that you have complete lineage and traceability of what has happened in every system that stores these records. That is a consistent pattern and a consistent architecture that we've recommended to our Oracle customers everywhere.
Coming to the advent of large language models, and now with generative AI, we don't necessarily believe that it should get a special pass to go straight into your data and do whatever it wants to do. We believe that, at the very least, it should go through the same set of defensive layers that we've built around the data pre-AI so that you are safeguarded, just like a human being who tries to access data that they may or may not have authorization for. The AI model needs to go through those same layers, but now we're building much more robust protections, specifically for AI, on top of that.
To answer your question, I don't believe—
Liberal
The Chair Liberal Ben Carr
Sir, I'm afraid I'm going to have to end there. We went quite a bit over, but there will be an opportunity for you to return to those remarks.
Thank you very much, Mr. Ste‑Marie.
Ms. Borrelli, the floor is yours for five minutes.
Conservative
Kathy Borrelli Conservative Windsor—Tecumseh—Lakeshore, ON
Thank you, Chair.
Thank you to our witnesses today. We certainly appreciate your testimony for this important study.
My question is for Mr. Dhillon. After yesterday's announcement from the Privacy Commissioner regarding an investigation into OpenAI privacy concerns, do you believe that strong enough action was taken? Do you believe we will see more cases similar to this if we deploy AI before proper legislation is in place?
Executive Director, Centre for Designing Change
About a decade ago, I was hearing Ottawa pushing for Transport Canada to ease off on policies around autonomous vehicles, which, in theory, are machine learning and AI, but since the advent of large language models, it's all about bringing on the regulations, bringing on the policies. That's the essence of my book.
I refer back to the description I use of what we currently have as being a digital species. I've even heard that the U.S. is now considering bringing in regulations.
I feel, at this moment, that when it comes to privacy and safety around AI in general, including large language models, it should be very tight, but it has to be done on a global level because, again, economically, we're in a race. If some of our competitors are not willing to do so, it's a conversation internally with our partners on where we stand.
Conservative
Kathy Borrelli Conservative Windsor—Tecumseh—Lakeshore, ON
Given that our AI is so heavily integrated within foreign firms, do you believe that it can pose a greater risk for us?
Executive Director, Centre for Designing Change
When we use OpenAI's ChatGPT, the data is actually going back to their data centres. It's not staying here with us, so there is a sovereignty issue, absolutely.
Considering that we have the godfather of AI in Ontario and in Quebec—because I believe there are two—and considering that we have some of the brightest minds in AI research, I think there is a golden opportunity for Canada to support its competitors—these are the large language models—where we have more oversight on the data that is then used and generated.
Conservative
Kathy Borrelli Conservative Windsor—Tecumseh—Lakeshore, ON
One concern we've heard throughout this study is this: AI systems are increasingly influencing decisions in hiring, lending, health care, policing and even public services, often without Canadians fully understanding how these systems operate or how decisions are being made. At the same time, many of these systems are being developed by large foreign technology companies with limited public transparency. The Canadian data collected usually falls under foreign legislation.
How do we ensure that Canadians retain oversight of AI decisions and ownership of our own data?
Executive Director, Centre for Designing Change
The simple answer to that is to not necessarily use other so-called competitor products that don't belong to Canada. I don't think that's the easiest answer, because it is moving so quickly. Most companies and organizations are already developing and using large language models—now the prominent description of AI.
This goes back to my second response. Canadian companies—some of which are listed as part of this standing committee—need to be supported in order for us to compete. At the moment, it looks like there's a two-horse race between the U.S. and China, in terms of companies developing large language models. The frustrating thing for me is that we are the global centre of AI research: How do we not have a dog in this race?
With these changes, and with deglobalization, there is a golden opportunity for Canada to break its barriers on restriction and truly compete globally, but it's going to need support.
Liberal
The Chair Liberal Ben Carr
Thank you, Ms. Borrelli.
Madame O'Rourke, the floor is yours for five minutes.
Liberal
Dominique O'Rourke Liberal Guelph, ON
Thank you, Chair.
Thank you to the witnesses.
I'd like to go to Ms. Piovesan first.
My concern is that we're waiting for this AI framework and thinking one framework or one law will do it all, but we're hearing about ChatGPT. Then we're going to get to agentic AI. Then we're going to get to superintelligence. Some witnesses said that we should have an AI conversation in every single standing committee, which I tend to agree with.
Is it one law? Is it several laws? Is there any way this framework will be comprehensive enough? My concern is that, regardless of what we table, people will say that it's too much or not enough. Then we won't move forward.
What are your thoughts on the best approach at this time?
Managing Partner, INQ Law
I tend to agree with you. It's not going to be just one framework. It's an overarching governance framework or strategy. An approach to what it means for Canada to properly govern high-impact AI systems in our jurisdiction is important.
I'll remind the committee that we already have laws in place governing certain aspects of artificial intelligence. Depending on the sector, we might actually have more guidance available to those operating.... For example, OSFI guidance has come out. They updated the model risk-management approach for financial services recently, in partnership with the Global Risk Institute and a series of financial operators. They have the Agile framework and the EDGE principles—again, so financial services have a governance framework for the use of AI in various contexts.
We can leverage what sectors are doing in order to inform what the centre ought to do, and to fill gaps. That is an important approach I raised in the ETHI committee last year—looking at different jurisdictions and how they have been modelling, or attempting to put together, an overarching framework. That framework doesn't have to be perfect. It has to create a narrative around what AI governance looks like and what we're drawing from, in terms of sectoral and existing provincial-territorial laws. Then fill in gaps to provide that clarity.
Liberal
Dominique O'Rourke Liberal Guelph, ON
Thank you.
I participated in an Inter-Parliamentary Union webinar on AI. They said, much to Mr. Jahangir's point, that data is the most important asset. I'm hearing that data is the new oil. We also heard from witnesses that it may not be possible to have that comprehensive auditability Mr. Jahangir referred to.
Ms. Piovesan, you said that we need an AI assurance market. We're hearing that AI can be verified at the moment it's launched, but with agentic AI or superintelligence, we don't know what it's going to do next.
Do we have that capacity now? What safeguards do we put in place, and how does this evolve over time?
Managing Partner, INQ Law
The AI assurance market is meant to address that exact point. We put in place the required professional skills, the technology that's needed, the process, the policy, the law and the framework to get us ready for agentic and superintelligence. That's exactly what the AI assurance market is meant to do.
It's also really core to who we are. Going back to 2017, Canada was the first to come out with an AI strategy and threw money behind it. I very much echo Mr. Singh Dhillon's point about how we are the leaders in AI research. We are also the leaders, in many ways, in AI ethics, AI governance and AI law and policy. It's not law on the books, but the thinking behind it. We should leverage the different values and skills we bring to Canada to get us ready for rapidly emerging technology as it changes.
Liberal
Dominique O'Rourke Liberal Guelph, ON
Mr. Jahangir, do you have something to say about the comprehensive auditability? Does it exist now?
We don't want to be researching and getting everything so perfect that we miss the boat on the commercialization and adoption.
Vice-President, AI Solution Engineering, Oracle
We are working on specific initiatives and projects to meet auditability and traceability requirements. A lot of the ability to do that kind of audit, that governance, really starts with what the end-user is doing. What is the usage pattern of the AI? We want to ensure we're catching every interaction point and every transaction downstream from that initial user's prompt to a model. We are building traceability and auditability as a fundamental requirement. It's not an afterthought.
From a practical perspective today, we are seeing this. Again, it's evolving on how we're doing it, but it is certainly a foundation upon which we are creating certain levels of predictability and guardrails for end-users who are using this.
Liberal
The Chair Liberal Ben Carr
Thanks very much.
Mr. Ste‑Marie, you have the floor for two and a half minutes.
Bloc
Gabriel Ste-Marie Bloc Joliette—Manawan, QC
Thank you, Mr. Chair.
Ms. Piovesan, in your presentation and in your exchanges with my colleague Ms. O'Rourke, you talked about assurance, assurance companies and the assurance market.
Since assurance regulation falls under provincial jurisdiction, not federal, have you had any discussions with provincial governments about this?
If so, what were their reactions?
Managing Partner, INQ Law
There is a need for coordination in Canada. We cannot have a fragmented assurance market in our country. We're moving towards greater mobilization and harmonization across Canada. Every jurisdiction has an important contribution to make. In fact, we could leverage some of the lessons from Quebec's law 25 and bake them into part of this assurance market. We really have to guard against over fragmentation. It will be extremely challenging, and it will create an uncompetitive ecosystem in Canada.
Bloc
Gabriel Ste-Marie Bloc Joliette—Manawan, QC
Thank you very much.
Representatives from TELUS came to tell us that, when it comes to data protection, three things are important.
First, our privacy laws must be enforced.
Second, no foreign law should apply to our data in the specific case where a country demands that data collected here by a company be transferred to it.
Third, no country should be able to interfere with the proper functioning of the system. It's a question of who controls the switch.
Ms. Piovesan, in your opinion, should the government stop sharing sensitive information with companies that do not meet these three conditions?
Managing Partner, INQ Law
If I understand the conditions properly, regarding the sharing of information, again, turning to law 25 from Quebec, we have a precedent where you can undertake—in the Quebec context you are required to undertake—an assessment of data sharing. It's a privacy impact assessment where personal information of Quebec citizens or residents is going to move outside Quebec borders. That's a precedent that we have.
We see this in the EU under the General Data Protection Regulation. Where there are particular data transfers, there is a requirement for an assessment. In our existing law. we already have a requirement to establish contractual assurances for any data flow. We've seen this.
There are precedents for enabling and requiring, in fact, an assessment as well as taking into consideration how data flows outside a particular jurisdiction and certainly out of Canada.