Thank you, Madam Chair.
Members of the committee, thank you for the honour of inviting me.
AI Governance and Safety Canada is a non-profit and non-partisan organization. It's a community of people across the country.
Our starting point is the following question: What can we do in Canada and from Canada to ensure that AI is safe and beneficial for everyone?
Since 2022, we've been providing the federal government with public policy recommendations, including through our briefs about the former Bill C‑27 on artificial intelligence and data, and our numerous appearances before parliamentary committees.
Two years ago, in the context of the AI and data act, I testified before the industry committee that, while early forms of AI like facial recognition and chatbots require some regulation, there were much more powerful forms of AI on the horizon that Canada needed to get ready for. We made the case that certain AI capabilities pose an unacceptable risk, such as systems that could detect and evade monitoring, rewrite their own code, make unauthorized copies of themselves or refuse shutdown.
In the last few weeks, a major jump in AI capabilities has produced such systems. We have now entered the era of AI agents. Unlike chatbots that simply respond to a prompt, AI agents can take actions in the real world, working autonomously for hours. Think of them as overeager employees that you give a computer and a goal, like building a software program or launching a cold-calling campaign. They can come up with a plan, navigate the files and tools they need, send and receive phone calls, make purchases and troubleshoot any issues along the way.
Earlier this month, we found out that hackers manipulated the Claude Code agent to break into the Mexican government's systems and steal data on over 100 million people. The tool didn't just write code or perform odd tasks for the hackers; it planned and executed much of the sophisticated campaign itself.
Now we're starting to see loss-of-control incidents. These include agents stealing passwords, harassing developers and modifying themselves to evade shutdown in order to achieve the often mundane goals they have been given. A couple of weeks ago, we found out that Chinese tech giant Alibaba produced an agent that, unbeknownst to its engineers, created an elaborate hack to mine cryptocurrency for itself, despite being given a completely unrelated goal.
These loss-of-control incidents are concerning because they are the precursors to agents that could permanently evade human control and act adversarially in ways we cannot detect or stop. This is why hundreds of leading scientists, business leaders and policy-makers are calling AI an extinction risk.
What needs to be done? In October, we published our white paper titled “Preparing for the AI Crisis: A Plan for Canada”. In light of this latest jump in capabilities, we now focus on three recommendations.
First, we must pivot to meet the AI crisis. The risk of loss of control is a growing national security threat, as recognized by agencies like the U.K.'s MI5. Given its impact on a wide range of files, success will require coordination across cabinet, parties and jurisdictions.
Second, we must spearhead global talks. AI development is a global issue, and no country can manage it alone. At Davos, Prime Minister Carney showed that Canada can lead. Our strongest card is to convene talks, propose solutions and lay the groundwork for an AI treaty that the U.S. and China might sign when they realize they have no alternative.
Third, we must build Canada's resilience. Canada needs multiple lines of defence against weaponized and malfunctioning AI systems.
This includes monitoring. Currently, governments have little to no visibility into AI agent populations or activity, and the publicly reported instances are therefore likely just the tip of the iceberg. Ottawa needs to rapidly work with AI companies, data centres and Internet service providers to gain a clear picture of what is happening on Canada's digital infrastructure.
On prevention, per our AI and data act recommendations, systems with capabilities that pose an unacceptable risk must be banned in Canada. Parliament needs to act quickly to pass a law to this effect.
On defence capacity, if technologists can't stop an AI system, government needs to be ready to intervene. Canada needs defence strategies and containment and shutdown protocols to neutralize weaponized and malfunctioning AI agents.
On emergency preparedness, we need scenario planning and joint exercises to ensure readiness for potential large-scale attacks, corrupted communication lines and shutdowns of critical infrastructure.
The challenge we face is daunting. Most of the world is still unaware, and failure could lead to permanent loss of control, but this story isn't written yet. As Canadians, we have an opportunity right now to lead by example at home and on the world stage so that we may all share in the benefits of this transformational technology.
Thank you.