Mr. Chair, committee members, thank you for inviting me to speak to you today. I'm honoured.
AI Governance and Safety Canada is a non-partisan not-for-profit organization and a community of people across the country. We start by asking the following question: What can we do in and from Canada to ensure that AI is safe and benefits everyone?
Since 2022, we've been making public policy recommendations to the federal government, such as our submission on the AI and data act bill and our many appearances before parliamentary committees.
Two years ago, in the context of the AI and data act, I testified before this committee that while early forms of AI, like facial recognition and chatbots, require some regulation, there were much more powerful forms of AI on the horizon that Canada needed to get ready for. We made the case that certain AI capabilities pose an unacceptable risk because they could lead to dangerous weaponization or loss of control scenarios: systems that, without the instruction or authorization of their users, can detect and evade monitoring, rewrite their own code, make copies of themselves, spawn other AI systems, commandeer resources or refuse shutdown.
In the last few weeks, a major jump in AI capabilities has produced such systems. We have now entered the new paradigm of AI called “AI agents”. Unlike chatbots, which simply respond to a prompt, AI agents are systems that can take actions in the real world, working autonomously for hours and overcoming hurdles along the way. Think of them as an employee that you sit down at a computer and tell to accomplish a goal such as building a software program or launching a cold-calling campaign. They come up with a plan, navigate the files and tools they'll need, send and receive emails and phone calls, make and receive payments and debug any issues they come across.
Last week, we found out that hackers manipulated Claude Code to break into Mexican government systems and steal data on over 100 million people. The tool didn't just write code or perform odd tasks for the hackers. It planned and executed most of the sophisticated campaign itself.
Also, now we're starting to see loss of control incidents. These include agents stealing passwords, harassing developers and modifying themselves to evade shutdown in order to achieve the often mundane goals they have been given. Over the weekend, we found out that Chinese tech giant Alibaba produced an agent that, unbeknownst to their engineers, had created an elaborate hack to mine cryptocurrency for itself, despite being given a completely unrelated goal.
These loss of control incidents are concerning because they are the precursors to agents that could permanently evade human control and act adversarially in ways that we cannot detect or stop. This is why hundreds of leading scientists, business leaders and policy-makers are calling AI an “extinction risk”.
What needs doing? In October, we published our white paper titled “Preparing for the AI Crisis: A Plan for Canada”. In light of this latest jump in capabilities, we now focus on three actions Canada can take.
Number one is to pivot to meet the AI crisis. AI development is now a national security emergency and needs to be treated as such. Given its impact on a wide range of files, success will require coordination across cabinet, parties and jurisdictions.
Number two is to spearhead global talks. AI development is global, and no country can manage it on its own. At Davos, Prime Minister Carney showed that Canada can lead. Our strongest card is to convene talks, propose solutions and lay the groundwork for an AI treaty that the U.S. and China might sign when they wake up to the crisis and realize that they have no alternative.
Number three is to build Canada's resilience. Canada needs multiple lines of defence against weaponized and malfunctioning AI agent systems.
This includes, first, prevention. Per our AI and data act recommendations, capabilities that pose an unacceptable risk must be made illegal in Canada. This means that you need to place an immediate moratorium on the latest generation of AI agents. Note that heads of Anthropic and Google DeepMind recently stated that they are willing to pause AI development if other companies do the same.
Second is monitoring. Currently, governments have little to no visibility into AI agent populations or activity. This means the incidents that have been publicly reported are very likely just the tip of the iceberg. Ottawa needs to urgently work with AI companies, data centres and Internet service providers to gain a clear picture of what is happening on Canada's digital infrastructure.
Third is defence capacity. Our national security teams need to rapidly develop defence strategies and containment and shutdown protocols to neutralize weaponized or malfunctioning agents.
Last is emergency preparedness. We urgently need scenario planning and joint exercises to ensure readiness for potential large-scale attacks, corrupted communication lines and shutdowns of critical infrastructure.
To make a COVID analogy, the release of the latest AI agents is like that initial outbreak in the wet market in Wuhan, China. Most of the world is still unaware of its implications, but if Canada acts quickly and decisively, we can not only prepare ourselves and help mitigate the emerging global crisis but also ensure that Canadians share in the benefits of this transformational technology.
Thank you.