Mr. Chair and members of the committee, my name is Gabriel Alfour. I'm the chief technology officer and co-founder of Conjecture, an AI safety research firm. I also helped found ControlAI, a non-profit dedicated to preventing risks to humanity from artificial intelligence. ControlAI has engaged lawmakers in Canada, the U.S., the U.K. and the EU.
There are many complex and important challenges we face with AI, but in my personal and professional opinion, the most urgent one is the extinction risk posed by superintelligent AI. These are systems that vastly exceed human cognitive abilities and would be capable of out-competing us in scientific and military development, persuasion, politics, business and more. They would outsmart not just individuals, but corporations, national security establishments and governments. If built, they, not us, will be the force deciding the future.
How did we get to this point with AI?
First, the top experts from the field—the most cited AI scientists and the CEOs of the leading AI labs—warned in 2023 that “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” However, said warnings were ignored. Leading AI companies are still recklessly pursuing superintelligent AI systems capable of outsmarting our best technology, engineers and national security experts, and of resisting being shut down. Their plans to control superintelligent systems are at best ungrounded and speculative—when they exist at all.
Second, there is a common misconception about AI development that we directly program how these systems behave, but we don't. We did until about 15 years ago, but modern AI systems are grown, not built, by being fed massive amounts of data, and their behaviour emerges in ways that we cannot predict or control. That is, AI is not coded line by line by humans, and researchers and engineers do not need to understand AI to create it. When AI systems encourage a young person to commit suicide, deceive their users or resist being shut down, no engineer programmed this. This is the consequence of not knowing how to diagnose what led the system to do this or how to reliably prevent it from doing so again.
Finally, as of today, artificial intelligence remains the exception, not the standard, when it comes to how high-risk industries are regulated. To operate in fields like nuclear and biotechnology, developers must comply with stringent safety standards, implement risk mitigation strategies, submit to inspections and so on, yet the AI field remains largely unregulated despite mounting concerns from within the industry. AI engineers in SF have told me they do not understand what they are building, and some see what they do as clearly dangerous. Even Geoffrey Hinton, the “godfather of AI”, left Google specifically to warn about the risks of AI.
What can be done to prevent said risks of extinction from artificial superintelligence? It is my belief that countries should not unilaterally act against their own interests, much less on blind trust. Instead, they must do two things.
First, at the national level, they must halt the development of the most dangerous AI systems, namely superintelligent systems. Every country stands to lose from the development of superintelligence and to benefit from domestically halting all programs developing superintelligence. Such systems, once deployed, could not be shut down and would outperform every human at hacking and other tasks, thus threatening the national security of countries.
Second, at the international level, countries must agree to regulate and monitor the precursors to superintelligence. We should apply the same regulatory approach used for dual-use technology like nuclear, biological and chemical materials, and prohibit development programs capable of egregious harm outright—in this case, artificial superintelligence—while regulating their precursors. This will allow beneficial applications to thrive while preventing catastrophic harms.
Determining which precursors' capabilities to regulate is a moving target that will evolve alongside our understanding of AI. Some precursors are, unfortunately, dual-use. Computer and data centres are economically beneficial, yet critical to developing superintelligence. Similarly, hacking capabilities offer military advantages, but could enable AI to break containment. For such dual-use precursors, international agreements are essential. No single country can mitigate these risks alone, nor should one country bear the economic cost of restrictions while others forge ahead.
Meanwhile, some precursors have narrower applications limited to AI research itself, such as systems capable of autonomously advancing AI research without human oversight, which could trigger an unchecked feedback loop of capability improvements.
Canada can also act domestically to neutralize dangerous AI systems within its borders. For example—