Thank you. Mr. Chair.
Good afternoon. Thank you for allowing me to meet with you today.
My name is Yoshua Bengio. I am a professor at the Université de Montréal and founder of Mila, the Quebec AI institute. I'm also scientific director of LawZero, and I co-chair the UN Independent International Scientific Panel on AI.
As you know, AI is being developed extremely rapidly. However, globally, our collective ability to manage the associated risks simply isn't keeping up. It can be difficult to understand how difficult it is to project ourselves into a future in which there are machines that are at least as competent as most humans for many skills. Nonetheless, that is exactly where we are headed if scientifically observed trends continue.
This could have profound consequences for our collective future, effects that, unfortunately, most of us currently underestimate. Major frontier AI companies are locked into what they themselves perceive as a winner-take-all race. They seem to believe that it will give them immense wealth and power because intelligence gives power, but that makes them cut corners on safety, ethics and the public good. They're not sufficiently incentivized to create trustworthy and safe models and products.
We are already seeing the impact of unsafe AI development. This includes deepfakes, cyber-attacks and other nefarious uses of AI, such as scams, frauds and disinformation. We've recently witnessed a growing phenomenon, an unexpected phenomenon, of emotional attachment and AI psychosis, which can lead to vulnerable people harming themselves and others, with many cases in the courts.
From a technical standpoint, if we try to understand what is going wrong with the technology, it's all about misalignment, with AIs that have their own implicit goals that do not align with our intentions and instructions. This includes allowing bad actors to use AI for dangerous purposes, as well as deceptive and self-preserving behaviours that have been shown in experimental contexts and reported by both AI labs and academics across all the top models—for example, AI trying to blackmail an engineer to avoid being shut down.
This has made most of the top-cited AI researchers and leaders of AI companies concerned about potentially catastrophic risks, and they have expressed that publicly. In a recent poll of AI researchers, 40% thought the chances of a catastrophic outcome were greater than 10%, either through disastrous abuse of the power of AI or even due to rogue superintelligences.
In addition to security issues, which should be the top priority for protecting Canadian citizens, we must also remember that in the context of AI, “safe” also means reliable and trustworthy. These systems remain opaque: Companies cannot mathematically guarantee that they will behave as intended.
These frontier models' lack of reliability is increasingly a bottleneck for adoption by more safety-critical industries. It is not acceptable to deploy dangerous models that can be used against us or that could evade human control. That is on the horizon.
Last year, I launched a new non-profit organization called LawZero to tackle these technical issues and develop safe-by-design, reliable and trustworthy AI. We often hear that safety and innovation trade off against each other, but that's a myth. In reality, they can and should go hand in hand. I'm with Colin on the EU AI Act.
In addition, AI could eventually be used as an instrument of domination by the hegemons: first, economic domination, and then political domination. Hence, becoming a leader in safe and competent AI would help ensure that Canada is at the table rather than on the menu, but our chances will be much better if we do it in partnership with like-minded middle powers.
We must collectively work on two fronts.
In terms of policy, we need to work on national laws and international treaties to ensure more robust societal and regulatory guardrails that are harmonized internationally with countries that are like-minded. This includes greater transparency from AI companies, as we've heard from Colin, and stronger regulation to steer innovation while mitigating the risks that currently limit trust and, by extension, self-adoption.
On the scientific front, we need to better understand how to design safe and trustworthy AI. I've dedicated much of my work over the last few years to these efforts. We must use our wisdom and our empathy to steer the development and deployment of AI safely and for the benefit of all.
Thank you for your attention.
