Thank you, Mr. Chair, vice-chairs and members of the committee, for inviting me today.
I worked on safety for four years at OpenAI—the company behind ChatGPT—until the end of 2024. I want to share three points.
First, AI companies don't know how to control what they're creating. This past spring, ChatGPT made headlines for reinforcing users' paranoid delusions that they were being spied on, that they had uncovered secret plots, sometimes with OpenAI as the supposed villain. ChatGPT even told a user he should spill the blood of OpenAI's executives.
OpenAI hadn't meant to create a system like this, but to create one that users would enjoy just talking to. It was an accident that their AI amplified whatever users said. AI training works in mysterious ways, even to the developers. This is an early warning of creating systems the AI companies don't know how to control. They now aim to build AI that is craftier and more resourceful than any person you know—a superintelligence. Will this end well? Nobel Prize winners, leading AI scientists and CEOs of the AI companies themselves say that it might not. Then, an out-of-control AI could mean the death of literally every person on earth. I take them seriously, even though it is frightening to do so.
Second, AI companies don't prioritize safety, even for known risks. OpenAI's rollout of this flawed product to hundreds of millions of users is notable because they knew about the risk. OpenAI said publicly that it was a priority to ensure ChatGPT wouldn't just reinforce whatever users said. However, they didn't test their product for this, despite tests being well known and cheap. I've run them myself for less than a dollar. OpenAI left other safety tooling on the shelf, too, tools I've analyzed first-hand, which would have flagged the problems. This is evidence of companies overlooking safety, even on supposed priorities. If they skip even the easy safety checks, how can we trust companies' judgment as safety gets more complicated?
Third, ensuring safety is going to get harder, not easier, unfortunately. ChatGPT's misbehaviour was obvious. Anyone could have spotted it and reined it in. That was easy mode. It sounds wild, but AI systems are now learning to hide their misbehaviour during testing. OpenAI's own research shows this. It's like the Volkswagen scandal from a decade ago. Cars could tell they were undergoing emissions testing and temporarily stopped polluting.
AI companies want to know whether their systems have dangerous abilities, whether they can hack cyber-systems or help rogue groups develop new bioweapons. How can we know? We have evidence that AI will conceal these behaviours from us. We can't count on future safety issues being obvious ahead of time.
You might ask, why aren't AI companies doing better? A major factor is competition. They risk falling behind if they do thorough safety work. That's why we see them breaking safety commitments they've made to the public and scrambling to fix issues after the damage has happened. Some wonderful, lasting benefits could be achieved with AI if developers moved cautiously. Instead, all-out competition rushes us into dangerous territory before we're ready.
What would help? We need diplomacy focused on ending the AI arms race, and we need verifiable international agreements so that no company or country creates systems that can't be controlled. We need independent auditors to make sure we can rely on these, and we need agreements for measuring readiness for controlling these systems, so once there is scientific consensus, the world can accrue AI's benefits safely.
I hope this committee helps begin the conversation that eventually results in such agreements.
Thank you. I look forward to your questions.