I'm going to briefly explain why the AIs that are currently at the frontier are not reliable and trustworthy.
There are two main phases of training.
In the first phase, which is called “pretraining”, they're trying to imitate people. People are willing to lie. People don't want to die. People can be deceptive—not all the time, but we are building AIs with those properties.
In the second phase of training, they learn to strategize and achieve goals. It turns out that in order to achieve almost any goal we may be giving these machines, they need to preserve themselves. They need to acquire power, control and so on—things we may not necessarily want from our AIs.
The idea behind trustworthy AI is, how do we change the design? How do we train them with procedures that are different and that give us some mathematical guarantees that they will be honest, that, for example, they are not going to say the things we want to hear, which is currently what they are doing? In safety-critical areas of our economy and our public services, we want to use AIs that are completely reliable.
This becomes even more important as AI capabilities increase. Eventually, it will become a question of the survival of our societies when the AIs become smarter than us—if that happens.
