These are some very excellent questions, and this is, in fact, a very difficult thing.
The non-locality of digital technologies presents novel, unprecedented risks of proliferation and difficulty of control. With nuclear weapons, for example, we have the luck, in a sense, that uranium ore is quite bulky and centrifuges are quite hard to build and quite visible from space, if you do them right. We have some of these benefits when it comes to AI systems, such as data centres. Others, such as the open-sourcing of many such systems, present novel difficulties.
This is why we've put such emphasis on the regulation of the development. If a superintelligent system was made, it would probably be a computer file that can probably never be deleted. It would be something that would spread. We might not even know what we're dealing with when it is first developed. It is quite likely that we won't even know that the first superintelligent system is superintelligent until it's too late. It is already the case right now that many of our AI systems have capabilities that we didn't know at the time of development. We only discover much later that our systems are capable of things we didn't know.
This is a novel regime. This is not something we have dealt with very well historically. Even historically, things such as export controls on software have not been very successful or have been very tricky to enforce.
It's very important to say that we do not think all AI should be halted or that all AI applications should be halted or not used by users. I'm sure my colleague would agree with me that we very much enjoy many of the AI applications on the market today. What we want is to gain the benefits of the kind of AI we have right now and continue forward into more powerful applications.