Yes.
To mirror the EU AI Act, systems that are high and medium in the unacceptable category are systems that, for example, refuse to be shut off, systems that deceive the user and systems that will modify themselves without notice. For example, if you give an AI system a task of doing something and it calculates that it needs to modify itself in order to achieve that task, that should be an unacceptable capability, because suddenly your system is a very different system from what was created, and the risk profile changes dramatically.
Another one is autonomous self-replication. If you ask your system to do something and it calculates that it needs to make a bunch of copies of itself on different servers so that if the first server it's on goes under, it can still keep running, that's a problem, because suddenly your model is no longer just on your computer. It's also on 10 other computers, and you don't necessarily have access to them.
In our recommendations for the AI and data act, we go into details on which are the biggest ones, but unprompted self-modification, commandeering of resources—if your model starts stealing in order to achieve its task—are all behaviours we're starting to see happen in test settings, and if we allow them, then we're very much in a vulnerable position.
