I think that's a great question.
I think we're already past many dangerous limits. For instance, we already have very persuasive systems. If we wanted to ensure that systems cannot manipulate people, we already have systems that are good at this. The same thing is true for hacking, for instance. If we wanted to ensure that current systems are not good at hacking, that is already lost. We already have systems that are good at hacking.
Now we're only measuring how much better they're getting, how superhuman they're getting and how much faster than people they're getting. We've already passed a few points that are quite dangerous. We're already in a tightening regime, edging closer to places from where we cannot really recover. This is the type of stuff we're talking about when we talk about measurements.
Another one that is relevant is how AI can autonomously develop itself. Right now we have companies that use AI more and more to develop AI. We have fewer and fewer humans in the loop. This is one of the other things we try to measure: how few humans are needed to develop AI. This is a measure of interest because it tells you when it could kick-start a runaway loop, which is basically a loop in the development of AI where it develops faster than we can even see it coming. These are the types of measurements we usually care about.
