Yes, I certainly agree. I often find myself saying that there seems to be—and it's like slang—a general missing mood here when it comes to where the future of AI is going. Setting aside the risk of loss of control, which I'm worried about, the current AI companies that are taking these risks very seriously are basically talking about building a technology—and you can call it AGI or something else—to have computers do the “thinky thing” we do that allows us to build rockets to go to the moon and to develop novel science. You can think of this as automating automation. Setting aside those loss of control risks, this is still something that would make all cognitive labour economically redundant and that would automate automation.
These companies expect to be able to build the systems that are approaching this in some small number of years. Maybe there's some advancement they don't have that's going to come in the way, and it could be 10 years. Five years ago, we thought AI systems that would be able to talk to us as they do today were 20 years away. Someone came up with a new idea with a transformer, and then all of a sudden, we made bigger AI systems, and they were talking to us. There's a forecasting question here that's hard, but more money is going into these systems than ever before, and we're making advancements.