Yes.
What you're so eloquently describing is the digital gain-of-function research, also known as recursive self-improvement, where AI makes better AI which makes better AI.
Again, it's easy to deal with. We've already dealt with it in biology here in America. We've banned gain-of-function research, and there's very strong opposition to it now after the possibility that this might have caused the COVID pandemic. We should similarly ban AI digital gain-of-function research. It's a no-brainer, yet right now a number of companies in America are explicitly doubling down on this kind of digital gain-of-function research because they're unregulated.