Anthropic warns about AI that builds itself and bets on a “slowdown” to avoid losing human control

Anthropic has warned about the latest advances in artificial intelligence (AI), referring specifically to the “recursive autoimprovement” models, that represent a loss of control for humans by developing completely autonomously, which is why it advocates a global “slowdown” in promoting this technology.

The company has explained that to advance the development of AI, the technology companies in the sector, such as Anthropic itself, are increasingly delegating more of the work to the AI ​​systems themselves, which helps speed up the work of experts. For example, much of the code is written by Claude himself, with the engineer directing and reviewing.

In this sense, he has clarified that, if taken to the extreme and are “provided with the necessary processing capacity,” this practice will lead AI systems to autonomously design and develop “their own successor”, what is known as a practice of “recursive autoimprovement”.

Although that point has not yet been reached, Anthropic has assessed that “could come sooner than most institutions are prepared for,” representing a risk and loss of control of this technology by humans. Faced with this, it has opted to carry out a global “slowdown” in AI development.

As he has shared in a statement on his blog, although an AI capable of self-building would represent “a transcendental advance in the history of technology”, with great utility in fields such as science or health, among other areas, Recursive self-improvement could also “increase the risk of humans losing control over AI systems.”

Faced with this, Anthropic has stressed that “it would be beneficial for the world to have the option to temporarily slow down or pause development of cutting-edge AI so that social structures and alignment research adapt to the advancement of technology.

Likewise, he has clarified that this slowdown must go hand in hand with a “global coordination” mechanism“. To address it, the company has indicated that the Anthropic Institute will carry out research in collaboration with other experts and companies in the sectorin order to create measures that allow building the necessary systems for a slowdown or pause.

In this way, they will create agendas that ensure cutting-edge AI developers can verify that other companies in the sector have also stopped or slowed down its development under equal conditions. Specifically, to ensure that “no malicious actor could take advantage of a slowdown coordinated to advance in secret”, while not slowing down innovation.

Following this line, the technology company has also indicated that it will require multiple well-equipped laboratorieswhich will agree stop “under the same conditions”, as well as verifying that the others have actually stopped.

To exemplify that these global pause agreements for a common good are possible, the company has referred to the Treaty on Intermediate-Range Nuclear Forces, although he has admitted that in this case it took time “decades” in the making “both infrastructure and trust”, time that is not available to AI, due to its rapid advances.

With all this, Anthropic has indicated that they will carry out their research, in addition to organizing conversations with policy makers, researchers, civil society and other companies so that they can “contribute to answering” these questions in relation to recursive self-improvement. Thus, they will publish the results soon to investigate these issues together.