Anthropic calls for global pause in AI development before humans lose control - SiliconANGLE

You are using an outdated browser. Please upgrade your browser to improve your experience.

SHARE

UPDATED 19:39 EDT / JUNE 04 2026

POLICY

Anthropic calls for global pause in AI development before humans lose control

by<br>Mike Wheatley

SHARE

Anthropic PBC has sounded the alarm on the rapid pace of artificial intelligence development and is calling on the world’s top AI laboratories to consider slamming on the brakes.

In a blog post today, the company warned that AI systems are getting close to the point where they may soon be able to improve themselves without human oversight, and said reaching this threshold could lead to massive societal disruption.

The startup’s solution is for a globally coordinated agreement to temporarily pause or at least slow down the pace at which new frontier models are being developed. In the blog post, Anthropic’s head of internal research Marina Favaro and head of policy Jack Clark argued that a pause would provide the world with the breathing room it needs to adjust to the pace of AI’s rapid growth.

The two authors said model advances appear to be getting closer to a theoretical concept known as “recursive self-improvement,” which refers to AI systems that are capable of independently improving themselves and expanding their own capabilities, simply by writing their own code. They acknowledge that this hasn’t happened yet and isn’t necessarily inevitable, but they also warn it could happen – much sooner than anyone is prepared for. According to Clark, there’s reason to believe that some models could be capable of recursive self-improvement within just two years.

“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” the authors suggested.

There are reasons to listen to Anthropic when it makes such a dire warning. The company recently concluded a massive, $65 billion funding round that brought its valuation close to $1 trillion, and this week it filed paperwork with the Securities and Exchange Commission stating its plans to list on the public market. It has emerged as one of the frontrunners in an intense competition for market supremacy with ChatGPT developer OpenAI Group PBC, which is also expected to launch an initial public offering soon.

However, Anthropic, which has carefully cultivated an image of erring on the side of caution and advocating for public safety, has also been accused of leveraging policy to try to slow down the advances of its competitors. David Sacks, a venture capitalist and an informal adviser to U.S. President Donald Trump, has previously accused the company of running a “regulatory capture agenda.” He argues that the company uses fear-mongering in a calculated way to encourage heavy-handed regulations that would essentially ban lower-cost open-source models in order to boost the popularity of its own proprietary algorithms.

Whether such a slowdown is even feasible is highly debatable. Enforcing a global pause on AI development would likely be near-impossible. Anthropic conceded that it would likely require something similar to the Cold War-era treaties that slowed down the pace of nuclear weapons proliferation, but said that it’s a lot easier to mask AI training runs than it is to conceal missile silos. Unless some kind of ironclad verification regime is put in place, with cooperation from countries such as China, any pause would risk creating a scenario where competing nations simply ignore the pause and take the lead in AI development.

Regardless, the authors said they intend to push forward with the idea. Anthropic Institute, which is the company’s in-house research organization that’s dedicated to understanding and shaping AI’s impact on the world, will continue collaborating with others on ways to “help build systems that a credible slowdown or pause would require.”

The company said it will try to organize a broader debate with policymakers and AI researchers to try and answer questions about the likelihood of self-recursive improvement and what a verification system might look like.

“In the absence of a coordinated, global slowdown, we are left with the current situation: powerful technology being developed at breakneck speed by a variety of actors in a variety of countries, locked in a competition with one another where commercial and geopolitical rivalries are drowning out the larger existential-to-the-species aspects of the technology being built,” the authors wrote.

Image: Anthropic

A message from John Furrier, co-founder of SiliconANGLE:

Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network , where technology leaders connect, share intelligence and create opportunities.

15M+ viewers of theCUBE videos ,...