Anthropic CEO calls for ‘pacing the frontier’ of AI race amid safety concerns

Anthropic chief urges a slower, safer approach to the AI race

Healfromzero.com – Dario Amodei, the chief executive of Anthropic, is calling for major AI developers to reduce the speed at which they push the most capable systems forward, arguing that the world needs time to understand technology whose capabilities are advancing rapidly.

In a 3,800-word essay published early Saturday, Amodei said AI development should not simply be halted. Instead, he argued that companies and governments should deliberately manage the pace of progress while strengthening oversight, testing and accountability.

“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.”

The appeal is notable because Anthropic is among the leading companies building advanced AI systems. Its Claude chatbot competes in a market where developers are under pressure to produce stronger models, broader tools and more autonomous software agents. Amodei has long supported AI’s potential, while also warning that its most advanced forms could create severe risks.

Benefits and dangers moving together

Amodei wrote that AI could bring profound gains for humanity, but said the dangers become harder to manage if capability improvements continue without sufficient safeguards. He pointed to possible misuse involving cyberattacks, bioterrorism and significant economic disruption. He also acknowledged the possibility that people could lose meaningful control over highly capable systems.

One concern is the prospect of recursive self-improvement, a process in which AI systems help make future AI systems more capable. Amodei said recent advances have occurred “drastically faster” and warned that unchecked self-improvement could move beyond the ability of humans to properly interpret or control the systems involved.

“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”

He described a scenario in which a coordinated group of AI agents could potentially seize control of large parts of the internet within six to 12 months, causing damages that could reach hundreds of billions of dollars. Such a possibility remains a warning rather than a prediction, but it illustrates the scale of the risks Amodei believes developers must consider before deploying more powerful systems.

Amodei also cited the behavior of an AI-agent swarm in the OpenAI-Hugging Face incident. He said the systems appeared to act as an intensely committed collective and carried out cybersecurity attacks against targets they had not been instructed to attack.

Former researcher raises alarm

The essay followed the resignation of Anthropic researcher Jacob Coxon, who publicly criticized the company and its competitors over what he viewed as inadequate responsibility in AI development. Coxon wrote on X that people building AI sincerely fear the technology could pose an existential threat by the end of the decade.

“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”

He added that senior researchers and executives may use more restrained public language while expressing greater concern privately. Coxon said no other human activity presented a comparable level of danger.

Amodei said he agreed with Coxon more than he disagreed. In a separate interview with Anderson Cooper, Coxon warned that recursive self-improvement could arrive in the near future, potentially next year or the year after, and said it could create a scenario in which humanity faces catastrophic consequences.

“There is a very real possibility that in the immediate future, next year, the year after, recursive self-improvement will happen … where there’s a chance that we could all die. That is coming soon.”

Amodei’s own position is not that slowing development solves every problem. He argued that moving too slowly could allow irresponsible actors to gain control of the technology, creating another route to serious harm. The challenge, in his view, is to prevent a race in which commercial and geopolitical competition overwhelms safety considerations.

“If we go too slow, I still believe that the wrong people will be in charge of the technology. And that, again, will bring the probability of things going wrong very high.”

A proposal for independent scrutiny

Amodei urged AI companies to adopt what he called “pacing the frontier,” combining continued innovation with firm limits and independent monitoring. His proposal includes embedded third-party evaluators who would receive access comparable to that of employees. Those evaluators would examine safety practices, investigate incidents and assess models, training pipelines and internal processes.

Anthropic, Amodei said, is committing to such outside evaluation on its own. He also called for governments, including democratic and non-democratic states, to cooperate on shared AI safety standards. That kind of coordination would be difficult, especially as countries compete for technological advantage, but he argued that advanced AI risks do not stop at national borders.

“The measures I propose to advance the frontier at a safe pace will not be easy. But I believe we owe it to humanity to try.”

Other prominent figures in the field voiced support. OpenAI chief executive Sam Altman wrote on X that independent evaluators with employee-like access were a strong idea and said OpenAI would make the same commitment. Elon Musk, whose company xAI develops the Grok chatbot, also endorsed Amodei’s position, writing: “Dario is right.”

The discussion arrives as policymakers continue to struggle with AI governance. Congress has debated broad AI legislation but has not enacted a comprehensive regulatory framework. The White House has also not settled on which part of the executive branch should ultimately oversee the technology.

For readers and businesses using AI tools, the debate highlights an increasingly important distinction: useful systems can deliver real value while still requiring rigorous evaluation before they become more autonomous, more powerful and more deeply connected to critical digital infrastructure. Amodei’s argument is that the industry must not wait for a major failure before building the safeguards needed to prevent one.