Policy on the AI Exponential \ Anthropic<br>Try Claude

Policy on the AI Exponential<br>AI is advancing at exponential speed, and the policymaking process was built for a slower world.<br>We are sharing two policy proposals to prepare for AI progress. The first, our Advanced AI Framework, offers a roadmap for governing increasingly capable systems, from transparency and independent evaluation to government authority to block or deter dangerous deployments. The second, our Economic Policy Framework, turns to the question of how to prepare workers and the economy for AI’s impact and ensure the financial benefits of AI are broadly shared. Together they cover two sides of the same challenge: steering the technology responsibly as it advances, and preparing society for what it brings.

01Advanced AI Framework<br>02Economic Policy Framework

Anthropic's Advanced AI Framework

Read the advanced AI Safety Framework →

We are publishing our proposal for how governments should address catastrophic risks from the most powerful AI models, including granting them the legal authority to block or deter dangerous deployments.<br>AI capabilities are on a sharp upward trend. A few years ago, AI models could barely write code. This year, Claude Mythos Preview discovered thousands of high-severity vulnerabilities, including in every major operating system and browser. Evidence strongly suggests this trend will continue—and with it, the risk of catastrophic harm will increase, too.<br>This calls for government action and for regulation—regulations that are carefully designed to prevent government overreach and protect innovation. When a model poses risks of this kind, the government should have the legal authority to block or deter its deployment—beyond what exists in current law or in existing proposals in Congress–with civil penalties tied to global annual revenue that escalate with repeated violations. Frontier AI developers should have to test these models, be transparent to the public about their findings, submit them to independent evaluation, and maintain a robust security program.<br>These rules should apply only to models trained using more than 10²⁵ floating-point operations (FLOPs), developed by companies earning more than $500M in AI-related revenue or spending more than $1 billion on AI R&D. They would address four kinds of catastrophic risk:<br>Biological risk. If AI systems are released without safeguards, it could become substantially easier to develop biological weapons. The same capabilities that accelerate drug discovery can be used to make it cheaper and easier for attackers to develop dangerous viruses.<br>Cyber risk. Frontier AI models can now find critical software vulnerabilities at large scale. Used defensively, these capabilities can secure critical systems, but they also raise the stakes for protecting essential infrastructure like hospitals and the energy grid.<br>Loss of control risk. As AI systems improve, it could become much harder to control systems that act outside of their developers’ control.<br>Automated R&D. AI systems are automating the research and development of AI itself, which could further amplify the three above risks.<br>Several recent state laws require companies to describe their safety practices and share them publicly. Anthropic has supported these laws. But the rapid pace of acceleration means that transparency alone is no longer sufficient. Governments need to play a more substantial role.<br>Our framework is written primarily with the US federal government in mind. But addressing AI risks cannot wait for action in Washington. We do not believe Congress should preempt state law unless it enacts a federal law that is at least as strong as the framework we are proposing today. Preemption is powerful, so it should be surgical, allowing states to regulate AI on all issues—such as child safety and consumer protection—which fall outside the specific safety functions a federal law covers.<br>Our framework is available in full here Below, we summarize its key recommendations.<br>Requirements for frontier developers<br>Transparency. Frontier developers should test their models and publish a summary of the results. They should publish a safety framework explaining how they evaluate catastrophic risks, and system cards evaluating the capabilities and risks of their frontier models. This is already required by California and New York law. But our framework goes further by requiring that developers also publish regular risk reports that describe their overall risk posture, and regularly engage independent evaluators.<br>Independent evaluation. Frontier developers should engage at least one qualified independent evaluator to publish a review of their evaluations and risk reports. In parallel, governments and industry should develop the ecosystem of qualified independent evaluators by setting standards for them and ensuring they have funding and sufficient access to frontier models.<br>Security. Model weights and training infrastructure are...