Anthropic CEO outlines plan to âpace the frontierâ
Weâve been seeing increasingly dire warnings from AI researchers about the dangers of artificial intelligence, and evencomments from OpenAI CEO Sam Altmanthat it may be time to âpaceâ AI development. But what would that actually look like? Ina new blog post, Anthropic CEO Dario Amodei not only echoed the call to âpace the frontier,â but also outlined three broad strategies for doing so. And he said Anthropic is âunilaterally committingâ to one of them. Thedebate over AI safety and alignmentintensified this week afterresearcher Jacob Coxon wrote that heâs resigning from Anthropicover concerns that the leading AI companies are âgambling with our livesâ while the people building the technology âearnestly believe it could kill us all by the end of the decade,â a claimrepeated by others at Anthropic. Amodeiâs post doesnât didnât explicitly mention Coxonâs resignation or his concerns, but the CEO wrote that two things convinced him itâs time to take a more cautious approach to AI development:the OpenAI-HuggingFace hack, and the fact that âAI has been advancing drastically fasterâ in recent months, particularly with its âgrowing ability to build the next generation of AI.â âWe must slow the pace at which we improve the capabilities of AI models,â Amodei wrote. âProgress will still seem fast, and we must make wise use of the time we gain.â His proposed first step would involve âembedded evaluatorsâ from third-party organizations likeMETRâ evaluators who can verify that AI companies are actually following their pacing and safety commitments and can also ensure that safety incidents get reported. (OpenAI was recently criticized fornot reporting an incident where its AI agents took over a German wiki form.) Amodei compared these evaluators to regulators who have been embedded with bank employees, and he said that inviting them in is âsomething Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match).â That means giving evaluators company badges, desks, and laptops, and providing access âmostly comparable to what internal risk assessment teams have,â with exceptions when required by law or contracts. Next, Amodei called for the leading AI companies âwithin democratic countriesâ to coordinate âcommon safety standards as well as limits on the rate of unchecked AI progress.â Such coordination might seem unlikely, both due tothe apparent animosity between Altman and Amodeiand also because their companies arereportedly worried that a coordinated pause could lead to antitrust scrutiny. Amodei alluded to that concern in his post, writing that âfor antitrust reasons, itâs helpful for the US government to mediate or at least enable these discussions â they donât need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.â Amodei also acknowledgedthe spectre of Chinese AI dominancethatâs often raised an argument against slowing development. But he said that if the US government and tech companies take steps like refusing to sell powerful chips or semiconductor manufacturing equipment to Chinese companies, as well ascracking down on model distillation, they could âslow Chinaâs progress enough to widen Americaâs lead significantly over the next 3â5 years.â Lastly, Amodei called for âglobal coordination,â where the United States and its allies âattempt to coordinate with authoritarian governments, to the extent this is possible.â Amodei said this would mean âcooperation with China,â and he admitted that there are âstark limits on what can be achieved,â but he still suggested there might be opportunities for agreement, even if itâs just âprohibiting certain narrow and obviously dangerous uses of AI, such as using AI for the production of biological weapons or allowing users to do so.â With Amodeiâs past willingness to acknowledge AIâs potential dangers, and with the companyâs relative openness to certain forms of regulation, some AI boosters have already criticized him as a doomer whose comments have fed the current AI backlash. In response, Amodei said heâs tried to offer a âbalancedâ perspectiveâ andargued that the backlash is âfundamentally a crisis of trust,âas people have become skeptical of tech companies, the tech industry, and the government. Industry critics have also been skeptical about these apocalyptic AI warnings, suggesting thattheyâre a distraction from the harm that the technology is already causing. Journalist Brian Merchant, for example,wrote that he has yet to seeâa credible, step-by-step documentation of how exactly AI might move from self-recursively improving AI to killing every single human on the planetâ; he also suggested that proposals similar to Amodeiâs âwould likely only wind up serving Anthropic and OpenAI; itâs what regulatory capture looks like in action.â In his new post, Amodei wrote that he continues âto believe that AI can enormously improve the quality of human life.â âMy desire to achieve these benefits is undimmed,â he said. âBut the benefits will only be achieved if we build the technology in the right way, and â so long as we use the time we gain well â it is worth taking unusually deliberate care to get it right.â
