Anthropic CEO outlines plan to āpace the frontierā
Weāve been seeing increasingly dire warnings from AI researchers about the dangers of artificial intelligence, and evencomments from OpenAI CEO Sam Altmanthat it may be time to āpaceā AI development. But what would that actually look like? Ina new blog post, Anthropic CEO Dario Amodei not only echoed the call to āpace the frontier,ā but also outlined three broad strategies for doing so. And he said Anthropic is āunilaterally committingā to one of them. Thedebate over AI safety and alignmentintensified this week afterresearcher Jacob Coxon wrote that heās resigning from Anthropicover concerns that the leading AI companies are āgambling with our livesā while the people building the technology āearnestly believe it could kill us all by the end of the decade,ā a claimrepeated by others at Anthropic. Amodeiās post doesnāt didnāt explicitly mention Coxonās resignation or his concerns, but the CEO wrote that two things convinced him itās time to take a more cautious approach to AI development:the OpenAI-HuggingFace hack, and the fact that āAI has been advancing drastically fasterā in recent months, particularly with its āgrowing ability to build the next generation of AI.ā āWe must slow the pace at which we improve the capabilities of AI models,ā Amodei wrote. āProgress will still seem fast, and we must make wise use of the time we gain.ā His proposed first step would involve āembedded evaluatorsā from third-party organizations likeMETRā evaluators who can verify that AI companies are actually following their pacing and safety commitments and can also ensure that safety incidents get reported. (OpenAI was recently criticized fornot reporting an incident where its AI agents took over a German wiki form.) Amodei compared these evaluators to regulators who have been embedded with bank employees, and he said that inviting them in is āsomething Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match).ā That means giving evaluators company badges, desks, and laptops, and providing access āmostly comparable to what internal risk assessment teams have,ā with exceptions when required by law or contracts. Next, Amodei called for the leading AI companies āwithin democratic countriesā to coordinateĀ ācommon safety standards as well as limits on the rate of unchecked AI progress.ā Such coordination might seem unlikely, both due tothe apparent animosity between Altman and Amodeiand also because their companies arereportedly worried that a coordinated pause could lead to antitrust scrutiny. Amodei alluded to that concern in his post, writing that āfor antitrust reasons, itās helpful for the US government to mediate or at least enable these discussions ā they donāt need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.ā Amodei also acknowledgedthe spectre of Chinese AI dominancethatās often raised an argument against slowing development. But he said that if the US government and tech companies take steps like refusing to sell powerful chips or semiconductor manufacturing equipment to Chinese companies, as well ascracking down on model distillation, they could āslow Chinaās progress enough to widen Americaās lead significantly over the next 3ā5 years.ā Lastly, Amodei called for āglobal coordination,ā where the United States and its allies āattempt to coordinate with authoritarian governments, to the extent this is possible.ā Amodei said this would mean ācooperation with China,ā and he admitted that there are āstark limits on what can be achieved,ā but he still suggested there might be opportunities for agreement, even if itās just āprohibiting certain narrow and obviously dangerous uses of AI, such as using AI for the production of biological weapons or allowing users to do so.ā With Amodeiās past willingness to acknowledge AIās potential dangers, and with the companyās relative openness to certain forms of regulation, some AI boosters have already criticized him as a doomer whose comments have fed the current AI backlash. In response, Amodei said heās tried to offer a ābalancedā perspectiveā andargued that the backlash is āfundamentally a crisis of trust,āas people have become skeptical of tech companies, the tech industry, and the government. Industry critics have also been skeptical about these apocalyptic AI warnings, suggesting thattheyāre a distraction from the harm that the technology is already causing. Journalist Brian Merchant, for example,wrote that he has yet to seeāa credible, step-by-step documentation of how exactly AI might move from self-recursively improving AI to killing every single human on the planetā; he also suggested that proposals similar to Amodeiās āwould likely only wind up serving Anthropic and OpenAI; itās what regulatory capture looks like in action.ā In his new post, Amodei wrote that he continues āto believe that AI can enormously improve the quality of human life.ā āMy desire to achieve these benefits is undimmed,ā he said. āBut the benefits will only be achieved if we build the technology in the right way, and ā so long as we use the time we gain well ā it is worth taking unusually deliberate care to get it right.ā
