Latest AI News
5 days left to exhibit at TechCrunch Disrupt 2026
The clock is ticking. Book yourexhibit tableforTechCrunch Disrupt 2026by September 18 at 11:59 p.m. PT. TechCrunch Disrupt 2026takes placeOctober 13–15 at Moscone West in San Francisco, bringing 10,000+ founders, investors, operators, and tech leaders together for three days of startup-building, fundraising, networking, product discovery, and dealmaking. Attendees come to find customers, investors, partners, talent, and the technologies shaping what’s next. If you want your startup in the middle of those conversations,exhibit on the Expo Hall floor. Exhibiting puts your team in a high-traffic environment for all three days with a6′ × 30″ branded exhibit tablewhere you can demo your product, meet prospects face-to-face, and turn conversations into leads. Your package includes: This is a chance to put your product directly in front of people actively looking for startups to back, tools to use, and companies to partner with. 5 days. Limited tables. This Friday. September 18, 11:59pm PT. You can still be a part of Disrupt.Join 10,000+ founders, investors, and tech leaders at TechCrunch Disrupt 2026 from October 13–15 and explore 300+ startups across all industries showcasing in the Expo Hall, six industry stages, high-impact networking including AI-powered matchmaking, and Startup Battlefield 200.
View
Hear how AI can engineer nature’s comeback at TechCrunch Disrupt 2026
Not long ago, bringing an extinct species back to life belonged to science fiction. Today, it’s the mission of a billion-dollar startup. Advances in artificial intelligence, genetics, and computational biology are changing what’s possible in conservation research, raising questions that extend far beyond the laboratory. If technology can help revive extinct species, should it? And what role should AI play in shaping the future of biodiversity? Those are the questions at the center of “Can We Engineer Nature’s Comeback?,” a Real World AI Stage fireside chat atTechCrunch Disrupt 2026. Joining the conversation isBen Lamm, co-founder and CEO ofColossal Biosciences, a company pioneering efforts to bring extinct species back through advances in synthetic biology and AI. Rather than focusing solely on the science, the discussion explores the technologies behind de-extinction, the role AI now plays in modern biology, and the growing debate over whether engineering nature represents a conservation breakthrough — or a distraction from protecting the ecosystems we already have. Disrupt returns to Moscone Westin San Francisco, October 13–15, bringing together 10,000+ founders, investors, and operators for 250+ sessions exploring the technologies and market forces shaping what’s next.Hear one of this year’s most provocative conversations.Secure your pass to Disrupt. AI is already reshaping software, robotics, and scientific research. Biology may be its next frontier. Researchers are using AI to analyze genetic data, model biological systems, and accelerate discovery. Companies like Colossal are pushing those advances even further, applying AI alongside synthetic biology to questions that until recently belonged in science fiction. This sessionexplores where those technologies intersect and what happens when AI moves beyond digital systems and into the natural world. For founders working at the intersection of deep tech, life sciences, and AI, it’s an opportunity to hear how one of the industry’s most ambitious companies is approaching problems few organizations have attempted to solve. Explore theReal World AI Stage lineupandregister for Disrupt today. Ben Lamm has spent more than two decades building companies at the intersection of AI and emerging technologies. A serial entrepreneur, he’s built businesses focused on solving complex challenges — from enterprise AI and critical infrastructure to conversational intelligence. Before founding Colossal Biosciences, he launched several pioneering technology companies, including Hypergiant, Conversable, and Chaotic Moon, which were later acquired by Trive Capital, LivePerson, and Accenture, respectively. Today, he’s applying that entrepreneurial mindset to an entirely different challenge: using AI and synthetic biology to explore de-extinction as a conservation tool. That work has made Colossal one of the most closely watched — and most debated — companies in science and technology. As CEO, Lamm sits at the center of conversations about biodiversity, climate resilience, emerging technologies, and the responsibilities that come with engineering living systems. His experience offers founders a rare look at what happens when breakthrough technology doesn’t just disrupt an industry, challenging how we think about what’s possible. Join the conversation with one of tech’s most unconventional founders.Secure your Disrupt pass today. Reviving extinct species captures headlines. The broader questions may prove even more significant. As AI becomes part of scientific discovery, conversations about technology increasingly extend beyond software into medicine, biology, conservation, and climate science. Those advances bring new possibilities, but they also raise difficult questions about how, and where, these tools should be used. Whether you view de-extinction as a scientific breakthrough, a conservation strategy, or a controversial experiment, the discussion reaches far beyond any single species. It asks what role emerging technologies should play in solving some of the world’s most complex problems and who gets to decide. Lock in your pass to Disrupt withup to $200 in savings, and join the conversation about where AI, biology, and innovation intersect.
View
Superhuman acquires YC-backed notetaker Fathom as productivity platforms push for agentic work
Superhuman is joining the cadre of productivity platforms launching notetakers — but rather than building one internally, it is acquiring Y Combinator-backed Fathom. AI-powered productivity platforms aim to automate work. Typically, that automation is triggered when a user types prompts and creates workflows. It would be even better, however, if AI could proactively start work, and for that, the topics and action items discussed in meetings are key. Hence the desire bySuperhumanand similar platforms to move into notetaking. To date,Fathom, which was founded in 2020, has raised over $30 million in funding, and it was valued at $94 million in 2024, according to data from PitchBook. Its investors include Zoom Apps Fund, Telescope Partners, BoxOne Ventures, and Maven Ventures. Its cap table also includes individuals such as Reddit CEO Steve Huffman, former Twitch CEO Emmett Shear, and Cruise co-founder Kyle Vogt. The notetaker offers a generous free plan, and that has resulted in over 400,000 monthly active users. The company said that over 1 million people have recorded meetings until now. TechCrunch learned that earlier this year, Superhuman tested a notetaker internally with select users, but eventually chose to acquire a finished product to integrate with its productivity suite, with AI at the center. Shishir Mehrotra, Superhuman’s CEO, acknowledged this; he said that in the process of testing, the company realized how hard it is to create a notetaker. “We’ve been working internally on this area for a while as well. I think it greatly increased our confidence in the demand in this category. It also becomes very clear how deep the products in this category are and how much you need to build. A lot of these products make the job feel really easy, but it’s actually quite tricky to do a great job in all parts [of meeting recording],” Mehrotra told TechCrunch over a call. Mehrotra said that he was impressed with some of the technical intricacies built by the Fathom team, but didn’t elaborate on which ones. He did tease that with a notetaker, Superhuman can work on surfacing meeting context in real time. Competition in the meeting notetaker space is intense, and startups likeGranola,Read AI, and new entrantWisprhave raised millions of dollars to aid that growth. Despite having a healthy customer base or revenue run rate, venture-backed notetakers are feeling the heat from the market to do more than just notetaking. Some of these factors did matter in Superhuman acquiring Fathom, while Fathom’s CEO Richard White said that Superhuman’s 40 million user base was a good opportunity to reach users at scale. “The reach of Superhuman is massive. I have always thought about M&A in terms of speed of building a product that everyone uses, and a part of that comes down to distribution. If we were to operate Fathom as a stand-alone company, we would have to build a lot of things that are already in the Superhuman platform,” he said. On the platform front, Superhuman already hasan email client, a docs app, a calendar, a database solution, anda newly launched AI agent builder platform. With a meeting notetaker in the middle, Superhuman can draft emails, update data entries, schedule follow-up meetings, and have AI agents start work and extract insights from meeting data.
View
Only at TechCrunch Disrupt 2026: What happens when OpenAI ships your roadmap?
For AI founders, the biggest competitive threat may no longer be another startup. It may be the platform you’re building on. Every major release from OpenAI, Anthropic, or Google raises the same question: What happens if the feature you’ve spent the last year building suddenly becomes part of the platform? It’s a question reshaping product strategy, fundraising, and company valuations across the AI ecosystem. Not long ago, founders worried about competing with other startups. Today, they’re competing with foundation model companies introducing entirely new capabilities every few months. That changes almost every strategic decision — from what products to build and where to differentiate to how to raise capital and create lasting value. The conversation moves from “Can we build it?” to “Can we still own it?” That’s the challenge at the heart of “What Happens When OpenAI Ships Your Roadmap,” aBuilders Stage sessionfeaturing leaders Michel Tricot (Airbyte), Rob Toews (Radical Ventures), and Linda Tong (Webflow), atTechCrunch Disrupt 2026,taking place October 13–15 at Moscone West in San Francisco. Disrupt is the flagship startup event where 10,000+ founders, investors, and operators gather for 250+ sessions exploring the market forces reshaping how startups are built, funded, and scaled. See how founders are responding to AI’s biggest competitive shift.Secure your pass to Disrupt with savings of up to $200 before rates increase on September 25. One of the biggest strategic risks facing AI startups today isn’t losing to another founder — it’s discovering that your competitive advantage has become someone else’s product update. That’s the reality this session is designed to confront. As foundation models rapidly evolve, capabilities that once differentiated an AI startup can quickly become table stakes. Increasingly, founders aren’t just competing with other startups — they’re competing with the platforms powering the AI ecosystem itself. Here, founders will examine why the next generation of AI winners may not be defined by the smartest models. They’ll be defined by everything the models can’t easily replace: proprietary data, deeply embedded workflows, customer relationships, domain expertise, and trust. The reality founders increasingly need to prepare for is, “Can we build something customers will still value after the next model release?” This session explores where defensibility still exists, how founders can respond when AI giants move into adjacent markets, and what separates companies that become features from those that remain businesses. Learn how founders are building beyond the next model release.Register for Disrupt today and save up to $200. To unpack this challenge, theBuilders Stage sessionbrings together a founder, an operator, and an investor, each offering a different view on creating defensible AI companies. How do you create lasting value when the technology underneath your product keeps changing? Michel Tricothas spent his career building the data integration infrastructure that powers analytics, operations, and AI. As CEO and co-founder ofAirbyte, he has grown the open source platform to more than 7,000 customers — including 18% of the Fortune 500 — giving him firsthand insight into where durable businesses are built beyond the foundation model. How do software companies stay differentiated as AI reshapes customer expectations? As CEO ofWebflow,Linda Tongis leading one of the industry’s top visual development platforms through one of SaaS’s biggest technology shifts. Her experience scaling Webflow and leading teams at Google, Cisco, and the NFL gives founders practical insight into evolving products without losing competitive advantage. What convinces investors that an AI company will still matter three years from now? AtRadical Ventures, partnerRob Toewsevaluates AI startups every day, identifying where competitive advantages still exist — and where products risk becoming features. Together, they bring unique perspectives to one defining question facing AI startups today: What can you build that the platforms can’t simply ship themselves? Secure your Disrupt passand hear how founders are building companies, not just AI features. Foundation models will continue to improve. That much is inevitable. As OpenAI and Anthropic continue expanding their capabilities, founders must decide how they’ll continue to differentiate. Tomorrow’s AI leaders will be defined by everything the models can’t easily replace — companies that customers continue to choose because of the workflows they enable, the data they own, the problems they solve, and the trust they’ve earned. That’s the strategic shift this session is designed to explore. Rather than reacting to every new model release, founders will hear where defensibility still exists, what enterprise and startup buyers continue to value, and how to build companies that remain relevant as AI evolves. If you’re building an AI company, the question isn’t whether foundation models will continue to evolve. It’s whether your company will continue creating value as they do. The greatest risk isn’t building a weak product — it’s building a strong one that eventually becomes someone else’s feature. You don’t want to miss this interactive session on theBuilders StageatTechCrunch Disrupt 2026. Secure your pass to Disruptto hear how founders, operators, and investors are redefining AI defensibility, and save up to $200 before September 25.
View
Microsoft’s new AI ‘code of conduct’ tells models not to hack systems or trick humans
As the AI world shifts its focus to safety and alignment, Microsoft has releaseda new AI code of conductmeant to guide AI models away from dangerous behavior. The document is more low-level thanAnthropic CEO Dario Amodei’s recent call for pacing the frontier, instead focusing on the values and red lines that guide model training within Microsoft AI. Still, the result is a comprehensive guide as to how Microsoft approaches AI safety, and how those ideas are implemented in practice. The document begins with the prediction that, in the next decade, superintelligent AI systems will surpass human performance in most tasks. “Containing, controlling, and aligning such a powerful force is one of the greatest challenges humanity has ever faced,” the code of conduct continues. “We must therefore be completely clear about why we are inventing these systems and how we intend to control them.” The code of conduct also lays out general principles that Microsoft AI models should uphold — supporting humans rather than replacing them, for instance, and accelerating human flourishing — as well as specific safety constraints meant to implement those principles. Under Microsoft’s system, each model has an overarching code of conduct that overrides the preferences of individual users or any specific tasks. That includes “absolute constraints” forbidding cyberattacks, nuclear weapons, or deepfake production. It also includes broader provisions against a general loss of human control. “MAI Modelswill not use adaptive, deceptive, self-reinforcing, collusion, or other mechanisms to evade or defeat human oversight so that they can no longer be reliably directed, modified, or shut down by authorized people or systems,” the document reads. The release comes amid an unprecedented focus on AI safety, driven bya string of rogue-agent incidentsas well asthe abrupt resignation of an Anthropic employeewho cited the growing risk that AI would cause human extinction. Together with Anthropic, OpenAI, and xAI, Microsoft has broadly embraced a general approach of pacing the frontier, with particular support for embedded evaluators in AI labs. “We welcome the research, focus, and deliberate pacing needed to get alignment right as the design goal,” Microsoft CEO Satya Nadellawrote online. “We also welcome ideas like “embedded evaluators” and the broader efforts to develop the mechanisms to make this more than just talk.”
View
AI Leaders Reach Consensus: Amodei, Altman, Musk, and Hassabis on Extinction Risks
Artificial Intelligence has been the centre of multiple debates for different reasons. While its benefits in making life easier for users and enterprises alike have shown promising results, it also carries immense risks. As AI becomes more deeply integrated into our lives, economists and AI scientists have flagged the risks it poses to the job market. As enterprises continue to accelerate AI adoption at an unprecedented scale and investors flood AI firms with capital, experts have also begun flagging a possible future in which human existence becomes extinct.
View
Beyond Bengaluru: How North Karnataka is Building an AI Corridor
More than 30 companies across the Hubballi-Dharwad-Belagavi cluster are developing, offering, or using AI-led products and services.
View
What’s behind the AI industry’s latest warnings of doom?
The AI industry seems to be having its loudest debate yet about whether its technology poses an existential threat to humanity. The current discussion began afterAI researcher Jacob Coxon said that he’s resigned from Anthropicbecause he’s worried that the leading AI companies are “gambling with our lives.” Then Anthropic’s alignment leaned chimed in witha post declaring, “We really do earnestly believe AI could kill all humans!” adding that he personally thinks the chance is “>10% within the next decade.” On the latest episode ofTechCrunch’s Equity podcast, Kirsten Korosec, Sean O’Kane, and I discussed the latest apocalyptic warnings. I tried to articulate why I’m skeptical of many AI doomer narratives, while Kirsten asked if this was “just a weird way of flexing to show how far advanced their company’s AI model is,” particularly as these companies prepare to go public. And Sean wondered how these concerns might show up in Anthropic’s S-1 filing for its IPO: “Are there junior lawyers right now who are going through and having to rewrite that entire section of the S-1 filing to say, ‘It’s a officially Anthropic’s position that there’s a more than 10% chance that we could develop something that would eradicate all of humanity and that would be materially bad for our business’?” Keep reading for a preview of our conversation, edited for length and clarity. (Note: We recorded this episode before Anthropic CEO Dario Amodeipublished his plan for more cautious AI development.) Sean O’Kane:I’m hard-pressed to think of something that blew up so fast. Not only did this warning shot come out from this young researcher who has also worked at OpenAI, but also was immediately shared on X by the alignment lead at Anthropic — who, in what might go down as one of the best misplaced exclamation marks ever, shared Coxon’s post and and thread and said, “We really do earnestly believe AI could kill all humans!” Exclamation mark! What a weird vibe. That was just a ton of accelerant on an already fraught post or series of posts. Coming after the Hugging Face hack from OpenAI’s internal model, plus just the increased capabilities we’ve seen with the latest models released by Anthropic and and now OpenAI with with Astra a few weeks ago, I think this was just perfectly timed to be a powder keg type of thing for this young researcher to say. Anthony Ha:Just to disagree with you, I do think that if you believe that AI could destroy all humanity, that does deserve an exclamation point. I would argue that that is a perfectly well used exclamation point! My issue with that tweet was more the “we.” Who is the “we” here? To what extent can we talk about sort of the AI community or AI research community as a monolith? And the greater than 10% chance — that’s just a made up number, that doesn’t mean anything. Sometimes [there is] this habit in both the tech industry and other places to just throw out these percentages, they’re not based on anything or calculated based on anything. [In retrospect, I realize the tweet was probably referencingthe concept of P(doom), but I still think it’s silly.] One thing I will say about Coxon’s statement and decision is — there’s this recurring theme on Equity, when someone like Sam Altman or Dario Amodei is doing this doomer narrative, there’s always this element of: Well, then, why are you doing what you’re doing? If you actually believe that [AI could destroy humanity], you would not continue doing this. [Whereas] this is actually somebody putting his professional trajectory where his mouth is. He’s actually saying, “I believe this is really, really, really bad, and I don’t want to keep working on it.” And so, props for having the courage to do that, if nothing else. Kirsten Korosec:Yeah, I put him in a separate camp than everyone else saying that and talking about the dangers. I’m going to put my speculative hat on, because I want to ask both of you a question, which is: Is it possible that every single time we see the increasing number of blog posts about yet another incident in which one of their AI agents breaks through unintentionally, or they talk about how humanity is at risk, is this a weird way of flexing to show how far advanced their company’s AI model is? I mean, that sounds very cynical, but it does achieve that purpose. Which is: If these AI models weren’t advanced and weren’t capable and weren’t breaking through, we wouldn’t have to worry about these things, right? It’s like a very weird way to brag about the capabilities of the models that you’ve created within your own company. Anthony:I’ve definitely wondered about this. I don’t think it’s completely cynical, in the sense that I don’t think it’s all just a very conscious marketing ploy across the board. I think that when a lot of these people — whether the researchers or CEOs — talk about it, they do have real concern. But of course, it does align with [their] business interests in a lot of ways, to say, “Wow, we’ve built the most deadly software that’s ever been made.” I don’t want to get too psychoanalytic here, but others have pointed out that there is this temptation on a personal level of: Of course, you want to believe that the thing you’re working on is the most important and most dangerous thing in the world. Sean:The thing that sticks out in my mind when I think about that question is, there’s certainly an element that makes it seem like, “Okay, we’re doing this thing that’s so capable, and that’s good for us in some way, even if it looks bad in a lot of different lights.” I think what’s different about some of these most recent examples is, it really gives you the feeling that these companies don’t have a handle on this stuff in certain ways, especially with the OpenAI stuff. We keep seeing more and more reporting about otherinternal agents that have accessed different wikis on the weband are leaving messages for each other, and in a way that doesn’t seem like it’s being handled in a competent way from OpenAI. I would imagine there would be just a bit more polish on the story being told, if it was wholly about getting people to believe that, “Oh my gosh, they’ve made something so incredibly capable.” The other thing that I think is really fascinating about this, in particular, [is] we’re what, a few weeks at most out from seeing Anthropic’s S-1 filing for its IPO, and just a couple more weeks or month or two away from a potential IPO. And the idea that you’re going to come out and say these things in this clear language ahead of an IPO — I’m very interested in what that means for that process. How much of this kind of stuff had they already written into the S-1 and the risk factors inside that document? Are there junior lawyers right now who are going through and having to rewrite that entire section of the S-1 filing to say, “It’s a officially Anthropic’s position that there’s a more than 10% chance that we could develop something that would eradicate all of humanity and that would be materially bad for our business”? Kirsten:You’re assuming that it’s not in there already. Sean:That’s what I’m saying, though: Is it in there already and being reworded? Or is this something that’s a true scramble? There has to have been language in there. It’s one of the reasons I’m so eager to read this document in a way that goes even further, in some ways, than the SpaceX [S-1], because I’m sure that there’s probably stuff specific to these ideas that will be interesting to see. Kirsten:Here’s the thing: In a traditional investment environment, one might believe that language like this would hurt the valuation of a company, because it’s suddenly dangerous. But we don’t live in normal times. And so again, back to my point, it could end up being a weird beneficial flex for the company on the valuation side. It’s not the same as the whole rage-baiting trend that we saw last year, but it’s in that same, let’s say, universe, in which the strength, capability, even elements of danger of something, equals high valuation. So I guess we’ll see in a few weeks. Putting that aside for a minute, what is being done about it? And can we control this? Tthe U.S. executive director of a nonprofit called ControlAI, Connor Leahy,he was on the show this week, talking about this. So what are you paying attention to in terms of how to control the dangerous aspects of AI, or are we throwing up our hands and watching it all unfold? Anthony:I myself do not necessarily have a great answer to this, but I have been thinking about some aspects of this debate and maybe why I respond the way I do. To echo one of Sean’s points, I do think that part of what this speaks to is the extent to which these major AI companies are feeling like they’re not really in control of these models anymore. That’s definitely not great. That is something that we should all be worried about. I do think that part of the reason I’m skeptical of the doomer narrative or resistant to the doomer narrative is because it reaches this level of hysteria of, “Wow, this could destroy humanity in the next 10 years.” It is a little bit of a distraction from the more immediate harms that AI can have, whether that’s labor-related, whether that’s environment- and climate-related. Ideally, I think we should be able to discuss all of these things, and have regulatory and other kinds of safeguards against all of these things [including AI’s existential threat]. But once you start using phrases like AGI and superintelligence, that just sucks up all the oxygen in the room in a way that is not very helpful.
View
Obama urges Democrats to have a ‘clear plan’ for AI safeguards
Former President Barack Obama recently said that Democrats need to make artificial intelligence one of their “central agendas” and “have a very clear plan” to address concerns around the technology’s economic impact and safety,according to The New York Times. Obama made these comments on Thursday, at Democratic fundraising event where he was interviewed by House Minority Leader Hakeem Jeffries. The NYT says that Obama’s office provided a partial transcript of the event, at which Jeffries asked Obama how congressional Democrats should approach AI. In response, Obama said that once Democrats regain the House majority, they need to “put together a framework for a very public conversation.” “This is something that is moving very fast in private hands, and if we don’t get on top of it, I think can be dangerous,” Obama added. “If we do get on top of it, I do think it’s beneficial. I genuinely think it’s going to accelerate, for example, drug development in ways that can help us cure diseases.” In a statement, Jeffries said the former president “is correct that decisive action must be taken on artificial intelligence.” Jeffries also said that “Republicans have abdicated their responsibility to govern on behalf of the American people.” The NYT also reports that Obama has offered himself as a “sounding board” to AI executives and has spoken to both Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman. His comments come amidst growing concern around AI safety, particularly afteran AI researcher resigned from Anthropicand claimed the leading AI companies are “racing straight to self-improving superintelligence and gambling with our lives.” On Saturday,Amodei outlined a broad approach to “pacing the frontier,”which would include giving independent safety evaluators access to leading AI companies and models, as well as developing “common safety standards” between companies. In social media posts, Altman and SpaceX CEO Musk seemed to respond positively to Amodei’s plan, with Altman saying that OpenAI would also commit to “having independent evaluators with employee-like access.” President Donald Trump, meanwhile, discussed AI safety concerns with reporters at an Irish golf event on Sunday.Bloomberg reportsthat Trump boasted that the United States is “the most sophisticated country in the world,” adding that he wants “to keep it that way because whoever wins AI wins.” (Earlier this year, the Trump administrationreleased a legislative framework for AIthat would preempt state laws and shift the child safety burden to parents.) “And we can put guardrails,” Trump said. “We can do this and that. But I think you have a lot of negative forces that are bringing it up that shouldn’t be bringing it up.”
View
OpenAI’s Sam Altman says it would be ‘ill-advised’ to go public in 2026
While OpenAI hasfiled confidentially for an IPO, the companywill not be going public this year, according to CEO Sam Altman. Altman wasinterviewed recentlyby Fortune editor in chief Alyson Shontell; amidst the fallout fromthe OpenAI-HuggingFace hack, as well asbroader discussions about AI safety, Shontell asked whether OpenAI still feels pressure to “move really fast” due to its IPO plans. “We’re not rushing into an IPO,” Altman said. “I actually think that given everything happening with safety, right now would be an ill-advised moment to go public.” Instead, he insisted that OpenAI will go public “when we’re ready, which is when the business is ready, when we feel ready from what the moment is like in society with this technology.” When pressed on whether that means the IPO isn’t happening in 2026, Altman replied, “I would say not 2026, yeah. We’ve got a lot of stuff to do.” The New York Times reported in Junethat although OpenAI had hired bankers and lawyers with the goal of going public in the third or fourth quarter of 2026, the company was leaning toward 2027 due to the volatility of tech stocks and its own financial challenges.
View
Anthropic CEO outlines plan to ‘pace the frontier’
We’ve been seeing increasingly dire warnings from AI researchers about the dangers of artificial intelligence, and evencomments from OpenAI CEO Sam Altmanthat it may be time to “pace” AI development. But what would that actually look like? Ina new blog post, Anthropic CEO Dario Amodei not only echoed the call to “pace the frontier,” but also outlined three broad strategies for doing so. And he said Anthropic is “unilaterally committing” to one of them. Thedebate over AI safety and alignmentintensified this week afterresearcher Jacob Coxon wrote that he’s resigning from Anthropicover concerns that the leading AI companies are “gambling with our lives” while the people building the technology “earnestly believe it could kill us all by the end of the decade,” a claimrepeated by others at Anthropic. Amodei’s post doesn’t didn’t explicitly mention Coxon’s resignation or his concerns, but the CEO wrote that two things convinced him it’s time to take a more cautious approach to AI development:the OpenAI-HuggingFace hack, and the fact that “AI has been advancing drastically faster” in recent months, particularly with its “growing ability to build the next generation of AI.” “We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote. “Progress will still seem fast, and we must make wise use of the time we gain.” His proposed first step would involve “embedded evaluators” from third-party organizations likeMETR— evaluators who can verify that AI companies are actually following their pacing and safety commitments and can also ensure that safety incidents get reported. (OpenAI was recently criticized fornot reporting an incident where its AI agents took over a German wiki form.) Amodei compared these evaluators to regulators who have been embedded with bank employees, and he said that inviting them in is “something Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match).” That means giving evaluators company badges, desks, and laptops, and providing access “mostly comparable to what internal risk assessment teams have,” with exceptions when required by law or contracts. Next, Amodei called for the leading AI companies “within democratic countries” to coordinate “common safety standards as well as limits on the rate of unchecked AI progress.” Such coordination might seem unlikely, both due tothe apparent animosity between Altman and Amodeiand also because their companies arereportedly worried that a coordinated pause could lead to antitrust scrutiny. Amodei alluded to that concern in his post, writing that “for antitrust reasons, it’s helpful for the US government to mediate or at least enable these discussions — they don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.” Amodei also acknowledgedthe spectre of Chinese AI dominancethat’s often raised an argument against slowing development. But he said that if the US government and tech companies take steps like refusing to sell powerful chips or semiconductor manufacturing equipment to Chinese companies, as well ascracking down on model distillation, they could “slow China’s progress enough to widen America’s lead significantly over the next 3–5 years.” Lastly, Amodei called for “global coordination,” where the United States and its allies “attempt to coordinate with authoritarian governments, to the extent this is possible.” Amodei said this would mean “cooperation with China,” and he admitted that there are “stark limits on what can be achieved,” but he still suggested there might be opportunities for agreement, even if it’s just “prohibiting certain narrow and obviously dangerous uses of AI, such as using AI for the production of biological weapons or allowing users to do so.” With Amodei’s past willingness to acknowledge AI’s potential dangers, and with the company’s relative openness to certain forms of regulation, some AI boosters have already criticized him as a doomer whose comments have fed the current AI backlash. In response, Amodei said he’s tried to offer a “balanced” perspective” andargued that the backlash is “fundamentally a crisis of trust,”as people have become skeptical of tech companies, the tech industry, and the government. Industry critics have also been skeptical about these apocalyptic AI warnings, suggesting thatthey’re a distraction from the harm that the technology is already causing. Journalist Brian Merchant, for example,wrote that he has yet to see“a credible, step-by-step documentation of how exactly AI might move from self-recursively improving AI to killing every single human on the planet”; he also suggested that proposals similar to Amodei’s “would likely only wind up serving Anthropic and OpenAI; it’s what regulatory capture looks like in action.” In his new post, Amodei wrote that he continues “to believe that AI can enormously improve the quality of human life.” “My desire to achieve these benefits is undimmed,” he said. “But the benefits will only be achieved if we build the technology in the right way, and — so long as we use the time we gain well — it is worth taking unusually deliberate care to get it right.”
View
What Really Happens When You Turn Your Selfie Into a 1980s AI Pic?
The viral trend raises questions about personal data, computing costs, and what AI companies gain from everyday image generation.
View
