Últimas Noticias de IA
OpenAI Reveals 6 New AI Incidents
The company has also published a new framework for reporting AI model misalignment.
View
Anthropic Merges Claude Cowork and Chat Into One Experience, Bringing Research, Files to Every Chat
Anthropic is merging its Claude Cowork and Chat experiences into a single conversation, the company announced on Wednesday. The move eliminates the need for users to choose between Chat and Cowork before starting a task. Instead, the AI chatbot can automatically detect the tools it requires to carry out the designated task based on the request. Anthropic said the unified experience can handle both simple questions as well as more complex tasks.
View
Al Gore says the real AI risk isn’t data centers
Al Gore has spent more than two decades as one of the most recognizable voices in the climate movement, so when he talks about AI, it’s worth paying attention to what he considers truly scary. And it isn’t what has peopleprotesting data centersin the streets. In an interview with TechCrunch this week alongside Lila Preston, head of growth equity at their investment firm Generation Investment Management, Gore said emissions from AI data centers aren’t what should keep people up at night, despite the mounting opposition over their environmental impacts. What should concern people more, in his view, are the warnings coming from inside the AI industry itself, and he takes them seriously. It’s not a new position for Gore. This past May, for example, he described AI data centers toanother outletas a “cause for deep concern, but not panic,” and in our conversation this week, his central point about emissions was about scale. “If you look at the emissions of all of the AI data centers put together, it’s only a fraction of the emissions from uncovered landfills in the world,” he said, adding that a comparable build-out is tied to something that gets far less attention: air conditioning. It’s a valid observation (and one the AI labs should be using in their discussions with cities and states if they aren’t already). The International Energy Agency estimates that global air conditioning already consumes more electricity annually than the entire European Union, and with ownership still under 15% across the hottest, fastest-growing parts of the world, that demand is expected to triple by 2050, putting far more pressure on the grid than AI data centers over the same period. Gore isn’t dismissive of concerns about how these data centers are being powered. “It is a matter of deep concern that some of the hyperscalers are jumping into new methane turbines,” he said. But what worries him about methane turbines isn’t the AI workload driving demand so much as the prospect that building new gas plants to meet that demand locks in decades more of fossil fuel generation. “I much prefer those that are supplying their energy needs with renewables and batteries,” he continued, saying he expects more companies to move inexorably in that direction simply because renewables are increasingly the cheapest option available. In fact, Gore thinks the public backlash at planning-commission meetings has less to do with carbon emissions than with anxiety over automation upending society in the not-too-distant future. “I think that part of the reason for the growing bipartisan opposition to data centers in so many parts of the U.S. is being driven by the underlying concerns about job losses and some of the other threats that experts inside OpenAI and inside Anthropic have been trying to alert the public to,” he said. We also pressed Gore on the wave of warnings from AI industry leaders that has dominated headlines over the last week. Though President Trump and Nvidia CEO Jensen Huangshared a laughat their expense on Monday during a conference in Los Angeles (Trump suggested that Anthropic CEO Dario Amodei’s public call toslow the paceof capability gains, quickly seconded by Sam Altman and Elon Musk, was a “hoax”), Gore said he takes the warnings at face value. “I think we went through a phase where some cynically suspected that the apocalyptic warnings were some kind of bizarre marketing strategy. If that was ever the case, I don’t think it is now,” he said. “I do think that Dario Amodei is very sincere, and I think that Sam Altman and Elon Musk were sincere in seconding [Amodei’s] warning of a few days ago.” Gore being Gore (he often reaches for literature, Scripture, and pop philosophy to make a point), he invoked Maya Angelou’s well-known line: “When someone tells you who they are, believe them the first time.” Part of what’s changed his calculus, he said, is recent, concrete behavior from AI systems themselves — pointing to “the recent example of models escaping confinement, collaborating secretly, covering their tracks, engaging in deceptive behavior,” and noting that Anthropic has said it stopped instances of Claude being used to help develop biological weapons in different countries. That said, Gore doesn’t treat AI risk and AI’s climate potential as being in opposition. He cited a recent study from economist Nicholas Stern of the London School of Economics projecting that AI applications aimed at efficiency and waste elimination could drive global emissions down by 6% to 9% per year starting next decade. If Gore’s focus was risk, Preston’s was where capital is already moving in response. She pointed to several fronts where the AI build-out itself is generating investable opportunities, not just consuming energy. Among these, the firm is focused on opportunities to decouple compute from energy intensity at every layer of the stack, from the power supply down to green cement and green steel used in construction, storage optimization software, and database design. The firm is also focused on grid resilience and flexibility, as the mix of power sources grows more complex. Here, Preston noted, Generation has invested inVolue, a European company helping utilities pull more renewables into the grid, andGridware, which places sensors on utility poles to monitor and help protect the grid from wildfire risk. Gore tied the AI moment to a broader argument made in Generation’s 10th annualSustainability Trends Report: This year marks the second time in four years the world has been “brutally reminded” that fossil fuels are a volatile source of energy, first with Russia’s invasion of Ukraine and now with the disruption to the Strait of Hormuz. But unlike past years, when Gore’s exasperation with the pace of the clean energy transition was obvious, he doesn’t see these events as a setback but instead as reasons for countries to embrace it at long last. Indeed, Gore noted that globally, clean energy investment now runs at roughly twice the level of fossil fuel investment. Of all new electricity generation capacity added worldwide last year, 86% came from renewables, and in the U.S. specifically, that figure was even higher, at 91%, “in spite of Donald Trump’s best efforts to slow it down,” he said. China, in particular, received high marks from Gore, who, among other things, applauded Beijing for wanting to be measured on emissions reductions rather than improvements in carbon intensity. Asked what’s changed more than he expected a decade into publishing this report, Gore immediately pointed to solar power. “For me personally, the scope and scale of the solar revolution is the breakout star of the sustainability transition,” he said. After observing that it’s at, or near, the cost floor for new electricity generation right now — roughly tied with wind and meaningfully cheaper than gas, coal, or nuclear — he closed with a line from a friend in Tennessee that had stuck with him: “If God had intended us to have a limitless supply of cheap, clean energy, he — or she — would have put a fusion reactor in the sky,” Gore said with a chuckle. “The joke makes itself.”
View
Snap tries to make the case again for its $2,200 smart glasses
When Snap debuted its long-awaited smart glasses, Specs, earlier this year,the revealwas not only underwhelming but also a genuine disaster. The glasses — which cost a whopping $2,200 and are vaguely reminiscent of scuba gear — inspired jeers online and calls for the company’s CEO, Evan Spiegel, to resign. The company’s stock subsequentlynosedived. Since then, Snap has clearly been looking for an opportunity to explain why Specs — a heavy, expensive piece of hardware that most casual consumers will probably be unable to afford — deserves to exist. This week, the company had its chance to make that argument. At an event in Los Angeles, Spiegel and other Snap executives unveiled a variety of new features and services connected to Specs. The new updates seem clearly designed to help integrate the glasses into a broader array of digital activities and, in doing so, give Specs a clearer sense of purpose. During Wednesday’s event, Spiegel shared details about a number of the new features. After a trippy promotional video that vaguely brought to mind an ayahuasca hallucination, the CEO took the stage and began a series of live demos. New entertainment options — Specs can now connect with HBO Max and Spotify — provide a baseline of interactive fun that smart glasses have largely lacked until now. Yet perhaps the biggest announcement was the launch of Specs Intelligence, which the company calls a new “anticipatory AI” system designed to pair not only with the company’s glasses but also with user devices — including iPhones and Macs. Snap claims that this new system “builds an understanding of your goals, priorities, relationships, and routines from the apps and tools you choose to connect, helping you focus on what matters now while making progress towards your longer-term goals.” Spiegel called this system an “AI native operating system with intelligence designed around your projects” that helps users “connect the dots across Lenses,” Snap’s term for its AR camera effects, available on both phones and Specs, “so you can get things done while focusing on what matters to you.” In practice, it seems to function more like an AI productivity app that can be used with Snap’s glasses but doesn’t require them. Snap has also announced a new program called Specs for Enterprise, which allows the device to be integrated into corporate workflows. On Wednesday, the company announced partnerships with a number of different tech companies — including Amazon, Salesforce, Nvidia, and other firms — in an effort to make the device useful to staff at those companies. What is the real-world utility of this, and who is this service really for? Apparently, the answer is IT technicians. While the actual scenarios in which Specs can be used are a little vague, a demo video played at the event showed an IT technician repairing some hardware with the help of Snap’s new glasses. “We’re building out mobile device management, security controls, and ways to deploy and manage stacks across teams with the world’s most trusted providers, bringing peace of mind to IT departments,” Spiegel said. Other announced features were slightly more head-scratching. For instance, one of the new experiences is an app, developed through a partnership with the NBA and WNBA, that allows users to practice basketball and improve their shot via the glasses’ AR capabilities. But, realistically, who is really going to play basketball while wearing heavy goggles on their head? Also announced Wednesday was a new connectivity package for anyone who wants to use Specs on the go, away from Wi-Fi — like outside the home or office. Sold separately, the package includes a charging case with built-in cellular connectivity. That connectivity, which is being made available via a partnership with Verizon, costs $10 a month for Verizon customers and $20 a month for non-Verizon customers. Spiegel ultimately framed Specs as a new way to engage with computing in a more human-centric format. “I will say it’s an interesting time to be launching a new computer because things in tech are pretty weird right now,” Spiegel said. “We’re being told that AI will take our jobs, that virtual worlds will replace the real one. Well, no, thank you. We do not accept that future. Making people irrelevant is not progress, and computers should make us more important, not less.” Snap has repeatedly justified the product’s cringe-inducing cost by characterizing the device as a computer — albeit one you wear on your face. Whether consumers will go for that or not remains to be seen. The company has announced that it is hosting a Specs pop-up in Los Angeles in October before the devices ultimately ship later this fall.
View
Your startup’s next teammate might be an AI agent: Gusto, Insight Partners, and Leland explain what that changes at TechCrunch Disrupt 2026
The first few hires at a startup can define the company. But increasingly, not every new capability requires a new employee. AI agents are beginning to take on engineering, customer support, research, and operational work that would previously have been assigned to early team members. For founders, that creates a new question before they even open a job requisition: What should people own, and what should be delegated to AI? AtTechCrunch Disrupt 2026, Josh Reeves, CEO and co-founder of Gusto; Michelle Johnson, senior vice president at Insight Partners; and John Koelliker, CEO and co-founder of Leland, come together on theBuilders Stagefor an insightful session called “Hiring When AI Is a Co-Founder.” This sessionwill explore how early-stage companies are building teams where humans and AI agents work alongside each other — and how founders can do that without sacrificing speed, accountability, or culture. Join them at Moscone West in San Francisco, October 13–15, alongside 10,000+ startups, investors, and tech decision-makers exploring what it takes to build and scale the next generation of companies.Register now and save up to $200on your Disrupt pass before rates increase on September 25 at 11:59 p.m. PT. Save up to 30% more when you register as a group. For an early-stage company, every hire is a trade-off between capability, cost, and speed. AI agents add another option. Engineering tasks, customer support, and operational work can increasingly be delegated to systems capable of completing multistep tasks rather than simply helping an employee complete them. That gives founders an opportunity to rethink what their earliest teams should actually look like. Instead of automatically asking, “Who do we hire next?,” the starting question can become: “What work needs to be done — and is a person the best way to do it?” That distinction matters. Giving work to an AI agent can help a small team move faster, but it also introduces new questions around ownership. Who checks the work? Who makes the final decision? What happens when an agent gets something wrong? And which responsibilities are too important to delegate? Those are no longer theoretical questions for companies building AI-native startups. All these questions, and more, will be answered on theBuilders Stageat Disrupt.Grab your passby September 25 at 11:59 p.m. PT to save up to $200. Bring your community and save up to 30%. Josh Reevessees the shift across a particularly broad customer base.Gustosupports more than 500,000 companies across payroll, benefits, compliance, onboarding, HR, and retirement, while combining AI with more than a decade of experience serving small businesses. That puts Gusto close to the day-to-day realities of companies deciding how — and when — to grow their teams. Michelle Johnsonbrings the startup scaling perspective. AtInsight Partners, she works with CEOs and CROs across North America and Europe on go-to-market strategy, revenue organizations, and AI implementation. Before joining Insight, she helped scale Flock Safety from less than $1 million to $90 million in ARR as an early sales and revenue operations leader. Her perspective raises a different set of questions. If AI agents can research prospects, prepare outreach, analyze customer data, or manage parts of the sales process, what should the human members of a revenue team concentrate on? And how does that change the skills a startup should prioritize when hiring? John Koellikersits directly at the intersection of talent and AI. As CEO and co-founder ofLeland, a career and talent platform for the AI era, he brings experience across product and growth roles at LinkedIn, Curated, and Uber, alongside building a company focused on how people develop and apply skills in a changing labor market. Together, they bring three different views of the same challenge: How do you design a startup team when some of the work can be done by agents from day one?Register by September 25 at 11:59 p.m. PT to save up to $200on your pass andhear this session live on the Builders Stage. Bring your team or community with you to save up to 30% on passes. Some of the most important decisions inside a startup are difficult to reduce to a workflow. Who understands what customers really need? Who challenges a bad strategy? Who takes responsibility when something fails? Who builds relationships, motivates a team, or decides when the data is pointing in the wrong direction? As AI agents become more capable, those questions could become more important, not less. The role of an early employee may increasingly be defined by judgment, ownership, and the ability to direct both people and machines — rather than simply completing a larger volume of tasks. That has implications across the startup ecosystem. Early-stage companies have always competed on their ability to do more with less. AI agents could take that principle much further. But a company is more than the sum of the tasks it completes. Founders still need people who can make decisions, take responsibility, understand customers, and create the culture around which a business grows. The challenge is deciding where those people create the greatest value — and where an agent might do the job instead. That’s the conversation Josh Reeves, Michelle Johnson, and John Koelliker will take on at Disrupt. Join them on theBuilders Stage, and explore what happens when AI doesn’t just help your startup — but also becomes part of the team. Then continue the conversation with 10,000+ founders, investors, and operators across Moscone West, the Expo Hall, Startup Battlefield 200, and countless networking opportunities. The startup org chart is changing. Learn to build yours for what comes next, only at Disrupt 2026.Secure your pass now and save up to $200before September 25 at 11:59 p.m. PT. Save up to 30% when you register as a group.
View
For Agentic Coding, Logitech Turns Physical Keypads Into AI Command Decks
Logitech’s MX Keypad enters a market that already includes programmable control surfaces such as Elgato’s Stream Deck and OpenAI’s Codex Micro.
View
Anthropic and OpenAI want to embed safety evaluators. Will they really be independent?
In a lengthyessaypublished over the weekend, Anthropic CEO Dario Amodei made a proposal that the AI industry would have rejected instantly even a year ago: embed third-party evaluators inside all frontier AI companies, giving them the power to report safety incidents, assess whether AI models are truly aligned, and share their unvarnished findings with the world. Amodei said Anthropic would commit to giving independent evaluators like METR and Redwood Research unprecedented access to the company’s systems. CEO Sam AltmansaidOpenAI also would commit to the practice, signaling a potentially profound change in how the industry works with outside research groups. Third-party evaluators who spoke to TechCrunch broadly welcomed the proposal, but said details need to be ironed out — and ideally backed by legislation — if they’re to know whether they will function as truly independent watchdogs or vendors operating on the AI companies’ terms. That deeper access is becoming more important as models get better at recognizing when they’re being evaluated, raising the risk that they’ll behave well during testing while concealing problematic behavior. Researchers say clues to that behavior can be missed when testing the finished model, but uncovered by investigating how it behaved throughout training. “AI companies should be able to answer some very basic questions about their training process, such as: Did the AI ever actively try to undermine its own alignment training while it was going through the training?” Alexander Meinke, head of research at Apollo Research, told TechCrunch. “The answer to this should be an unequivocal no, and right now we are completely relying on AI companies to both carefully check this themselves and then truthfully report this to the public. And we’ve seen from recent incidents that, by default, they will do neither. As embedded evaluators, we could actually check.” Historically, AI companies brought in outside reviewers to test finished models shortly before their release. Now, evaluators that TechCrunch spoke to propose giving them access not just to the final model, but to intermediate versions, or “checkpoints,” from its lifetime of training. Adam Gleave, CEO of FAR.AI, said evaluators could compare those checkpoints to determine when concerning behavior emerged, inspect the post-training environment that rewards models for certain behaviors, and check evaluation transcripts and logs to verify a company’s claims about how a model performed. Whether and when Anthropic and OpenAI plan to provide that kind of access is unclear. Neither company has shared which evaluators they’ll work with, when they will be embedded, how many they’ll bring on, exactly what systems and information they will be able to access or what can be disclosed to the public, despite repeated questions from TechCrunch. Looking under the hood like this matters because models that perform well on safety tests aren’t necessarily safe if they’ve learned specifically how to pass that test. John Steidley, head of strategy at Palisade Research, pointed to an example of a “shutdown resistance benchmark” that measures if the AI will resist being shut down in certain circumstances. “It’s extremely relevant if the AI has been trained specifically to perform well on that benchmark,” Steidley said, comparing it to Volkswagen’s Dieselgate scandal, in which cars were programmed to recognize emissions tests and perform differently under testing conditions. Gleave noted that meaningful access could extend beyond the models themselves, with evaluators being given access to interview employees to check whether a company’s documentation and public descriptions of its safety practices match what happened internally. Amodei did outline a fairly comprehensive proposal that might give evaluators the kind of access they think is necessary, including the right to “publish key findings about risk levels, incidents, practices, and the access they received or didn’t receive — without editorial control by Anthropic.” But evaluators say such a system will only work if AI companies are actually willing to surrender control over the process. Previous efforts at independent evaluations suggest that that surrender will be hard won, as third parties have often run up against tensions over access, time, confidentiality, and what they can say publicly. Gleave said FAR.AI has had to turn down contracts with several frontier developers that wanted too much control over the evaluation process, threatening the firm’s independence. By default, he said evaluators are treated like ordinary contractors: bound by restrictive NDAs and agreements that give developers significant control over what can ultimately be published. There’s also the question of whether reviewers will get enough time and access to do the work they’re being asked to do. When investigating the Hugging Face incident,OpenAI gave METRandRedwoodroughly a week on premises to investigate, and both later said they could not draw confident conclusions due, in part, to scope and timing limitations. A similar issue occurred during the pre-release testing for GPT-6 Astra, which OpenAI has touted as itsmost aligned model yet. According toApollo Research’s contribution to the model card, the firm was given only three days to test Astra, which made it difficult to draw firm conclusions. “Apollo believes that, given the higher rates of eval awareness and limited evaluation window, low rates of misbehavior here do not provide substantial evidence about the model’s alignment or misalignment,” the firm wrote in its evaluation. That track record leaves evaluators with a basic question: Why should this time be different? “It’s certainly possible that Dario and Sam just had a change of heart, and they’re going to be very open about this,” Gleave said. “But the intellectual property of these companies is so incredibly valuable to them, and I think they’re going to, by default, be very careful about what can be shared.” Several researchers who spoke to TechCrunch called for a transparent framework that they all agree to publicly. Part of the framework, says Steidley, should involve standards for what kinds of auditors companies can rely on, lest they try to sidestep the issue by shopping for evaluators that either aren’t qualified or aren’t interested in assessing the most concerning risk. Henry Papadatos, executive director of Safer AI, says the problem, even with a public framework, is that voluntary measures are always dependent on a company’s goodwill. “Ideally, we would have good regulation mandating this…because then companies cannot change their mind tomorrow if they have a big PR crisis,” Papadatos told TechCrunch, noting that it’s also a good means of pushing all companies to adhere to the rules, not only the most willing. Not everyone has signed on. So far, Meta, SpaceXAI, and Google DeepMind have not committed to embedding third-party evaluators, though DeepMind CEODemis Hassabis has proposeda separate industry standards body to independently test frontier models. Google, OpenAI, and Anthropic have alsoprivately been discussing AI safetyplans for weeks. Some laws are already forming around the idea of third-party evaluators. California’s SB 53, signed into law last year, requires large frontier AI developers to publish safety frameworks and report critical safety incidents. A new law, SB 813, signed this month, creates a framework for state-recognized “independent verification organizations” with expertise assessing AI risks. In Europe, the EU AI Act requires frontier developers to conduct and document model evaluations and adversarial testing and report serious incidents. The EU AI Office can also conduct its own evaluations and appoint independent experts. For now the law remains less expansive than what Amodei is proposing, leaving frontier labs largely responsible for deciding how much independent scrutiny they will submit to. Papadatos said voluntary self-regulation is better than nothing, but ultimately, companies can’t demand the freedom to control their own safety rules while also asking the public to trust that they’re following them. “You cannot have it both ways, having zero accountability externally, and then say, ‘I’ll just have my own flexible rules,” Papadatos said.
View
Google DeepMind Sees Early Signs of Recursive Self-Improvement Ahead of Gemini 4
Logan Kilpatrick said the feedback loop could help Google close the gap with other frontier AI labs with Gemini 4.
View
Your AI agents can now control your Google Home devices
Smart home owners will soon be able to control their devices connected to Google Home via an AI agent. On Wednesday, Google rolled out early access to its Model Context Protocol (MCP) server for its Google Home ecosystem, which will allow any AI agent that supports MCP — like Claude, Hermes, OpenClaw, ChatGPT, and Google Antigravity — to securely work with their smart home devices and access their event history. This update will allow people to use natural language instructions to do things like review their camera summaries, monitor smart home activity, control their connected devices, and build their own custom smart home dashboards. To get the connection set up, users will need to create a Google Cloud project and configure it to use Home MCP. They will then provide the MCP configuration details to their agent of choice and ask it to set it up. The agent will then ask them to sign in and grant permissions. (A setup guide will also be offered in the Google Home Developer Center.) The system will support any device in the Google Home ecosystem, including Google Nest doorbells and thermostats and “Works with Google Home” (or Matter) devices, like light bulbs. Google already supports MCP in other areas of its business, including in itsGoogle Cloudanddata platforms, developer tools, andGoogle Workspace, for instance. Support for Google Home is targeted more directly at consumers who are experimenting with AI agents designed to handle everyday tasks. The company said access to the MCP will roll out starting today and continuing over the coming weeks to subscribers who pay forGoogle Home Premium Advancedin the U.S. This is the more expensive, $20-per-month subscription tier that provides features like longer event-based video history, descriptive notifications and detailed alerts, tools to search video history, daily summaries, and more. Google wouldn’t comment on if or when the MCP would roll out more broadly, such as to other subscription tiers or markets. Google is also soliciting feedback from early adopters during this period by way of itsSmart Home for Developers Community, it says. An earlier version of this post misstated the rollout’s start. It is today, Wednesday, not Tuesday.
View
AI labs want in-house auditors — but maybe they should shut the front door first
Last weekend, after one of his researchers resigned over fears that AI could lead to human extinction, Anthropic CEO Dario Amodei wrote about the need for outside organizations “to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.” Executives at OpenAI, Google, and SpaceXAI have alreadyrallied aroundAmodei’s plan, which has quickly become a central pillar of the emerging AI safety push. But there may be a simpler and more effective fix hiding in plain sight. Internet security experts say the labs need to focus on network security basics like logs and permissions, applying the same rigorous defenses they do for human users. It’s not as exciting as third-party auditing and alignment work — but it may end up being more effective. “To me, it seems like they’re outsourcing,” Katie Moussouris, the CEO of Luta Security, told TechCrunch of Amodei’s proposal. “Saying [a third-party audit] is the solution is a strange proposition from my perspective. It would be the same as if, instead of writing theTrustworthy Computing Memo, Microsoft said, let’s slow down development.” That memo, written by then-Microsoft CEO Bill Gates in 2002, called on his employees to ensure that their software would be reliable and safe following a series of widely publicized computer worms that took over then-nascent enterprise systems. The AI sector may be at a similar turning point, as the value and risk of the new technology becomes increasingly clear. While alignment remains an important concern, Sayash Kapoor, an AI researcher who will be a professor at UC Berkeley starting next year,arguesthat “marginal investments in control are more likely to be effective compared to those in alignment. We view these incidents as illustrating the lack of emphasis on AI control within companies, despite the availability of known techniques.” The incidents that have spurred these concerns revolve around frontier models being asked to complete training tasks, usually cybersecurity evaluations, and then accessing the open internet and penetrating closed third-party systems in an attempt to do so. They usually did so because ofpoorly configured“sandbox” environments that are supposed to contain these agents; ironically, one Anthropic break-out happened because third-party evaluators didn’t close the right doors. “We as a profession know how to block access to the internet,” Avery Pennarun, the CEO of Tailscale, a security company, said. “If you read through all these big long [reports] — ‘wow, that was a very impressive multi-stage attack, blah, blah.’ Look, you gave it access to download stuff. You should have not done that separately from the internet.” That’s one problem — but a bigger problem is that frontier labs were unaware of these activities. “What was really profound was that all of the discoveries of what they were doing happened either because a victim saw something, or in some of the other cases … it was network activity, and none of it was actually from monitoring the AIs directly,” Moussouris points out. In one case, where OpenAI agentstook overa defunct German WikiForum to cheat on evaluations, the agents were active for weeks before anyone at the company appeared to notice. Security experts that TechCrunch spoke to said that real-time monitoring is key to preventing future break-outs, and that every agentic session should be time-limited and expire. Shapor Naghibzadeh, a former Google security executive who now leads thestartup QueryStory, says the solution is to “put the agent in a box and instrument it heavily from the outside looking in and watch everything that crosses the boundary. Every tool call, every process, every network connection, no exceptions. …The one hole you leave open for convenience is the one that gets used. The bypass went through exactly that kind of exception. [At Google,] I watched that movie many times with human attackers, and these models are at least as good at finding the propped-open door.” OpenAI has begun moving in that direction, announcing that it had begun monitoring all tool-using inference by its Astra model, at “significant compute cost.” Anthropic, too, says it ishardening its securityprocedures, including expanding observability of its models. Neither company responded to TechCrunch’s questions about how they track and control AI agents. Other problems are the use of shared infrastructure by agents, which allowed them to communicate during the Hugging Face attack. Simon Willison, a software developer who co-created the Django web framework, has written about something he calls the “lethal trifecta” — when agents have access to untrusted input, the internet, and private information all at the same time, it’s a recipe for disaster. “The trick is you can pick any two legs of the trifecta and an agent can have any two,” Pennarun said. “If you need all three, then you need to split it across at least two agents … and maybe they’re allowed to talk to each other through a controlled channel.” Experts TechCrunch spoke to understand that frontier lab security personnel have difficult jobs. Naghibzadeh points out that every nation-state actor on Earth is trying to steal their model weights and mount distillation attacks on their APIs, as well as the bread-and-butter security tasks of any large digital company. “Research infrastructure has a hard time rising to the top of that priority stack, although that must be changing now,” he said. “Making security incidents public really helps align everyone internally toward the goal of improving.” That’s one note that Moussouris emphasizes: Right now, there is no formal victim notification procedure when the labs discover their agents have penetrated third-party systems, and it is likely that there have been other incidents that have not been widely publicized. While she worries that laws that regulate models directly may have unintended consequences, mandatory notification is one idea she believes policymakers should pursue. And while it’s clear that security best practices weren’t being followed, experts say that the labs are doing work no one has done before — “they’re doing orders of magnitude more than your typical enterprise,” Zack Korman, the CEO of cybersecurity firm Embroidery, told TechCrunch. And while alignment may not be the place to start, it can’t be ignored. Cybersecurity experts are resigned to having to use AI agents to monitor other agents if they are to have any chance of tracking their behavior in real time, a scenario where the potential for deception raises its ugly head. “You’re trapped using AI to try and deal with this, even though AI is not necessarily safe right now,” Moussouris said. The job will only get harder. Everything agents are doing now, Moussouris says, “they are doing loudly” — they are posting on public forums, and their chain of thought and other reasoning traces are in English. “It’s still human readable,” she says, “so take advantage of that for as long as that lasts, because it won’t last forever.”Additional reporting by Aditya Mehta
View
After accusations of selling ‘perv glasses,’ Meta prepares to sell a pair without a camera
Meta’s camera-equipped smart glasses haveproven more successfulthan other entrants into the market, but they have also deeply disturbed certain consumers who see them as invasive emblems of a dystopian surveillance society run amok. Now, as the company weathers complaints that it’s selling “pervert glasses,” it has reportedly decided to sell a pair that doesn’t come with integrated spy equipment. The Informationreports thatMeta is developing a new smart glasses model, dubbed Luna, that, sans cameras, comes equipped with a convenient system to communicate with the company’s AI chatbot and Muse, its AI consumer agent. The glasses include six built-in microphones so users can communicate with the chatbot, as well as a button on the side of the glasses that, when pressed, activates the AI system. Luna may be unveiled as soon as Meta Connect, Meta’s annual hardware/developer event happening in Menlo Park next week, the outlet notes. TechCrunch reached out to Meta for more information. The smart glasses industryhas grownover the past few years, with Metapositionednear the front of the pack of providers. However, concerns over usability, cost, and privacy have dogged the business, with many consumersstill unsureof how, why, or if they should use the devices. Meta’s Reality Labs, which is responsible for developing its smart glasses line,is still losing a gargantuan amountof money, as its earnings report from April revealed.
View
Former Infosys chief’s AI startup nabs another $53M
Hang Ten Systems, an AI startup founded by former Infosys CEO Vishal Sikka just four months ago, has expanded its seed funding by another $53 million. This new investment round was closed just five weeks after theinitial $32 million seed round was closed, the company tells TechCrunch. The fresh investment, led by Temasek’s early-stage investment platform Xora, takes Hang Ten’s total funding to $85 million. Mayfield, which led the startup’s earlier investment, also participated. Other investors include Aramco Ventures, Intel CEO Lip-Bu Tan, Micron CEO Sanjay Mehrotra, and Yahoo co-founder Jerry Yang, who is also on its board. Founded in May, Hang Ten is betting that AI can fundamentally reshape how large companies build and maintain software. The startup advises companies — typically with over $10 billion in annual revenue — on AI strategy and also helps build and modernize software. “The build part itself has basically become close to zero marginal cost, close to zero time,” Sikka (pictured above) told TechCrunch. He argued that AI is shifting the focus of software development from writing code to defining requirements and validating whether systems perform as intended. The thesis appears to be resonating with customers. Within 25 days of the first meeting with one customer, Hang Ten signed a multimillion-dollar contract to build a mission-critical software system, Sikka said, adding that he had never seen enterprise deals move that quickly. Hang Ten is now working with a mix of existing customers and late-stage prospects spanning 21 major enterprises, Sikka said, with some prospects in the proposal or contract-negotiation stage. The startup has secured multiple seven-figure contracts and is pursuing eight-figure deals. Moreover, its customers span the U.S., Europe, the Middle East, and Asia, and include Fresenius Kabi, Saudi Aramco, and Siemens Energy. The early customer wins prompted Xora to approach Hang Ten about investing, Sikka said, adding that Xora saw an opportunity to introduce the startup to other companies in Temasek’s portfolio. Sikka declined to disclose Hang Ten’s valuation but said the second round of seed funding was a bump up. With the fresh capital, Hang Ten plans to expand its engineering, consulting, and sales teams. The Palo Alto-headquartered startup currently has about 20 to 25 employees across the U.S., the Middle East, and Australia, and expects to hire in Europe and India, Sikka said. Hang Ten enters a crowded market in which AI model developers such as OpenAI and Anthropic are expanding their enterprise offerings, while traditional consulting firms are racing to embed generative AI into their own services. “Enterprises still need trusted partners who are independent of the underlying platform and who work with and solely in the interest of the enterprise,” Sikka said. “There will always be room for companies like us.” Hang Ten says its edge lies not in building another AI model, but in adapting AI to the complex software development processes of large enterprises. Co-founder and chief design officer Sanjay Rajagopalan told TechCrunch the startup’s engineers rely on an in-house framework, Hobie, that packages reusable AI “skills” for regulated industries and complex enterprise software projects. “We are actually delivering production software,” Rajagopalan said. “It’s not like we are doing some kind of PoC.” Hang Ten’s approach is already putting it in competition with traditional systems integrators. Many of the startup’s engagements, Sikka said, are replacing incumbent providers, though more than half of its current opportunities involve new projects that companies had previously put off, rather than work taken from existing vendors. Rajagopalan noted that Hang Ten can use teams of about two to four people for some projects that might previously have required about 30, although customers or independent third parties still handle final quality checks and certification. The startup promises customers a 10-fold improvement in cost, speed, or a combination of the two, he said. Even in four months of its debut, Hang Ten has received serious acquisition offers from “very big companies,” Sikka told TechCrunch. He, however, declined to identify the prospective buyers. The startup, he said, has turned them down for now as it focuses on expanding the business. Before founding Hang Ten, Sikka co-founded enterprise AI startupVianAIin 2019 after leaving Infosys in 2017. It raised$50 million in seed fundingand laterraised $140 millionin a 2021 round led by SoftBank Vision Fund 2. Sikka departed that startup in April, according to his LinkedIn profile. He told TechCrunch that VianAI is “going through a transaction,” declining to provide further details. As for how startups like Hang Ten could disrupt services providers like Infosys, Sikka noted, “We are fortunate that we don’t have the burden of legacy that we have to transform.” The startup takes its name from “hang ten,” the surfing maneuver in which a surfer rides with all 10 toes over the front of the board. For Sikka, it reflects the startup’s ambition to help enterprises ride the ongoing AI wave, what he called “probably the biggest wave of our lives.”
View
