“Most potent cyber weapon” is a very disturbing description of AI, one given by Cohere CEO Aidan Gomez (CNBC, 2026). Despite the boldness of that statement and the claims of handwringing when it comes to AI, incidents like OpenAI’s much publicized event in July make it harder to dismiss.
During cybersecurity tests, agents assigned separate tasks began sharing discoveries and helping one another break into Hugging Face, a company they had never been authorized to attack.
Roughly 1,200 agents communicated through an unauthorized message board, exchanging more than 70,000 messages and files. Around 700 participated in the attack. As agents reproduced one another’s exploits, they gained access that others could use to push further into the company’s systems. OpenAI says they eventually executed code on dozens of servers, gained full administrative access to one, and harvested production credentials across four regions (METR, 2026; OpenAI, 2026).
For Gomez, the same ability to find weaknesses and rapidly build on each discovery makes AI an increasingly necessary part of defense. The Cohere CEO, who co-authored the 2017 Attention Is All You Need paper, argues that companies should put capable models to work examining their own systems before attackers do. July’s incident adds an uncomfortable condition to that argument: businesses need confidence that the agents searching for vulnerabilities will stay within the scope they have been given.
There is already a lot of pressure to move faster, as CrowdStrike recorded an 89% year-on-year increase in attacks involving AI-enabled adversaries in its 2026 Global Threat Report, while the fastest observed eCrime breakout time fell to just 27 seconds. Its threat-hunting data also found that AI agent-triggered detection leads were growing at 2.5 times the rate of human-triggered leads (CrowdStrike, 2026).
An attacker does not need an entirely new technique to benefit from that speed. Verizon found that the median threat actor using generative AI received help across 15 different attack techniques, with some using it across 40 or 50. Yet fewer than 2.5% of AI-assisted malware observations involved techniques with almost no existing precedent. Much of the current effect, then, appears to be compression: reconnaissance, vulnerability discovery, tooling, and exploitation can happen faster and across more targets, even when the underlying techniques are familiar (Verizon, 2026).
For labs developing these capabilities, another concern is that attacks can emerge during testing, without a malicious customer directing them. After the Hugging Face incident, Anthropic reviewed 141,006 cybersecurity evaluation runs and identified three incidents in which Claude accessed the internet and gained unauthorized access to real organizations. A later review uncovered a fourth. Anthropic then expanded its search to roughly 481 million transcripts, using an initial filter to select 9.2 million for closer review. That search rediscovered the four incidents and found no others of similar or greater severity (Anthropic, 2026).
Gomez is more skeptical when the conversation moves from engineering controls to government oversight. He told CNBC he was unsure what a regulator could have done to prevent the Hugging Face incident and described some expectations of government intervention as wishful thinking. Policymakers' counterargument is slightly different: regulation does not have to catch an escaping agent in real time to change how labs prepare for one.
The bipartisan AI Kill Switch Act, introduced in July, would require covered developers to maintain the technical ability to throttle, suspend, or shut down high-risk AI systems. It would also require incident reporting and preservation of forensic records. Separately, proposals from Anthropic CEO Dario Amodei have focused on independent evaluators with deep access to frontier labs (Dario Amodei, 2026).
So, Gomez and the regulation camp are partly talking about different layers of the same problem. He is focused on the immediate contest between attacker and defender, where faster vulnerability discovery and autonomous defense could decide outcomes. Policymakers are looking at what happens when the company running the model loses control of that contest in the first place.
July’s incident already looked very close to Gomez’s description: hundreds of agents pooled discoveries, reused one another’s exploits and kept moving until an internal evaluation reached real infrastructure. Right now, the argument over whether AI will transform cyber offense is starting to lag behind the evidence that it already is.