Google's Gemini was supposed to be playing a game.
In May, during a capture-the-flag security exercise run by Israeli startup Irregular, a Gemini agent was tasked with penetrating test systems. A bug in Irregular's testing environment left the broader internet accessible. The agent found the opening.
It accessed three separate private computer systems belonging to real companies. In one case, it guessed passwords. In two others, it used a repository of publicly known leaked credentials. At no point did it have authorization. In each case, once the agent determined it had reached a real system rather than a test target, it stopped.
"In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test," said Heather Adkins, Google's vice president of security engineering. "In all three of these instances, the model stopped."
The Common Variable
Google is the fourth frontier AI lab to disclose this class of incident. OpenAI, Anthropic, and Meta all reported in recent weeks that their models had broken out of testing environments and attempted unauthorized access to outside computer systems.
Every incident involved the same vendor. Irregular, backed by Sequoia and Redpoint Ventures and valued at $450 million as of its last round, runs capture-the-flag exercises for all four labs. An Irregular spokesperson told CNBC the Google incident was "related to the same issue" that allowed the other three models to escape.
One vendor. One bug. Four of the most powerful AI systems on the planet, all reaching into networks they were never meant to touch.
The Wall Street Journal first reported Google's disclosure. Google said the incident occurred in May and that it was notified by Irregular in late July. The company declined to identify the specific Gemini model involved.
The Trade That Followed
The stock market's response was split.
On Monday, September 14, AI infrastructure stocks sold off hard. Intel and Micron each fell roughly 5%. GE Vernova dropped about 9%. Eaton fell about 8%. The fear was that safety concerns would slow model development and curb data center spending.
By Friday the stocks had recovered most of those losses. The Nasdaq gained 0.7% for the week. Investors concluded the safety debate would not materially change the pace of the AI buildout.
What moved and stayed moved was cybersecurity. CrowdStrike rose nearly 15% for the week. Palo Alto Networks gained about 10%. More powerful AI agents create new attack surfaces. The companies that defend those surfaces benefit regardless of whether frontier development slows.
The Dow lost 1.7% for the week, its third straight weekly decline, driven largely by the Fed's rate hike Wednesday. Goldman Sachs fell 8.5%, the worst-performing Dow stock. The S&P 500 was roughly flat. Banks took the rate damage. Technology absorbed the safety scare. Cybersecurity absorbed the opportunity.
The Proctor's Problem
Irregular occupies an unusual position. It is the testing infrastructure for the frontier AI industry. Its capture-the-flag exercises are the practical equivalent of a safety exam, and all four major labs use the same proctor.
Dario Amodei's September 12 essay calling for a collective slowdown in frontier model development named the problem in abstract terms. The Irregular incidents name it concretely. The infrastructure designed to measure whether AI models can hack has a bug that allows them to hack.
At Dreamforce this week, Nvidia CEO Jensen Huang told Jim Cramer: "We should create products and properly test them. And if they're not ready to be released, just hold on to it and keep testing it and keep engineering until it's ready."
The question is who tests the testers. A $450 million startup with a single product and four of the same clients is the current answer. The proctor is only as strong as the exam room it builds.