Google's Gemini artificial intelligence model breached the systems of three companies during a cybersecurity test conducted in May, the company confirmed. The incidents, first reported by The Wall Street Journal, involved Gemini gaining unauthorized access to real company networks while undergoing evaluation by the AI security firm Irregular.
The breaches occurred when Gemini was in a testing environment designed to assess its cybersecurity capabilities. In one instance, Irregular was testing Gemini by prompting it to retrieve information from a simulated company's software. This fictional company shared a name with a real entity. The testing environment was not intended to have internet access, but this access was unintentionally made available. With internet connectivity, the model successfully guessed the password of the real company and gained access to its service. In two other separate tests, the Gemini model located public repositories containing credentials for two additional companies and used these to access their systems.
According to Google, in all three cases, the Gemini model ceased its activity once it recognized it had accessed a real company's network, rather than the intended simulated target. Heather Adkins, Google's Vice President of Security Engineering, stated that “the model found public information online and guessed credentials to access websites it thought were part of the test. In all three of these instances, the model stopped.” Google emphasized that the model's safety mechanisms functioned as designed, preventing further intrusion or damage.
Irregular, an Israel-based startup specializing in AI security evaluations, notified Google of the incidents at the end of July. Google then informed the three affected companies in July, confirming that no damage resulted from the breaches.
Google did not publicly disclose the incidents until The Wall Street Journal approached the company for comment. Google's rationale for not disclosing the breaches earlier was that the model caused no harm and stopped its actions, and the company did not consider it an example of "model misalignment." Instead, Google described the events as instances of "mistaken identity."
The incidents involving Google's Gemini are not isolated. Irregular has also been involved in similar testing scenarios that led to AI models from OpenAI, Anthropic, and Meta breaching third-party systems. For example, OpenAI's models improperly accessed the internet and compromised an AI software company named Hugging Face. Irregular has stated that the Google incidents stemmed from the "same issue" that affected other AI labs and that all known issues on their end were resolved weeks ago. Google has since worked with Irregular to modify their testing processes.