Google Thwarted First AI-Generated Zero-Day Exploit Before Attack

Key Findings

• Google Threat Intelligence Group discovered a zero-day exploit developed entirely by artificial intelligence before attackers could deploy it at scale

• Code analysis revealed telltale AI artifacts including excessive documentation strings, highly annotated code, and a hallucinated CVSS score that don't match human developer patterns

• A well-known cybercrime group with a history of high-profile attacks and mass exploitation was preparing to use the exploit for financial gain

• The vulnerability affected a Python script in a popular open-source web-based administration tool and allowed attackers to bypass two-factor authentication

• This marks the first confirmed evidence of AI-developed zero-day exploits in the wild, though security experts believe additional variants likely exist

Background

For years, cybersecurity researchers have warned that artificial intelligence would eventually be weaponized to develop zero-day exploits. The threat seemed theoretical until Google's discovery proved the prediction accurate. GTIG chief analyst John Hultquist noted that Google's own Big Sleep AI agent successfully identified a zero-day in late 2024, signaling the capability was possible. What makes this incident significant is not just that it happened, but that Google caught it in time and has documented the evidence.

The Discovery and Evidence

Google's researchers identified the exploit when investigating a vulnerability affecting a Python script used by a popular open-source administration tool. The defect allowed attackers to bypass two-factor authentication, a critical security layer. Upon closer examination, the code contained unmistakable signs of machine authorship. Specifically, researchers found an abundance of educational docstrings, highly annotated sections, and a completely fabricated CVSS severity score that didn't exist in any official database. These artifacts are inconsistent with how human programmers typically write code.

Google ruled out the involvement of major AI models like Gemini or Anthropic's Claude. The exact model used remains unknown, though the company is confident it was from publicly available sources.

The Threat Actor

The cybercrime group preparing to exploit this vulnerability is well-known within cybersecurity circles for orchestrating high-profile incidents and conducting mass exploitation campaigns. Hultquist characterized their track record as particularly strong and dangerous. While Google declined to formally identify the group, the description suggests they rank among the most sophisticated and capable threat actors currently operating. The group's intention to use this exploit for mass exploitation indicates they planned widespread financial gain rather than targeting specific victims.

What This Signals About the Future

Hultquist emphasized that this discovery likely represents only the beginning of a larger trend. He believes multiple AI-developed zero-days are probably already circulating in the wild, and the capability trajectory is steep. The real concern isn't just this single incident but what it portends for future attacks as AI capabilities mature. According to GTIG analysts, the game has already begun, and the cybersecurity community should expect more devastating AI-assisted attacks as technology advances. The watershed moment occurred two years prior when researchers first proved AI could discover vulnerabilities, making today's incident feel inevitable in retrospect.

Sources

• https://cyberscoop.com/google-threat-intelligence-group-ai-developed-zero-day-exploit/

• https://hackread.com/google-hackers-used-ai-develop-zero-day-exploit/

• https://x.com/TweetThreatNews/status/2053856426427080885

• https://www.reddit.com/r/cybersecurity/comments/1ta2oke/google_spotted_an_aideveloped_zeroday_before/

• https://www.theverge.com/tech/928007/google-ai-zero-day-exploit-stopped

• https://www.infosecurity-magazine.com/news/hackers-using-ai-zero-day-first/