AI Shatters All Records in Autonomous Cyber Capability Benchmarks

Key Findings

• Claude Mythos Preview and GPT-5.5 have dramatically exceeded all tracked benchmarks for autonomous cybersecurity capabilities

• Task completion timelines have accelerated to double approximately every four months, compared to previous estimates of five to eight months

• Claude Mythos Preview became the first model to complete both of the UK's AI Security Institute cyber range simulations

• Palo Alto Networks identified 26 critical vulnerabilities across 130+ products through AI scanning, compared to typical monthly volumes under five

• Researchers cannot yet determine if this represents a sustained acceleration or an isolated capability leap

Background

The UK's AI Security Institute and Palo Alto Networks released independent studies Wednesday showing that frontier AI models have shattered performance expectations in autonomous cybersecurity tasks. The AISI conducts pre-deployment evaluations for the British government, while Palo Alto Networks tested Claude Mythos as a launch partner for Anthropic's Project Glasswing. Both organizations had been tracking how quickly AI systems could complete complex cyber tasks, using human expert completion times as a baseline measure.

The Acceleration Pattern

The AISI had established a doubling trend in November 2025 showing frontier models' capabilities were expanding every eight months. By earlier this year, that estimate shortened to five months. The latest findings from Claude Mythos Preview and GPT-5.5 have now outpaced even that accelerated trajectory, with an estimated four-month doubling time since late 2024. Separate research from METR, a nonprofit tracking AI software task performance, arrived at nearly identical conclusions.

Cyber Range Breakthrough

The clearest evidence came from the AISI's structured simulations of multi-stage attacks against small, undefended enterprise networks. A checkpoint of Claude Mythos Preview completed "The Last Ones," a 32-step simulated corporate network attack, in 6 of 10 attempts. More significantly, it solved "Cooling Tower" in 3 of 10 attempts, becoming the first model ever to complete this previously unsolved challenge. GPT-5.5 solved "The Last Ones" in 3 of 10 attempts. These aren't theoretical exercises — they represent realistic attack scenarios that would typically require skilled human operators.

Real-World Vulnerability Discovery

Palo Alto Networks' testing revealed the practical implications of this advancement. When Claude Opus 4.7 and GPT-5.5-Cyber scanned the company's product portfolio, they identified critical vulnerabilities at an unprecedented scale. The AI models uncovered 26 CVEs representing 75 distinct security issues across more than 130 products in a single testing cycle. This dwarfs the company's typical monthly volume of fewer than five CVEs discovered through conventional methods. The models proved particularly adept at converting discovered vulnerabilities into exploitable attack paths in near-real-time. All critical issues in Palo Alto's SaaS products were patched, with patches released for customer-operated systems.

Measuring Uncertainty

The AISI emphasized that its findings come with meaningful caveats. The estimates rest on analysis of relatively few models, and the hardest tasks in their test suite have limited human comparison data for validation. However, the institute found the overall trend robust — removing any single model from the analysis shifted the estimated doubling time by less than a month. The methodology itself has been validated through independent research reaching similar conclusions, lending credibility to the rapid acceleration being observed.

Enterprise Response Requirements

Palo Alto Networks outlined four urgent priorities for organizations as AI cyber capabilities continue advancing. First, enterprises must identify and patch vulnerabilities faster than attackers can exploit them. Second, organizations should reduce attack surfaces and deploy AI-powered scanning to catch misconfigurations. Third, detection and response tools require deployment across all systems with machine learning continuously monitoring for threats. Fourth, security operations teams must be rebuilt to respond in minutes rather than hours, because AI-orchestrated attacks may unfold at machine speed.

Looking Forward

The AISI is already developing more challenging evaluations to keep pace with advancing capabilities. The institute is building new cyber ranges and incorporating active cyber defenses to better reflect real-world conditions where attackers employ their own countermeasures. Whether the current acceleration represents a sustained shift or a temporary spike remains the critical unanswered question among researchers, but the performance gap between these frontier models and their predecessors has left little room for complacency in the cybersecurity community.

Sources

• https://cyberscoop.com/ai-autonomous-cyber-capability-benchmarks-broken-gpt5-claude-mythos/

• https://x.com/TheCyberSecHub/status/2054693240520224902