Epoch AI: Open Models Trail on Cyber by 6-7 Months

The best AI models whose weights anyone can download are roughly six to seven months behind the best closed models on cybersecurity tasks, according to a new estimate from research group Epoch AI. Epoch is clear that the figure rests on thin data. The top open model’s score comes from just three benchmark results, and some of the closed-model results were matched to models by inference.

The headline finding

In a data insight published on 16 October, Epoch AI researchers Jaeho Lee and Alexander Barry write that Z.ai’s GLM-5.3 is the highest-scoring open-weight model on the Cyber ECI, a cybersecurity-focused version of Epoch’s Capabilities Index.

“Its score of 160 is above that of OpenAI’s GPT-5.4, released in March 2026. However, it still trails Claude Mythos Preview and GPT-5.5, both from April 2026,” they write. “This places the open cyber frontier roughly where the closed frontier stood 6–7 months ago.”

Epoch reaches that figure by finding the closed models whose scores sit either side of GLM-5.3’s: GPT-5.4, released on 5 March 2026, scores below it, and Mythos Preview, released on 7 April 2026, scores above it. As of Epoch’s data cutoff on 8 October, those releases were seven and six months old.

Meanwhile, the closed frontier has moved on. It is “currently led by Anthropic’s Claude Opus 5.5 and OpenAI’s GPT-6 Astra, both with a Cyber ECI of 171.”

How the index works

The Cyber ECI estimates each model’s cyber capability from its results on cybersecurity benchmarks, on the same scale as Epoch’s general index. Epoch says its October update expanded the index “from 56 to 76 models and from 20 to 35 cybersecurity benchmarks”, giving 324 model-benchmark results. The data comes from the UK AI Security Institute, the US Center for AI Standards and Innovation (CAISI), the evaluation company Irregular, model developers’ own reports and other public sources. Only models with at least two cyber results are included.

The caveats

Epoch lists several limitations, and they matter for how much weight the 6-7 month figure can bear:

GLM-5.3’s score rests on three results. All three come from CAISI’s September 2026 assessment of the model, and they are the only cyber results available for it. Epoch’s table gives GLM-5.3 a 90% interval of roughly 156 to 167, which overlaps with several closed models.
Some US model identities were inferred. “CAISI did not name the US models it evaluated,” Epoch says, so it attributed those results to the models it judged most likely to have been leading at the time, such as GPT-6 Astra and Claude Mythos Preview.
The two leaders cannot be cleanly separated. Opus 5.5’s score comes from four benchmarks reported by Anthropic. Opus 5.5 and Astra share no benchmarks, and their confidence intervals “overlap almost entirely”.
The October refit lowered top scores. Claude Mythos 5 fell by 3.9 points and GPT-5.6 Sol by 4.7, so Epoch warns that Cyber ECI values in its earlier publications “are not directly comparable to the new values.”

Epoch also says it expects methodology changes that will improve how it estimates uncertainty in differences between models.

Why it matters

Cyber capability is one of the main reasons labs and governments worry about releasing powerful model weights openly: once weights are public, safeguards can be stripped out. A gap of around half a year suggests open models are not far behind, but also that the most capable cyber systems remain closed. Given how few results some scores rely on, the estimate is best read as a rough marker, not a precise measurement.

What this means

Epoch estimates open-weight models are about 6-7 months behind closed models on cyber benchmarks, with Z.ai’s GLM-5.3 the top open model on a score of 160.
The estimate rests on limited data. GLM-5.3’s score comes from three CAISI results, some US models were identified by inference, and the latest refit lowered several top scores.
Older Cyber ECI figures should not be compared directly with the new ones.