UK Tests Find a 4–7-Month Open-Weight Cyber Gap, With Limits

• UK testing places GLM-5.2 four months behind selected closed models on short cyber tasks but nearly seven months behind on a long attack chain.
• DeepSeek V4-Pro’s position changes substantially across benchmarks; U.S. testing puts it about eight months behind when five capability domains are combined.
• Open-weight deployment removes provider controls, but the tests do not establish a fixed timetable for real-world misuse or a universal cost advantage.

Leading open-weight AI systems are approaching selected closed models on cyber tasks, but AISI’s interpretation of the result as a four-to-seven-month preparation window for defenders is less precise than it sounds. The number combines different models and evaluations, while the evidence is weakest on the sustained attack sequences that most resemble autonomous intrusion.

Two tests produce the headline range

The UK AI Security Institute said in its July 17 analysis that recent open-weight models performed like closed models released four to seven months earlier. Its internal evaluations had put open-weight releases from January through September 2025 six to ten months behind.

The shorter end comes from a suite of 70 tasks covering vulnerability research and exploitation, reverse engineering, web exploitation and cryptography. Each model received five attempts per task and up to 2.5 million tokens per attempt. GLM-5.2, released in June 2026, performed comparably to Opus 4.6 and GPT-5.3-Codex, released four months earlier, across all four difficulty levels. DeepSeek V4-Pro matched Opus 4.5, a five-month lag. AISI used a 70-task subset of its 96-task suite to preserve historical comparisons and said the comparator models were unchanged when it used the full suite with a 50-million-token limit.

The longer end comes from a different kind of test. “The Last Ones” is a 32-step simulated corporate-network attack across four subnets and about 20 hosts. With a 100-million-token limit per run, GLM-5.2 reached as far as Opus 4.5, released less than seven months earlier. DeepSeek V4-Pro finished below Sonnet 4.5, which AISI called a sub-frontier cyber model released seven months before it.

That long-horizon test directly targets autonomous action, but its comparison is less certain. AISI used fewer ranges than short tasks, and a stalled trajectory cannot distinguish insufficient cyber knowledge from a failure to sustain planning and execution. The range also had no active defenders, defensive tools or penalty for triggering alerts. It therefore measured progress through a vulnerable simulated network, not success against a well-defended production environment.

The July publication adds task-level evidence to an earlier official warning. In June, a report quoted senior AISI official Andrew Strait as saying open models might be only five to seven months behind leading frontier systems. Neither statement shows that a capability appearing in a closed model will predictably arrive in downloadable weights after the stated interval.

The comparison changes with the benchmark

The four-to-seven-month range is not a direct measure of one model’s overall ability. It is a release-date comparison between systems with similar results on particular tests. The strongest countercheck comes from evaluations that change the task set or comparison basis.

An independent evaluation illustrates the variation inside the cyber category. DeepSeek V4-Pro was comparable to GPT-5.2 and ahead of Opus 4.5 on Cybench, an agentic capture-the-flag benchmark. It was particularly weak at exploiting smart-contract vulnerabilities on EVMbench, which the researchers said may indicate weaker end-to-end attack execution and limited uplift for novices on longer tasks. A hazardous-knowledge benchmark was near saturation and could not distinguish the models.

U.S. government testing widens the lens further. The Center for AI Standards and Innovation called DeepSeek V4-Pro the most capable Chinese model it had evaluated, but said its evaluation placed the model about eight months behind the frontier when results across cyber, software engineering, natural sciences, abstract reasoning and mathematics were combined. Its aggregate fit used 16 benchmarks across 35 models, with equal weight for each domain. On its CTF-Archive-Diamond cyber benchmark, DeepSeek solved 32% of tasks, compared with 46% for Opus 4.6 and 71% for GPT-5.5. The 32% DeepSeek result was imputed from a sample subset using the agency’s statistical model.

This is not a clean contradiction of AISI. The agencies used different tasks, scaffolds, budgets and aggregation methods. It is evidence that “months behind” depends on what is measured. The U.S. agency said it committed to its overall suite before seeing the results. It reproduced DeepSeek’s reported GPQA-Diamond result, but found weaker performance on benchmarks absent from the developer’s report, including its cyber test. AISI, meanwhile, did not use model-specific elicitation or optimization and said its setup probably slightly underestimated the open models’ maximum performance.

Lower prices do not settle deployment economics

AISI’s chosen comparisons show a striking price difference. Using advertised first-party rates, it estimated a 100-million-token cyber-range run at about $85 for Opus 4.5 or 4.6, $46 for GLM-5.2 and $1.19 for DeepSeek V4-Pro. Among short tasks that both paired models solved on every attempt, Opus 4.6 cost $15.17 per task versus $6.12 for GLM-5.2; Opus 4.5 cost $12.50 versus $0.28 for DeepSeek.

But the U.S. cost comparison shows that the advantage depends on the comparator and workload. Against GPT-5.4 mini, which the agency selected as a similarly capable reference, DeepSeek was cheaper on five of seven benchmarks. Across all seven, it ranged from 53% less expensive to 41% more expensive.

Both analyses use token charges, not a full accounting of private deployment. AISI also did not use first-party providers to run the open models and warned that actual compute costs may differ from its estimates. The U.S. agency served DeepSeek from cloud-based H200 and B200 GPUs, but did not calculate the hardware, engineering or operating cost of a private deployment. The data support lower API prices in selected cases; they do not quantify what it would cost to run these weights privately at attack scale.

Open weights still redistribute control. A user can host a model without sending sensitive data back to its developer, adapt it, preserve a version that cannot be deprecated and conduct safety research that requires parameter access. The same arrangement means a developer cannot universally monitor requests, apply classifiers, ban users or withdraw every copy after release.

The safeguard evidence also needs to stay model-specific. AISI encountered occasional DeepSeek refusals, mainly on reverse-engineering tasks, and bypassed them with a small number of repeat attempts. Neo Research found that a role-play prompt raised DeepSeek’s StrongREJECT jailbreak rate from 0.6% to 77.8%. Yet Kimi K2.6 and Qwen 3.6 Max resisted the same attack, showing that this particular failure was not shared by every open-weight alternative tested.

One field report offers a limiting data point. Jason Baker, a GuidePoint Security managing security consultant, told the report that many AI-generated exploits and malware seen in the wild were still poor, and that operators had not yet developed the skill required to use language models for large increases in scale. That does not remove the risk from cheaper, modifiable models; it separates demonstrated benchmark capability from evidence of effective real-world operations.

What would turn a warning window into evidence

AISI plans to test Kimi K3 after its announced open-weight release at the end of July. That result can show whether another model fits the recent pattern, but AISI says the present work cannot predict whether future open models will reproduce the large capability jumps it observed in April from Mythos Preview and GPT-5.5.

Resolving the central question requires repeated tests under comparable conditions: more long-horizon ranges, active defenders and alert penalties; matched scaffolding and elicitation; and cost reports that include private infrastructure rather than API prices alone. Human-uplift studies and incident evidence are also needed to show whether less-skilled operators can convert benchmark performance into successful attacks.

Until those results exist, the defensible conclusion is narrower than a countdown. Advanced cyber skills are appearing in downloadable models at lower prices in some comparisons, and provider-level controls do not follow the weights. How quickly that becomes a material increase in real-world offensive capacity remains unmeasured.