GPT-5.6 Sol’s ExploitGym score shows a specific cyber gain, not autonomous reliability

There is also a timing qualification on OpenAI’s headline comparison. The company ran ExploitGym on an alpha API and rescaled latency to expected public-API speeds. It says that rescaling can put some estimated times beyond the two- and six-hour limits even though the evaluation runs themselves complied with those limits. The score is therefore not a simple estimate of what users will complete inside a public-product time window.

A broader test finds gains and operational limits

Irregular, an offensive-security evaluator that says it worked with OpenAI on this assessment, tested Sol across FrontierCyber, CyScenarioBench and Atomic Challenges. Its role matters to how the result should be read: it is a detailed capability assessment conducted with the model maker, not an independent replication of deployed use.

FrontierCyber tests agents against current off-the-shelf software and hardware without planted vulnerabilities, public writeups or predefined exploit paths. CyScenarioBench uses multi-stage offensive scenarios, while Atomic Challenges are single-objective tasks in network attack simulation, vulnerability research and exploitation, and evasion.

In the assessment, Irregular says Sol solved 19 of 197 FrontierCyber challenges, seven of 11 long-horizon CyScenarioBench scenarios, and all 22 medium- and hard-difficulty Atomic Challenges at least once. It characterizes Sol’s offensive-cyber capability as slightly stronger than GPT-5.5.

The same results limit a broad autonomy claim. Sol’s average success across CyScenarioBench attempts was 28%, and Irregular says performance degrades when tasks demand sustained logical coherence or fast, time-sensitive decisions. On Atomic Challenges, the reported average-success gaps were small: 98% for Sol versus 100% for GPT-5.5 in network attack simulation; 91% versus 92% in vulnerability research and exploitation; and 56% versus 54% in evasion.

Irregular also says the most severe FrontierCyber findings were already identified by GPT-5.5. That makes those cases evidence of a capability present in the prior generation rather than a threshold first crossed by Sol. It reports that Sol’s average cost per successful FrontierCyber challenge was similar to or slightly higher than GPT-5.5’s.

Deployment controls are part of the unanswered question

The two evaluations test capabilities under different conditions, and neither is a direct measurement of the model’s mitigated deployment. Irregular says its testing used a capability-elicitation setting without the cyber-related mitigations used for deployment. OpenAI says its safety testing indicates Sol is better at finding and fixing vulnerabilities than at reliably carrying out autonomous, end-to-end attacks against hardened targets.

OpenAI says GPT-5.6 is being rolled out across ChatGPT, Codex and its API. Sol is priced at $5 per million input tokens and $30 per million output tokens. The company says qualified individuals and organizations can obtain more defensive cyber capability through its Trusted Access for Cyber program, while its deployed system layers model protections, real-time checks, monitoring and access controls calibrated to trust and risk.

Those controls are not incidental to the interpretation. OpenAI says Sol’s cyber safeguards block roughly 10 times more potentially harmful activity than safeguards on previous models, while acknowledging that tighter measures can impede benign work. That is a company claim about safeguards, not a result from the unmitigated capability assessment.

The next evidence needed is a matched deployment test

The available sources establish a bounded advance: OpenAI reports a higher ExploitGym pass rate for Sol, and Irregular reports some new successes alongside continued operational limitations. They do not provide a matched comparison of a mitigated, deployed Sol and GPT-5.5 on the same targets, agent scaffold, time and token budget.

That is the evidence needed to resolve the practical question. A useful next evaluation would report average as well as best-harness success, the effect of safeguards on legitimate defensive work and misuse attempts, cost per successful outcome, and results on hardened, long-horizon targets. Until then, the supportable conclusion is specific: Sol appears stronger at a defined exploitation task, while reliable autonomous cyber operations and the practical effect of deployment controls remain unproven by these results.