GPT-6 Astra vs Claude: One Model, Two Scores, and What Actually Lost

GPT-6 Astra scored 62.7% on ARC-AGI-3. It also scored 99.9% on ARC-AGI-3. Same model, same benchmark, same week — the difference was the harness. That single fact is the best available lens on the whole Astra-versus-Claude question, because almost every claim in this comparison depends on how the test was wired up. Here is what holds regardless.

ARC Prize, who build and run the benchmark, published both. On the semi-private evaluation with their <strong>standard, provider-neutral harness</strong>, Astra scored <strong>62.7%</strong> at a compute cost of $26,098. On a new <strong>provider-adapter harness</strong>, built to use OpenAI's native context-management features, the same model scored <strong>99.9%</strong> at $18,817.

The adapter preserves opaque reasoning state between requests and compacts longer conversations, letting the model reuse prior work instead of rebuilding its reasoning each call. That is a real capability, not a trick &mdash; and it produced a higher score at lower cost. But it is a capability of the model plus OpenAI's plumbing, not of the weights alone.

Both numbers are legitimate. The 62.7% is state of the art on the standardised rig, comfortably ahead of Claude Opus 5's 30.2% and GPT-5.6 Sol's 7.8%. Quoting only the 99.9%, though, implies a result the model does not produce under neutral conditions, and quoting only the 62.7% understates what the system can do in production.

Strip out the harness ambiguity and there is a consistent pattern: Astra's advantages are concentrated in long-horizon, tool-heavy, environment-driven work rather than in general knowledge or single-turn reasoning.

• ▸<strong>Novel-environment reasoning.</strong> Even at the conservative 62.7%, ARC-AGI-3 is double Opus 5's 30.2%. ARC Prize also reported Astra surpassing human performance on 96% of levels, using fewer moves than a typical human.
• ▸<strong>Terminal and systems work.</strong> 57.7% on Terminal-Bench 4.0, which tests agents on software engineering, system configuration and data analysis in a terminal.
• ▸<strong>Reverse engineering.</strong> 88.0% on SRE-Bench at a single attempt, working on binaries without source.
• ▸<strong>Long-context retrieval.</strong> 100% on MRCR v2 8-needle in the 256K&ndash;512K band and 96.3% from 512K to 1M &mdash; most million-token models degrade badly up there.
• ▸<strong>Token efficiency.</strong> Roughly 70% better than its predecessor on coding tasks, which is why cost per task can fall even as price per token rises.

On the two independent composites that cover both vendors, Anthropic leads.

• ▸<strong>Intelligence Index:</strong> Claude Fable 5.1 scores 66 at max effort with default fallback, the highest published figure. Astra scores 61.
• ▸<strong>Coding Agent Index:</strong> Fable 5.1 in Claude Code leads at 70; Astra in Codex scores 67, level with Claude Opus 5 and the original Fable 5.
• ▸<strong>Cached input pricing:</strong> Fable 5.1 reads cached tokens at $0.25 per million against Astra's $1, a fourfold gap on the line item that dominates agent loops.
• ▸<strong>No repricing cliff:</strong> Astra doubles its input rate on any request above 272,000 input tokens. Claude has no equivalent threshold.

These are aggregate measures, with all the caveats that implies &mdash; and the Coding Agent figures compare complete systems rather than bare models. But they are the same measures applied to both vendors by one third party, which is the fairest comparison publicly available.

Put those two lists side by side and the shape is clear. Astra leads on tasks that run long, touch an environment, and reward persistence: terminals, binaries, novel environments, enormous contexts. Claude leads on aggregate reasoning quality and on the economics of repeated agent turns.

Which matters more is a question about your workload rather than about the models. An agent that runs for hours against a real system is squarely in Astra's territory. A high-volume loop that re-reads a large cached prefix every turn is squarely in Claude's, and the fourfold cache difference will show up on the invoice long before a five-point index gap shows up in your output.

The two ARC-AGI-3 runs came with compute costs attached, and that detail deserves more attention than it got. The 99.9% run cost $18,817. The 62.7% run cost $26,098. The better-scoring configuration was also the cheaper one, because preserving reasoning state between requests means the model rebuilds less work on every call.

That inverts the usual assumption that higher scores come from throwing more compute at the problem. Here the gain came from better plumbing, not a bigger budget &mdash; which is exactly why OpenAI has been shipping context-management features and why Codex is gaining settings that keep notes across context windows rather than compressing them into a single summary.

For anyone building on these models, the transferable lesson is that state management is a first-class performance lever, comparable in effect to changing model. Most teams spend far more time comparing models than improving how their loop carries context between turns, and the ARC numbers suggest that is the wrong allocation of effort.

The ARC-AGI-3 split is a useful permanent lesson. When you see a model score quoted, three questions decide whether it means anything for you.

• ▸<strong>In what harness?</strong> Provider-native plumbing can move a result by tens of points, as ARC Prize demonstrated on a single model in a single week.
• ▸<strong>At what effort and cost?</strong> The 99.9% run cost $18,817 and the 62.7% run $26,098. Scores without compute budgets are not comparable.
• ▸<strong>Measured by whom?</strong> Vendor tables and independent leaderboards routinely differ on the same named benchmark for legitimate methodological reasons &mdash; repeat counts, time budgets, sandboxes.

We pull the composite-index version of this apart in <a href="/blog/fable-5-1-is-it-really-that-good">what a number-one ranking actually means</a>, and the price-matched comparison in <a href="/blog/gpt-6-astra-vs-fable-5-1">Astra against Fable 5.1</a>.

Astra did not launch generally. Access went first to organisations in OpenAI's application-based cybersecurity programme, with ChatGPT Plus, Pro, Business and Enterprise following over subsequent days, and the most advanced capabilities gated by tier. On the API it is available as GPT-6 Astra and through Amazon Web Services.

That is a consequence of Astra being the first OpenAI model to reach the company's internal &ldquo;Critical&rdquo; cybersecurity threshold. OpenAI paused two weeks of deployment-focused reinforcement-learning training before release, reported that its largest planned frontier run remained on hold, and says the released model refuses 91.5% of cyber jailbreak attempts against 59% for its predecessor.

For a comparison piece this matters more than it sounds: a staged rollout means many readers cannot run the head-to-head themselves yet, and benchmark numbers gathered by organisations with early access may reflect configurations that are not generally available. Another reason to hold published scores loosely until you can reproduce them on your own account.

Both of these models cost $50 per million output tokens, neither publishes weights, and neither runs on hardware you own. For the frontier tasks described above, that is a reasonable trade. For the bulk of what actually gets sent to them &mdash; boilerplate, renaming across files, format conversion, first-draft tests, summarising diffs, answering questions about a codebase &mdash; it is not.

That work runs acceptably on open-weight models on hardware you already have, at no cost per token: a 32B-class model fits a 24&nbsp;GB card at Q4_K_M, a 14B fits 16&nbsp;GB, an 8B runs on 8&nbsp;GB. Our <a href="/models">Model Explorer</a> ranks what fits your specific machine, and the <a href="/gpu">GPU pages</a> break it down card by card. Keep the frontier model for the ten percent that genuinely needs one, and the choice between Astra and Claude stops being an expensive decision.