FrontierCode v1.1 has surfaced a counterintuitive finding that contradicts what many developers assume about AI coding assistants: Claude Opus 5 at medium reasoning effort delivers nearly the same accuracy as the highest-scoring configurations tested, at ~1.5x baseline compute versus ~3x for high effort. For teams evaluating AI coding benchmarks to optimize their workflows and budgets, this data changes how they should weigh cost against accuracy when choosing between Anthropic and OpenAI models and configuring test-time compute scaling.
Disclosure: This analysis is based on benchmark data from FrontierCode v1.1. At the time of publication, the benchmark methodology, task set, and evaluation harness have not been independently verified by this publication. The models "Claude Fable 5" and "GPT-5.6 Sol" referenced below have not been confirmed in public Anthropic or OpenAI model listings; they may represent pre-release, internal, or hypothetical designations. Readers should treat all scores and cost figures as provisional until independently reproducible methodology is publicly available. The author has no disclosed financial or contractual relationship with Anthropic or OpenAI.
Table of Contents
- What FrontierCode v1.1 Actually Tests
- Claude Opus 5 Benchmark Results at Every Effort Level
- How Claude Opus 5 Compares to Fable 5 and GPT-5.6 Sol
- Why Medium Effort Wins: Understanding Test-Time Compute Scaling
- Practical Takeaways for AI-Assisted Coding Workflows
- Key Takeaways
What FrontierCode v1.1 Actually Tests
Benchmark Design and Task Categories
FrontierCode v1.1 evaluates frontier language models on realistic software engineering tasks. Unlike HumanEval, which focuses narrowly on standalone function completion, or SWE-bench, which tests models against real-world GitHub issues with verified human-authored patches, FrontierCode v1.1 spans a broader taxonomy of coding challenges. These include code generation, debugging, refactoring, and multi-step reasoning about codebases.
Note on methodology: At the time of writing, no public repository, paper, or methodology document for FrontierCode v1.1 has been identified. The task count, evaluation harness, pass@k configuration, sampling parameters, and identity of the evaluators have not been disclosed. Readers should be aware that the scores below cannot be independently reproduced without access to this information.
The benchmark reports two distinct score sets: a "main set" and an "extended set." The main set captures core coding competency across standard task categories. The extended set broadens coverage to include longer-context tasks and more complex multi-file interactions. Both sets use pass rates (pass@1 unless otherwise noted), representing the percentage of tasks a model configuration solves correctly on a binary pass/fail basis. This dual-set structure matters because performance gaps between the two often reveal how well a model handles different task characteristics rather than just baseline coding fluency.
Note: Extended set scores run consistently higher than main set scores across all models and effort levels tested. Available benchmark documentation does not explain this pattern. Readers should not assume the extended set is inherently harder based on task descriptions alone; task weighting, scoring normalization, or category-level difficulty calibration may account for the difference.
What "Reasoning Effort" Means in Practice
Reasoning effort, as evaluated in FrontierCode v1.1, refers to test-time compute scaling: how many "thinking" tokens a model generates before producing its final answer. Low, medium, and high effort settings correspond to increasing amounts of internal chain-of-thought processing. In practical API terms, these map to parameters that control the depth of reasoning a model performs on each request. For the Anthropic API, reasoning effort is controlled via the thinking parameter's budget_tokens field. For OpenAI, the equivalent is the reasoning_effort parameter (values: low, medium, high). Verify these parameter names against current API documentation before implementation, as they may change between API versions.
The cost and latency implications are direct: higher effort means more tokens consumed and longer response times. A high-effort query can use roughly twice the tokens of a medium-effort query, which in turn uses more than a low-effort query. For developers making hundreds or thousands of API calls per day, this dimension of model configuration becomes the primary cost driver, affecting both budgets and how responsive interactive workflows feel.
Claude Opus 5 Benchmark Results at Every Effort Level
The Numbers: Low, Medium, and High Effort Compared
FrontierCode v1.1 tested Claude Opus 5 across all three reasoning effort tiers on both the main and extended sets. The results reveal a clear pattern.
At low effort, Claude Opus 5 scores noticeably below its potential, as expected. The model produces faster but less thorough responses, missing correctness on tasks that require multi-step reasoning. Medium effort pushes Opus 5 to 53.4% on the main set and 63.6% on the extended set, its peak scores across all effort levels tested. High effort, however, does not reward the extra compute: scores plateau or marginally decline relative to medium effort, a difference within the approximation range of these measurements. This is the critical finding. Cranking reasoning effort to maximum does not yield better coding output from Opus 5 in this evaluation. The additional thinking tokens introduce diminishing or even negative returns.
Cranking reasoning effort to maximum does not yield better coding output from Opus 5 in this evaluation. The additional thinking tokens introduce diminishing or even negative returns.
FrontierCode v1.1 Performance Table
The table below presents the full benchmark data across all tested models and effort configurations. Relative compute cost is expressed as a multiplier against the lowest-effort baseline.¹ ²
| Model | Effort Level | Main Set (%) | Extended Set (%) | Relative Compute Cost |
|---|---|---|---|---|
| Claude Opus 5 | Low | ~48 | ~58 | 1x |
| Claude Opus 5 | Medium | 53.4 | 63.6 | ~1.5x |
| Claude Opus 5 | High | ~53 | ~63 | ~3x |
| Claude Fable 5 | Low | ~49 | ~59 | 1.2x |
| Claude Fable 5 | Medium | ~54 | ~64 | ~2x |
| Claude Fable 5 | High | ~55.3 | ~65.4 | ~3.5x |
| GPT-5.6 Sol | Low | ~47 | ~57 | 1x |
| GPT-5.6 Sol | Medium | ~52 | ~62 | ~1.8x |
| GPT-5.6 Sol | High | ~53 | ~63 | ~3.2x |
¹ Values prefixed with "~" are approximate. Exact scores have not been published. Narrative comparisons between approximate values (e.g., high vs. medium effort deltas) should be interpreted with appropriate uncertainty.
² Relative compute cost methodology is undisclosed. It is not confirmed whether these multipliers reflect total tokens consumed, thinking tokens only, or another metric. Claude Fable 5's higher baseline cost at low effort (1.2x vs. 1x) may reflect a larger base model or different minimum token budget, but no explanation has been provided. Actual API cost savings will vary by provider pricing and output token ratios.
Efficiency highlight: Claude Opus 5 at medium effort achieves scores within 1-2 percentage points of the top-scoring Fable 5 high-effort configuration (based on estimated scores), while consuming roughly half the compute. This positions Opus 5 medium as the most compute-efficient configuration tested on FrontierCode v1.1.
How Claude Opus 5 Compares to Fable 5 and GPT-5.6 Sol
Claude Fable 5: Higher Ceiling, Higher Cost
Claude Fable 5 at high effort scored highest in the FrontierCode v1.1 evaluation, reaching approximately 55.3% on the main set and 65.4% on the extended set. These represent the ceiling across all configurations tested. But the gap between Fable 5 at high effort and Opus 5 at medium effort is narrow: approximately 1-2 percentage points on both sets, based on estimated scores.
The compute cost to capture that marginal gain is substantial. Based on the table's estimated multipliers (3.5x / 1.5x), Fable 5 at high effort consumes approximately 2.3x the compute of Opus 5 at medium, though both figures are estimates pending methodology disclosure. For teams running large volumes of coding queries, this translates to more than double the estimated token consumption for a small accuracy improvement. Fable 5 high effort pays off only in scenarios where even marginal accuracy gains carry outsized value, such as high-stakes automated refactoring pipelines where correctness rates compound across many operations. (See the safety caveat below for safety-critical applications.)
GPT-5.6 Sol and Cross-Provider Context
FrontierCode v1.1 also evaluated GPT-5.6 Sol from OpenAI. At high effort, it scores in a range comparable to Claude Opus 5 at medium effort, landing at approximately 53% on the main set and 63% on the extended set. Under the same evaluation conditions, this places it competitively with but not ahead of Opus 5's medium-effort configuration, assuming methodological parity across providers. These cross-provider comparisons assume identical evaluation conditions for all models. Differences in system prompt handling, API defaults, or tokenization may affect results.
Anthropic's Opus 5 shows a better accuracy-to-cost ratio on coding-specific tasks within this benchmark, though several caveats apply. FrontierCode v1.1, like all benchmarks, measures a specific distribution of tasks. Real-world coding work varies dramatically in structure, ambiguity, and domain. Models that score similarly on benchmarks can behave quite differently on production codebases with proprietary libraries, unconventional patterns, or domain-specific constraints. Benchmark scores should inform model selection but not serve as the sole decision criterion.
Performance-per-Compute Visualization
Plotting relative compute cost on the X-axis against FrontierCode score on the Y-axis produces a characteristic diminishing returns curve. The curve rises steeply from low to medium effort, then flattens dramatically from medium to high. Claude Opus 5 at medium effort sits near the inflection point where the curve bends, meaning each additional unit of compute beyond this point yields progressively less improvement. Fable 5 at high effort pushes slightly higher on the Y-axis but far to the right on the X-axis, illustrating the disproportionate cost of those final percentage points. (Note: A chart illustrating this relationship is not included in this article; the description above summarizes the pattern visible in the tabular data.)
Why Medium Effort Wins: Understanding Test-Time Compute Scaling
The Diminishing Returns Problem
The general principle behind test-time compute scaling is that allowing a model more internal reasoning steps should improve output quality. In practice, the relationship between thinking tokens and correctness is not linear. Early reasoning steps tend to catch obvious errors, resolve ambiguities, and structure the approach. Additional reasoning beyond a certain threshold can introduce failure modes.
The model may second-guess correct initial solutions or overengineer responses. This explanation is consistent with the data but not confirmed by this evaluation. Without interpretability analysis or mechanistic study, the precise cause of the plateau or marginal regression at high effort remains an open question.
Coding tasks are particularly susceptible to diminishing returns from extra reasoning. Unlike open-ended generation, where quality exists on a spectrum, coding benchmarks using binary pass/fail test execution grade correctness as all-or-nothing. A function either passes its test suite or it does not. Extra reasoning on a coding problem may degrade output by overcomplicating solutions that were already correct at a simpler level of analysis.
What This Means for the "Just Crank It Up" Instinct
Many developers default to the assumption that maximum reasoning effort equals best results. FrontierCode v1.1 data directly contradicts this for Claude Opus 5 on coding tasks. Medium effort represents an optimal allocation where the model has enough reasoning budget to solve complex problems without entering the diminishing-returns zone.
Medium effort represents an optimal allocation where the model has enough reasoning budget to solve complex problems without entering the diminishing-returns zone.
Other teams have reported similar patterns informally when scaling test-time compute across different model families, though systematic published evidence for this as a general principle is limited. The practical implication is clear: treat reasoning effort as a tunable parameter to optimize, not a dial to max out.
Practical Takeaways for AI-Assisted Coding Workflows
When to Use Medium Effort (Most of the Time)
For general coding tasks, including code generation, debugging, refactoring, and test writing, medium effort on Claude Opus 5 represents the optimal default. Switching from high to medium effort across a typical development workflow of 200-500 daily queries can reduce estimated token consumption by ~50% based on benchmark compute multipliers; actual API cost savings will vary by provider pricing and output token ratios.
When Higher Effort Still Makes Sense
Specific scenarios justify higher reasoning effort. Complex multi-file architectural decisions (tasks touching 3+ files or requiring coordinated changes across several hundred lines), novel algorithm design, and tasks with highly ambiguous specifications may benefit from the additional deliberation. Safety-critical code may warrant additional compute, though benchmark accuracy improvements do not constitute safety assurance; human review and formal verification remain necessary regardless of model performance on coding benchmarks. The key is selective escalation rather than blanket maximization.
The Fable 5 vs. Opus 5 Decision
Here's how to choose. Budget-constrained teams should default to Claude Opus 5 at medium effort. Accuracy-maximizing workflows where marginal gains justify the spend should consider Fable 5 at high effort. Both decisions should also account for latency requirements, context window needs, and the specific task distribution of the target workflow, factors that benchmark scores alone cannot capture.
Key Takeaways
- Claude Opus 5 at medium reasoning effort is the most compute-efficient configuration tested on FrontierCode v1.1, scoring 53.4% (main) and 63.6% (extended).
- High effort shows diminishing or slightly negative returns for Opus 5 on coding tasks, disproving the idea that more reasoning always improves output. The observed difference falls within the approximation margin of reported scores.
- Fable 5 at maximum effort scores only about 1-2 percentage points higher than Opus 5 medium (based on estimated scores), at roughly twice the estimated compute cost.
- Default to medium effort for everyday coding tasks. Selectively escalate to high effort only for complex, ambiguous, or safety-critical work, with the understanding that benchmark accuracy gains do not replace human review or formal verification.
- Real-world task distributions, latency constraints, and context window requirements all factor into optimal configuration; benchmark performance alone should not dictate model and effort-level selection.
- All scores and cost multipliers in this analysis draw from FrontierCode v1.1 data whose methodology has not been publicly disclosed. Treat these findings as provisional until reproducible methodology is available.
