Claude Opus 5.5 tops SemiAnalysis GDPx agent benchmark with 12%
SemiAnalysis has published GDPx, a new benchmark for long-running AI agents, and its first leaderboard puts Anthropic's Claude Opus 5.5 in first place with a headline score of 12.0%. Google's Gemini 4 Carbon follows at 10.9% and OpenAI's GPT-6 Astra at 9.6%, while Meta's Muse Wonder Full, Claude Fable 5.1, GPT-6.1 Sol and Claude Sonnet 5.5 sit in a tight group between 9.0% and 8.2%. The benchmark was released on November 2, 2026 by Max Kan, Andrew Megalaa and Dylan Patel of SemiAnalysis, together with a public leaderboard and data explorer at gdpx.semianalysis.com.
The low numbers are the point of the design. GDPx contains 206 tasks spread over 11 professional domains, from chip design and cybersecurity to finance, legal work and quantitative trading, and each task runs inside a realistic working environment for up to eight hours. The authors describe it as "a true frontier-level benchmark: even the best model scores just 12%." The ranking covers 13 models with published headline scores, including open-weight systems from DeepSeek and Zhipu and xAI's Grok 4.7, which finishes at 3.9%.
··········
HOW THE SEMIANALYSIS GDPX BENCHMARK IS BUILT.
SemiAnalysis chose its domains by asking where frontier AI is most likely to produce revenue, and it deliberately left out sectors such as government, real estate and insurance. The 11 domains are recursive self-improvement, chip design, cybersecurity, biology, finance, legal, quant trading, hardware design, materials science, alignment and software engineering. The tasks were supplied by 14 specialist companies that build reinforcement-learning environments, and every partner had to pass SemiAnalysis's automated quality checks on the first attempt with at most minor mistakes. The authors say the same checks, run against Terminal Bench 4.0, flag nearly half of that benchmark's tasks as broken, which is the standard they want GDPx to be judged against.
The environments are large. A finance task, for example, places the agent in a data room of roughly 4,000 files replicating a private-equity acquisition, and a hardware task asks it to route a 12-layer DDR5 tester board with a 676-ball FPGA, 2,520 connections and 767 components. Models run in their native agent harness where one exists, such as Claude Code for Opus 5.5 and Codex for GPT-6 Astra, and in the open-source OpenCode harness otherwise. The default time limit is eight hours per task.
........
Domain | Task partner(s) | Tasks |
Recursive self-improvement | Extrapolate | 10 |
Chip design | Aria, Arcverex, Ergodex | 34 |
Cybersecurity | Arithmetic | 18 |
Biology | LatchBio | 13 |
Finance | Living Context | 24 |
Legal | Chakra | 20 |
Quant trading | Third Moment | 10 |
Hardware design | Beautiful Machines | 21 |
Materials science | ValenceForge, Emulated | 23 |
Alignment | Fleet | 18 |
Software engineering | Datacurve | 15 |
........
Scoring follows a few explicit rules. The headline figure is pass@1 averaged over three attempts per task, every domain carries the same weight, and partners inside a domain are also weighted equally. Grading is binary or partial depending on the field; in finance and legal, binary grading gave nearly every model zero, so the leaderboard uses average partial scores there. Because each of the 11 domains counts for about 9.1 points of the total, a headline score of 12% is roughly equivalent to fully solving one domain and a small part of a second, which gives a sense of how far the tasks sit beyond current agents.
SemiAnalysis positions GDPx against OpenAI's GDPval, released in September 2025, which it calls "woefully inadequate for evaluating today's frontier models." Its argument is that every GDPval task is precisely specified and non-interactive, while real professional work requires an agent to explore an environment, find the relevant material and decide what the request really demands.
··········
CLAUDE OPUS 5.5, GEMINI 4 CARBON AND GPT-6 ASTRA ON THE GDPX LEADERBOARD.
The published leaderboard gives headline scores for 13 models. The table below lists them in order with the gap to the leader in percentage points; per-domain values appear on the site only as charts and are therefore omitted.
........
Rank | Model | Developer | GDPx score | Gap to leader |
1 | Claude Opus 5.5 | Anthropic | 12.0% | 0.0 pts |
2 | Gemini 4 Carbon | 10.9% | 1.1 pts | |
3 | GPT-6 Astra | OpenAI | 9.6% | 2.4 pts |
4 | Muse Wonder Full | Meta | 9.0% | 3.0 pts |
5 | Claude Fable 5.1 | Anthropic | 8.5% | 3.5 pts |
6 | GPT-6.1 Sol | OpenAI | 8.4% | 3.6 pts |
7 | Claude Sonnet 5.5 | Anthropic | 8.2% | 3.8 pts |
8 | Gemini 4 Argon | 7.3% | 4.7 pts | |
9 | Muse Wonder | Meta | 6.8% | 5.2 pts |
10 | DeepSeek V4.1 Flash 1017 | DeepSeek | 5.4% | 6.6 pts |
11 | GPT-6 Luna | OpenAI | 5.1% | 6.9 pts |
12 | Grok 4.7 | xAI | 3.9% | 8.1 pts |
13 | GLM 5.3 | Zhipu AI | 2.7% | 9.3 pts |
........
According to SemiAnalysis, Opus 5.5 is the strongest all-round model and leads quant trading, hardware design and chip design. Gemini 4 Carbon is the best biology model and edges ahead in software engineering, and the authors describe it as "a real improvement from Google and the only model in the same ballpark as Opus and Astra." GPT-6 Astra leads recursive self-improvement and materials science, and Meta's Muse Wonder Full is, in the authors' words, "the best available cyber model today by a meaningful margin."
The ranking also shows how much model tier matters inside each family. Anthropic's top model scores 3.5 points above Claude Fable 5.1 and 3.8 points above Claude Sonnet 5.5, Gemini 4 Carbon is 3.6 points ahead of Gemini 4 Argon, and GPT-6 Astra is 4.5 points ahead of GPT-6 Luna. Readers comparing these models for coding and agent work can set the new numbers against our Claude Opus 5.5 versus Claude Fable 5.1 comparison of pricing and benchmarks and our report on Gemini 4 Carbon against Gemini 4 Argon and Claude Opus 5.5.
Two qualifications from the authors affect how the table should be read. First, SemiAnalysis says Opus 5.5 and GPT-6 Astra have "sandbagged" headline scores because their safety classifiers interfere with tasks in cybersecurity, biology and alignment, which is why the site also offers a ranking that excludes those three domains; the values of that alternative ranking are published only as charts. Second, the benchmark includes a cost-versus-performance view in which, according to the authors, GPT-6 Luna and DeepSeek compete at the low end, Opus wins the high end and OpenAI's models occupy the middle. No dollar figures for that comparison are given in the text.
··········
WHERE FRONTIER AGENTS FAIL IN GDPX FINANCE, LEGAL AND TRADING TASKS.
The public example tasks that SemiAnalysis describes in detail are more informative than the headline numbers for anyone planning to use agents in professional work. In finance, the 24 tasks are built around a single replicated deal in which a private-equity fund buys about 62% of a snack company for $880 million plus an earnout, and the agent must produce or revise spreadsheets, decks, memos or PDFs covering LBO models, debt covenants, purchase-price adjustments and quality-of-earnings work. In the public quality-of-earnings task, no model passed across 25 attempts. GPT-6.1 Sol averaged 5.6 of 11 checks, Opus 5.5 and GPT-6 Astra averaged 5.4, and GLM 5.3 and Grok 4.7 stayed under 2; the best single run passed 8 of 11.
The finance errors are the kind an experienced analyst would catch in review. Ten of 15 runs by Opus, Astra and Sol treated a waived premium-audit adjustment as a settlement and understated trailing EBITDA, fifteen runs wrongly put the sell-side banker's success fee into net debt, and most runs missed owner perks that appeared only in ledger memos. Seventeen runs kept compensation for a family member who had logged no hours.
Legal work produced similar results. The 20 legal tasks from Chakra cover eight practice areas, including M&A, venture capital, capital markets and data privacy, and are graded against expert-written rubrics, with each output graded five times. In a public task asking the agent to redline a Series B charter for the lead investor, none of 20 runs produced a usable redline. Mean partial scores were 44% for Opus 5.5, 25% for GPT-6 Astra, 2% for Gemini 4 Argon and 0.5% for GLM 5.3, and the best run, by Opus 5.5, reached 57%. All 20 runs missed a subsidiary bypass in the charter.
Quant trading is graded with a statistical test on hidden trading days, so a strategy cannot score by taking more risk. Of 111 graded runs, 70 did worse than the reference strategy. In one AAPL task, average rewards were 0.31 for Opus 5.5, 0.23 for GLM 5.3, 0.19 for Grok 4.7 and 0.11 for GPT-6.1 Sol, and no model solved it. The authors give a telling example: one Opus 5.5 run earned $2,540 against the reference strategy's $1,399 but took 1.3 times the risk and scored only 0.23, while another cut its trading so aggressively that a $2,221 validation result turned into $84 on unseen days.
··········
GDPX RESULTS IN CHIP DESIGN, HARDWARE, BIOLOGY AND SOFTWARE ENGINEERING.
Chip design is the only domain with three task partners, which SemiAnalysis justifies by the AI spending it expects in the field from 2027. The tasks range from writing self-checking SystemVerilog testbenches to optimizing RTL for power, performance and area and repairing timing on a placed design. In a public timing-repair task on an Ibex processor core, GPT-6 Astra averaged 0.34 against 0.135 for Opus and 0.07 for Fable, and the task partner Ergodex reports that OpenAI models consistently beat Anthropic models on this type of engineering change order. Opus 5.5 still leads the chip design domain as a whole.
Hardware design tasks come from real enterprise projects and use tools such as SOLIDWORKS, Siemens NX, KiCad, Vivado and Cadence. In a public task to convert a 3D-printed compressor housing for injection molding, none of 12 runs by GPT-6 Astra, Opus 5.5, Grok 4.7 and GLM 5.3 achieved a verified full pass, although five of the six Opus and Astra submissions passed the clearance and edit-perturbation checks. A GLM run displaced the housing by more than 11 mm, and a Grok run cut four screw holes in the wrong direction.
In biology, LatchBio supplies 13 tasks graded deterministically against real experimental data, without rubric judges. The failures are statistical: treating replicates as independent, confusing instrument artifacts with biology and using all supplied data without filtering. In the public antibody task, Opus 5.5 fell into the trap of trusting inflated chip-binding measurements in every rollout. In software engineering, Datacurve's tasks are written by open-source maintainers, and the authors report that 76% of failed frontier-model rollouts involved requirements around the main request, such as preserving existing behavior, integrations and edge cases. Models tended to write tests that confirmed their own code rather than tests that checked the original requirements.
··········
WHAT GDPX MEANS FOR TEAMS CHOOSING CLAUDE, GEMINI OR GPT AGENTS.
For companies deciding which model to put behind an agent, GDPx offers three practical signals. The first is that the ranking at the top is close: 2.4 points separate Opus 5.5 from GPT-6 Astra, and domain leadership is split among Anthropic, Google, OpenAI and Meta, so the right choice depends on the type of work more than on the headline order. A team running chip-verification or timing work has reason to test GPT-6 Astra alongside Opus 5.5, while a biology team has reason to include Gemini 4 Carbon.
The second signal concerns supervision. Even the best runs in finance and legal work produced deliverables that an expert would reject, and the failure patterns, from misclassified adjustments to missed charter provisions, are the details a reviewer has to check. The benchmark supports using frontier agents to draft and assemble long pieces of professional work, with a qualified person reviewing the result before it is used.
The third concerns cost and tier. Mid-tier models such as Claude Sonnet 5.5 and GPT-6.1 Sol score within about four points of the leader, and SemiAnalysis's cost analysis places the cheapest competitive options with GPT-6 Luna and DeepSeek. Teams that run large volumes of agent tasks can treat the gap between tiers as a measurable trade-off against price, and our Claude Opus 5.5 versus GPT-6 Astra comparison sets out the published per-token prices of the two leading closed models.
··········
SEMIANALYSIS GDPX: CONFIRMED RESULTS AND OPEN QUESTIONS.
The established facts are the benchmark design and the published headline scores: 206 tasks in 11 equally weighted domains, eight-hour tasks run in native harnesses, pass@1 over three attempts, and a leaderboard led by Claude Opus 5.5 at 12.0%, Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%. The task-level examples, from the finance quality-of-earnings case to the Series B redline, are documented by SemiAnalysis with per-model figures.
Several points remain open. Per-domain scores, the ranking without cybersecurity, biology and alignment, and the cost comparison are published only as charts, so their exact values cannot be checked from the text. Three attempts per task is a small sample, and the authors acknowledge that task selection involved some judgment when several partner tasks passed quality control. Most tasks are private, which limits independent reproduction, and the publication includes no response from the model developers. As a first measurement, GDPx shows a frontier that is close at the top, uneven across domains and still far from completing long professional tasks without human review.
··········
FOLLOW US FOR MORE.
·····
DATA STUDIOS
·····
datastudios.org
