Even the best AI completes only 12% of real jobs in SemiAnalysis's new GDPx benchmark

SemiAnalysis published a new agentic benchmark, GDPx, on November 2, placing 13 frontier AI models inside simulated workplaces and asking them to actually do 206 professional tasks — and the best model completed only 12.0%. Anthropic's Claude Opus 5.5 led at 12.0%, Google's Gemini 4 Carbon came second at 10.9%, OpenAI's GPT-6 Astra third at 9.6%, and the lowest-ranked model, Z.ai's GLM 5.3, managed 2.7%. That is not a story about models getting worse; it is a story about the test getting honest: GDPx grades jobs, not exam answers. (Cover photo is a file photo of an AI data centre.)

What it tests: 11 high-return domains, eight hours per task

GDPx spans chip design, hardware design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement, materials science, alignment and software engineering — 11 domains chosen for economic return on investment rather than share of GDP. Its finance tasks reconstruct a real private-equity deal, a fund buying roughly 62% of a snack manufacturer for about $880 million plus an earnout, spread across some 4,000 anonymised deal files the model must work through; cybersecurity tasks run against live services carrying real unpatched vulnerabilities. Tasks carry an eight-hour default limit, and models run in their native tooling where available — Claude Code for Opus 5.5, Codex for GPT-6 Astra, Antigravity for Gemini 4 Carbon — with the headline score averaging first-attempt pass rates over three attempts and weighting every domain equally.

Different leaders by domain; finance and legal at zero

Opus 5.5 won quant trading, hardware design and chip design outright; Gemini 4 Carbon led biology and software engineering; GPT-6 Astra took recursive self-improvement and materials science. But under binary grading — a task counts only if completed end to end — every model scored zero in both finance and legal, which is why SemiAnalysis reports partial-credit scores for those domains instead. In quant trading, 70 of 111 graded runs underperformed a simple reference strategy, and in alignment-focused tasks, 196 of 200 runs broke at least one behavioural rule. Costs varied as widely as capability: Opus 5.5 averaged $51.41 per task, Gemini 4 Carbon roughly a third of that at $16.29, and GPT-6 Astra was the most expensive at $53.99 per task for a score 2.4 points behind the leader.

How to read the 12%: three caveats

First, GDPx is a paid SemiAnalysis product whose partner-built tasks have not been independently audited, so on day one every number is the publisher's own reporting. Second, SemiAnalysis itself says Opus 5.5 and GPT-6 Astra are held back by their safety classifiers on cybersecurity, biology and alignment tasks, making the 12.0% figure conservative for those models. Third, GDPx is positioned as the successor to OpenAI's GDPval from September 2025, which SemiAnalysis argues belongs to the pre-agent era because it hands the model all context up front with no interaction; the firm plans updates every three to six months, so the next reading will show which way the line moves.