N環球 N GLOBE
US Technology

METR update: frontier AI models now finish week-long tasks about half the time

· N GLOBE

Non-profit AI evaluator METR published its latest quarterly assessment on November 20: OpenAI's GPT-6 Astra and Anthropic's Claude Opus 5.5 now have a 50% time horizon of about 40 and 39 hours respectively, up from 16 hours six months ago. But on open-ended tasks requiring research judgment, models still lag well behind humans.

Non-profit AI evaluation organisation METR published its latest quarterly capability assessment (Time Horizon 2, or TH2) on November 20. Its headline finding: OpenAI's GPT-6 Astra and Anthropic's Claude Opus 5.5 now have a 50% time horizon of about 40 hours and 39 hours respectively — meaning that, faced with a task that would take a skilled human expert a full working week, the leading models now stand roughly a fifty-fifty chance of completing it on their own. Six months ago, that figure was 16 hours.

What 40 hours means

METR's time horizon does not measure how long a model can run continuously. It measures task difficulty: the length of task, in human-expert hours, at which a model succeeds about half the time. METR says the figure is doubling roughly every 3.6 months, with a 95% confidence interval of 2.08 to 4.94 months. The report also notes that on tasks they do complete, models are typically several times faster than humans — the metric tracks the capability frontier, not working hours.

Top models beat hired experts on three benchmarks

On three automatically scored benchmark tasks, the best-performing model beat every specialist hired for the same task: Reverse Conway, 0.208 for the top model against 0.137 for the best recruit; Minecraft Bot, 0.175 against 0.055; and Train a Program, 0.443 against 0.042. The hired specialists logged 39, 160 and 45 hours of total work across the three tasks. The TH2 suite adds 36 newly authored tasks across 12 task families, combined with 222 older tasks in TH1.1 and 75 MirrorCode tasks, spanning software engineering, machine learning and cybersecurity.

Judgment remains the weak spot

On open-ended research tasks without a clear scoring standard, the models clearly lag. In the 'Making METR Tasks' test, no model produced an evaluation design that METR staff judged immediately usable; in a test scoring philosophical arguments, models also fell short of humans. AI graders were also found to be too generous to stronger models: in one test, Opus 5.5 scored GPT-6 Astra's work about 0.2 points out of 1 higher than a human rater did. In short, models can now match or beat skilled humans where there is a right answer — but where research taste and subjective judgment are needed, genuine autonomy is still some way off.

The uncertainty behind the numbers

METR discounts its own figures: only 9% of human task times come from a real timed baseline, the rest are projections, and the 95% interval around the 40-hour point estimate runs from about 12 hours to well over 1,000 hours. The report says public evidence is too thin to rule out a large speed-up in AI research and development soon, and that the team 'remains significantly uncertain about central questions.' MIT Technology Review has previously warned that the time-horizon chart is widely shared — and widely misinterpreted.

The assessment is seen as a key coordinate for judging AI's path toward autonomous research. METR also runs the embedded safety assessments OpenAI announced on November 16, continuing to monitor the capability frontier. The conclusion cuts both ways: progress on verifiable programming tasks is striking, but judgment — the ability to do research without a clear answer key — has not yet been crossed.

💬 Comments

No comments yet. Be the first!