Claude Fable 5.2 Completes 41% of Real Upwork Projects on Scale's Remote Labor Index — Roughly Double Its Rivals
Claude Fable 5.2 completes 41.3% of real paid Upwork projects on Scale Labs' Remote Labor Index — roughly twice Opus 5.5 and GPT-6 Astra, and far above the 2.5% best score at the benchmark's launch.
Anthropic’s Claude Fable 5.2 now tops Scale Labs’ Remote Labor Index (RLI) with a 41.3% automation rate — the share of real, paid freelance projects an agent completes at least as well as the human professional who originally delivered the work. That is close to twice Opus 5.5 (21.25%) and GPT-6 Astra (20.83%), 2.3 times Fable 5.1 (17.92%), and far above the 2.5% that the then-leading agent, Manus, managed at the time the leaderboard launched.
What the benchmark actually measures
RLI is unusual among agent benchmarks: every project is a real economic transaction. The benchmark’s dataset — 240 attempted projects, 230 of them private for scoring — was sourced from 358 verified Upwork freelancers across 23 domains, with a total economic value of $143,991. Projects average $632.60 in human cost (median $200) and 28.9 hours of professional work (median 11.5 hours), and deliverables span documents, audio, video, 3D models and CAD files.
Scoring is deliberately unglamorous: trained expert evaluators — not the original clients — compare the agent’s deliverable against the human gold standard and ask whether “a reasonable client” would accept it. A majority vote of three evaluators, with 94.4% inter-annotator agreement, decides. Each agent gets a standardized harness, a $30 generation budget per task and access to image, speech and video tools. Manual expert grading and a private task set make the benchmark harder to shortcut than automated suites, though the source itself notes a residual contamination risk on its 10 public projects.
The scope is narrower than “freelance work” at large: RLI explicitly excludes projects requiring direct client interaction (customer service, tutoring), physical labor, or long-term evaluation (SEO), covering 23 of Upwork’s 64 domains. A 41.3% automation rate is expert acceptance of benchmark deliverables — not a measure of actual customer acceptance or of the share of all freelance work that can now be automated.
Why the jump matters
Near-floor scores at the benchmark’s launch made RLI a useful counterweight to benchmarks where frontier models post 80-90%+. If agents could not finish a $200 logo job to a client’s standard, talk of economic displacement was premature. Fable 5.2’s 41.3% does not close that gap, but it moves the conversation from “agents can’t do real work” to “agents complete a large minority of real work, and the leader is pulling away.”
The competitive picture shifted in one release. Fable 5.1 and Opus 5.5 both scored in the high teens and low twenties; Fable 5.2 more than doubled its predecessor in a single generation, while GPT-6 Astra and Opus 5.5 — the models many operators route end-to-end agent work to today — sit at roughly half its rate. For anyone selecting a model for unattended, deliverable-producing jobs, RLI is now the most direct public evidence available.
The result also lands amid a wave of Fable 5.2 coverage. Ethan Mollick’s hands-on assessment — filmmaking, business launches, literary scholarship, original research — concluded the model is “almost” AGI: expert-grade on many digital tasks, but unreliable at noticing its own missteps, from joyless course design to a self-generated 87,500-word rulebook (One Useful Thing, Nov 9). The Rundown’s November 10 issue says Anthropic “just released” the model at 20% cheaper and up to 30% faster than 5.1, with Anthropic describing its capability restraint as “pacing, not pausing” (The Rundown AI). RLI grounds that narrative in paid, expert-graded work rather than demos.
The honest caveat: 59% still fail
RLI’s published failure-mode analysis — drawn from the benchmark’s evaluation of agents generally, not a model-specific breakdown of Fable 5.2’s remaining 58.7% — clusters around (categories overlap, so shares total more than 100%):
| Failure mode | Share of failed projects |
|---|---|
| Poor quality (“child-like” output) | 45.6% |
| Incomplete deliverables | 35.7% |
| Technical and file issues | 17.6% |
| Inconsistencies across files | 14.8% |
These are production problems as much as reasoning problems: truncated videos, corrupt file formats, a 3D house whose design changes between views. Mollick’s independent testing points the same direction: Fable’s internal checks catch errors but never ask whether the work is too long, too repetitive, or no fun. Independent external evaluation is exactly what surfaces that gap, which is why Anthropic giving METR employee-level access matters as much as leaderboard scores.
The operator read
For teams running agent fleets, three takeaways. First, the model you route end-to-end deliverable work to now has a measurable, economically grounded ranking — and it changed decisively this release. Second, even the leader fails the majority of projects: keep human review gates on anything client-facing, a lesson that also showed up in our coverage of underspecified-prompt benchmarks. Third, the failure modes above are dominated by quality, completeness and file-delivery problems rather than fabricated facts — promising for harness engineering and verification steps, though neither source proves those fixes alone close the remaining gap. As we’ve argued in reading agent benchmarks in 2026, RLI’s manually graded, real-money design makes it one of the few scores worth tracking at face value.
The benchmark’s limitation cuts the other way too: manual expert evaluation is time-consuming and expensive, which the source says makes it difficult to test new agent frameworks at high frequency. But a jump from a 2.5% floor to 41.3% is a trajectory signal, not noise.
