Anthropic’s Fable 5.2 tops freelance AI benchmark with 41.3% pass rate

Anthropic’s latest model tops a benchmark measuring complete freelance project completion, but the metric reveals significant limitations in AI readiness for real-world independent work. The result matters for assessing whether AI can actually replace or augment remote labor at scale.

• Fable 5.2 achieved 41.3% pass rate on the Remote Labor Index, nearly double Opus 5.5’s 21.25%.
• The RLI test dataset contains 240 projects across 23 freelance categories including documents, audio, video, 3D models, and CAD files.
• Passing requires a finished deliverable meeting professional reference standards, meaning 58.7% of evaluated projects fell below acceptance.

• 41.3% Fable 5.2 pass rate on Remote Labor Index versus 21.25% for Opus 5.5
• 240 Freelance projects in the RLI dataset spanning 23 work categories

CryptoSlate reported that Anthropic’s Fable 5.2 model leads the Remote Labor Index leaderboard, a test of AI performance on complete freelance projects. According to Anthropic’s November 9 release announcement, Fable 5.2 scored 41.3% on the RLI benchmark with safeguards enabled, a designation Anthropic labeled “High effort.” The leaderboard, checked on November 12, lists Fable 5.2 ahead of the next-highest performer, Opus 5.5, which achieved 21.25%. The gap between the two models underscores how few AI systems currently clear the RLI’s demanding acceptance standard.

What the Remote Labor Index actually measures

The RLI dataset, described in a research paper submitted in October 2025 by researchers from the Center for AI Safety and Scale AI, consists of 240 complete projects spanning 23 freelance work categories. Each project includes a client brief, the necessary input files, and a professional human reference deliverable that sets the quality bar.

The projects encompass documents, audio, video, 3D models, and computer-aided design files, making the benchmark a test not only of instruction comprehension but also of file production and finished-product quality.

Human reviewers, working independently and using majority voting to reach conclusions, compare each AI deliverable against both the project brief and the professional reference standard. A project passes only when the AI’s work satisfies the brief at least as well as the reference, at a quality a reasonable client would accept. Partial results or useful prototypes that fall short of the professional standard do not pass, even if they demonstrate relevant capabilities.

Fable 5.2’s lead reveals the gap between benchmark success and practical readiness

Fable 5.2’s 41.3% pass rate means that nearly six in ten evaluated projects failed to meet the professional acceptance threshold.

The RLI’s design reflects real-world constraints that broader AI benchmarks often overlook. The original study excluded work requiring client interaction, teamwork, or physical labor, as well as projects whose outcome depends on client feedback before evaluation. These exclusions mean the RLI tests a subset of remote work, not the full spectrum of freelance activity. Anthropic’s disclosure that Fable 5.2 achieved this result at “High effort” with safeguards enabled does not specify the exact execution setup, per-project spending, or the degree of human assistance involved in the evaluation.

The pass-rate metric alone does not measure job displacement, freelance platform earnings, or the actual cost of deploying such a system in production. Benchmark results differ from real employment outcomes and do not reflect what humans might earn or what companies might pay for AI-generated work on actual freelance platforms.

The BlockWest read. The RLI reveals a wide gulf between leading-model capability and the bar for autonomous execution on real work. For allocators and hiring teams assessing AI’s near-term utility, a 41% completion rate on curated projects signals that AI augmentation of freelance labor remains months or years away from reducing headcount, and that human oversight remains essential infrastructure.

Anthropic has not disclosed whether it plans to optimize Fable 5.2 specifically for RLI task categories or whether further versions will target the 58.7% failure rate. Scale AI and the Center for AI Safety may expand the dataset or refine evaluation methods, which could shift the competitive ranking or reveal whether improvements are genuine capability gains or artifacts of benchmark-specific tuning.

BlockWest is a news publication. Nothing here is investment advice. Read our disclaimer and editorial policy.