Claude Code subagent effort (TokenSave study)Partly tested🌏 World

TokenSave's 288-run study: asking for subagent search left recall almost unchanged and raised the Opus bill by up to 221%

Claude Code 2.1.318 added subagent effort (`CLAUDE_CODE_SUBAGENT_EFFORT`). TokenSave (dataset under johnisking; Japanese write-up by @Yunany90) ran 288 pre-registered headless trials on 12 “find every call site” questions across 6 OSS repos, Haiku 5.5 / Opus 5.5, no-instruction vs forced Explore subagents at low/medium/high. Result: recall stayed ~97–100% (<2 pt gaps); cost rose Haiku +74–116%, Opus +143–221%; with no instruction, 0/72 runs called a subagent. We recomputed the CSV: 288 rows, balanced cells, Opus C2/C0 mean cost ≈3.21× (+221%), C0 subagent_calls sum 0. We did not re-bill a live Claude Code session.

Visit site Claude Code subagent effort (TokenSave study) ↗
Pricing
Paid
Region
Global
Where it's from
Overseas
Added
Nov 7, 2026
Tested on
Nov 7, 2026
Version we used
token-cost-by-language @ 1acf67e; Claude Code 2.1.318 per study; dataset DOI 10.5281/zenodo.24014489
Categories
CodingProductivityResearch
Original author
TokenSave / 조재현 @Yunany90 ↗
Our test
Partly tested
Cost / time
Dataset and write-ups are free (CC BY 4.0). Your own Claude Code usage still bills via Anthropic; this study shows casual subagents can sharply raise per-run cost.
Source
Qiita @yunany90 ↗X @LiLensss ↗GitHub johnisking/token-cost-by-language ↗

The task

When doing ordinary “find every call site” searches in Claude Code, decide whether to force Explore subagents in the prompt and whether to hand-tune subagent effort.

Steps

  1. Read the results page or repo: https://tokensave.app/data/claude-subagent-effort and GitHub `claude-subagent-effort/` (PREREG.md, questions.json, CSV/JSON, scorers). License CC BY 4.0.
  2. For everyday code search: state what to find; don't casually add “use subagents / Explore”. With no instruction the harness almost never calls them (0/72 here).
  3. Reserve subagents for large monorepos or tasks you personally A/B for cost vs recall; don't default `CLAUDE_CODE_SUBAGENT_EFFORT` to medium/high.
  4. If you drive headless/`claude -p` with subagents: 2.1.318 emits multiple result events—join all of them; scoring only the first tanks accuracy to single digits.
  5. To reproduce: pin repo commits, pre-register answer keys, allow read/search tools only; watch the 5-hour subscription limit creating empty runs.

Prompt

In this repo, find every place RequestAbortedError is created; list file paths and enclosing function/method names. Use Grep and Read yourself and write the final answer—do not launch subagents.

What the author says

TokenSave / @Yunany90: 288 pre-registered runs show forced subagent search barely moves recall (~97–100%) while raising Haiku cost +74–116% and Opus +143–221%, with ~2–3× latency; subagents bill at the main model; with no instruction none were started. Chinese amplifiers include @LiLensss.

Our test

Cloned johnisking/token-cost-by-language (commit 1acf67e), read claude-subagent-effort/README.md and PREREG.md, and recomputed claude-subagent-effort.csv/.json in Python. Cross-checked Qiita @Yunany90 and the tokensave.app page. No Claude Code install, no re-run of the 288 headless jobs, no subscription-limit behaviour check.

What we found when we tried it:CSV 288 rows; 144 each Haiku/Opus; 36 per condition. Question-mean recall: Haiku C0/C1/C2/C3 ≈ 0.982/0.970/0.985/0.987; Opus ≈ 0.997/0.993/0.990/1.000. Mean cost: Haiku $0.0056/$0.0098/$0.0105/$0.0121; Opus $0.1589/$0.3861/$0.5093/$0.4825. C0 subagent_calls sum 0; Opus C2 vs C0 ≈ +221%. Matches the published figures.

How to get in

Public GitHub / Zenodo / Hugging Face / tokensave.app; Japanese Qiita longform; Chinese summary via @LiLensss. Needs GitHub access.

Who it suits

Daily Claude Code users who care about bill and latency; people wrapping Claude Code in CI/headless scripts.

Pitfalls

  • Questions are mid-size “find the full list” tasks where base models were near ceiling—huge monorepos may differ.
  • One prompt wording and Claude Code 2.1.318 only; re-test after version changes.
  • Headless multi-result events are easy to score wrong—read the scoring notes before reproducing.
  • We recomputed the public CSV/JSON only; we did not re-spend API on 288 runs.

More in this category