$2,893 of compute, and where it went.
Every model call in the study, priced at the providers' published API rates: 2.33 billion tokens over 200 agent-hours, in 34 research sessions, 24 Claude Code sessions with their subagents, and 1,844 benchmark runs. Running the same research and benchmark yourself would cost about this much before counting your own time.
Where the $2,893 went
Show the numbers
| Part of the work | Cost at API prices |
|---|---|
| Research sessions behind the 34 research documents (estimated from the reports) | $1,483 |
| Reviews, writing, building and analysis, in Claude Code sessions | $801 |
| Benchmark runs of Claude Code | $542 |
| Other models: graders, labellers and task writers | $53 |
| Claude helper calls inside the benchmark | $13 |
| Total | $2,893 |
When the money was spent
Show the numbers
| Day | Claude Code sessions | Benchmark runs | Total |
|---|---|---|---|
| 16 Sep | $101.39 | $0.00 | $101.39 |
| 17 Sep | $0.00 | $0.00 | $0.00 |
| 18 Sep | $0.00 | $0.00 | $0.00 |
| 19 Sep | $0.00 | $0.00 | $0.00 |
| 20 Sep | $0.00 | $0.00 | $0.00 |
| 21 Sep | $0.00 | $0.00 | $0.00 |
| 22 Sep | $0.00 | $0.00 | $0.00 |
| 23 Sep | $0.00 | $0.00 | $0.00 |
| 24 Sep | $15.50 | $0.00 | $15.50 |
| 25 Sep | $22.41 | $0.00 | $22.41 |
| 26 Sep | $0.00 | $0.00 | $0.00 |
| 27 Sep | $0.00 | $0.00 | $0.00 |
| 28 Sep | $0.00 | $0.00 | $0.00 |
| 29 Sep | $0.00 | $0.00 | $0.00 |
| 30 Sep | $0.00 | $0.00 | $0.00 |
| 1 Oct | $14.32 | $0.00 | $14.32 |
| 2 Oct | $0.00 | $0.00 | $0.00 |
| 3 Oct | $0.00 | $0.00 | $0.00 |
| 4 Oct | $166.82 | $0.00 | $166.82 |
| 5 Oct | $121.17 | $30.50 | $151.67 |
| 6 Oct | $0.00 | $0.00 | $0.00 |
| 7 Oct | $0.00 | $0.00 | $0.00 |
| 8 Oct | $14.38 | $35.70 | $50.08 |
| 9 Oct | $109.53 | $0.00 | $109.53 |
| 10 Oct | $25.23 | $57.48 | $82.71 |
| 11 Oct | $49.03 | $70.41 | $119.44 |
| 12 Oct | $61.92 | $137.98 | $199.90 |
| 13 Oct | $0.00 | $0.00 | $0.00 |
| 14 Oct | $0.00 | $0.00 | $0.00 |
| 15 Oct | $56.92 | $210.11 | $267.03 |
| 16 Oct | $0.00 | $0.00 | $0.00 |
| 17 Oct | $0.00 | $0.00 | $0.00 |
| 18 Oct | $0.00 | $0.00 | $0.00 |
| 19 Oct | $42.86 | $0.00 | $42.86 |
Most tokens are cache reads; most of the bill is not
Show the numbers
| Token type | Tokens | Share of tokens | Cost | Share of cost |
|---|---|---|---|---|
| Cache reads | 1,996.3 m | 95.8% | $417 | 31.1% |
| Cache writes | 73.8 m | 3.5% | $630 | 47.0% |
| Output | 13.4 m | 0.6% | $293 | 21.8% |
| Input | 0.0 m | 0.0% | $0 | 0.0% |
Cheap tokens in bulk and dear tokens in small numbers cost about the same
Show the numbers
| Token type | Tokens | Cost | Per million tokens |
|---|---|---|---|
| Cache reads | 1,996.3 m | $417 | $0.21 |
| Cache writes | 73.8 m | $630 | $8.54 |
| Output | 13.4 m | $293 | $21.92 |
| Input | 0.0 m | $0 | $5.69 |
The first request of a session costs several times any later one
Show the numbers
| Request | Cache writes | Cache reads | Output | Input | Total | Sessions |
|---|---|---|---|---|---|---|
| 1 | $0.0855 | $0.0020 | $0.0003 | $0.0000 | $0.0878 | 312 |
| 2 | $0.0339 | $0.0042 | $0.0003 | $0.0000 | $0.0384 | 312 |
| 3 | $0.0236 | $0.0050 | $0.0002 | $0.0000 | $0.0288 | 312 |
| 4 | $0.0452 | $0.0056 | $0.0002 | $0.0000 | $0.0510 | 312 |
| 5 | $0.0185 | $0.0068 | $0.0008 | $0.0000 | $0.0261 | 309 |
| 6 | $0.0171 | $0.0075 | $0.0007 | $0.0000 | $0.0253 | 250 |
| 7 | $0.0163 | $0.0079 | $0.0005 | $0.0000 | $0.0248 | 223 |
| 8 | $0.0150 | $0.0085 | $0.0008 | $0.0000 | $0.0243 | 200 |
| 9 | $0.0122 | $0.0092 | $0.0009 | $0.0000 | $0.0224 | 175 |
| 10 | $0.0112 | $0.0101 | $0.0009 | $0.0000 | $0.0221 | 143 |
| 11 | $0.0104 | $0.0108 | $0.0008 | $0.0000 | $0.0220 | 123 |
| 12 | $0.0099 | $0.0114 | $0.0007 | $0.0000 | $0.0220 | 104 |
| 13 | $0.0087 | $0.0120 | $0.0006 | $0.0000 | $0.0212 | 86 |
| 14 | $0.0086 | $0.0130 | $0.0004 | $0.0000 | $0.0220 | 70 |
| 15 | $0.0104 | $0.0137 | $0.0005 | $0.0000 | $0.0246 | 59 |
| 16 | $0.0103 | $0.0139 | $0.0003 | $0.0000 | $0.0246 | 53 |
| 17 | $0.0104 | $0.0148 | $0.0002 | $0.0000 | $0.0254 | 43 |
| 18 | $0.0066 | $0.0152 | $0.0011 | $0.0000 | $0.0229 | 35 |
Research
| Item | Count |
|---|---|
| Research sessions, one per research document | 34 |
| Words in the research documents | 192,785 |
| Web searches and page fetches by those sessions | about 1,600 and 900 |
| Sources cited, deduplicated | 1,040 |
| Public repositories reviewed from source, each with a written verdict | 46 (36 in full, 10 in part) |
Benchmark
| Item | Count |
|---|---|
| Configurations specified for testing | 107 |
| Benchmark runs of Claude Code | 1,844 |
| Tokens used by those runs | 411,692,001 |
| Agent-hours of those runs | 49.4 |
| Tasks in the final harm-check bank | 69 |
Most tokens are cached context that the model re-reads on every turn; they are cheap per token but dominate the count. The research sessions searched and read the web; the Claude Code sessions reviewed the repositories, wrote the setup and the benchmark, and analysed the results; the benchmark runs are Claude Code building the test tasks.