Cloudflare Cut Its AI Code Reviewer's Bill 68% by Hunting Cache Writes, Not Buying Bigger Models

Cloudflare published an engineering post on October 19 that every operator of a large agent fleet should read twice. Its AI code reviewer — a coordinator handing merge requests to specialist code-quality, security, performance, documentation and standards reviewers — saw the average cost per review creep from $1.19 in April to about $2.46 by the end of August. Nothing architectural had changed. The tokens just drifted upward, a few percent at a time, until the bill had doubled. By mid-October, after a deliberate optimization campaign run largely by its own coding agents, the same review costs $0.77 — down 68% from the $2.43 the reviewer averaged over its two most expensive weeks, with the end-of-August figure at about $2.46 — and the median review finishes in 1m 32s instead of 4m 09s (Cloudflare Blog).

The story matters because the cost curve Cloudflare documents is not a Cloudflare problem. It is the default behavior of any agentic pipeline that calls a frontier model in a loop, and the right countermeasure is the same one we recommend in Cost per Completed Task Beats $/MTok: price the finished job, not the token.

The quiet doubling

Cloudflare’s own numbers show the drift was mostly invisible in the moment. Holding the mix of review tiers fixed, cost per review rose steadily from $1.22 in March to $2.32 in August. Part of it was mix shift: more changes landed in the most expensive “full review” tier (59% in March, 65% in August) because better models let engineers produce bigger diffs. But the larger part was model changes — moving every reviewer onto Anthropic models in August alone raised cost per job from $1.38 to $2.03, and putting the internal standards reviewer on Claude Opus 4.7 took it from $0.25 per review to $0.88.

The subtler trap was caching. The cache hit rate improved from 85.7% to 97%, while tokens per review more than tripled from 0.91 million to 2.99 million. A high hit rate on three times the tokens still costs a lot: before optimization, output tokens were only 23% of spend, while cache writes (43%) and cache reads (34%) — context the model had already seen or was about to see again — made up 77% of the bill. A per-review cost metric that pages a human when it drifts, as outlined in AI Agent Cost Alerts: Anomaly Thresholds, Paging, and Auto-Pause, would have caught this months earlier.

What the agents actually found

The single biggest win came from instrumenting the first LLM call of every session. Starting September 14, the reviewer logged token usage plus a hash of the first system block as sent to the provider. The result: the first request hit a cached prefix only 14%–26% of the time, even though every session begins with 11k–21k tokens of identical tool definitions and agent prompt.

The cause was mundane. OpenCode v1 built one system-prompt block — agent prompt, then an environment section with the working directory and date — and placed the only cache breakpoint at the end. Because the shared prefix never ended on shared bytes, a review in one repository could never reuse what a review in another repository had written. Splitting the block at a marker after the shared part, giving that part its own breakpoint, and moving repo-specific data behind it (plus inlining the standards reviewer’s reference index so it stopped burning three turns on tool-call reads) took cost per review from $2.47 to $2.02 in a day.

A second wave of unglamorous fixes — sub-reviewers could previously see (and in 57%–76% of sessions, tried to call) a tool they weren’t allowed to use, then wasted a call recovering; 64% of code-quality turns made exactly one file-read call; OpenCode handed the result back for a summary turn nobody read; and 358 of 446 sampled cancelled jobs kept running to completion because GitLab’s executor never signaled them — together removed about $0.20 per review. Three reviewers now share one cache prefix by sending byte-identical first requests, with the second and third waiting up to 20 seconds for the first’s cache write to start streaming.

The punchline: when Cloudflare split the savings, code changes accounted for only $0.65 of the $1.67 saved. Model and effort changes accounted for $1.06. Dropping to medium effort, moving to Opus 5.5, and — after a 50/50 split test — moving Sonnet reviewers to Sonnet 5.5 (62% cheaper per run on code quality at equal per-token price, purely from fewer tokens) did more than all the engineering. That is exactly the discipline described in Kill Low-Effort Retry Loops: Price Effort per Completed Task: effort is a cost dial, and only a split test tells you where to set it.

Cheaper, but did it get worse?

The quality section is what makes the post credible. Critical-and-warning findings fell 40%–71% depending on reviewer, so Cloudflare validated two ways: did authors change the flagged lines, and did blinded cross-provider model judges rate sampled findings? Code quality at high effort held its rate of acted-on findings (34% → 59% action rate, roughly flat findings-that-led-to-change per 1,000 reviews) and over-rated severity fell 70%. But medium effort genuinely lost findings — code quality at medium produced about a quarter the acted-on findings of high — and performance lost about 64% of important findings, so both are being retested at high effort. The operator lesson is explicit: a cheaper bill is not automatically fine, and neither is a noisier one.

The same day, Cloudflare also announced Clairic, a preview “System One” extraction model that answers typed fields (text spans, numbers, dates, estimates, forecasts) in a single non-generative pass on Workers AI. It is a different announcement, but the same economics: stop routing structured sub-steps through a generating LLM when a one-pass lane will do.

The checklist for your fleet

Cloudflare’s own closing advice, worth adopting verbatim:

1. Check whether each session’s first request actually reads from the cache. Log the first call’s token usage and a hash of the system block as sent. If your shared prefix doesn’t end on a cache breakpoint, you’re paying cache-write prices on every session.
2. Count calls per task. Every extra call re-sends the whole conversation. Cloudflare’s code-quality reviewer went from 6.2 calls per run to 1.7 on Sonnet 5.5 at medium effort — 3.4 at the high-effort setting Cloudflare actually ships.
3. Strip tools the agent can’t use, and stop the loop when the work product is posted.
4. Split-test models and effort levels rather than assuming; per-token price explained only about $0.20 of Cloudflare’s savings.
5. Measure findings quality per dollar, not just findings — acted-on rate and severity calibration are the metrics that survived.

If your agent costs have been quietly climbing with “no changes,” this post is the debugging manual. Both articles are live on the Cloudflare blog as of October 19, 2026.