Benchmarks
The numbers, and exactly how we got them.
Measured runs, each published with its method and its caveats — plus one clearly-labeled ROI projection built on them. Where a benchmark hasn't been run yet, this page says so instead of showing you a chart.
The bottom line
projectionGanvil is $20/mo and sends Claude ~4× less input. On a $200 Claude Max plan, that becomes one of two things you actually feel:
~2.5×
more output tokens
if you max out a capped plan — the code you actually ship. (~2× at lighter input.)
~$80–100/mo
back in your pocket
if you pay per token — same work, smaller bill (~39% off input-heavy sessions).
Either way, $20 buys back multiples of itself. The case study below shows exactly why — and every input-token figure under it is measured, not asserted.
Why now
Token consumption is going vertical.
Coding agents re-read context every turn, run more turns, and carry longer histories — so the tokens behind a day of work are compounding fast. Prices are falling, but nowhere near fast enough to keep up with the volume.
~7×
industry tokens processed, year over year
5×
Microsoft tokens / quarter, YoY (100T+ per quarter)
24×
projected growth by 2030 (Goldman Sachs)
−80% / +7×
token price fell ~80%, but volume grew ~7× — bills still climb
The escape isn't cheaper tokens — it's fewer tokens per unit of work. That is exactly what's measured below.
Sources: Microsoft & Google earnings (2026); Goldman Sachs AI-agents forecast; YipitData / io-fund token-volume estimates. External market figures, not Ganvil benchmarks.
The case study — how $20/mo becomes 2.5× output or ~$100 back
projectionTake a $200/mo Claude plan. The one assumption that drives everything is how input-heavy your work is — and agentic coding is very input-heavy: you re-send repo context and tool results every turn, while the output (the diff) is small. So input is the bulk of the spend, which is exactly where Ganvil's 4× input cut (≈75% less, measured below) bites hardest.
| Input share of spend | Input $ | After 4× (¼ tokens) |
|---|---|---|
| 50% · conservative | $100 | $25 |
| 65% · typical agentic | $130 | $32 |
| 80% · context-maximal | $160 | $40 |
If your plan is capped (Claude Max) and you max it out
A usage cap meters cost-weighted tokens — Claude charges 5× more for output than input, so a turn costs about input×1 + output×5. Cut input 4× and each turn gets cheaper against the cap, so you fit more turns — and every extra turn is more output:
80% input · turn cost 0.8 + 0.2 = 1 → (0.8/4) + 0.2 = 0.4 → ~2.5× more output tokens
65% input · → ~2× more output tokens
That's 2.5× the code you actually ship for the same plan — output is the expensive, valuable half.
Two ways to book the value
A · same work, smaller bill
Do the same work, just pay less. Typical (65% input): your $200 bill drops to ~$102 of frontier + $20 Ganvil = ~$122/mo. Scales with how input-heavy you are.
~$78/mo saved · ~39% off
B · same bill, more work — extend the plan
the oneKeep spending $200; your input goes 4× further, so you do ~2.5–3× the work. Valued as the frontier capacity you'd otherwise have to buy: $130 input × 4 = $520 + $70 output = ~$590/mo of work for $220 all-in.
~$370/mo surplus · 2.7× return
Range across 50–80% input-heaviness: B extends the $200 plan by ~$280–460/mo of frontier work. On a $100 Claude Max 5× plan the same math is ~$175/mo surplus (~2.5×). It's a projection built on the measured ~75% input cut below — your savings dashboard shows the real per-month number.
Caching caveat: Claude Max caches repeated context for up to an hour (reads ~0.1× cost and don't count toward your rate limit), so where your context stays cached it's already cheap — treat the input-side figures as an upper bound for cache-heavy sessions. The durable savings are on what caching can't hold: cache writes (every edit/turn, and 2× on the 1-hour TTL), misses (branch switch, expiry, cold start), the dynamic search/pack gather, and output (never cached). Ganvil's deterministic output is also more cache-friendly, so it raises your hit rate.
The caching-proof lever · offload the writing
projectionOutput tokens cost Claude 5× input and are never cached — so the most durable win is to stop paying Claude to write. Ganvil's on-device model generates the code; Claude only verifies it — says "correct," or makes minimal edits. It reads the code once as cheap input and emits almost nothing.
It pays off the moment the local model clears ~20% accuracy — below that, Claude re-reads the bad code and rewrites most of it, so it costs more. Above it, savings climb with accuracy:
| Local-model accuracy | Claude's output cost | Output cost cut | $/mo on a $200 plan* |
|---|---|---|---|
| 20% · break-even | 5.0× O | 0% | $0 |
| 50% | 3.5× O | 30% | ~$21 |
| 70% | 2.5× O | 50% | ~$35 |
| 80% | 2.0× O | 60% | ~$42 |
| 90% | 1.5× O | 70% | ~$49 |
| 100% · "correct" | 1.0× O | 80% | ~$56 |
*Output-offload alone, ~$14 of output work per 5× on a typical input-heavy $200 plan. Generation-heavy work saves more (~$36–96/mo at the same accuracies). None of this is touched by caching — output is never cached — and it stacks with the input savings above. Projection; the binding question is whether a small on-device model clears the accuracy bar on your code, so Ganvil routes it only what it can nail and sends the rest to Claude directly.
Measured · 2026-07-27
Token savings vs a careful file+grep baseline
Three real coding tasks, same repo (commit 66ff0c7), three gather styles. Tokens = ceil(chars / 4); hybrid RAG off. The machine was under heavy load at run — latency was degraded, but token counts are load-independent.
Naive dump
Whole listed files + full contents of up to 30 grep-hit files — no paste cap.
Careful typical
Whole hot files + first 4k of up to 12 extra grep hits, 48k-char cap.
Ganvil MCP
Real tools: search + context_pack + outline + get_symbol (48k cap), via stdio.
| Task | Careful tokens | Ganvil MCP tokens | Careful ÷ MCP | % less |
|---|---|---|---|---|
| dedup | 12,000 | 3,305 | 3.6× | 72% |
| handle_msg | 12,000 | 3,476 | 3.5× | 71% |
| claim | 12,000 | 2,333 | 5.1× | 81% |
| Total | 36,000 | 9,114 | 4.0× | 75% |
Averages ~75% less input context vs a careful file+grep gather (per-task 71–81%). Against a naive whole-file dump it's ~11.8× fewer / ~91% less. Live-model generation wasn't re-run in this batch; prior live API runs showed prompt tokens down without a clear gen-time win on short completions.
Corpus: the Ganvil repository itself at commit 455bb84c5c57 (135 files), imported via gan import and served by gan mcp from the imported store. Run 2026-07-08.
A driver speaks MCP over stdio and issues a realistic mix of ten tool calls, recording each response's size. Each call is paired with a counterfactual: the bytes the equivalent read-grep-read workflow ingests, measured from the actual repository. For callers, that means the summed size of every file containing the queried name — which is what "grep, then read the hits" costs. Tokens estimated at 4 characters per token on both sides.
The counterfactuals are deliberately conservative. They exclude wrong-file reads, retries, and re-reads across turns, and they assume the grep workflow finds the right files on the first pass. Real agents don't.
| MCP GANVIL CALL | Response tokens | Counterfactual tokens | Saving |
|---|---|---|---|
| outline | 1,738 | 4,814 — read the file | 2.8× |
| get_symbol (one function body) | 889 | 4,705 — read its file | 5.3× |
| callers (resolve_rev, depth 1) | 141 | 14,820 — grep + read 8 hit files | 105× |
| impact (transitive + tests) | 2,012 | 15,210 — grep + read hit files | 7.6× |
| search (ranked, degraded mode) | 406 | ~5,700 — grep + read candidates | 14× |
| structural_grep (.unwrap() in ganvil-core) | 2,012 | ~22,200 — grep + read to filter false positives | 11× |
| context_pack (task bundle, 2k budget) | 2,012 | 26,420 — read top 6 relevant files | 13× |
| symbol_diff (HEAD vs worktree) | 14 | ~2,000 — read the text diff | 143× |
| blame_symbol | 74 | ~5,000 — read log -p of the file | 68× |
| log_search | 283 | ~1,500 — read full log output | 5.3× |
| Total | 9,581 | 102,410 | ~10.7×* |
*Session total for this counterfactual method only (tool surface vs assumed perfect read-grep-read). Product tagline uses the more conservative ~4× / ≈75% less input context from end-to-end gather benches—not “10× faster code generation.”
EFFICACY
Real-world savings sit above these figures.
Counterfactuals exclude retries and wrong-file reads; deterministic tool output turns repeat queries into prompt-cache hits; and context_pack replaces multiple LLM retries from your coding agent during exploration with a single call.
Headline claim we stand behind: ~4× less input (≈75%).
On end-to-end gather paths (MCP pack vs dump+grep), context tokens drop on the order of ~2–6×, often ~4×. That is about 75% less input context—not a promise that model decode wall-clock is 4× faster.
Per-tool table below is a different (stricter) method.
Each row compares one MCP response to a read-grep-read counterfactual. Some tools look extreme (callers, symbol_diff) because the counterfactual is many whole files. We do not market “10× codegen speedup” from those outliers alone.
What ~75% less input means for you.
Result: you write way more code with the same plan quota, or you reduce API costs for the same throughput—because less of every turn is spent paying to re-read the repo. How much depends on how input-heavy your sessions are (output-heavy jobs save less on total bill).
What we do not claim (yet).
We do not claim full E2E “write code 10× faster” wall-clock. Live short completions often show similar gen time; prefill savings matter more on large prompts. See method + caveats above.