GTM-Bench
An open benchmark of how often AI models get facts about real companies wrong.
Pilot, October 2026: seven models, five of them Claude, no OpenAI or Google. Code not public.
3–30% of facts wrong from memory, best model to worst
Wrong when it answered, company facts from memory
| Label | Value |
|---|---|
| Opus 5.5 | 3% |
| Sonnet 5.5 | 5% |
| Sonnet 5 | 8% |
| Haiku 5.5 | 8% |
| Nemotron 3 Super | 14% |
| Haiku 4.5 | 22% |
| Gemma 4 31B | 30% |
- Where
- Personal project
- Role
- Design and build
- Built with
- Python, SEC EDGAR, Anthropic and Ollama Cloud APIs
What I made
Sales AI tools claim accuracy and rarely measure it in the open. Every answer here is scored against public SEC filings, so anyone can check a single result.
Seven models took four tests on the same frozen data: six facts per company from memory across 240 public companies, fit against a written customer rule, buying signals in 106 8-K filings, and a cold-email first line.
240public companies, 80 per size tier
4tests, on the same frozen data
What came out of it
Remembering isn't solved
From memory, Opus 5.5 was wrong on 3% of what it answered and Gemma 4 31B on 30%.
Reading nearly is
Given a filing, all seven scored 84% to 95% on tagging buying signals.
Small companies are where it breaks
Gemma 4 31B went from 19% wrong on the largest companies to 39% on the smallest, while Opus 5.5 held at 3%.
Thin packs bring out invented detail
From a thin pack, Haiku 4.5 invented detail in 94% of first lines and Opus 5.5 in 30%. Judges agreed 79% of the time, so these are approximate.
Where it's going
The full version covers every major vendor, adds each new model on the same frozen data, and keeps a public history anyone can rerun.
- NowSeven-model pilotFive Claude models, two open-weight.
- Every major vendorGPT and Gemini on the same data.
- Public codeRaw answers and the rerun command.
- Release-day rerunsEach new model adds one row.