From Diff to Primitive: An LLM Auditor that Found 8 WebKit CVEs

[ Abstract ]

Zenshin, an LLM auditor that only hands over reproduced WebKit bugs: call-graph reachability, cross-model debate, and an ASAN binary as the final judge. Eight WebKit CVEs so far, plus the funnel numbers and where it still falls flat.

Point an LLM at a few million lines of WebKit and it will happily hand you a wall of plausible-sounding bugs, almost all false. This talk is about the machine I built (Zenshin) that instead hands me *reproduced* ones — validated by an actual call graph, argued down by a cross-model debate, and only accepted once an AddressSanitizer binary agrees. It has found 8 Safari CVEs so far, with 2 more awaiting fixes. I'll walk through the full pipeline, the design decisions that turned "plausible" into "proven," real bugs from diff to exploitable primitive, the numbers (candidate → crash → primitive funnel, cost, false-positive rate), and — just as important — where it falls flat and why turning a crash into impact is still a human job. If you're trying to get *real* memory-safety bugs out of an LLM on a large C/C++ target, this is what worked and what didn't.