Google AI Found 45 V8 Bugs to Fuzzer's 37 in 6 Months

Google’s AI bug hunter found slightly more security flaws in Chrome’s JavaScript engine over six months than the automated testing tool Google has relied on for years, but the two mostly found different bugs. That is the main finding of a study published on 27 October by Project Zero, Google’s in-house team of security researchers.

The Project Zero post, written by researcher Samuel Groß, compares BigSleep, an AI agent that reads and reasons about code, with Fuzzilli, a fuzzer that bombards V8, the engine that runs JavaScript in Chrome, with automatically generated test programs. Its conclusion is that companies should not swap one for the other. “We recommend vendors perform both fuzzing and agentic analysis of their code to maximize the number of vulnerabilities that get reported before they impact users,” it says.

In brief

From April to September 2026, BigSleep found 45 actionable vulnerabilities in V8 and the Fuzzilli fuzzer found 37, with only about a quarter found by both, Project Zero says.
In a separate test, the AI agent rediscovered about 62% of 112 older fuzzer-found bugs, but struggled most with the bugs that had been hardest for the fuzzer.
Google says the AI model used is already outdated, and the cost of each approach per bug cannot yet be compared.

Six months of bug reports from both systems

Both tools run as automatic pipelines that check new changes to V8 for bugs. The study only covers April to September 2026, because the automatic BigSleep pipeline only went live in April. In that window, Project Zero counted 441 raw reports from BigSleep and 316 from Fuzzilli in Google’s bug tracker.

After filtering out duplicates, non-security crashes and reports that did not need fixing, BigSleep came out at 45 actionable vulnerabilities and Fuzzilli at 37. “So in terms of vulnerabilities, it’s roughly a tie (small advantage BigSleep) at the moment,” the post says. Of the 81 distinct vulnerabilities, 20 were found by both systems, an overlap of roughly 25%. A slight majority of those shared bugs were found by Fuzzilli first.

Project Zero does not say which tool is cheaper. It writes that “neither system is particularly calibrated for an optimal cost-per-bug ratio”, so the cost per bug cannot fairly be compared yet.

Can the AI find the fuzzer’s old bugs?

Because the live comparison probably understates the overlap, the team also ran an experiment. It took Fuzzilli bugs found between early 2023 and mid-2026, kept the vulnerabilities that could be reproduced on a standard build, and removed duplicates, leaving 112. For each, it pointed BigSleep at the code change that most likely introduced the bug and gave it 25 attempts, using the same model the production pipeline was running at the time, Gemini 3.6 flash.

BigSleep rediscovered about 62% of the original bugs and found a different, since-fixed bug for another 12% or so of the code changes. It found nothing for about 26%. With two extra prompting approaches, the share of correct finds rose to about 70% and some bug was found for about 84% of starting points, though the post cautions that similar tweaks to a fuzzer could also raise its hit rate. Project Zero researcher Natalie Silvanovich summed it up on X: “These experiments estimated overlap as closer to 60%.”

The team checked whether the model had simply memorised old bugs from its training data. It found no statistically significant memorisation for bugs from before 2026, as the model’s knowledge cutoff is March 2026, but warned this could differ for other models.

Why Google is keeping its fuzzers running

The most practical finding is the one that cuts against the hype. If AI agents found the bugs that fuzzers miss after days of searching, companies could move that computing budget elsewhere. Project Zero tested this by measuring how long each bug had taken Fuzzilli to find, and the result “actually points in the other direction”. In its words: “Bugs that are hard to find for a fuzzer are likely also hard to find for an AI agent. As such, those fuzzers should probably be kept running for now.”

The overlap was also significantly smaller for bugs in V8’s JIT compilers, the parts that turn JavaScript into fast machine code, which the post suggests is where Fuzzilli is most specialised. The case studies point the same way. BigSleep used reasoning to find a variant of an old bug that would likely have been too complex for a fuzzer to trigger, while one Fuzzilli bug that no AI run rediscovered came from two separate optimisations interacting, which the post says would be hard to spot by reading just one part of the code. Google’s own Keyword blog said in July that “fuzzing continues to be especially effective at finding bugs that arise from long-range interactions between disparate parts of our codebase”.

This is one team’s study of one codebase with one AI system, not a general verdict. It also has a short shelf life. Project Zero writes that “the model used for our experiment is by now already outdated and outperformed by newer generations”, and the production pipeline’s model has been upgraded several times since April. The 45-to-37 lead could widen quickly, which is why the study’s real value is the method rather than the scoreline.

Part of a wider shift in how bugs are found

BigSleep is already part of Chrome’s daily routine. Google said in July that BigSleep and its CodeMender tool run “every 24 hours across all CLs”, Google’s term for code changes, and that “in May alone, we blocked over 20 vulnerabilities from reaching production”. Project Zero’s X account announced the new comparison on Tuesday evening.

Other AI labs are making similar claims about their own tools. Anthropic said this month that Claude proved 239 open-source entry points crash-free in 123 widely fuzzed open-source libraries. The flood of AI-found bugs is also changing disclosure rules: Bitcoin Core ended year-long waits for bugs LLMs can easily find. On the other side, Epoch AI estimates that open-weight models trail closed ones on cyber skills by six to seven months, which matters because the same skills can help attackers. Outside researchers are still collecting bounties too, as when OpenAI paid Xint $6,000 for a reasoning leak.