I was looking back at CVEs that had already been published, some of them remediated and fixed. A fix isn’t always sufficient, so I was using Atomic, our autonomous penetration testing capability, to see if it could re-exploit them.
When I set up the software to test each CVE first, I noticed I wasn’t able to exploit some of them. Atomic was saying the same thing: there’s no vulnerability here.
That led me to check deeper into the CVEs themselves. They are approved CVEs. They’re just not exploitable under the conditions we tested.
What a failed reproduction does and doesn’t tell us
I should be careful with that last part. Not being able to reproduce something isn’t proof it’s false. The test environment could be missing something, or the trigger could be wrong. A disputed finding isn’t automatically a bad one either, and a rejected CVE isn’t automatically a fabricated one.
What it does tell you is that the claim hasn’t been established. When I test a CVE, the result is one of three things: confirmed, not reproduced under the tested conditions, or contradicted by the evidence. These ones fall into the second group.
I’m not the only one seeing this. In July 2026, six CVEs were published against SQLite, and two of them scored 9.8. JFrog’s researchers found that some of the functions the reports referenced didn’t exist in the affected versions, and the proofs of concept didn’t reproduce. MITRE rejected the records at the end of July.
Why this is happening
A lot of it isn’t coming from a malicious point of view. I think it’s really an issue of triage.
Before, a lot of weak reports were blocked, because if you didn’t include a screenshot, that wasn’t enough proof. Now AI tools can use a browser, take screenshots and replicate that proof. The evidence looks very believable. Some people are using AI to find these issues and not validating them themselves. They’re just going with what the AI said, and the report gets past the guards that are meant to stop it.
The people reviewing reports are overworked, too, because everyone’s using AI to find these. GitHub reported that private vulnerability reports on its platform went from about 550 a week in January 2026 to more than 3,000 a week for most of May. It received almost 4,000 CVE requests in May, nearly ten times the year before. GitHub said the quality of those requests held up, so more volume doesn’t mean more bad reports. It does mean less time for each one.
The process also leans toward assignment. CNA Operational Rules don’t require a working public proof of concept. When a CVE Numbering Authority (CNA) can’t make a clear determination after a reasonable, good-faith effort, the rules direct it to assign. There are good reasons for that. But it means a CVE ID tells you a claim is being tracked, not that someone has reproduced it.
Who carries the cost
Maintainers get it first. Daniel Stenberg, who maintains curl, one of the most used tools on the internet, has written about being swamped with AI-generated reports. Every one of them has to be verified manually. The curl project ended its paid bug-bounty program in January 2026.
Inside a company, it depends on size. At a small company, there’s usually no tier one support. It’s a developer handling reports as an additional task, for the same pay, and it bleeds into their actual job. At a mid-size company, tier one support takes them on. But they’re often covering tier two and three issues as well, so they’re stuck choosing. Either way, something gets missed. It affects the company from the bottom up.
Then there’s fatigue. If you’re getting ten of these a day, or even five, you get backlogged with the ones that aren’t real, and a real one sits there that nobody has looked at. Vendors can make mistakes too. A tired reviewer can say something isn’t vulnerable when it is, or the opposite. You end up with false negatives and false positives.
What I check before acting on a CVE
A severity score tells you how serious a vulnerability would be. It doesn’t tell you whether it’s there in your environment. When a CVE comes in, these are the questions I work through:
- Does the vulnerable function exist in the version you run?
- Can external input reach that code in your deployment?
- Are the attack prerequisites realistic for your environment?
- Can a controlled test trigger the claimed behavior, and does it show the claimed impact?
- What is the record’s current status across sources: rejected, withdrawn or disputed?
No single answer settles it. But working through them tells you whether you need to act now or take a closer look at the report.
Where Atomic is useful
This is where Atomic has been useful for me. It’s built on agentic AI, so it plans, adapts and pursues attack hypotheses as evidence emerges. Every action it initiates passes through the Edgescan Harness first. The Harness is architecturally separate from the AI model, and it enforces defined scope, permissions, approvals and policies, with non-destructive testing by design. Findings are validated with evidence before they reach an analyst, and our security experts step in when a case needs deeper human assessment. I still make the call on what the evidence means, but I’m not relying on the advisory text alone.
If we’re going to use AI to find more potential vulnerabilities, we should be using it to check the claims that come in, too.
For more information, read the research paper, AI Made Vulnerability Reports Cheap. Verification Isn’t.
