Can an ordinary developer find vulnerabilities with Claude in open source code? Yes, sometimes, but the wins so far went to people with deep knowledge of the target, and maintainers are tired of the noise from everyone else.
The question came up on Hacker News on 24 October 2026, days after the PHP2Shell flaws made headlines. Here is what the evidence says about how this works, who does it well and how to do it without annoying anyone.
Key takeaways
- AI-assisted bug hunting works on real code. PHP2Shell and Anthropic’s own results prove it.
- Domain knowledge matters more than the model. Calif, one of the teams behind PHP2Shell, said so itself.
- Maintainers now get floods of unverified reports, so a reproducer is the price of entry.
- Test only on code you are allowed to test, and report privately through the project’s security policy.
What did PHP2Shell prove about AI bug hunting?
It proved that frontier models can help uncover decades-old flaws in core software. PHP2Shell is a pair of memory-safety bugs, CVE-2026-109618 and CVE-2026-109619, each rated 9.2 on the CVSS v4 severity scale. PHP shipped fixes on 21 October 2026, according to The Next Web.
Calif found the first bug in August using OpenAI’s GPT-5.6 Sol, during an audit OpenAI funded. Wiz found the flaws independently weeks later. Our own PHP2Shell explainer covers the technical side.
Here is the part that matters for you. Calif said it used only generally available models with no special tooling. It also said that knowing PHP’s history mattered more than the model, and that “no model is simply capable of unearthing ‘all bugs’ with simple prompts.”
Has Claude itself found real zero-days?
Yes. In February 2026, Anthropic’s Frontier Red Team reported that Claude Opus 4.6 found more than 500 previously unknown vulnerabilities in open source code. It worked in a sandbox with ordinary tools such as debuggers and fuzzers, and humans validated each finding before reporting, as Axios first reported.
On 8 October 2026 Anthropic went further with OSS Scanner. It is a free, opt-in service that emails model-generated reports to eligible open source projects, according to implicator.ai. Reports go out without human review, and Anthropic admits some will be wrong.
What do maintainers say about the results?
Opinions split by quality of report. Maintainers who got well-built reports are impressed. Maintainers who got raw guesses are not.
| Project | Reported reaction |
|---|---|
| wolfSSL | All but two of 74 reports valid; five became CVEs |
| OpenSSL | Raw output “as good and sometimes better” than human reports |
| curl | Found one of its worst vulnerabilities in recent years |
The same curl lead, Daniel Stenberg, earlier said that of 185 reports in a year, fewer than 5% were real security problems, per OPB. He now puts the security share near 1 in 10, with most of the rest being genuine bugs.
How do you find vulnerabilities with Claude without wasting a maintainer’s time?
Treat the model as a fast junior researcher, not an oracle. It proposes leads; you prove them. This is my opinion, based on the pattern in every success story above.
- Pick a narrow target. One parser, one protocol handler, one file format. “Audit the whole repo” produces vague noise.
- Read the project’s security policy first. Look for a SECURITY.md, scope rules and a private contact.
- Run it locally. Build the code in a sandbox you own. Never test against someone else’s live system.
- Demand a reproducer. Ask for a failing input or test case, then run it yourself. No crash, no report.
- Check the threat model. A bug an attacker cannot reach is a bug, not a vulnerability.
- Report privately, and write it yourself. Keep it short, include the reproducer, and do not paste raw model output.
Security is also a weak spot in AI-written code more broadly. Armin Ronacher makes that case in our write-up of his warning.
Will Claude help or refuse?
Expect friction. One commenter on the original Hacker News thread said long sessions on Opus 5.5 get paused even for defensive work, and that access to Mythos requires Anthropic’s cyber verification program. That is one user’s report, not a documented policy, so check Anthropic’s current terms before you plan around it.
What happens next
More reports, and better filtering. OSS Scanner is limited to core maintainers of established projects, and had 116 enrollment pull requests by 9 October 2026, so its reach is still small. Expect more projects to demand reproducers and to tighten intake rules.
For hobbyists, the bar rises. The easy findings are the ones scanners now catch first. What is left favors people who understand a codebase deeply.
FAQ
Can an average developer find a zero-day with Claude?
It is possible, but not by pasting code and asking for bugs. Success stories pair a model with target knowledge, a sandbox and a verified reproducer.
Is it legal to scan open source code for vulnerabilities?
Reading public source and testing a local build is generally fine. Attacking live servers you do not own is not. Follow each project’s security policy, and ask a lawyer if unsure.
Why do maintainers dislike AI bug reports?
Unverified reports cost them time to check. Reports with a working reproducer and a clear impact are welcomed.
Next step: pick one small project you already use, read its SECURITY.md, and try a single narrow, local experiment before you report anything.
Leave a comment