Updated at 2:26 p.m. on October 15, 2025 to remove an incorrect reference by Chris Wyosopal to kernel structs and verifier hooks in the presentation example code.

In August, a relatively unknown security researcher named Agostino “Van1sh” Panico gave a talk at hacking conference Defcon. The 45-slide deck presenting several vulnerabilities in Linux kernel technology extended Berkeley Packet Filter (eBPF) was initially well-received. But open source security experts soon found gaps in the analysis.

At the end of September, news emerged in a thread on Openwall's oss-security mailing list exposing Van1sh for using AI in his work. Crucially, it found that “none of the problems” in eBPF Van1sh had reported to them, and later showcased in the Defcon talk, amounted to real security issues.

“I don't know if this was intended as trolling, or [if it was] just an attempt to deliver something in time for the talks when lacking actual results, even if this meant making things up,” wrote Alexander Peslyak – also known as Solar Designer – founder of Openwall, who contributes to Linux security through projects such as Linux Kernel Runtime Guard.

Upstream project maintainers getting hit with AI-generated vulnerability reports is not new, but this incident has garnered the attention of the security industry.

How did this latest AI slop report end up at Defcon, how big is the problem, and what is being done to stop them getting through?

Sane Slides

The slides as part of the talk, "Jailbreaking the Hivemind: Finding and Exploiting Kernel Vulnerabilities in the eBPF Subsystem" look legitimate at first glance, but this doesn't mean the research they refer to actually existed, Peslyak notes. “The combination of a little knowledge and AI is a more dangerous thing,” he wrote on the thread. 

Peslyak describes how the Linux kernel introduced eBPF a few years ago. Initially, the new functionality was partially available to unprivileged users, so its potential vulnerabilities were also exposed for attack by people trying to gain root. 

This changed in around 2022 when a means to prevent unprivileged users gaining access –  kernel.unprivileged_bpf_disabled – was set by default in mainline kernels and by major distros. Now, eBPF programmes can’t be loaded unless the user already has high privileges on the system.

“On modern systems with default settings, none of these issues are supposed to be exposed for attack at all,” Peslyak tells The Stack.

Aside from this, he highlights a number of red flags indicating the use of AI. Firstly, reporting 11 eBPF issues at once, as Van1sh did, is unusual. 

“Van1sh's own slides list only three past CVEs,” Peslyak, says.  

“It would in fact take extraordinary research to find 11 at once,” he tells The Stack. “Extraordinary claims should require extraordinary evidence.”

Join peers following The Stack on LinkedIn

Another red flag indicating the use of AI or LLMs was the fact the individual CVSS scores didn't fit in the High vs. Critical ranges specified – and the Critical threshold was different from CVSS definition.  This is consistent with “how LLMs may fail to count while producing otherwise plausible content”.

Other experts concur with Peslyak’s analysis. Chris Wysopal, chief security evangelist at security outfit Veracode said "Hallucinated findings often look polished - complete with CVSS tables and citations – but they collapse under replication."

"Requiring runnable artifacts and independent repro steps is the reliable way to keep this noise out of security research," he added.

On the Main Stage


While it’s curious how AI slop could gain a main stage talk at Defcon, it’s difficult to weed out this content when a speaker applies. At the initial stages with no slides created, nothing was so obviously wrong that anyone could confidently reject the talk. 

“It is perfectly normal that not everything is ready for review and the reviewers don't go this deep just to decide on accepting or rejecting a talk proposal,” an anonymous industry source tells The Stack. “Maybe this will change now, with for example, organisers being more sceptical of submissions by people without much reputation.”

But the talk was always going to be found out, says Dane Sherrets, staff innovations architect at HackerOne. “Defcon is one of the most prestigious stages in infosec, so it should be assumed that any code is going to be reviewed afterwards. Presenting non-working AI slop code at Defcon is like presenting a cure for the common cold at a medical conference and thinking that nobody is going to examine the study.”

“Defcon is in touch with the speaker who is committed to updating and releasing code consistent with the talk as presented as timely as possible,” a source familiar with the matter at Defcon told The Stack.

AI Slop in Research 

AI slop in security research is a growing issue that is on researchers’ radars. However, not all AI uses are bad. 

AI has promise for helping security teams find and remediate vulnerabilities quickly. Indeed, 67% of security researchers now use AI or automation tools to speed reconnaissance and cut repetition, says Sherrets. 

And some even think the negative impact of AI will be small. The issue of AI slop isn’t a big deal yet, Peslyak tells The Stack. To his knowledge, around two or three time wasting reports out of the 59 sent to Openwall's distros list this year so far were AI/LLM-generated.

“AI is becoming a major disruption to established vulnerability handling processes, but it is too early to tell what overall effect it will have. I’m hoping it will eventually be net positive.”

See also: 22,600+ emails = 599 vulnerabilities. Security disclosure triage is HARD

Yet others are concerned the AI slop problem is rapidly getting worse. Daniel Stenberg, founder and lead developer of Curl and libcurl is seeing AI slop in around 20% of submissions to the Curl bug bounty programme. 

He describes a collection of vulnerability reports submitted to Curl that he says were AI assisted. “All of them bad, some of them *very* bad.”The risk for conferences such as DefCon is that more slop is accidentally accepted as the AI output quality increases, he warns.The slop surge also forces everyone to raise their guards and increase the work in making sure submitted proposals are "real", Stenberg points out. “It forces everyone to spend more time and energy to vet future proposals and speakers.”

Daniel Card, an independent security consultant, agrees. “The AI slop effect is real and consumes valuable time and resources.”

Stopping AI Slop Reports

To remedy the issue, many bug bounty programmes are combining automated validation with human oversight to filter out AI slop, says Sherrets. “Modern triage systems now review historical data, compare new disclosures against past patterns, and flag likely duplicates or hallucinations before they reach security teams. The most advanced setups even guide researchers at the point of submission, rejecting low-quality or irrelevant findings early on.”

What led to the AI slop talk at DefCon is still not entirely clear. Was Van1sh motivated by fame and kudos, was he in a rush, or was he really a troll orchestrating a publicity stunt to make a statement? While it appears he has been in contact with Defcon since the incident, The Stack has attempted to contact Van1sh numerous times through multiple channels and has received no response. 

Yet even without this knowledge, there are still lessons to be learned. In conclusion, Card emphasises the need to recognise the value LLMs bring, but to be aware of the risks as the level of AI slop inevitably grows.

“The level of AI slop, from what I see, is insane. Like any tools, they need to be used well. Mistakes happen, but if people are out to deceive, these tools are creating a nightmare.”

The link has been copied!