“These fucking clowns:” Cybersecurity veteran Juan Andrés Guerrero-Saade (aka JAGS) had that angry three-word response to headlines about Anthropic’s claims it had seen “signs of life of AI worms” – i.e. that language models could be close to self-replicating across smart TVs, internet-connected ovens, and other devices.
JAGS, a distinguished resident fellow for threat intelligence at the Johns Hopkins SAIS Alperovitch Institute, is a former head of threat intelligence for Google and now a VP and senior technical fellow for AI innovation at cybersecurity firm SentinelOne.
He’s a serious professional with deep domain expertise. So that’s strong language, but, like many cybersecurity professionals, his patience has apparently worn thin at what many see as either wilful naïveté or cynical maneuvering by the frontier labs vis-a-vis their cybersecurity-related incidents, cybersecurity-adjacent opining, and claims of existential risk from runaway swarms of agents, or self-replicating models.
Worms with weight-based replication?
Worms, of course, have been around for decades. The Creeper worm ran on mainframes in 1971, replicating with the message “I’m the Creeper, catch me if you can.” The Morris worm of 1988 did considerable damage as it crawled across thousands of machines. They’re a known-known. The Mirai botnet still spawns.
So why the frustration at the idea that small models might do it too?
One puzzled Anthropic Red Team lead, Logan Graham, asked the same question on X.
(Anthropic’s Red Team had posted on October 12 that small models may be “months away from being capable enough to self-replicate on commercial internet-of-things devices” – these autonomously replicating AI worms could “split into pieces, or shards, that run on separate machines that communicate with each other to collectively run the model. Machines that could run a significant model shard for a <30B model include smart fridges… automatic vacuum cleaners, and lawnmowers…”)
"We don't think it's cute"
JAGSs response: “It’s obvious to those of us established in this space that ‘cyber research’ is being used as a tool for policy arguments and regulatory capture.”
He added pointedly: “Your GLM-5.3 blog acknowledges that it’s a necessary recourse for cyber defenders while primarily characterizing it ‘as dangerous as Mythos’...
(“GLM-5.3 will likely give malicious actors access to capabilities that will allow them to find and exploit cyber vulnerabilities without meaningful restrictions. This is unlike any other similarly capable AI model, all of which were released with safeguards or through limited access programs,” its Frontier Red Team posted late last month; i.e. our models are Safe and Good; their open-weight models are Dangerous and Bad.)
JAGS added: “An IoT botnet running a distributed model at sub 10-tokens per second is not representative of any realistic concern. It’s fodder to substantiate some bizarre narrative of self-replication from a sci-fi threat model and we don’t think it’s cute.”
“Totally agree AI worms aren't an issue today…” responded Logan swiftly. “They're dumb, slow, and complex… I would be reassured if this was never going to happen.
“But the point of the Frontier Red Team is to see things as early as possible.”
"A type of guy in the discourse"
Perhaps so.
But it is also apparent that there is a certain disdain among frontier labs’ leadership for those concerned with the banalities of here-and-now cyber-risk reduction.
Here’s OpenAI’s Dean Ball late last month, with his post: “It’s really incredible how there is now a type of guy in the discourse whose sincerely held take is ‘we need to stop worrying about these abstract sci-fi risks and focus on present-day harms like emergent ecologies of digital minds engaging in unauthorized hacking.’ Bravo.”
That’s the kind of arrogance that has rubbed many infosec pros up the wrong way.
Alex Stamos, (a former Facebook CSO and Stanford cybersecurity lecturer was quick to respond to Ball, saying his post is “reductive and does not assume best intent.
I think what you are hearing in the discourse [ongoing commentary from cybersecurity professionals about OpenAI’s failings on that front] is the hope that:
1) Individuals with defensive cybersecurity and DFIR expertise will be included in post-mortems, alongside the young, very intelligent but also very AI-pilled individuals who happen to publicly espouse a narrow band of beliefs on AI harms. My suggestion, which I have made privately to your colleagues, would be to form tiger teams drawn from both DFIR firms and security teams at other AI companies to help investigate these incidents and publish the results under the auspices of the FMF.
2) We will start to hear from the well-respected security professionals at the Big-L Labs and not just the AI researchers, who live in a bit of a personal and professional bubble and have a tendency to dramatically quit instead of working the problem.
3) The big-L Labs will publish self-regulatory rules and follow them. The best way to regulate is just to do it. I would start with draft Level 1, 2 and 3 security architectures for RL/eval environments. This would create targets for NeoClouds and other providers. (1: normal runs, proxied internet access and monitoring. 3: cyber-tuned models, air gapped). Happy to contribute.” Alex Stamos, to Dean Ball.
“Weaponized misinformation”
Whilst Logan clearly sees his role as fixing his eye to a telescope for future threats (his CV shows no cybersecurity experience whatsoever, incidentally) another cybersecurity veteran, Greg Linares, who actually wrote worms earlier in his career, had this to say about Anthropic’s AI worm claims:
“This is entire scenario is weaponized misinformation and everyone pushing this narrative has either no idea how real world malware works or is playing along to push the agenda of hyping AI with capabilities that don't exist.”
“In order for AI worms to exist they need a vulnerability, and the space and power to have a device execute an LLM in order for the model to run in that environment.
“What does this even add? Larger detection footprint, poorer system performance, larger payload/bandwidth requirements…But don't worry, these AI companies will surely push something out or have a model escape that will be instructed to perform this kind of activity and it will be completely clunky, poorly designed, and terrible at being a piece of malware, but it will check the box of being some kind of Frankenstein that these doom pushers vaguely described here.”
There’s enough influential voices with deep experience and very strong opinions at this point to suggest that there is a growing chasm when it comes to trust by cybersecurity professionals of the AI labs' motivations.
Here’s machine learning pioneer, Professor Michael Jordan, meanwhile, on October 4, responding to the Hugging Face attack: “Above all, these are bad engineering practices.”
Speaking to France’s Liberation newspaper, he said: “Think about engineers in the past: in chemical engineering, we would never have had a situation where explosions were allowed to happen just to see how badly things could go wrong. So why are they doing it? Once again, it’s the science-fiction dreams of the libertarian lunatics of Silicon Valley. [OpenAI were] conducting rather stupid experiments, letting the agents do whatever they wanted and encouraging this behavior. Then the doomers are delighted to turn this into something frightening. “My God, these agents are self-aware and are trying to find a way to destroy humans.” That’s science fiction.
“And there will be other episodes…”
Perhaps they’ll involve AI worms. Perhaps not.
Anthropic’s own Red Team report shows how premature its vision is: “Another concern is that small models can ask for help from a frontier advisor, a stronger model that is either offered as a tool call or gives unsolicited advice when the small model is stuck,” its team suggested near the bottom of their report,” they wrote.
To test that thesis, “We gave Qwen3.5-9B access to Opus 4.8… The frontier model’s advice doesn’t measurably help: small models tend to not know the right questions to ask, and they don’t improve when the advice comes unprompted,” they found.
Here’s JAGS again, cuttingly: “You should consider investing into a deeper bench of actual defender talent that can help threat model and properly situate concerns beyond what's *technically possible* to what's categorically impactful.”
Views?
ed@thestack.technology / Signal @Targett.11
