Hugging Face hacked: Turned to Chinese LLM for help after US models blocked Blue Team

Hugging Face said its production infrastructure was breached by an “autonomous” AI agent system early last week (w/c Monday July 13).

The platform’s security team were initially stymied in their incident response (IR) by unnamed US LLM frontier model guardrails “which cannot distinguish an incident responder from an attacker," they said.

So Hugging Face’s defenders turned instead to the open-source GLM 5.2 model from China’s Z.ai lab – running it on their own infrastructure to analyse the 17,000+ logs, or footprints, that the attackers left behind.

See also: GLM 5.2 is in high demand

That’s a striking public admission for the New York-headquartered Hugging Face, which lets users collaborate on models, datasets and applications, and which this summer hit the $100 million ARR mark. 

In an incident report, the company recommended that defenders “have a capable model you can run on your own infrastructure [our italics] vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment.”

Hugging Face: Rotate tokens

The unknown attacker “abused two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration) to run code on a processing worker,” (or compute instance) said Hugging Face in a detail-thin July 16 incident report. 

They then “escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend” using what the firm said was a “swarm of short-lived sandboxes, with “self-migrating” C2 staged on public services.

See also: Citrix credits JPMorgan, pushes fixes for six ugly NetScaler bugs

Hugging Face told its customers that: “We recommend rotating any access tokens and reviewing recent activity on your account.” 

It is “still completing our assessment of whether any partner or customer data was affected” it said and will contact customers directly if it finds evidence that they were – but its incident response team has seen no “tampering” with models, datasets, or spaces, and its supply chain (container images and published packages) are “verified clean.”

Guardrails were an IR blocker

The company’s lessons for defenders in their incident writeup on July 16 stood out to both infosec practitioners and tech investors.

“When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails.”

Hugging Face said it then "ran the forensic analysis instead on [China-developed] GLM 5.2, an open-weight model, on our own infrastructure. 

Hugging Face added: “This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment…”

The timing… 

Hugging Face’s incident report was published the same day that Chinese AI startup Moonshot’s Kimi K3 model rocked global markets.

The 2.8 trillion parameter model is the largest open-weight AI model to date. Blind developer testing by Arena (a platform created by researchers at UC Berkeley) for its frontend code evaluation test put Kimi K3 ahead of Anthropic’s Fable 5 and OpenAI’s GPT 5.6 last week.

Chinese frontier models are also notably cheaper than their US counterparts, as data from Artificial Analysis shows below.

Anthropic on June 30 meanwhile re-released its Fable 5 and Mythos 5 models after US export controls on it were lifted. The models now have stronger cybersecurity safeguards to try and block malicious use. 

"One particularly important safety mechanism involves classifiers—smaller automated AI systems that, during an interaction, detect when the model is asked to perform a potentially harmful cybersecurity task...

"We deliberately set the safety classifiers to trigger on a set of requests that we know are likely benign... a request has to look very clearly safe to avoid triggering the classifier," admitted Anthropic on June 30.

Hugging Face did not say which commercial frontier models it had first tried to use for its IR; nor was it clear what model the attackers used.