Startup's agent hacked McKinsey AI - exposing huge volumes of sensitive data
$20 in tokens and two hours to expose 46 million chat logs, 728,000 private files and proprietary RAG documentation
Security
McKinsey’s internal chatbot “Lilli” exposed over 46 million chat logs, 728,000 private files and proprietary RAG documentation to hackers.
The vulnerability was due to 22 exposed endpoints that didn't require authentication – one of which turned out to have a SQL injection flaw.
The security incident was identified and disclosed by a startup called Codewall, which pointed an offensive AI security agent at the company.
It took just two hours and $20 in tokens to breach the multinational consultancy and gain access to highly sensitive data, Codewall’s founder Paul Price, a former cybersecurity consultant for Schillings told The Stack.
He added that most large organisations are ill-prepared for the improvements in AI agents at offensive security – and how they can identify overlooked attack surface and autonomously work to iterate through the next steps of their campaigns after establishing an initial beachhead.
He added that AI itself had chosen McKinsey as a target – he’d prompted a model to identify a range of high-profile organisations with public cybersecurity disclosure guidelines and a few other parameters to focus on.
"When it found JSON keys reflected verbatim in database error messages, it recognised a SQL injection that standard tools wouldn't flag” Codewall said.
“From there, it ran fifteen blind iterations — each error message revealing a little more about the query shape — until live production data started flowing back,” the company added in a blog published on March 9.
McKinsey AI app hack
Codewall’s agent appears to have dug pretty deeply, exposing what the company said in its blog was “the entire knowledge base feeding the AI, with S3 storage paths and internal file metadata. This is decades of proprietary McKinsey research, frameworks, and methodologies — the firm's intellectual crown jewels — sitting in a database anyone could read.”
It was also able to access a claimed “1.1 million files and 217,000 agent messages flowing through external AI APIs — including 266,000+ OpenAI vector stores, exposing the full pipeline of how documents moved from upload to embedding to retrieval,” Codewall said. “The agent [also] chained the SQL injection with an IDOR vulnerability to read individual employees' search histories, revealing what people were actively working on…”
What is Lilli?
Speaking to The Stack in an early 2025 interview, McKinsey’s then-Chief Technology and Platform Officer, Jacky Wright, described Lilli as helping the company’s consultants improve data collation and ask questions of the firm’s knowledge base and datasets in natural language.
“We’ve processed all of the firm’s knowledge into vector stores by using OpenAI embedding models, with semantic and keyword search as well as LLM (OpenAI GPT-4o)-generated metadata to pull the best chunks of information for a user’s question” she explained at the time.
More specifically, the firm uses Cohere’s re-ranker to “further refine the search results” and then runs the top results through a proprietary filtering/relevancy check process: “Once the highest value chunks are identified we synthesize the final answer and include citations back the underlying source documents” she told The Stack by email last year.
Codewall’s founder: We followed best practice
Founder Paul Price told The Stack: “McKinsey test their systems rigorously and their own tools and third-party pen tests didn't pick this up for 2+ years. We also ran OWASP ZAP [a widely used OSS web application scanner maintained by Checkmarx] against it, which failed to find the issue.”
Questioned on whether he was concerned that this kind of unsolicited agentic penetration test might land him in legal hotwater, Price told us:
“We followed their responsible disclosure policy which (in our view) counts as explicit authorization to test within their guidelines. We followed their policy and standard ethics, and immediately reported the issue to their team along with requesting safe harbor. Their team also proof read and amended our blog post before agreeing on a mutual publication date. They were all very amenable and professional.
A McKinsey spokesperson told The Stack: “McKinsey was recently alerted to a vulnerability related to our internal AI tool, Lilli, by a security researcher. “We promptly confirmed the vulnerability and fixed the issue within hours.
“Our investigation, supported by a leading third-party forensics firm, identified no evidence that client data or client confidential information were accessed by this researcher or any other unauthorized third party.”
They concluded in an emailed comment: “McKinsey’s cybersecurity systems are robust, and we have no higher priority than the protection of client data and information that we have been entrusted with.”