Just 5% of AI pilots have delivered value for businesses, an MIT report claimed this month, giving way to a flurry of headlines questioning, "Has the AI bubble burst?", and wiping billions off the stock market as NVIDIA and Palantir shares dropped.

The preliminary report, from MIT's Project NANDA studying "decentralised agentic web infrastructure", titled The GenAI Divide - State of AI in Business 2025, posed serious questions about AI's ability to fundamentally change business operations, while also providing advice on how best to overcome challenges.

Paired with comments from OpenAI's Sam Altman that the industry is in an AI bubble and reports Meta froze AI hiring amid a restructuring, it's no wonder investors were spooked. MIT, keen to limit the blast zone, also quickly hid its report behind a Google form asking "What is your specific interest?"

But with the dust, slightly, settled, what does the study really say, and what does it mean for the path forward on GenAI?

Yes, 95% of AI pilots are struggling, but...

While some media coverage may have been overhyped, the MIT report did find that 95% of organisations have seen no return on GenAI projects, despite lavishing $30-40 billion on enterprise AI investment.

It also found that tools like ChatGPT and Copilot, explored by at least 80% of organisations, had driven individual productivity but not P&L performance, with enterprise-grade systems "quietly rejected".

See also: Walmart lays ground for "AI super-agents" but yet to see sales lift

However, reading past the headline, what should stand out to CIOs second-guessing rollout of their new internal LLM is the claim by researchers that organisations deploying AI carefully and strategically have seen millions in value as a result, creating what it calls the GenAI Divide.

"Organizations that have crossed the GenAI Divide are beginning to see selective workforce impacts in customer support, software engineering, and administrative functions...
"These early results suggest that learning-capable systems, when targeted at specific processes, can deliver real value, even without major organizational restructuring."

Where you focus is key

The study found the majority of companies are pointing their AI tools in the wrong direction, with 50% of GenAI budgets targeted at sales and marketing while back-office automation provides better return on investment (ROI).

Procurement officers keen to demonstrate tangible impact are often missing the point, it warned, by "directing resources toward visible but often less transformative use cases", with just 5% of custom enterprise AI tools reaching production.

As Alan Jacobson, Chief Data and Analytic Officer, Alteryx, commented to The Stack: "It takes significant expertise to design and build LLM workflows that are robust/reliable.

"The rolling of underlying foundational models to new versions, changing business conditions, and the fundamental methodological approach of how LLMs work can make repeatable results elusive without strong expertise.

"There's also the problem of a knowledge gap between the analysts who understand the business and are best positioned to create solutions and the data science experts who understand how to implement LLMs. And there are VERY few of these data scientists around the world. Top LLM experts are commanding eye-watering salaries measured in hundreds of millions of dollars..." 

See also: Not the CIO’s job? Getting your organisation AI agent ready – “the promise and the peril”

There is still room for experimentation though, researchers said their interviews showed "the most effective AI-buying organisations" didn't wait for the perfect use case or exact CEO approval, but found success in "distributed experimentation, vendor partnerships, and clear accountability."

While it may have gained more attention, MIT's research is in line with what the industry has already heard, with the high failure rate of AI projects well documented.

Anthony Villa, CMO at AI management services company inloop.studio, said he has seen MIT's findings in practice, "Right now many leaders are rushing into AI pilots without first doing the difficult work of discovery and understanding which problems are truly worth solving... The companies that succeed are the ones that pause for structured discovery before committing to tech."

See also: The Big Interview: BNY CIO Leigh-Ann Russell

A Stanford study from February 2025 also found AI developers were more successful with AI projects given clear leadership, and focussed on a task with obvious, similar benefits for a group of employees.

After four years of observation, its researchers said successful projects needed to keep a tight scope, and ensure good communication between AI developers and the employees they are building for.

What do AI builders say?

Bill Conner, CEO of AI architecture provider Jitterbit said the report "adds to what we already know" that GenAI is not "a guaranteed growth engine".

He said: "Success with AI is less about bold promises and more about disciplined integration. The winners in this next phase will be those who treat AI not as a one-off pilot, but as a long-term practice of layering intelligence into every corner of the enterprise.”

According to MIT's report, organisations building AI solutions in partnership with vendors were also twice as likely to succeed then those building internally, finding success "when they decentralise implementation authority but retain accountability."

See also: Behind the scenes of Barclays’ technology overhaul

The news will be welcome for the growing list of AI vendors such as custom AI builder Cogna, with its CEO Ben Peters stating "Too often, proof-of-concepts fail because they’re launched without a real problem to solve.

"Add a bias toward building in-house, and the odds get even worse. Problems worth solving are often complex and need reliable systems, which tends to mean blending AI with fixed guardrails using classic software. That's not a skillset typically found in internal teams."

The link has been copied!