Skip to content

How Glean calculates ROI from AI developer tools

The best practices for using AI coding tools are evolving, and they'll probably vary across different roles on a software team.

How Glean calculates ROI from AI developer tools
Photo by Juanjo Jaramillo / Unsplash

Business leaders have been trying to measure and evaluate the productivity of their expensive software developers for decades, with varying degrees of success along the way. The explosion of AI coding agents over the last year made it even more challenging to properly assess developer output, especially when companies want to maximize productivity while minimizing token spend.

"It is incontrovertible that there are large portions of people that are getting tremendous, like 10x value out of [AI tools] in terms of what they're able to produce," said Tony Gentilcore, co-founder and head of engineering at Glean, in a recent interview with The Stack.

But those gains have not been distributed equally across Glean's development organization (and many others, to be fair), which led Gentilcore on a quest to perfect a new metric for measuring developer productivity. Many of the old principles still apply — shipping code is not the same as shipping product — but the methodology needs to be reinvented for the AI era, he argued.

The new math

This content is for members only

Subscribe
Add The Stack on Google