The use of AI to write code is “widespread but highly uneven”, an international team of academics have found, with US developers the keenest to trust LLMs to help them up their work rate.
Newer contributors were more likely to use AI, the researchers found, while the use of AI resulted in a boost in the number of commits.
However, other research this week suggests that a substantial chunk of the code being generated by AI is being put into production without oversight, presenting vulnerability and security concerns, and this proportion is only likely to increase.
The study, by Simone Daniotti, Johannes Wachs, Xiangnan Feng, and Frank Neffke of the Vienna-based Complexity Science Hub, first reported by The Register, used a “neural classifier to spot AI-generated Python functions in 80 million GitHub commits (2018-2024) by 200,000 developers and track how fast--and where--these tools take hold”.
It found that by December last year, AI accounted for 30.1% of Python functions from US contributors, outstripping Germany’s 24.3%, France’s 23.2%, and India’s 21.6%.
The researchers added that they saw “noticeable growth spikes in AI-generated code soon after key generative AI releases such as GitHub Copilot, the original ChatGPT, and GPT-4.0, highlighting how breakthroughs in LLMs prompt rapid uptake by developers worldwide.”
They also said other studies suggested higher AI adoption by less experienced programmers. “This would, other things equal, compress productivity inequality between junior and senior workers. Unlike most previous work, we find no significant differences between men and women.”
Daniotti and his co-researchers estimated that this amounted to an “annual value of AI-assisted coding in the US at $9.6 to $14.4 billion, rising to $64 to $96 billion if we assume higher estimates of productivity effects reported by randomized control trials.”
How much?
Putting a definitive value on the size of the US software sector is tricky. But CompTIA in its most recent outlook cites a Gartner estimate of $1 trillion for spending on software worldwide. Meanwhile, The US International Trade Association estimates computer systems and design related services added $489.2 billion value to the US economy in 2023.
So, even if we take the lower value reached by Daniotti and co, LLMs are chewing through a substantial chunk of value, that would traditionally have been huamn generated.
But there are swings and roundabouts. Ideally, someone should be reviewing that code. Research by Cloudsmith released this week suggests this is definitely not the case.
Its Artifact Management Report suggested that half of codebases are now AI generated. But, it also found that two thirds of developers “review AI generated code before every deployment”. Which means that a third of AI code is not being reviewed before heading to production – even though 40% of developers believe that code generation presents the “greatest risk” from AI generated input.
Moreover, even that level of oversight is unlikely to be sustainable, given AI’s remorseless growth, and pressures on development teams.
The link has been copied!
