Inference budgets are overrunning by "orders of magnitude" - what now?
Goldman Sachs: "Industry is countering by redistributing value [via] open source distilled models and the focus on roadmaps weighted to proprietary SLMs..."
Many companies are overrunning their initial AI inference budgets “by orders of magnitude,” according to equity analysts at Goldman Sachs.
That will be little surprise to many CTOs and engineering leaders, but the scale at which costs are running away from many organisations is significant.
Inference costs were approaching 10% of engineering headcount cost in one software company the investment bank surveyed – and the firm said it expects them to reach parity with total engineering headcount cost soon.
That’s according to a 42-page research paper for clients, seen by The Stack.
Goldman’s team met with 30 private companies and 10 public companies – including Autodesk, CrowdStrike, Rubrik, Workday, and Zscaler, as well as leading VCs to discuss the trends they are seeing around inference use – and the extent to which software firms have a “moat” against AI dominance.
“The industry is countering”
This post is for paying subscribers only
Join peers managing over $100 billion in annual IT spend and subscribe to unlock full access to The Stack’s analysis and events.
