Holy mackerel: You blink one minute, GCP is haemorrhaging cash and about to join “killed by Google”; you blink the next and it’s a blooming $70 billion run-rate proper hyperscaler; hot on AWS’s heels and possibly about to be responsible for putting ChatGPT out of its misery too...
Google Cloud’s parent Alphabet is on an absolute tear: It just reported a record $403 billion in annual revenues; 75% of GCP’s customers are using AI and some 350 customers each processed more than 100 billion tokens in December alone. (Boy does that burn committed spend.)
It has also driven extraordinary AI workload efficiencies....
Here’s what keeps Sundar Pichai up at night however...
The StackEdward Targett
A few hundred companies with money to burn are merrily burning it on inference, as Alphabet’s earnings show. Not everyone is so cavalier or optimistic that all this alchemy will turn the dross of their data into gold.
We spoke to the SVP of a large-but-frugal financial services firm this week who must, on this one, go unnamed. A segment worth capturing:
“We do a little POC [of an AI application], and we show that it’s 50% more expensive than doing it the way we did it. We started with over 400 AI use cases, and we pruned it down to three. [Then you’ve got people saying] ‘ok but that Neocloud gives you 50% cheaper on the GPU!’ Yeah, and you just added 10x the latency, plus the egress fees. Yeah. That’s not a win…”
This mystery SVP told The Stack their focus is on “how can we do excellent refactoring of these models to run in whatever hardware is available” and steer engineers away from “just going for the easy button of the bigger model, which is very expensive on hard hardware to [get].”
That’s not the focus of the story below, but it is interesting context to it from the infrastructure side. nb: We’ll have more on running AI locally on commodity hardware, both on the infrastructure and toolchain front soon. Want to feed into that discussion? Get in touch.
The StackThe Stack
Right, let’s get stuck into some newsy highlights.
A brief call to action first!
The Washington Post this week fired 300 reporters including amazing technology reporters. The New Stack (not us) also laid off scores of reporters. We are bucking the trend. We'd like to commission them and keep hiring. Subscriptions help. It's £25/m or £250/y and unlocks a lot of stories and interviews.
When containers first came along, it wasn't always clear if that was a good enough isolation boundary, especially to people used to VMs; Kata Containers promise stronger isolation with the performance and scalability of containers. The coming V4 release is poised for trusted AI at scale and sovereign cloud. Mary Branscombe took a deep dive.
The StackMary Branscombe
Phillip de Wet has a disturbing take (with some expert insight) into how Iran cut off its Internet (with some BGP control and a grip on a tiny number of ASes that serve it) as it conducted one of the most grotesquely overlooked massacres in recent history.
The StackPhillip de Wet
NATO has disbanded its CIO office and let go of its CIO…
The StackEdward Targett
The UK’s Ministry of Defence is now hiring for a CIO at a healthy £270k-£300k after influential and longstanding technology chief Charles Forte moved on.
The StackThe Stack
Noah Bovenizer looks at the backdooring of popular Notepad++
The StackNoah Bovenizer
And unlike Birmingham Council’s car crash attempt to shift to Oracle Fusion, ExxonMobil’s CEO is rather happy with this SAP migration…
The StackNoah Bovenizer
All of that and a host more to read and explore on the homepage.
Coming up soon: A deep-dive into LiveKit, apropos the news that France is basing its sovereign video conferencing software on its OSS stack; a fresh look at Cilium, post-Cisco buyout of Isovalent; that feature on running LLMs and SLMs on-premises and a lot more. Make sure you're subscribed!
Want to learn how you can work with The Stack commercially? We hit an extraordinarily influential audience of major technology budget holders, across a mix of verticals globally. Get in touch here.
Thanks for reading, as ever. Ed