AWS has waived a $250,000 data egress bill for a company abandoning its S3 cloud storage environment in favour of an on-premises Pure Storage flash array – making good on a March 5, 2024 promise for all customers.

David Heinemeier Hansson (widely known as “DHH”), the co-owner and CTO of Basecamp-maker 37signals, said that the move ended a 10-year relationship with AWS and would save a “$1.5m/year S3 hosting bill!”

“The savings will rack up pretty quick from there, once we've amortized the $1.5m outlay on the combined 18 petabytes of capacity we've bought from Pure across our two data centers. After that, the yearly cost will be below $200,000. Much easier to swallow than $1.5m/year!”

AWS had pledged on March 24 that “customers globally, are now entitled to free data transfers out to the internet if they want to move to another IT provider” – although there are some caveats to that commitment. 

As The Stack noted at the time, the offer only applies for customers moving “all of their data off of AWS” and shutting down their entire AWS environment.  Customers wishing to do this need to contact customer support, get approval, get credits to make the exit and do it in 60 days.

There are also exclusions to be aware of. AWS’s own services won’t be available to help you freely: “Data transfer out from specialised data transfer services, such as Amazon CloudFront, AWS Direct Connect, AWS Snow Family, and AWS Global Accelerator, are not included.”

Petabytes off AWS S3 - without its help

DHH’s team handled the transfer building “a Rails app in a few days that does it using Solid Queue jobs and deploys with Kamal” he explained. 

“We're just getting out over the open internet.

“We have 40gbit, so can do the transfer in a few weeks…”

(Kamal is an open-source tool to deploy web applications developed by DHH’s team. It is “Capistrano for Containers, without the need to carefully prepare servers in advance… You can boot a brand new Ubuntu (or whatever) server, add it to the list of servers in Kamal, and it’ll be auto-provisioned with Docker, and run right away. Docker’s layer caching also speeds up deployments with less mucking about on the server.”)

Answering a string of questions from befuddled cloud-native engineers amazed that on-premises deployments are like, totally, actually viable, he noted that “we already had the power, the space, and the network. So the marginal cost for those items were zero. But even if it hadn't been, there's a lot of [headroom in saving $1m+] to host half a rack in two DCs!”

“Pure Storage ships with a S3 compatible API. So it was basically drop-in.”

Ceph cluster + HDDs?

“I wonder—could a finely tuned Ceph cluster with 20TB HDDs + NVMe cache deliver the same at 40% less? True sovereignty and zero lock-in TCO are still underrated” one commentator mused aloud on LinkedIn. 

DHH was not convinced: “We priced it out. 

“Hard drives use a lot more power, so in our installation, the increased power commits we would need tipped the scales towards flash. Besides, HDDs wear out on average after 2 years and 7 months, according to Backblaze's detailed statistics. At 10PB, that's a lot of swapping! But at a different scale… drives can indeed make sense,” he replied. 

DHH had triggered a lot of talk in 2023 with a blog post “we’re getting off the cloud”. Responding to one critic wondering why his company was, two years later, still on AWS, he replied: “We left in 2023 with all our compute/databases/caches, but our storage contract with S3 is on a 4-year commit[ment], which doesn't expire until this summer…”

"Egress" is pay-as-you-go

The hyperscalers have all waived cloud exit fees now, amid pressure from regulators. AWS has rejected the notion that “egress fees” is a fair characterisation, telling UK regulators in 2023 that "the cost for inbound data transfers is… a one-time cost that can be incorporated into [data storage fees]. The same is not true for outbound data; some customers may not plan to transfer their data out, whereas some will transfer it millions of times daily… By charging for outbound data transfer instead of incorporating it into the fees for storage or other services, AWS can ensure that customers only pay for the services they use. 

“This is a key component of AWS’s pay-as-you-go pricing model, which enables customer flexibility and allocates costs based on actual customer use of AWS services," as the $117 billion cloud provider said in early 2023.

Waiving data “egress” solely for AWS account closures lets the cloud provider showcase to regulators that it is not “locking in” clients. 

Whilst cloud providers tout the flexible provisioning, removal of infrastructure wrangling and updating, accessible, standardised, easy development environments and cutting edge SaaS tools a click away, “cloud bill shock” has stung some customers and repatriation stories generated unique interest: Among The Stack’s most-read stories of recent years, for example, were our two pieces on insurer GEICO’s decision to move back on-premises; not least to slash a $300 million annual cloud bill. 

The link has been copied!