Can OpenStack really replace VMware in organizations that still need to run their own virtualisation?

The open source private cloud alternative is shaping up as a VMware escape hatch but most enterprises aren't showing off their migrations yet.

Tesco has filed a £100 million lawsuit against Broadcom for breach of contract over software updates for vSphere perpetual licences. The filing is one of the most aggressive responses to the licence changes Broadcom has issued to encourage customers onto VMware Cloud Foundation. 

Some organizations are sucking up the increased Broadcom licence costs, but many are looking for an alternative. Gartner expects over a third of the workloads currently running on VMware to move to other platforms by 2028. 

See also: Michelin swaps VMware for open-source Kubernetes

Broadcom may claim that no longer letting customers buy VMware licences from the hyperscale clouds it used to partner with is actually about  ‘licence portability’.

But not everyone wants an (exorbitantly expensive) private cloud or to jump ship to another vendor. (Front runners include Nutanix with the closest equivalent to vSphere, or a hyperscaler cloud like Azure with Microsoft’s mature Hyper-V hypervisor – and a new bulk migration tool.)

Who to trust next? 

Deciding what replaces your VMware workloads is much about trust in vendors – which the Broadcom’s changes may have quickly eroded – as it is about saving money.

“Pretty much every organization who is currently being confronted with the Broadcom changes is having that revelation of why they need to get off a vendor driven platform,” Kevin Carter, longtime OpenStack contributor and director at Rackspace, an OpenStack-based cloud provider, says.

“The open source nature of OpenStack makes it so that you can't have that rug pull which is powerful.” 

While few organizations new to OpenStack will choose the purely open-source solution (which would require more participation and contribution than most businesses can afford), Carter suggests they can pick one vendor while keeping their options open. 

“If the customer doesn't like us anymore. They can take their ball and go home and that’s not a technology cliff where they have to now figure out how to get off of this platform,” he added. 

Not your grandfather’s OpenStack 

OpenStack has long been viewed as a solution for enterprises with specific needs. The company was born out of NASA’s need for grid computing; telcos now often turn to OpenStack for virtualised network functions. 

Over the last few years, OpenStack has been busy modernising its hardware integrations and support for containers.

“The use cases have kept changing,” explains Goutham Pacha Ravi, chair of the OpenStack Technical Committee  and principal software engineer at Red Hat. “At the very beginning it was ‘give me a VM’ because everything was just needed to be virtualized.” 

For some, bare metal continues to be useful – and containers were a natural evolution. “We started seeing people wanting to be more efficient, wanting to run containers directly on top of virtual machines or bare metal,” Ravi said.

See also: GEICO repatriates from cloud to OSS, incl. OpenStack

OpenStack turned out to be a natural fit for container orchestration systems like Kubernetes “because there were APIs and you could pretty much act on these like you would on the commercial public cloud systems.” 

OpenStack’s ability to provide the same provisioning as VMware for other demands like network integrations, DNS, and storage, meant they were added to OpenStack’s offering.

Ravi told The Stack he’s seen the shift from people wanting to run containers on VMs “in a very hands-on way” to “treating them a lot like cattle.” He says people want containers they can “easily go deploy, tear it down, deploy it again, move it and scale it.”

New vendors, new integrations

As with most open-source projects, especially one as complex as OpenStack, many enterprise customers choose a commercial, supported version. 

There’s an increasing number of providers beyond the familiar Rackspace, Red Hat and Canonical, such as Platform9’s Private Cloud Director, Okestro Contrabass and Mirantis – plus migration tools like Hystax Acura as well as hardware integrated offerings from Huawei and Fujitsu. 

That’s alongside new integrations with enterprise hardware suppliers, Ravi notes. “A lot of different hardware vendors out there want to integrate and expose something, because there's an open source API to consume them.”

Ravi suggested Broadcom’s changes have pushed more vendors to pay attention to OpenStack. “When you have customers spooked, you also have vendors spooked,” he says.

“In comes an open-source community like OpenStack, where no single company can actually control that software.” He says infrastructure that doesn't require subscriptions or paid support may appeal to companies making long-term bets.   

Different but familiar

Based on multiple projects for cloud infrastructure, OpenStack has five core services but also a variety of extensions which can make getting started look challenging – but only because you need to think about what you want to achieve. 

There are options to simplify set up and configuration, like tools that deploy directly onto bare metal, packaged solutions from vendors like Red Hat, and playbooks with familiar infrastructure-as-a-service platforms like Ansible and Puppet to handle deployment.

These choices shouldn’t be unfamiliar to VMware users, Carter suggests. 

“When you're coming into the world of OpenStack, it's like ESX. You've got the giant menu of all the different things that Broadcom offers today and you don't need to have everything. If you don't need containerization management, don't deploy it. If you don't need Ironic for bare metal, don't deploy it.” 

As a long-term OpenStack contributor, he says Keystone (identity), Neutron (networking), Glance (an image library) and Nova (compute provisioning) are the key services needed for a successful OpenStack deployment, adding “that's a pretty simple stack to maintain.”

Start with the basics

Core compute, networking and storage capabilities are similar to VMware, but OpenStack has its own architecture and philosophy. 

“In VMware parlance, you’d be talking about NSX, and in OpenStack we're saying Neutron,” Carter points out. “All the lifecycle management things that are table stakes are absolutely there: live migration, resizing, adding resources to a VM, attach, hot detach and attach of storage networking.”

The project has sample configurations like the starter kit that deploys the five core services: Nova for compute, Glance to manage VM images, Keystone for identity, Neutron for networking, and Placement for resource management. Then you can add your choice of dashboards to work with various service APIs.

“One of the reasons why people are looking at OpenStack with renewed interest is because it offers a mature feature set,” OpenStack chair Ravi adds. The solution is cohesive despite the many offshoots because of strong governance. 

“Things can get complicated, because everybody wants to do something differently,” he admits but OpenStack handles it by “keeping things small enough in the core that is extensible enough and generic enough that it can serve different implementations.” 

Software defined everything

Carter admits that “VMware is not just a virtual machine solution; it's your data center in a box.” However, he says as a product, VMware has done a good job of convincing people they need everything in that box.

OpenStack does the opposite, giving users a breadth of choice on projects, integrations and enterprise storage providers that could be confusing in comparison. 

Having a software-defined data centre as the platform for your virtual infrastructure makes sense. It gives you the same capabilities as an AWS or an Azure, using your choice of off-the-shelf physical hardware to host everything VMs migrating on-prem apps, to new portable environments, alongside hosting cloud-native platforms like Kubernetes. 

Red Hat has focused marketing its OpenStack environment with this approach: OpenStack Services on OpenShift, emphasising its role as a private cloud for existing and new workloads.

There’s also the opportunity to mix and match your on-premises OpenStack implementation with provider infrastructure, to take advantage of additional compute when necessary, or to move workloads on-premises from a provider when data sovereignty is an issue (something that’s increasingly relevant with today’s geopolitical situations). 

Migrate and modernize

Carter is keen not to position OpenStack as a one-to-one replacement, especially not one running to catch up: “We see a lot of people today of saying ‘I need an alternative to VMware’, which is probably the right mindset, because what they're really saying is ‘I need an alternative to paying this massive bill’. But OpenStack isn’t the lesser VMware; its goal is not to be a less functional VMware, it’s to be a better OpenStack with its own ecosystem.”

He also cautions that some VMware products that don’t have a single, simple replacement: “Nothing is going to provide Aria Operations except Aria Operations. If you've built your business on a proprietary, unique solution, then as much as I would love to get you out of that ecosystem, those customers just need to write the check.”

“There's not an easy button to get off of something that is wholly VMware,” he said. At best, lift and shift can only be the first step in modernising applications for the cloud-native world. “Those companies need to do some introspection and figure out, what does life look like after Broadcom, and that's a tough question, because sometimes they don't know.”

Even if OpenStack can deliver that same virtualisation, organizations should be looking further ahead, agrees Sean Cohen, Red Hat director of product management and OpenStack lead. “The playbook we had for virtualization is not going to cut it for where we are right now.”

Organisations need to be able to handle multiple generations of technology, from managing VMs to cloud and AI native, Cohen says. “We have to have unified practices like platform engineering, I need to be able to have gitops, and a CI/CD pipeline so I can deploy everything and update everything and test everything automatically.”

Change on both sides

OpenStack has already started adding or improving features to help VMware users who are on that journey, Ravi notes. “We started seeing folks coming and telling us about their VMware use cases and the need to evolve those use cases, or the need to find parity between what VMware was offering, and what OpenStack could do.”

Sometimes that means changing your workloads to better fit the cloud naïve approach, but he also admits OpenStack developed new features off of filling the gaps between the OSS and VMware. 

Some of that is different networking requirements or missing integrations that are being worked on.

“We’re talking a lot more about disaster recovery. [Or] if they’re porting AI workloads from VMware, they have different expectations of how graphics cards and accelerators are shared.”

Customers want chargeback abilities

People moving from Broadcom to OpenStack also want “more than host and tenant isolation; chargeback is one of them, billing is one of them,” Cohen agrees. AI workloads are raising the demand for GPUs-as-a-Service.  

The recent Epoxy release added the ability to live migrate different GPUs and a new operator supports dynamic results allocation to optimise workload distribution, balance resources and reduce infrastructure hotspots, whether that’s migrating VMs dynamically or for data operations like evacuating workloads when you need to upgrade infrastructure.

OpenStack will soon offer feature packs (which Cohen compares to Microsoft service packs) that make adding new features less disruptive, and live kernel patching for applying critical security updates without rebooting, currently in preview, will be generally available in November.

Scaling the stack

The ability to scale from homelab to multi-region cloud puts OpenStack in an interesting position, as administrators and platform engineers can experiment on small systems with a handful of nodes, while running multi-data centre private and public clouds. 

Like Kubernetes, the OpenStack community sees its APIs as its strength, letting you run the same code on edge implementations as in data centres, and use the same tools to manage a single cluster of VMs as an entire hosting cloud.

That model makes OpenStack attractive to telcos, as they can migrate workloads where they’re needed: running in large data centres for capacity, or pushed out to small server racks on the edge of their network, as part of the hardware in cellular base stations.

The Open Infrastructure foundation quotes some large numbers: 45 million cores in production, but that includes a small number of extremely large organisations like Yahoo, Walmart and Workday who might be running OpenStack on over a million cores.

“Some of our largest customers are opening the largest networks in the US or in Europe or India,” the Red Hat director and OpenStack lead Cohen says. “We have very large deployments.” 

“It's unbelievable the number of telephone conversations that are actually running on top of OpenStack in some way,” Ravi adds. 

How big is big enough

“Where we see OpenStack really shines is ‘I used to run IaaS infrastructures, and I’m now looking at alternatives’,” Cohen suggests. “I need to build a private cloud, and that product cloud is going to have scale, and I have regions and different availability zones, and I need now to manage standard isolations, and I need charge back.”

Cohen warned any business up to 50 nodes shouldn’t consider OpenStack. “It's highly complicated. It's not designed to do the small stuff,” Cohen says. 

Red Hat suggests OpenShift (based on KubeVirt) for those organizations. However Rackspace is all in on OpenStack and Carter believes it’s adaptable enough to work for much smaller environments. 

Carter and Ravi both argued OpenStack would work in smaller environments pointing to university computing labs with a handful of nodes using the software. 

Raid explains, “The complexity itself starts growing with the scale and we're building for both.”

Edge computing will also bring smaller OpenStack installations, he suggests; “where there is a larger central site, but the edge sites themselves are small – these are separate OpenStack clouds that are actually connected together in a federated manner.” Even large customers like telcos also have very small systems: “we've seen examples of telcos putting OpenStack on top of routers and much smaller devices.”

Skills and certifications 

Smaller installations may mean easier migrations, argues Carter, especially if they have a more traditional virtualisation architecture. 

Customers who have been on VMware from the early 2010s, using flat networking, virtual machine management and simplified images, could have an easier time migrating. “You could take a three-node OpenStack hyperconverged environment, run your virtual machines, have converged networking, converged storage: totally flat, no layer three, no tenant networking, nothing fancy and address those older environments really easily.” 

Just because similar functionality is available doesn’t mean it will be as easy for organizations to run OpenStack in production, of course. 

VMware expertise is extremely common: “you can throw a stone into a crowd of IT people and hit somebody who is VMware certified and has been running VMware over a long period of time,” Carter points out. 

While Broadcom seems to view OpenStack and KubeVirt – or at least Red Hat – as its closest competition. Gartner is a little more cautious, pointing out that OpenStack skills are less common than VMware certifications.

But with a history going back to the early days of grid computing and technologies like Beowulf clusters, OpenStack relies heavily on Linux making it easier to pick up than it might sound.

“There’s a lot of open source innovations that OpenStack sits on top of,” Ravi agrees. “We’re talking the Linux container ecosystem, Linux itself and Linux virtualization ecosystem.”

Kubernetes expertise will be useful too, Cohen notes. “Every service in OpenStack is a microservice already; it’s deployed as a microservice in an operator.”

From operations to observability

“Historically, OpenStack was fairly difficult to wrangle,” Carter admits, “there were always lots of moving parts, lots of applications you had to manage.” He says OpenStack still had moving parts, but they’ve matured under a meaningful system. 

“Do you need an operations team to manage 10,000 nodes across the globe? Yes, probably. But if you've got four or five folks doing tangential operation efforts on a medium to small size cluster, you should be totally fine.”

“If you have very good administrative staff, you're going to have a great experience, whereas if you have more junior staff it might be a little bit of a bumpy road as you get started.”

That may make operating OpenStack reasonably routine for enterprises with Linux expertise but the fact that we haven't yet found typical mid-size organisations ready to talk about their experience of running it in production suggests those journeys are still ongoing rather than complete.

Mathematics of migration

The Rackspace take on the economics of migration suggests combined migration and operating costs will range from slightly to significantly less than the cost of the new Broadcom deal, depending on how quickly organizations can stop paying for VMware. how much amount of work you are willing to do,” Ravi says.

The figures are based on un-named Rackspace customers with deployments ranging from one or two sites running hundreds of VMs to 16 sites running thousands of VMs, reusing existing hardware.

“What if you stopped paying for Broadcom on day one? This results in a 66% cost reduction in year two, 76% reduction in year three. What if you can’t do 100% migration? Moving to a 50/50 split still results in a 26% reduction in cost by year three. With these savings users can replace hardware, or reduce the number of hosts,” he notes.

Time to start moving?

Recent changes like blocking hyperscaler clouds from reselling VMware licences underline how quickly organizations need to start making decisions; existing licences from those clouds will be up in less than 12 months time and that means either accepting higher prices or planning a move.

Customers may need to move not just their VMs but likely their underlying data store, which at a couple of petabytes would take months to migrate, Carter cautions. 

“Months to move is a scary place for enterprise IT when they have weeks to solve problems because they have not addressed it over the last three years. I think the majority of people in 2023 saw some writing on the wall and said, ‘I'm going to sign a three year deal right now’ and so they've got runway. But at the end of 2026, those deals become up for renewal and I think that is where we'll see that inflection point.”

“It's going to be a rough inflection point, and I think it'll be a mad scramble,” Carter warns. However, if you’re already running on AWS S3 or Azure Blobs, you can simply access them over their APIs from OpenStack, making migration a lot easier than it might seem at first glance.

Not everyone will be rushing, Ravi argues. “I've seen folks that have the time to modernize actually chart it out and plan this, because they're looking at the next five years to ten years, even 15 years, as to how they're transforming their businesses.” 

That means they can wait for OpenStack to implement a feature they need and make changes to their workloads in preparation. Smaller organisations who can adjust their expectations for storage and networking can move faster: “within the one year horizon – because they're able to realize the gains of moving that quickly.”

Although Broadcom’s price hikes have made headlines, they’re not the only pressure pushing organisations towards modernising their technology stack, Cohen says. 

“Everyone’s looking at cost optimisation. IT budgets and network budgets this year, and I would say even going into next year, remain flat,” adding companies were researching where they can cut back to “allocate funds to the cloud native journey and the AI native journey.”

The point of technology, after all, is what your business can do with it, he points out. “Technology modernization equals revenue. If you want to just keep the lights on and save costs, good luck saving your role. It won't cut it in today's next generation requirements.”