Collapsing the Kubernetes Stack: A Private Cloud OS Architecture for VMware Exit

The first decade of cloud-native infrastructure shipped on a clean assumption. The Kubernetes management plane and the infrastructure substrate were independent decisions made by independent teams. Compute, storage, networking, and the Kubernetes stack each carried their own license, their own upgrade path, and their own support relationship. Coordination across those layers was a tax the operations team paid in time and effort.

Three external pressures broke that assumption.

The first pressure is cost. Broadcom's acquisition of VMware closed in November 2023, and the price model shifted from perpetual licenses to subscription. Effective April 10, 2025, the minimum license purchase moved from 16 cores to 72 cores, which forced small VMware customers into a footprint they could not justify. Reports of price increases between 150% and over 1,000% are now well documented, with small businesses seeing 350% to 450% increases driven by the 72-core minimum alone.^[1]

The second pressure is the operational reality on the floor. CloudBolt's January 2026 survey of 302 enterprise IT decision-makers found that 86% of organizations are actively reducing their VMware footprint, while only 4% have completed a full migration. 72% are migrating workloads to public cloud IaaS, 43% to Hyper-V or Azure Stack, and 34% to SaaS replacements. 88% of respondents remain concerned about future price increases.^[2]

The third pressure is the rise of stateful workloads in Kubernetes. The CNCF 2024 Annual Survey found that 74% of organizations now use containers to manage stateful applications in production, with overall Kubernetes production adoption at 80% in 2024.^[3] The 2025 survey reported production adoption climbed further to 82%, the highest in the survey's history.^[5] Stateful container workloads carry persistent volumes that demand the same protection, replication, and recovery semantics as traditional VMs. They do not get those semantics from the upstream Kubernetes specification. They get them from the storage substrate underneath.

This paper makes a single argument. The right exit from VMware-plus-Kubernetes is not another hypervisor swap. The right exit is a Private Cloud OS that consolidates compute, storage, networking, data protection, and the Kubernetes integration into a single code base. The rest of this paper describes what that architecture looks like, where it changes the operational math for stateful Kubernetes workloads, and how to evaluate the choice on its merits rather than on procurement spreadsheets.

Most VMware exits today follow the same pattern. The procurement team prices Proxmox, Nutanix AHV, Hyper-V, or OpenStack against the renewed VMware quote, picks the cheaper option, and hands the result back to operations as a directive. Operations then plans a migration that preserves the rest of the architecture. The storage arrays stay. The networking products stay. The Kubernetes distribution stays. The overlay storage layer stays. The backup software stays. Only the hypervisor changes.

The CloudBolt data shows what this looks like at scale. The most common path is a phased, partial transition rather than a full migration.^[2] The architectural pattern repeats. The hypervisor changes, the rest of the stack persists, and the operational model that depended on the original stack persists with it.

Replace the hypervisor and the storage array still uses its own metadata, its own dedupe engine, its own snapshot logic, and its own management plane. The Kubernetes distribution still requires its own licensing, its own upgrade cadence, and its own integration matrix against the new hypervisor. The overlay storage layer still runs as a duplicate of the underlying vSAN, doubling capacity and management overhead. Each layer keeps its own caching, its own optimization logic, and its own resource allocation. Coordination across them still consumes the team time it consumed before.

The cost is not in the new platform. It is in the unwinding of dependencies the old architecture created. A hypervisor swap solves the licensing problem and leaves the coordination problem in place. The migration that procurement is planning addresses 30 to 40 percent of total infrastructure cost. The migration the operations team needs addresses the three-tax pattern that procurement does not see.

The question this paper asks is whether organizations leaving VMware can do something more than reduce a line item. The answer requires a different architecture, not a different hypervisor. It requires a Private Cloud OS that consolidates the Kubernetes stack at the same time.

The Three-Tax Model frames the problem in operational terms a CIO can use during a budget review.

VMware shops running Kubernetes today pay for the same workload three times. They pay Broadcom for vSphere licensing to host the cluster nodes. They pay a Kubernetes distribution vendor for the cluster management plane, whether that is Tanzu Kubernetes Grid, Red Hat OpenShift, or Rancher Prime. Many pay a third vendor for overlay storage because vSphere storage policies do not extend cleanly into Kubernetes without commercial Tanzu add-ons. Each contract has its own pricing model, its own renewal calendar, and its own support relationship. The integrations between them are not specified by any vendor and are not the responsibility of any one team.

The Private Cloud OS pattern collapses those three contracts into one. The data design splits into selecting one of two architecture decisions.

VergeOS implements the Private Cloud OS pattern with a native Kubernetes integration that ships as Helm charts from the verge-io repository on GitHub. The integration assumes the customer already runs a Kubernetes distribution they trust, whether that is RKE2, K3s, vanilla upstream, or a vendor distribution. VergeOS does not introduce a competing distribution. Rancher stays as the management plane the team already uses. The four components (the CSI driver, the Cloud Controller Manager, the Cluster Autoscaler, and the Rancher node driver with UI extension) are not features bolted onto a Kubernetes layer. They are native services that delegate storage operations, networking, node provisioning, and autoscaling directly to the VergeOS API.

Three architectural choices in VergeOS produce the consolidation that matters for Kubernetes workloads. Each is invisible from the management plane and consequential at the substrate.

VergeFS as a Hypervisor Service

Most vSAN technology runs as a virtual machine on top of the hypervisor. Each storage request from a pod passes through the cluster node VM, into the storage VM, back through the hypervisor, and out to the application. The round trip adds latency at every layer and requires dedicated networking to mask the inefficiency. VMware vSAN ESA mandates NVMe and 25Gbps networking for that reason.

VergeFS runs as a service inside the VergeOS hypervisor, not as a VM. There is no storage VM, no extra hop, and no separate communication path. The file system shares memory and scheduling with the compute services. Server-class flash drives inside the nodes replace dedicated storage arrays at roughly one-fifth the price.^[4] For Kubernetes persistent volumes, the result is a clean delegation path. The CSI driver talks to the VergeOS API. The API talks to VergeFS in the same memory space. The persistent volume gets inline deduplication, multi-tier placement across NVMe, SSD, and HDD tiers, and integrated snapshots without traversing a separate storage product.

One Code Base, Not Three Stacks Pretending to Be One

The VergeOS code base is fewer than 400,000 lines.^[4] The VMware ESXi, vSAN, and NSX stack that VergeOS replaces is more than 25 million lines, by VergeIO's measurement against the VMware product set.^[4] That ratio is not accidental. Three independent products that solve overlapping problems carry overlapping code, overlapping metadata, and overlapping operations. A single code base that solves the same problems carries each one once. Every feature that ships in VergeOS shares state with every other feature. Deduplication operates across compute, storage, networking, and replication paths. Snapshots are an operating-system primitive available to a single VM, an entire Virtual Data Center, or a Kubernetes persistent volume.

Native Kubernetes Support

The integration is the part that matters to platform engineers. Four Helm charts on the verge-io repository on GitHub provide the contracts Kubernetes needs from a platform underneath.^[7]

The CSI driver supports two backends. The NAS backend serves EXT4 volumes over NFS for ReadWriteMany access patterns. The block backend hotplugs vSAN block drives directly to VergeOS VMs for ReadWriteOnce access. Both delegate to the VergeOS API. A Kubernetes persistent volume snapshot becomes a vSAN snapshot, which means stateful workloads share the same DR and replication infrastructure that protects production VMs.

The Cloud Controller Manager populates node metadata such as provider ID, instance type, and internal IPs from the underlying VergeOS VMs. It also handles LoadBalancer service provisioning through VergeOS VNet NAT rules. The cluster does not need MetalLB or an external load balancer for typical workloads.

The Cluster Autoscaler adjusts node pool size based on pending pod resource requests. It uses Rancher's API to manage pools, and Rancher uses the VergeOS node driver to provision or delete the underlying VMs.

The Rancher node driver and UI extension complete the integration. The Docker Machine driver clones template VMs, injects SSH keys via cloud-init, configures CPU and RAM, attaches the VM to a network, and powers it on. The UI extension surfaces VergeOS-specific cloud credential and machine configuration forms inside the Rancher interface. An operator who knows how to provision a Rancher cluster on vSphere knows how to provision a Rancher cluster on VergeOS. The flow is identical.^[8]

Most VMware-plus-Kubernetes shops do not need new hardware to evaluate this architecture. The single most expensive mistake in a VMware exit is buying a fresh hardware fleet to host the replacement platform. VergeOS runs on existing servers including Dell VxRail nodes, HPE servers, Cisco UCS, Lenovo, and commodity x86 white boxes.^[4] The reference deployment has three components.

The migration workflow is incremental, not all-at-once. ioMigrate automates VMware VM rehosting, including networking configuration, into existing hardware running VergeOS. VMs move in batches. For Kubernetes clusters specifically, the pattern is parallel operation. New clusters land on VergeOS through Rancher. Existing TKG or Rancher-on-vSphere clusters continue to operate. Stateless services move first. Stateful workloads move after the new platform validates under real load. Even in large data center environments, the IT team gets productive on VergeOS within one to two weeks.^[4]

The architectural argument is theoretical until something specific happens to a workload. Three customer situations show what consolidation changes in practice.

Situation 01: Rancher Inside vSphere

The customer already runs Rancher to manage Kubernetes clusters inside vSphere. The control plane stays. The substrate underneath changes. Application teams see no change in their day-to-day Kubernetes operations. Platform teams see the vSphere line item disappear and the overlay storage line item retire. Migration is straightforward because the Kubernetes management contract above the substrate is preserved.

Situation 02: VMware Tanzu Kubernetes Grid

The customer built their Kubernetes practice on Tanzu Kubernetes Grid (TKG). Broadcom's restructuring of the Tanzu portfolio plus bundled vSphere licensing pressure has put their roadmap in question. This segment has the highest exit pressure and the deepest migration work. As well as the highest ROI. Storage Policy-Based Management projects vSphere policies into Kubernetes as StorageClasses. NSX-T provides networking. Tanzu Application Platform layers developer experience on top. All of this needs reconstruction on the destination platform.^[10]

The honest position is that the migration is a project, not a port. VergeOS is the platform built to absorb the work. The dominant pattern is parallel operation with iterative workload movement. New clusters land on VergeOS through Rancher. Old TKG clusters stay on vSphere. Stateless services move first. Stateful workloads move after the new platform validates under real load.

Situation 03: Bare-Metal Kubernetes

The customer runs Kubernetes directly on bare metal. No hypervisor, no live migration, no integrated DR, no shared snapshot model. Operational complexity scales with cluster count. This situation is not a VMware exit story. The customer never bought VMware, probably because of overhead concerns. It is an operational story. RKE2 and K3s clusters provisioned on VergeOS gain VM-level isolation, snapshot-based recovery, and unified storage policy across containerized and traditional workloads. The Kubernetes operations workflow stays the same. The platform underneath does the work that bare metal cannot. This segment has the highest gain in operational efficiency and likely environmental availability.

Stateful Kubernetes workloads are now the majority case. The CNCF 2024 Annual Survey reported that 74% of organizations use containers to manage stateful applications in production.^[3] The 2025 CNCF Annual Survey put overall Kubernetes production adoption at 82%, the highest in the survey's history.^[5] Persistent volumes are normal. Database pods are normal. Stream processing with stateful operators is normal.

The challenge is that the upstream Kubernetes specification does not provide the data protection semantics those workloads need. The cluster knows how to mount a persistent volume. It does not know how to take an application-consistent snapshot. It does not know how to replicate that snapshot to a DR site. It does not know how to roll back a stateful workload to a point in time. Those operations belong to the storage substrate or to a backup product.

In a multi-vendor stack, the substrate and the backup product do not coordinate. The overlay storage layer (Longhorn, Portworx, OpenEBS, Rook) takes its own snapshots inside its own metadata store. The underlying vSAN takes its own snapshots at the VM level. The backup product takes application-aware snapshots and stores them in its own catalog. Three snapshot tools, three retention policies, three restore paths. Veeam's own documentation makes the architectural point cleanly. Application-aware backup must capture the full state of the workload including persistent volumes, configurations, cluster metadata, and control plane components.^[6] The fragmentation is real.

VergeOS treats stateful Kubernetes data resilience as an architectural property rather than a feature. Four data points frame the scope of the problem.

The CSI driver delegating to the VergeOS API solves three problems at once. First, it removes the overlay storage layer. The persistent volume is a vSAN volume. No second storage system, no second metadata store, no second snapshot tool. Second, it gives the Kubernetes snapshot the full vSAN feature set. Inline deduplication operates across the volume. Multi-tier placement directs the volume to NVMe, SSD, or HDD as the StorageClass specifies. Replication and DR run on the same infrastructure that protects production VMs. Third, it produces a unified recovery path. A Kubernetes persistent volume snapshot is a vSAN snapshot. The IT team has one place to look when a stateful workload needs to be rolled back, replicated, or restored.

The architectural argument changes the day-to-day shape of Kubernetes platform operations. Three operational decisions shift the most.

Sizing

Sizing a VergeOS-plus-Rancher deployment is different from sizing a vSphere cluster with Tanzu Kubernetes Grid. The orchestrated stack sizes each layer independently and layers a coordination tax on top. VergeOS sizes the workload requirement once. For a typical VMware-plus-Kubernetes migration, plan on a 30 to 40 percent reduction in physical server count for the same workload density.^[4] A four-server VergeOS cluster covers most mid-market VMware-plus-Kubernetes footprints. A 12 to 20 server deployment covers most enterprise consolidations.

Licensing

VergeOS is licensed per physical server. Not per CPU socket, not per core, not by storage capacity, not by RAM, not by Kubernetes node count, not by container count. A server with 192 cores and 122TB of NVMe carries the same license fee as a server with 32 cores and 4TB of flash. The model rewards customers who invest in dense hardware. VMware's per-core licensing under the 72-core minimum punishes density. Tanzu's per-core licensing on top of that compounds the punishment. Overlay storage license fees scale with capacity. VergeOS does not.

The licensing scope includes everything. Compute, storage, networking, replication, snapshots, the CSI driver, the Cloud Controller Manager, the Cluster Autoscaler, the Rancher node driver, ioMigrate, ioReplicate, ioFortify, and the VergeIQ private AI capability all ship inside the same license.^[4] There are no additional charges for data protection, no separate add-ons for replication, and no metered services that scale with workload growth.

Testing Cadence

The operational benefit of consolidation shows up most clearly in testing. A multi-product stack tests the failover of each layer independently and rarely tests the integrated path. Integrated tests require coordinating three product owners. A single platform tests the integrated path natively. Stateful Kubernetes workload recovery is a three-click VDC operation. The team runs the test as a recurring practice rather than an annual exercise. Most VergeOS customers run upgrades inside the maintenance window without scheduling a multi-team coordination call.

A common question during a VergeOS-plus-Rancher evaluation is which existing infrastructure products the platform makes unnecessary. The answer is most of the orchestrated Kubernetes stack, fully in some places and partially in others.

The VMware-plus-Kubernetes exit is the most significant infrastructure decision most platform engineering teams will make this decade. The wrong choice swaps one hypervisor for another and preserves the operational model that built the three-tax problem in the first place. The right choice replaces the orchestrated stack with a single platform that operates as one system rather than three products coordinated by APIs.

VergeOS plus Rancher delivers that consolidation. The single code base runs the hypervisor, the storage, the networking, the data protection, the disaster recovery, the migration tooling, and the native Kubernetes integration as services rather than independent products. Server count drops 30 to 40 percent. The unified snapshot model gives stateful Kubernetes workloads the same DR and replication infrastructure that protects production VMs. The CSI driver delegates directly to the VergeOS API. The Cloud Controller Manager provisions LoadBalancer services through VergeOS VNet rules. The Cluster Autoscaler scales node pools through Rancher's API and the VergeOS node driver. Stateful workloads move on the customer's timeline.

The operational story is concrete. Migration in 60 to 120 days using ioMigrate. Team productivity in one to two weeks.^[4] Existing VxRail and commodity x86 hardware repurposes without a new fleet purchase. The licensing model charges per physical server and includes every feature, including the native Kubernetes integration. The Rancher control plane stays in place.

Organizations leaving VMware should evaluate the Private Cloud OS architecture against the orchestrated alternatives on the operational math, not the licensing math. Procurement sees the licensing line. Operations pays the coordination cost for the next five years.

The right next step is a paired proof of concept. Run VergeOS on existing VxRail or commodity hardware. Migrate a representative Kubernetes workload using ioMigrate and the Rancher node driver. Validate stateful workload recovery against the new platform's snapshot and replication infrastructure. Measure server count, recovery time per stateful workload, and team time over a 60-day window. The numbers will make the architectural argument better than this paper can.

VergeOS rewards organizations that consolidate the Kubernetes stack. It punishes the procurement habits that built the three-tax problem in the first place.