13.2 VMware Kubernetes Service & Workload Clusters

13.2 VMware Kubernetes Service & Workload Clusters

In modern enterprise IT environments, Kubernetes has become the de facto operating system for cloud-native microservices. However, operating disparate Kubernetes installations across bare-metal servers or unmanaged virtual machines introduces configuration drift, operational silos, and governance challenges. VMware vSphere Foundation (VVF) resolves these issues by integrating enterprise Kubernetes directly into the hypervisor layer. Through the Supervisor and the VMware Kubernetes Service (VKS), vSphere transforms standard ESXi clusters into an enterprise-grade Kubernetes control plane capable of orchestrating both native container workloads and fully conformant, multi-node Kubernetes clusters.

---

Architectural Overview: Supervisor & Tanzu Kubernetes Clusters

The architectural model of Tanzu on vSphere is structured into two primary tiers: the Supervisor (the infrastructure control plane) and Tanzu Kubernetes Clusters (the workload execution plane).

The Supervisor as the Management Plane

The Supervisor operates as a specialized Kubernetes control plane embedded directly into vSphere. When an administrator enables Workload Management on a vSphere cluster, three Supervisor Control Plane VMs are deployed onto the ESXi hosts. The Supervisor is primarily the infrastructure/service control plane; deploy general application workloads through the supported workload model. Instead, it serves as the infrastructure management layer, orchestrating vSphere resources, enforcing multi-tenant isolation through vSphere Namespaces, and running system operators—most notably the Cluster API (CAPI) and Cluster API Provider vSphere (CAPV).

VKS workload clusters (older material calls them Tanzu Kubernetes clusters)

A VKS workload cluster is a dedicated, fully conformant, upstream Kubernetes cluster provisioned inside a vSphere Namespace. Unlike the Supervisor, which is tied to vSphere release cadences, TKCs run standard, vanilla-compatible Kubernetes distributions tailored for developer consumption. Platform teams can deploy multiple independent TKCs within separate vSphere Namespaces, providing strong tenant isolation, independent Kubernetes version lifecycles, and dedicated administrative boundaries.

Cluster API (CAPI) manages the entire lifecycle of a TKC declaratively:

• Control Plane Node Pools: Provisioned as virtual machines running kube-apiserver, etcd, kube-controller-manager, and kube-scheduler. Deployments can utilize a single control plane node for development or three nodes for production high availability.
• Worker Node Pools: Compute nodes hosting containerized application workloads. Worker pools can be scaled up or down horizontally by editing the cluster manifest, with compute sizing governed by VirtualMachineClass resources and storage governed by StorageClass definitions.
• Declarative Upgrades & Self-Healing: CAPI performs non-disruptive, rolling upgrades of TKCs by provisioning new nodes with the target Kubernetes version, migrating workloads, and decommissioning old nodes. If a worker node crashes or becomes uncontactable, CAPI automatically provisions a replacement node.

---

Workload Models: vSphere Pods vs Tanzu Kubernetes Clusters

vSphere Foundation provides two distinct architectures for executing containerized workloads:

1. vSphere Pods (ESXi-Native Execution)

A vSphere Pod is an ultra-lightweight, hardware-isolated pod that runs directly on the ESXi hypervisor without requiring a traditional guest Linux operating system VM. To support vSphere Pods, ESXi utilizes two specialized technologies:

• Spherelet: A native ESXi daemon that functions as an ESXi-based kubelet. It communicates directly with the Supervisor control plane, translating Kubernetes pod specifications into native hypervisor commands.
• CRX (Container Runtime Executive) Runtime: An ultra-minimal hypervisor execution wrapper that encapsulates the container image. The CRX engine includes a stripped-down Linux kernel and minimal device drivers, booting in hundreds of milliseconds. It delivers hardware-enforced hypervisor isolation between pods, making vSphere Pods ideal for hostile multi-tenant environments or security-critical container workloads.

Version and workload scope: Supervisor Services can use vSphere Pods on supported VDS networking. General workload-pod support and networking depend on the selected configuration; do not say that every vSphere Pod always requires NSX. Verify the service and workload type.

2. Tanzu Kubernetes Clusters (Guest VM Clusters)

In contrast to vSphere Pods, a Tanzu Kubernetes Cluster deploys full Linux virtual machines (typically Photon OS or Ubuntu) configured as Kubernetes control plane and worker nodes. Developers receive standard cluster-admin privileges inside their TKC, allowing them to install custom Helm charts, operators, Container Network Interfaces, and service meshes without impacting hypervisor configuration. TKCs are supported on both standard vSphere Distributed Switch (VDS) networking (using HAProxy or Avi Load Balancer) and VMware NSX.

Architectural Comparison

---

Storage Integration: vSphere CSI Plugin & CNS

Stateful cloud-native workloads require persistent storage that survives container restarts and pod rescheduling. VMware Kubernetes Service integrates natively with vSphere storage through the Container Storage Interface (CSI) plugin for vSphere and Cloud Native Storage (CNS).

How vSphere CSI & CNS Function

1. StorageClass Definition: The platform administrator creates a Kubernetes StorageClass in the TKC that references an SPBM storage policy assigned to the parent vSphere Namespace (e.g., vsan-raid1-storage-policy).
2. Dynamic Volume Provisioning: When an application requests storage via a PersistentVolumeClaim (PVC), the in-cluster vSphere CSI driver intercepts the request and communicates with the CNS control plane inside vCenter Server.
3. First Class Disk (FCD) Creation: CNS provisions a First Class Disk (FCD)—an independent virtual disk that exists outside the context of any specific VM—on the underlying compliant datastore (vSAN, VMFS, or vVols).
4. Volume Attachment: As Kubernetes schedules the pod to a specific worker node, CNS instructs vCenter to attach the FCD to the worker VM's virtual SCSI/NVMe controller and formats the volume with a native filesystem (ext4 or XFS).
5. Volume Access Modes: vSphere CSI supports ReadWriteOnce (RWO) block storage for single-node attachments and ReadWriteMany (RWX) file storage when backed by vSAN File Services or NFS shares.

---

Networking & Load Balancing Architecture

VMware Kubernetes Service decouples pod-level networking from physical infrastructure routing using standardized networking plugins and load-balancing integrations:

Container Network Interface (CNI): Antrea

The default CNI for Tanzu Kubernetes Clusters is Project Antrea, an open-source networking plugin built on Open vSwitch (OVS):

• Pod-to-Pod Connectivity: Encapsulates intra-cluster pod traffic across worker nodes using Geneve or VXLAN overlay tunnels.
• Kubernetes NetworkPolicies: Enforces fine-grained Layer 3/4 firewall rules between pods, namespaces, and external endpoints directly at the OVS flow table level.
• Egress & Routing: Provides native egress IP control and NodePort routing without requiring external routing changes.

Load Balancing Integration

Kubernetes requires external load balancing to expose the Kubernetes API server endpoint and provide Virtual IPs (VIPs) for Service resources of type: LoadBalancer. TKS supports three load-balancing providers:

1. VMware NSX Advanced Load Balancer (Avi Networks): Enterprise-grade, software-defined load balancer offering multi-cloud traffic routing, SSL termination, and advanced analytics.
2. HAProxy: An open-source, lightweight virtual appliance deployed on standard VDS port groups, ideal for environments without NSX.
3. NSX Native Load Balancer: Embedded Layer 4 load-balancing services provided directly by VMware NSX Tier-1 routers.

---

Cluster Access and Workload Validation

Use the access method and CLI supplied for the installed Supervisor/VKS version. Obtain a namespace-scoped context, discover the available cluster classes and Kubernetes releases, and submit the supported Cluster API specification. Do not use an old TKG API example as proof that every version supports the same fields or authentication flags.

After cluster reconciliation, obtain the workload cluster context and verify its nodes. Namespace permission in vCenter and Kubernetes roles inside a workload cluster are different authorization layers. Assign supported bindings for the intended team and verify them with a representative identity; namespace edit permission is not a universal automatic cluster-admin grant.

Deploy a small workload into the workload cluster, inspect its pods and Service endpoint, and test a PVC using the approved StorageClass. Check pod readiness and actual client access. A running node VM is not proof of a healthy Kubernetes cluster, and a bound PVC does not prove an application mounted the expected path.

Before a Kubernetes upgrade, check the supported service/release combination and workload disruption requirements. Pod disruption budgets, spare worker capacity, persistent storage and application replication affect whether a rolling update preserves service. A controller replacing a node does not make every application highly available.

Official Reading

Use the official 2V0-16.25 objective guide for scope and Broadcom technical documentation for the installed 9.0 component. This independent lesson was reviewed October 18, 2026; actual deployment support depends on the applicable version and configuration.

Review Takeaways

• VMware Kubernetes Service (VKS) in VMware vSphere Foundation leverages the upstream Cluster API (CAPI) framework to manage the lifecycle of enterprise, CNCF-conformant Kubernetes clusters declaratively.
• VKS workload clusters (older material calls them Tanzu Kubernetes clusters) are provisioned inside vSphere Namespaces, providing multi-tenant isolation, dedicated control plane nodes, and horizontally scalable worker node pools.
• The Supervisor supports two execution models: vSphere Pods (running containers directly on ESXi hypervisors via Spherelet and CRX micro-VMs with hardware isolation) and Tanzu Kubernetes Clusters (full multi-node VM clusters for standard cloud-native workloads).
• Storage persistence in TKCs is delivered by the vSphere Container Storage Interface (CSI) plugin, dynamically binding PersistentVolumeClaims to First Class Disks (FCDs) governed by SPBM storage policies.
• VKS networking, access methods and workload permissions depend on the installed service and Supervisor versions; verify their supported configuration.