5.1 OKE Cluster Types, Node Options & Cluster Creation
Key Takeaways
Enhanced clusters support every OKE feature, including virtual nodes, add-on management, workload identity, and self-managed nodes, and come with a financially backed SLA.
Basic clusters support core Kubernetes and OKE features with a service level objective, not a financially backed SLA, and they can only use managed node pools.
Virtual nodes give serverless Kubernetes with hypervisor-level isolation for pods and pay-per-use pricing for the compute each pod consumes.
A basic cluster can be upgraded to an enhanced cluster if it is VCN-native, but an enhanced cluster cannot be downgraded to basic.
The Console creates an enhanced cluster by default, while the CLI and API create a basic cluster unless you specify otherwise.
What OKE Manages for You
OCI Kubernetes Engine (OKE), formerly Container Engine for Kubernetes, provides CNCF-conformant Kubernetes clusters. Oracle runs and patches the control plane: the Kubernetes API server, scheduler, controllers, and etcd. You decide how worker capacity is provided and how the cluster connects to your network.
Enhanced vs. Basic Clusters
| Capability | Enhanced cluster | Basic cluster |
|---|---|---|
| Core Kubernetes and OKE features | Yes | Yes |
| Managed node pools | Yes | Yes |
| Virtual node pools | Yes | No |
| Self-managed nodes | Yes | No |
| Workload identity (IAM for pods) | Yes | No |
| Fine-grained add-on management (versions, auto-update, optional add-ons) | Yes | No |
| Node cycling during updates and upgrades | Yes | No |
| Higher worker node limits and limit increases | Yes | No |
| Service commitment | Financially backed SLA | Service level objective only |
| Price | Charged per cluster per hour | Free |
A few rules that often appear in scenarios:
- An enhanced cluster must be VCN-native, meaning its Kubernetes API endpoint is integrated into your VCN.
- A basic cluster can be upgraded to enhanced if it is VCN-native and meets the prerequisites. You cannot downgrade an enhanced cluster.
- In a basic cluster you are responsible for upgrading the essential add-ons (such as CoreDNS, kube-proxy, and the CNI plugin), and you cannot disable them, choose versions, or install Oracle's optional add-ons through OKE.
- The Console creates enhanced clusters by default. The CLI and API create basic clusters by default unless you request an enhanced cluster.
- Scale limits differ by network type. Enhanced clusters that use VCN-native pod networking have a default limit of 2,000 managed or self-managed nodes; enhanced clusters using flannel can grow far larger with Oracle approval.
Three Kinds of Worker Capacity
Managed Nodes
Managed nodes are OCI Compute instances in your tenancy, organized into managed node pools. OKE provisions and updates the Kubernetes software on them, and you choose the shape, image, placement, and boot volume. Managed nodes support GPU, Arm (Ampere), and bare metal shapes, and you can pass cloud-init scripts to customize the operating system. Node pools can span availability domains and fault domains for resilience.
Virtual Nodes
Virtual nodes provide a serverless Kubernetes experience. Oracle manages, scales, and upgrades the node infrastructure, and you pay for the exact compute resources each pod consumes, plus a per-virtual-node charge, instead of paying for whole servers. Oracle states that virtual nodes provide hypervisor-level isolation for pods, so even untrusted workloads can run safely. Virtual nodes are available only in enhanced clusters and use VCN-native pod networking. Because there is no node operating system for you to customize, features that depend on host access, such as privileged DaemonSets or cloud-init tuning, are not available.
Self-Managed Nodes
Self-managed nodes are compute instances that you create and configure yourself and then join to an enhanced cluster. They suit specialized hardware or configurations that managed node pools do not support, such as some high-performance computing setups.
Images and Shapes for Worker Nodes
For managed nodes you choose:
- Image: an OKE image (an Oracle Linux image with Kubernetes components preinstalled, which shortens node startup), an Oracle Linux platform image, or a custom image built from them.
- Shape: VM or bare metal, AMD, Intel, or Ampere Arm, including flexible shapes where you set OCPUs and memory, and GPU shapes for AI and ML workloads.
Pick shapes per node pool, and run several node pools when workloads need different hardware, for example a general pool and a GPU pool with taints.
Creating a Cluster: Quick Create vs. Custom Create
| Workflow | What it does | When to use it |
|---|---|---|
| Quick Create | Creates the cluster and the network resources it needs (VCN, subnets for the API endpoint, workers, and load balancers, gateways, route tables, and security rules) using Oracle defaults | Labs, proofs of concept, fast starts |
| Custom Create | Uses network resources you already created and lets you set every option (CNI type, endpoint subnet, node pool details, encryption, add-ons) | Production clusters in a governed landing zone |
With Custom Create, you prepare the Kubernetes API endpoint subnet, the worker node subnet, the pod subnet (for VCN-native networking), and load balancer subnets, together with the security rules OKE requires between them. Users who create clusters also need IAM permissions, typically manage cluster-family in the cluster's compartment, plus permissions to use the network and compute resources involved.
Kubernetes API Endpoint Placement
The Kubernetes API endpoint lives in a subnet you choose:
- Public endpoint: the endpoint has a public IP in a public subnet. Restrict access with the subnet's security rules or NSGs so only approved CIDR ranges can reach port 6443.
- Private endpoint: the endpoint has only a private IP in a private subnet. Administrators reach it from inside the VCN, through OCI Bastion port forwarding, from Cloud Shell with private network access, or from on-premises networks over VPN or FastConnect.
Private endpoints are the usual choice for production because the API server is never exposed to the internet.
Accessing the Cluster with kubectl
You generate a kubeconfig file with the OCI CLI and then use kubectl:
oci ce cluster create-kubeconfig --cluster-id <cluster-ocid> --file $HOME/.kube/config --region us-ashburn-1 --token-version 2.0.0 --kube-endpoint PRIVATE_ENDPOINT
- Cloud Shell already has the OCI CLI and kubectl installed and authenticated, so it is the fastest way to reach a cluster whose endpoint it can reach.
- A local terminal needs the OCI CLI configured with an API signing key, plus network access to the endpoint.
The kubeconfig does not store a static password. Each kubectl call obtains a short-lived token from the OCI CLI based on your IAM identity, and Kubernetes RBAC then decides what you can do inside the cluster.
A bank needs Kubernetes with a financially backed control plane SLA, pod-level IAM permissions for OCI API calls, and pods that run without any worker operating system to patch. Which OKE design meets all three requirements?
An enhanced cluster with a virtual node pool, using workload identity for pod permissions
A basic cluster with a managed node pool, using instance principals on the worker nodes
A basic cluster with a virtual node pool, using API keys stored in Kubernetes secrets
An enhanced cluster with self-managed nodes, using instance principals on the worker nodes
An ML team needs Kubernetes workers on bare metal GPU shapes with custom kernel settings applied by cloud-init. Some nodes must be configured in ways that managed node pools do not support. Which choice fits?
Virtual nodes in a basic cluster, because Oracle installs the GPU drivers automatically
An enhanced cluster mixing managed GPU node pools with self-managed nodes
Virtual nodes in an enhanced cluster with a privileged DaemonSet that tunes the kernel
A basic cluster that joins self-managed GPU nodes for the special configurations
Sections you finish are checked off in the contents.