3.2 Deployments: Rolling Updates, Rollouts & Rollbacks

Deployments: Rolling Updates, Rollouts & Rollbacks

The Kubernetes Deployment controller provides declarative updates for Pods and ReplicaSets. It abstracts the manual management of underlying ReplicaSets, automating zero-downtime rolling updates, canary releases, pause/resume phases, and instantaneous rollbacks. For the CKA exam, administrators must understand the exact arithmetic behind maxSurge and maxUnavailable, master all kubectl rollout subcommands, and be able to diagnose stalled rollouts.

---

1. Deployment Architecture and Controller Hierarchy

A Deployment does not create Pods directly. Instead, it creates and manages one or more ReplicaSets, which in turn manage the Pod lifecycle.

+-----------------------------------------------------------------------------------------+
|                         DEPLOYMENT CONTROLLER HIERARCHY                                 |
|                                                                                         |
|                              +-----------------------+                                  |
|                              |      Deployment       |                                  |
|                              | (spec.replicas = 3)   |                                  |
|                              +-----------------------+                                  |
|                                          |                                              |
|                    +---------------------+---------------------+                        |
|                    | (Rollout Revision 1)                      | (Rollout Revision 2)   |
|                    v                                           v                        |
|         +---------------------+                     +---------------------+             |
|         | ReplicaSet (Old)    |                     | ReplicaSet (New)    |             |
|         | (spec.replicas = 0) |                     | (spec.replicas = 3) |             |
|         +---------------------+                     +---------------------+             |
|                                                                |                        |
|                                                     +----------+----------+             |
|                                                     |          |          |             |
|                                                     v          v          v             |
|                                                  [Pod 1]    [Pod 2]    [Pod 3]          |
+-----------------------------------------------------------------------------------------+

Pod Template Hashes

When a Deployment is updated, the Deployment controller computes a hash of spec.template (e.g., pod-template-hash=7bf4c7987b) and appends it to the newly created ReplicaSet name. This guarantees that Pods created by distinct revisions are uniquely identifiable.

---

2. Deployment Strategies: RollingUpdate vs Recreate

Deployments support two primary update strategies specified in spec.strategy.type:

1. Recreate

• Mechanics: Terminates all existing Pods simultaneously before creating any new Pods.
• Pros/Cons: Causes application downtime during the gap between old Pod deletion and new Pod readiness. However, it prevents two different versions of the application from running concurrently, which is critical for workloads that cannot handle shared database schema mutations.

2. RollingUpdate (Default)

• Mechanics: Gradually replaces old Pods with new Pods, ensuring continuous application availability without downtime.
• Configuration Parameters:
  • maxSurge: The maximum number of Pods that can be created above the desired number of Pods (spec.replicas). Can be an absolute number (e.g., 2) or a percentage (e.g., 25%). Percentage is rounded up to the nearest integer. Default is 25%.
  • maxUnavailable: The maximum number of Pods that can be unavailable during the update process. Can be an absolute number (e.g., 1) or a percentage (e.g., 25%). Percentage is rounded down to the nearest integer. Default is 25%.

+-----------------------------------------------------------------------------------------+
|                    ROLLING UPDATE ARITHMETIC (Replicas = 4, 25% / 25%)                  |
|                                                                                         |
|   - Desired Replicas: 4                                                                 |
|   - maxSurge: 25% of 4 = 1 -> Max Allowed Total Pods = 4 + 1 = 5                        |
|   - maxUnavailable: 25% of 4 = 1 -> Min Required Available Pods = 4 - 1 = 3             |
|                                                                                         |
|   Step 0: Old RS = 4 Pods, New RS = 0 Pods (Total: 4)                                   |
|   Step 1: New RS scales to 1 Pod (Total: 5 -> Max Surge reached)                        |
|   Step 2: New Pod becomes Ready; Old RS scales to 3 Pods (Total: 4)                     |
|   Step 3: New RS scales to 2 Pods; Old RS scales to 2 Pods                              |
|   Step 4: Continue until Old RS = 0 Pods, New RS = 4 Pods                               |
+-----------------------------------------------------------------------------------------+

[!CAUTION] maxSurge and maxUnavailable cannot both be 0 simultaneously. Setting maxUnavailable: 0 guarantees 100% capacity throughout the rollout but requires additional cluster compute headroom for surging new Pods.

Additional Timing Parameters

• minReadySeconds: Minimum number of seconds for which a newly created Pod must run without any of its containers crashing, and pass readiness checks, before it is considered available. Defaults to 0.
• revisionHistoryLimit: Number of old ReplicaSets retained to allow rollbacks (default: 10). Setting this to 0 deletes all old ReplicaSets, preventing rollbacks.
• progressDeadlineSeconds: Maximum number of seconds the controller waits for a rollout to make progress before surfacing a Progressing: False condition (default: 600s / 10 minutes).

---

3. Imperative and Declarative Rollout Management

Administrators must master the full spectrum of kubectl rollout commands.

# 1. Update image in a deployment (triggers rollout)
kubectl set image deployment/frontend web=nginx:1.25.4 --record=false

# 2. View live rollout status
kubectl rollout status deployment/frontend

# 3. View revision history
kubectl rollout history deployment/frontend

# 4. View details of a specific historical revision
kubectl rollout history deployment/frontend --revision=2

# 5. Rollback to the immediate previous revision
kubectl rollout undo deployment/frontend

# 6. Rollback to a specific target revision
kubectl rollout undo deployment/frontend --to-revision=1

# 7. Pause a rollout (useful for canary testing)
kubectl rollout pause deployment/frontend

# 8. Resume a paused rollout
kubectl rollout resume deployment/frontend

# 9. Perform a rolling restart without changing the Pod template
kubectl rollout restart deployment/frontend

Annotating Change Causes

To track why a rollout occurred, annotate the Deployment:

kubectl annotate deployment/frontend kubernetes.io/change-cause="Upgraded nginx to 1.25.4 for CVE-2024-xxxx patch" --overwrite

---

4. Canary Deployments and Blue/Green Strategies

While Deployments natively perform rolling updates, advanced deployment strategies can be constructed using Service label selectors.

+-----------------------------------------------------------------------------------------+
|                            CANARY DEPLOYMENT TOPOLOGY                                   |
|                                                                                         |
|                               +-----------------------+                                 |
|                               |   Service (Frontend)  |                                 |
|                               |  selector: app=web    |                                 |
|                               +-----------------------+                                 |
|                                           |                                             |
|                    +----------------------+----------------------+                      |
|                    | (Traffic: 80% / 4 Pods)                     | (Traffic: 20% / 1 Pod|
|                    v                                             v                      |
|     +-----------------------------+               +-----------------------------+       |
|     | Deployment: web-v1          |               | Deployment: web-v2 (Canary) |       |
|     | labels: app=web, version=v1 |               | labels: app=web, version=v2 |       |
|     | replicas: 4                 |               | replicas: 1                 |       |
|     +-----------------------------+               +-----------------------------+       |
+-----------------------------------------------------------------------------------------+

• Canary Strategy: Deploy a second Deployment (web-v2) sharing the same base label (app: web) but with fewer replicas. The Service load-balances across all matching Pods proportionally to replica counts.
• Blue/Green Strategy: Run two full-scale Deployments (web-blue and web-green). Point the Service selector to version: blue. To switch traffic instantaneously, update the Service selector to version: green.

---

5. Troubleshooting Stalled Rollouts

When a rollout does not complete, inspect the Deployment status:

kubectl describe deployment <name>

Common Failure Scenarios

1. Insufficient Quota / Capacity: Cluster lacks CPU, Memory, or PersistentVolumes to schedule surged Pods. Pods remain Pending.
2. ImagePullBackOff: Typo in the container image tag or missing image pull secret for a private registry.
3. Readiness Probe Failures: New Pods start, but readiness probes fail due to database misconfigurations or incorrect health endpoints. The Deployment controller stops rolling out because the new Pods never become Ready.
4. Deadlock from maxUnavailable: 100%: If all replicas are terminated at once and new Pods cannot start, total outage occurs.