3.2 Advanced Controllers (StatefulSets, DaemonSets, Jobs & CronJobs)

3.2 Advanced Controllers (StatefulSets, DaemonSets, Jobs & CronJobs)

While Deployments and ReplicaSets excel at managing stateless web services, cloud-native applications frequently require specialized lifecycle behaviors. Stateless pods are interchangeable and replaceable, but stateful databases, cluster infrastructure agents, and batch data processing require dedicated specialized controllers: StatefulSets, DaemonSets, Jobs, and CronJobs.

---

1. StatefulSets: Managing Stateful Workloads

Stateless applications treat Pods as ephemeral and interchangeable entities. In contrast, stateful applications (such as PostgreSQL, MySQL, Redis, Apache Cassandra, or Kafka) require persistent identity, sticky storage, and ordered cluster management. A StatefulSet is designed specifically for these requirements.

Core Guarantees of StatefulSets

1. Stable Network Identity: Each Pod created by a StatefulSet receives a deterministic, ordinal index starting from zero (pod-0, pod-1, pod-2). This identity remains sticky across rescheduling or node failures.
2. Headless Service Integration: StatefulSets require a Headless Service (clusterIP: None) to publish Pod network identities. Kubernetes generates a stable, direct Domain Name System (DNS) entry for every Pod in the format: <pod-name>.<service-name>.<namespace>.svc.cluster.local
3. Ordered Deployment and Scaling: By default, StatefulSets deploy and scale Pods sequentially in order (pod-0 must be Running and Ready before pod-1 is created). During scale-down, Pods are terminated in reverse ordinal order (pod-2, then pod-1).
4. Persistent Storage per Pod: StatefulSets use volumeClaimTemplates to automatically provision a separate PersistentVolumeClaim (PVC) for each Pod index (e.g., data-db-0, data-db-1). When a StatefulSet Pod is rescheduled, its associated PVC automatically re-attaches to the newly scheduled Pod.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-cluster
spec:
  serviceName: "redis-service"
  replicas: 3
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:7.2
        ports:
        - containerPort: 6379
          name: redis
        volumeMounts:
        - name: redis-data
          mountPath: /data
  volumeClaimTemplates:
  - metadata:
      name: redis-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 10Gi

---

2. DaemonSets: Infrastructure Node Agents

A DaemonSet ensures that all (or selected) worker nodes run exactly one copy of a Pod. As new worker nodes are added to the cluster, the DaemonSet controller automatically schedules a Pod onto them. When nodes are removed from the cluster, those DaemonSet Pods are garbage collected.

Typical Use Cases for DaemonSets

DaemonSets are intended for cluster-wide infrastructure services that must operate locally on every physical or virtual node:

• Cluster Log Collection: Running agents like Fluentd, Logstash, or Vector to aggregate node-level container log files.
• Node Monitoring & Observability: Deploying metrics exporters such as Prometheus node-exporter or Datadog agent to collect host CPU/RAM metrics.
• Container Network Interface (CNI) Plugins: Running networking software like Calico, Cilium, or Flannel that provides pod-to-pod networking on every node.

[!TIP] DaemonSet Pods can bypass node taints (such as node-role.kubernetes.io/control-plane:NoSchedule) by using explicit tolerations in their spec, allowing monitoring and networking daemons to run on control plane nodes as well.

---

3. Jobs: Batch Workload Execution

Deployments, StatefulSets, and DaemonSets are designed to run continuous, long-running processes that should never exit. Conversely, a Job manages batch workloads designed to run until a specified number of Pods terminate successfully (run-to-completion tasks).

Key Job Parameters

• completions: The total number of successful Pod completions required for the Job to be marked complete.
• parallelism: The maximum number of Pods that can execute concurrently during the Job execution.
• backoffLimit: The number of retry attempts before Kubernetes considers the Job failed (defaults to 6).
• restartPolicy: Must be set to either OnFailure (restarts container inside Pod) or Never (creates a brand-new Pod on failure).

Common Job applications include database migrations, nightly data exports, video rendering tasks, or batch machine learning model inference.

---

4. CronJobs: Scheduled Batch Workloads

A CronJob manages Jobs on a repeating time-based schedule, behaving like the traditional Linux crontab daemon. CronJobs use standard 5-field cron syntax (minute hour day-of-month month day-of-week).

# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of week (0 - 6) (Sunday to Saturday)
# │ │ │ │ │
# * * * * *

Concurrency Policies

When a CronJob's schedule triggers while a previous execution is still running, the concurrencyPolicy dictates how Kubernetes responds:

• Allow (default): Allows concurrent Job executions to run simultaneously.
• Forbid: Skips execution of the new Job if the previous Job has not completed.
• Replace: Cancels the currently running Job and starts a new Job in its place.

apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-backup
spec:
  schedule: "0 2 * * *"
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: custom-backup-tool:v1
            args: ['/scripts/backup.sh']
          restartPolicy: OnFailure

---

5. Workload Controller Comparison Matrix