9.2 Container Security: Image Scanning, Registries & Runtime Defense

9.2 Container Security: Image Scanning, Registries & Runtime Defense

Quick Answer: While virtual machines isolate workloads using hardware-level hypervisors running separate guest kernels, containers share the underlying host operating system kernel. Workload isolation is enforced through Linux kernel primitives: Namespaces (isolating what a process can see, including PID, NET, MNT, IPC, UTS, and User) and Control Groups (cgroups) (limiting what a process can consume, including CPU, memory, and process count). Because the kernel is shared, a single kernel flaw or misconfigured capability allows container breakout. Defense-in-depth mandates dropping Linux capabilities (e.g., CAP_SYS_ADMIN), applying seccomp syscall filters, adopting minimal base images (Google Distroless), enforcing cryptographic image signing (Cosign), locking registry tags to prevent mutation, and deploying eBPF-based runtime detection (Falco) to catch unauthorized system calls in real time.

Containerization has revolutionized software development and cloud operations. By packaging an application alongside its runtime dependencies, libraries, and binaries, containers provide lightweight, portable, and rapid application deployment across hybrid and multi-cloud environments. However, the fundamental architectural differences between virtual machine hypervisors and container engines create unique security challenges that feature prominently on the CCSK v5 exam.

Under CSA Security Guidance v5 (Domain 8), container security cannot be treated as a single checkpoint. It requires a continuous, multi-layered security strategy spanning the Build phase (supply chain hardening, vulnerability scanning, image signing), the Ship phase (private registry controls, admission policies), and the Run phase (kernel isolation primitives, resource constraints, and runtime behavioral monitoring).

---

Virtual Machines vs. Containers: The Core Isolation Difference

To understand container vulnerabilities, security architects must contrast how virtual machines and containers interact with physical compute resources:

┌──────────────────────────────────────┐     ┌──────────────────────────────────────┐
│         VIRTUAL MACHINE MODEL        │     │            CONTAINER MODEL           │
├──────────────────────────────────────┤     ├──────────────────────────────────────┤
│  ┌────────────┐      ┌────────────┐  │     │  ┌────────────┐      ┌────────────┐  │
│  │ App / Deps │      │ App / Deps │  │     │  │ App / Deps │      │ App / Deps │  │
│  ├────────────┤      ├────────────┤  │     │  └─────┬──────┘      └─────┬──────┘  │
│  │  Guest OS  │      │  Guest OS  │  │     │        │                   │         │
│  │   Kernel   │      │   Kernel   │  │     │        ▼                   ▼         │
│  ├────────────┴──────┴────────────┤  │     │  ┌────────────────────────────────┐  │
│  │    Hypervisor (Type 1 or 2)    │  │     │  │   Container Runtime (containerd)│  │
│  ├────────────────────────────────┤  │     │  ├────────────────────────────────┤  │
│  │   Host Hardware (CPU, RAM)     │  │     │  │    Shared Host Linux Kernel    │  │
│  └────────────────────────────────┘  │     │  ├────────────────────────────────┤  │
│  • Strong hardware isolation (VT-x)  │     │  │   Host Hardware (CPU, RAM)     │  │
│  • Separate kernel per VM            │     │  └────────────────────────────────┘  │
│  • Heavy footprint; slow boot time   │     │  • Shared kernel across all tenants  │
│  • High isolation boundary           │     │  • Lightweight; millisecond boot     │
│                                      │     │  • Kernel vulnerability threatens all│
└──────────────────────────────────────┘     └──────────────────────────────────────┘

• Virtual Machine Isolation: Type 1 hypervisors (e.g., KVM, Xen, ESXi) provide hardware virtualization. Each virtual machine runs its own dedicated guest operating system kernel. A privilege escalation vulnerability or kernel panic inside VM-A affects only VM-A; the hypervisor hardware boundary (enforced via CPU virtualization extensions like Intel VT-x or AMD-V) prevents access to neighboring VMs or the physical host.
• Container Isolation: Containers do not virtualize hardware and do not run independent kernels. All containers running on a worker node execute directly on the shared host Linux kernel. Container isolation is fundamentally a set of software boundaries maintained by the host kernel. If an attacker gains root privileges inside a poorly isolated container and exploits a vulnerability in the underlying host kernel, the attacker compromises the host and every other container executing on that machine.

---

Linux Kernel Isolation Primitives

Container runtimes (such as containerd, CRI-O, and Docker) rely on five foundational Linux kernel technologies to establish workload boundaries:

1. Linux Namespaces (Resource Visibility Boundaries)

Namespaces partition kernel resources so that each container process perceives an isolated instance of system resources. Linux implements several distinct namespaces:

• PID Namespace (Process ID): Isolates the process ID space. The primary container process views itself as PID 1 (the root init process inside the container), while on the host operating system, that same process is assigned a standard unprivileged PID (e.g., PID 42890). This prevents a container from observing or terminating processes running in other containers or on the host. Exam Danger: Never configure hostPID: true in production pod specifications, as this breaks PID isolation.
• NET Namespace (Network): Virtualizes network system resources, providing each container with its own virtual network loopback interface, IP routing tables, firewall rules (iptables/nftables), and socket lists. Exam Danger: Enabling hostNetwork: true attaches the container directly to the host's network namespace, allowing it to sniff host network traffic and bind to host ports.
• MNT Namespace (Mount): Isolates the filesystem mount points. The container sees only its own root filesystem (/) constructed from its container image layers and explicitly mounted volumes. Exam Danger: Mounting sensitive host directories (such as /var/run/docker.sock, /etc, or /) into a container completely invalidates mount isolation.
• IPC Namespace (Inter-Process Communication): Isolates IPC resources, including System V IPC message queues, semaphores, and POSIX shared memory. Processes in different IPC namespaces cannot communicate via shared memory.
• UTS Namespace (UNIX Timesharing System): Isolates the system hostname and NIS domain name, allowing each container to possess its own distinct hostname.
• User Namespace (UID/GID Mapping): Maps user and group IDs inside the container to different user and group IDs on the host. Most crucially, User Namespaces allow a container process to run as UID 0 (root) inside the container while being mapped to an unprivileged UID (e.g., UID 10001) on the host. If a process breaks out of the container filesystem, it possesses only unprivileged rights on the host, preventing host takeover.

2. Control Groups (cgroups): Resource Consumption Limits

While namespaces dictate what a process can see, Control Groups (cgroups) dictate what a process can consume. In multi-tenant cloud environments, an unconstrained container can consume all available CPU, memory, or disk I/O, causing a catastrophic Denial of Service for co-located workloads (the "noisy neighbor" syndrome).

• cgroups v1 vs. cgroups v2: Modern Linux distributions enforce cgroups v2, which provides a unified hierarchy for resource management and robust memory/IO throttling.
• Memory Limits (memory.max): Restricts the maximum physical RAM a container can consume. If a container exceeds its memory limit, the Linux kernel Out-Of-Memory (OOM) Killer terminates the container process (OOMKilled) rather than allowing the host node to crash.
• CPU Quotas (cpu.max): Restricts CPU bandwidth using Completely Fair Scheduler (CFS) quotas, ensuring containers cannot monopolize CPU cores.
• Process Count Limits (pids.max): Restricts the maximum number of concurrent processes a container can spawn. Enforcing a strict PID cgroup limit is the primary defense against Fork Bomb Denial of Service attacks, where malicious code spawns infinite subprocesses until the host kernel's process table is exhausted.

3. System Call Filtering via Seccomp

The Linux kernel exposes over 300 unique system calls (syscalls) allowing user-space applications to request kernel services (e.g., clone, fork, kill, mount, ptrace). Standard containerized applications (such as a Node.js API or Python microservice) require only a small fraction of these system calls (typically between 50 and 70) to function.

• Seccomp (Secure Computing Mode): A Linux kernel feature that intercepts and filters system calls executed by a process using Berkeley Packet Filter (BPF) rules.
• Default Seccomp Profiles: Modern container runtimes apply a default seccomp profile that blocks approximately 44 dangerous system calls by default, including:
  • reboot and kexec_load (prevents rebooting or replacing the host kernel).
  • sys_chroot and mount (prevents altering mount tables and bypassing chroot jails).
  • ptrace (prevents tracing and injecting code into other processes).
  • acct, settimeofday, stime (prevents altering system auditing and system clocks).
• Custom Least-Privilege Seccomp Profiles: In high-security environments, security teams record the exact syscalls utilized by an application in staging and generate a custom seccomp profile that enforces a strict whitelist, terminating any container process (SCMP_ACT_KILL) that invokes an unauthorized syscall.

4. Mandatory Access Control: AppArmor and SELinux

Standard Linux file permissions rely on Discretionary Access Control (DAC), which allows the owner of a file to modify permissions. Container runtimes augment DAC with Mandatory Access Control (MAC):

• AppArmor (Ubuntu, Debian): A path-based MAC framework that confines containers to specific file paths, network operations, and raw capabilities based on loaded profiles (e.g., docker-default). It blocks access to sensitive files under /proc and /sys regardless of whether the user is root.
• SELinux (RHEL, Rocky, Fedora): A label-based type enforcement MAC system. Under SELinux and sVirt, every process and file is assigned a security label (e.g., system_u:system_r:container_t:s0:c123,c456). Through Multi-Category Security (MCS), each container is assigned two unique, random categories (such as c123,c456). Even if a process achieves root privileges and breaks out of container namespaces, the SELinux kernel driver blocks access to host files or other containers because their MCS labels do not match.

5. Dropping Linux Capabilities (POSIX Capabilities)

Historically, Unix systems divided process permissions into a binary model: unprivileged users (UID > 0) or all-powerful superuser root (UID 0). Linux breaks down root privileges into approximately 40 discrete POSIX Capabilities.

When a container runs without restrictions, it inherits dangerous capabilities that an attacker can exploit to execute a container breakout:

• CAP_SYS_ADMIN: The ultimate "catch-all" capability. It permits mounting filesystems, modifying kernel parameters, and configuring cgroups. A container with CAP_SYS_ADMIN can trivially escape to the host.
• CAP_NET_RAW: Allows the construction of raw network packets and promiscuous sniffing, facilitating ARP poisoning and IP spoofing inside the container virtual network.
• CAP_SYS_PTRACE: Allows debugging and memory inspection of other processes.
• CAP_DAC_OVERRIDE: Bypasses all filesystem read, write, and execute permission checks.

[!IMPORTANT] The Least Privilege Capability Rule: In production container definitions, organizations must enforce the security directive: Drop ALL capabilities by default, and selectively add back only the specific capabilities strictly required for the workload.

securityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 10001
  capabilities:
    drop:
      - ALL
    add:
      - NET_BIND_SERVICE  # Allows binding to ports below 1024 if needed

---

Container Supply Chain Security: Base Images, Scanning & Signing

Compromised or vulnerable software packages embedded inside container images represent the most common entry vector for cloud-native breaches.

Minimal Base Images: Distroless vs. Full OS

Traditional container images were frequently constructed using full desktop or server operating system distributions (such as ubuntu:latest or debian:bullseye). These base images weigh hundreds of megabytes and contain hundreds of non-essential packages: shells (/bin/bash, /bin/sh), package managers (apt, dpkg), networking utilities (curl, wget, nc), and core utilities (tar, coreutils).

If an application running in such an image suffers a Remote Code Execution (RCE) vulnerability, the attacker immediately leverages /bin/sh to interact with the environment, uses curl to download cryptominers or reverse shell payloads, and uses package managers to install exploit utilities.

To neutralize this threat, modern supply chain engineering mandates Minimal Base Images:

• Alpine Linux: A lightweight Linux distribution (~5MB) built around musl libc and BusyBox. While small, it still contains a package manager (apk) and a minimal shell.
• Google Distroless Images: Container images that contain only the application and its runtime dependencies. They contain no package managers, no interactive shells (/bin/sh), and no OS utility binaries.
  • An attacker who exploits an RCE vulnerability in a Distroless Java or Go container cannot execute shell commands, cannot spawn an interactive terminal, and cannot run curl/wget. This drastically reduces the post-exploitation blast radius.
• Scratch Images: The ultimate minimal base (FROM scratch), representing an empty 0-byte filesystem. Used for statically compiled, self-contained binaries (such as Go or Rust applications) that require zero external shared libraries.

Static Vulnerability Scanning & Software Composition Analysis (SCA)

Container vulnerability scanning must be integrated natively into CI/CD build pipelines:

• OS-Level Vulnerability Scanning: Scans the operating system layers of the container image for known Common Vulnerabilities and Exposures (CVEs) registered in the National Vulnerability Database (NVD).
• Software Composition Analysis (SCA): Modern microservices depend heavily on third-party application language libraries (such as npm packages for Node.js, PyPI for Python, Maven for Java, and crates for Rust). SCA tools inspect package lockfiles (package-lock.json, pom.xml, requirements.txt) to identify vulnerable transitive dependencies.
• Software Bill of Materials (SBOM): Build pipelines must generate an automated, machine-readable inventory of all software components, libraries, and licenses embedded in the container image using open standards such as SPDX (Software Package Data Exchange) or CycloneDX. SBOMs enable rapid queries when zero-day vulnerabilities (e.g., Log4Shell) are disclosed.
• Automated Pipeline Quality Gates: Continuous integration pipelines must enforce strict security thresholds: any container build containing unpatched Critical or High severity CVEs (or vulnerabilities with known active exploits) must be automatically failed and blocked from proceeding to the registry.

Container Image Cryptographic Signing & Attestation (Sigstore / Cosign)

Vulnerability scanning is ineffective if an adversary can tamper with the image in transit or push an unauthorized image directly to production. Enterprises enforce Cryptographic Image Signing:

• Sigstore / Cosign: An open-source standard for container signing, verification, and provenance. Cosign signs container images using public-key cryptography or ephemeral, identity-based certificates (Keyless Signing via OIDC tokens, the Fulcio certificate authority, and the Rekor public transparency ledger).
• Supply-chain Levels for Software Artifacts (SLSA): Generates verifiable provenance attestations linking the compiled container image directly to its source Git repository, commit hash, and build environment, preventing tampering between source code and artifact release.

---

Private Registries & Admission Control Policies

Once an image is constructed and signed, it transitions to the Ship phase.

Private Container Registry Governance

Enterprises must never permit production environments to pull unvetted public images directly from public registries (such as Docker Hub). Production clusters must pull exclusively from secured, Private Enterprise Registries (e.g., AWS Elastic Container Registry [ECR], Azure Container Registry [ACR], Google Artifact Registry, or Harbor):

• Tag Immutability: In standard container registries, image tags (such as myapp:v1.2 or myapp:latest) are mutable pointers. An attacker with compromised credentials (or a rogue developer) can push a backdoored image with an existing tag name, overwriting the legitimate image. Production registries must enforce Immutable Image Tags, ensuring that once a tag is written, it can never be modified or overwritten.
• Continuous Registry Scanning: A container image deemed secure at the time of push may have new CVEs disclosed weeks later. Private registries must perform continuous asynchronous vulnerability scanning of stored images, automatically quarantining images that fall out of compliance.

Kubernetes Admission Controllers (OPA Gatekeeper & Kyverno)

How does a Kubernetes cluster guarantee that only approved, scanned, and signed images execute in production? Through Admission Controllers.

┌────────────────────────────────────────────────────────────────────────┐
│                     KUBERNETES ADMISSION CONTROL PIPELINE              │
├────────────────────────────────────────────────────────────────────────┤
│                                                                        │
│   [kubectl apply -f pod.yaml] ──► [Kube-apiserver]                     │
│                                          │                             │
│                                          ▼                             │
│                           ┌──────────────────────────────┐             │
│                           │ Authentication & AuthZ (RBAC)│             │
│                           └──────────────┬───────────────┘             │
│                                          │                             │
│                                          ▼                             │
│                           ┌──────────────────────────────┐             │
│                           │ Mutating Admission Webhook   │             │
│                           │ (Injects sidecars/proxies)   │             │
│                           └──────────────┬───────────────┘             │
│                                          │                             │
│                                          ▼                             │
│                           ┌──────────────────────────────┐             │
│                           │ Validating Admission Webhook │             │
│                           │ (OPA Gatekeeper / Kyverno)   │             │
│                           └──────────────┬───────────────┘             │
│                                          │                             │
│                  Evaluates Declarative Security Policies:              │
│                  1. Is image from approved private registry?           │
│                  2. Does image possess valid Cosign signature?         │
│                  3. Is runAsNonRoot set to true?                       │
│                  4. Are privileged containers blocked?                 │
│                                          │                             │
│                        ┌─────────────────┴─────────────────┐           │
│                      Pass                                Fail          │
│                        │                                   │           │
│                        ▼                                   ▼           │
│             ┌──────────────────────┐             ┌───────────────────┐ │
│             │ Persist to etcd      │             │ REJECT DEPLOYMENT │ │
│             │ & Schedule Pod       │             │ Return HTTP 403   │ │
│             └──────────────────────┘             └───────────────────┘ │
│                                                                        │
└────────────────────────────────────────────────────────────────────────┘

• Validating Admission Webhooks: When an API request to create or update a pod is submitted to the Kubernetes API server, it passes through validating webhooks before being committed to the etcd datastore.
• Policy Engines (OPA Gatekeeper / Kyverno): Declarative policy engines intercept pod manifests and evaluate them against organizational rules:
  • Rule 1: Images must originate strictly from enterprise.azurecr.io/* or 123456789.dkr.ecr.us-east-1.amazonaws.com/*.
  • Rule 2: The image signature must be verified against the corporate Cosign public key.
  • Rule 3: Pods configured with privileged: true, hostPID: true, or hostNetwork: true are instantly rejected.
  • Rule 4: Root execution (runAsNonRoot: false) or writable root filesystems (readOnlyRootFilesystem: false) are denied.

---

Runtime Container Defense: eBPF & Falco

Securing the build and deployment phases is critical, but zero-day vulnerabilities, application logic flaws, and compromised credentials can still lead to runtime compromise. In the Run phase, organizations must deploy Runtime Threat Detection.

The Failure of Traditional Antivirus on Containers

Legacy host-based antivirus (AV) and endpoint detection agents were designed for static virtual machines. They fail in containerized environments because:

• Containers are highly ephemeral, spinning up and terminating in seconds.
• Traditional agents run in user space and lack awareness of container namespaces, failing to map suspicious processes to specific pods or Kubernetes microservices.
• Traditional agents introduce high CPU overhead and memory footprint, making it impossible to run an independent agent inside every container container.

Extended Berkeley Packet Filter (eBPF)

Modern container runtime security is built upon eBPF (extended Berkeley Packet Filter). eBPF is a revolutionary Linux kernel technology that allows sandboxed, custom bytecode programs to execute directly inside the Linux kernel in response to kernel events and system calls, without modifying kernel source code or loading unstable third-party kernel modules.

• Zero Overhead & Safety: eBPF programs are verified for safety by the in-kernel eBPF verifier before execution (guaranteeing no infinite loops, crashes, or unauthorized memory access).
• Universal Observability: Because eBPF operates at the kernel layer, it observes every system call, process execution, file modification, and network packet across the entire worker node, regardless of which container namespace or cgroup generated it. It correlates kernel events directly with container IDs and Kubernetes pod metadata.

Behavioral Anomaly Detection via Falco

Falco (originally created by Sysdig and graduated under the Cloud Native Computing Foundation [CNCF]) is the de-facto open-source runtime security monitor for Kubernetes and Linux containers. Falco taps into kernel system call streams (using eBPF) and evaluates activity against a declarative rule set.

Falco generates high-priority security alerts when anomalous behaviors occur inside running containers, including:

1. Spawning an Interactive Shell: A production microservice container (e.g., an Nginx or payment processing pod) suddenly executes /bin/sh or /bin/bash.
2. Privilege Escalation Attempts: A container process attempts to modify ownership (chown), change permissions (chmod +s), or invoke unauthorized capability manipulation.
3. Unauthorized Network Connections: A container initiates an outbound connection on port 4444 or attempts to communicate with a known cryptocurrency mining pool or Command and Control (C2) IP address.
4. Sensitive File Access: Any process attempting to read /etc/shadow, write to /etc/pam.d, or read Kubernetes ServiceAccount tokens from /var/run/secrets/kubernetes.io/serviceaccount.
5. Writing to Non-Writable Directories: A process writing executable binaries into /bin, /usr/bin, or /lib.

Container Breakout Vectors & Defense Summary