2.2 Container Hardening, Signals, and Layer Caching

Minimal Base Images: Distroless vs. Alpine

Selecting the runtime base image directly dictates container security and runtime stability:

Why Distroless is the Production Gold Standard

Google's Distroless container images contain only your application and its direct runtime dependencies (such as CA certificates, timezone data, and necessary shared libraries). They lack shells and package managers.

If an attacker discovers a Remote Code Execution (RCE) flaw in your application code, they cannot invoke /bin/sh to execute shell commands, download malicious scripts via curl or wget, or install reconnaissance packages using apt or apk.

---

Security Hardening and Non-Root Execution

A container image with no configured USER commonly runs as root (UID 0); base-image and platform settings can select another user. Running containers as root violates the Principle of Least Privilege and introduces severe container breakout risks:

1. Container Breakout Vulnerabilities: If a kernel vulnerability exists, a process running as root inside the container could potentially gain host-level root privileges on Compute Engine or GKE Standard nodes.
2. Enforcing Non-Root Execution: Always declare a non-root user in your Dockerfile using the USER instruction or specify an explicit UID (e.g., USER 10001:10001). In Distroless, use the built-in USER nonroot (UID 65532).
3. GKE Pod Security Standards: GKE Autopilot applies managed security restrictions, primarily based on the Baseline Pod Security Standard with selected additional restrictions and partner exceptions. It does not universally prohibit root. Explicit non-root settings and Pod Security Admission policies provide additional protection. This Deployment illustrates those settings:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-processor
spec:
  replicas: 3
  selector:
    matchLabels:
      app: payment-processor
  template:
    metadata:
      labels:
        app: payment-processor
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        runAsGroup: 10001
        fsGroup: 10001
      containers:
      - name: payment-api
        image: us-central1-docker.pkg.dev/my-project/apps/payment-api:v2
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
          capabilities:
            drop:
            - ALL

---

PID 1, Signal Handling, and Graceful Shutdown

In Linux, the process with Process ID 1 (PID 1) is the init system. PID 1 has special responsibilities: it handles signal propagation and adopts orphaned child processes to prevent zombie process leaks.

The Shell Form vs. Exec Form Trap

A frequent source of production outages in Cloud Run and GKE is using the shell form instead of the exec form in CMD or ENTRYPOINT:

• Shell Form (CMD node server.js or ENTRYPOINT npm start): Docker runs the command as /bin/sh -c "node server.js". The shell process (sh) becomes PID 1, and node runs as a child process. Standard Linux shells do not forward OS signals (such as SIGTERM) to child processes!
• Failure Mode: When Cloud Run or GKE scales down or terminates a container, the platform sends a SIGTERM signal. Because /bin/sh does not forward the signal, the application never knows it is shutting down. Inflight HTTP requests are dropped, database transactions are cut off, and after the platform grace period (e.g., 10 to 30 seconds), the platform issues an uncatchable SIGKILL, abruptly crashing the workload.
• Exec Form (ENTRYPOINT ["node", "server.js"]): The application binary executes directly as PID 1, receives the SIGTERM signal immediately, stops accepting new connections, drains existing requests, and exits cleanly.

Shell Form (BROKEN):
  Platform SIGTERM ---> [ PID 1: /bin/sh ]  (Signal NOT forwarded!)
                               |
                        [ PID 2: Node.js ] (Never receives SIGTERM -> Hard killed by SIGKILL!)

Exec Form (CORRECT):
  Platform SIGTERM ---> [ PID 1: Node.js ] (Catches SIGTERM -> Drains requests -> Clean Exit)

Implementing Graceful Shutdown in Application Code

// Node.js Express Graceful Shutdown Implementation
const server = app.listen(process.env.PORT || 8080);

process.on('SIGTERM', () => {
  console.log('SIGTERM received. Starting graceful shutdown...');
  
  // Stop accepting new incoming requests
  server.close(async () => {
    console.log('HTTP connections drained. Closing database pools...');
    await dbPool.end();
    console.log('Shutdown complete. Exiting.');
    process.exit(0);
  });

  // Force shutdown if connections do not drain within 10 seconds
  setTimeout(() => {
    console.error('Forced shutdown: connections failed to drain in time.');
    process.exit(1);
  }, 10000);
});

---

Layer Ordering and Build Caching in Cloud Build

Docker builds images in sequential layers. When building an image, Docker and Cloud Build check if each instruction matches a previously cached layer. If any layer changes, all subsequent layers are invalidated and must be rebuilt.

Anti-Pattern: Premature Source Copy

## INEFFICIENT: Source code copied before installing packages
FROM node:24-alpine
WORKDIR /app
COPY . .                 # Any source code edit invalidates this layer!
RUN npm install          # Runs npm install from scratch on EVERY build (slow!)
CMD ["node", "server.js"]

Optimized Pattern: Cache Dependency Layers First

## EFFICIENT: Manifests copied and cached independently
FROM node:24-alpine
WORKDIR /app
COPY package.json package-lock.json ./ # Cached unless dependencies change
RUN npm ci --omit=dev                  # Reused from cache on code edits
COPY src/ ./src/                       # Only this layer rebuilds on code edits
USER node
CMD ["node", "src/server.js"]

---

Common Container Anti-Patterns

1. Using the :latest Tag in Production: Tags are mutable pointers. Deploying :latest means different container instances may pull different image revisions during horizontal autoscaling. Always deploy using immutable SHA256 image digests (e.g., image@sha256:abc...); semantic version tags also require enforced immutability to provide the same protection.
2. Baking Secrets into Image Layers: Storing API keys or certificates in ENV instructions or local files embeds secrets into the image layer history. Anyone with image read access can extract them via docker history or container inspection.
3. Running Multiple Unrelated Services per Container: Running an NGINX proxy, an SSH daemon, and a Python app in one container complicates process supervision, breaks health checks, and hampers horizontal scaling.

Official references checked October 18, 2026: Google Cloud 1, Google Cloud 2.