8.3 Dispatcher Caching Diagnostics, X-Cache-Info & Local Validation

8.3 Dispatcher Caching Diagnostics, X-Cache-Info & Local Validation

Core Principle: The AEM Dispatcher is the critical caching and security layer standing between end users and the AEM publish tier. A misconfigured Dispatcher leads to catastrophic publish tier overload, stale content delivery, or security vulnerabilities. Diagnosing caching failures requires mastering the X-Cache-Info HTTP response header, analyzing dispatcher.log decision trails, understanding invalidation scopes (statfileslevel), and validating Apache configurations locally using the AEM as a Cloud Service Dispatcher SDK.

---

1. Dispatcher Diagnostic Headers & X-Cache-Info

When troubleshooting caching behavior, developers inspect HTTP response headers emitted by the Dispatcher web server module (mod_dispatcher.so).

$ curl -I https://wknd.site/us/en/adventures.html
HTTP/1.1 200 OK
Date: Wed, 23 Sep 2026 14:30:15 GMT
Server: Apache/2.4.58 (Unix)
Server-Agent: Communique-Dispatcher
X-Cache-Info: caching disabled; response is not cacheable
Age: 0
Content-Type: text/html;charset=UTF-8

Key Diagnostic Response Headers

• Server-Agent: Communique-Dispatcher: Confirms that the HTTP response was routed through an active AEM Dispatcher module rather than hitting the backend AEM Jetty servlet engine directly.
• X-Cache-Info: Emitted by Dispatcher when debugging headers are enabled or in development/staging environments. Explains the exact reason why a document was cached, missed, or rejected.

Deciphering X-Cache-Info Status Values

---

2. Dispatcher Logging & Decision Analysis

To diagnose caching decisions in detail, configure the Dispatcher log level to debug in the Apache virtual host or Dispatcher farm configuration:

# In Apache httpd.conf / virtualhost configuration:
LogLevel dispatcher:debug

Anatomy of dispatcher.log Entries

1. Filter Rule Evaluation

[Wed Sep 23 14:30:15 2026] [D] [pid 1024] Checking filter rules for [/content/wknd/us/en.html]
[Wed Sep 23 14:30:15 2026] [D] [pid 1024] Filter rule [15] matches: /0015 { /type "allow" /url "/content/*" }
[Wed Sep 23 14:30:15 2026] [D] [pid 1024] Filter: ALLOW

2. Cache Lookup & Stat File Invalidation

[Wed Sep 23 14:30:15 2026] [D] [pid 1024] Found cached file [/mnt/var/www/html/content/wknd/us/en.html]
[Wed Sep 23 14:30:15 2026] [D] [pid 1024] Cache file is older than .stat file [/mnt/var/www/html/content/wknd/.stat]
[Wed Sep 23 14:30:15 2026] [I] [pid 1024] Cache-action: REFETCH from render [publish:4503]

3. Bypassing Cache Due to URL Parameters

[Wed Sep 23 14:30:15 2026] [D] [pid 1024] Query string present [utm_source=email]
[Wed Sep 23 14:30:15 2026] [D] [pid 1024] /ignoreUrlParams denied query string -> BYPASS CACHE

---

3. Diagnosing Chronic Cache Misses

When publish instances suffer high CPU or latency, chronic Dispatcher cache misses are one important cause. Four primary architectural factors cause unintended cache misses:

1. Uncacheable Query Parameters & /ignoreUrlParams

By default, Dispatcher considers any URL containing a query string (?key=value) to be dynamic and bypasses caching entirely.

Incoming Request: https://wknd.site/us/en.html?utm_source=newsletter&utm_medium=email
       |
       v
Dispatcher evaluates /ignoreUrlParams
       |
       +--> Parameter NOT in /ignoreUrlParams allow list?
       |         |
       |         v
       |    BYPASS CACHE -> Forwards request to AEM Publish
       |
       +--> Parameter IS in /ignoreUrlParams allow list?
                 |
                 v
            SERVE FROM CACHE (Returns /mnt/var/www/html/content/wknd/us/en.html)

The Solution: Configuring /ignoreUrlParams In conf.dispatcher.d/cache/default_invalidate.any (or the farm's /cache block), explicitly allow marketing and analytics parameters:

/ignoreUrlParams {
    /0001 { /glob "*" /type "deny" }
    /0002 { /glob "utm_source" /type "allow" }
    /0003 { /glob "utm_medium" /type "allow" }
    /0004 { /glob "utm_campaign" /type "allow" }
    /0005 { /glob "gclid" /type "allow" }
    /0006 { /glob "fbclid" /type "allow" }
}

2. Authorization Headers & Session Cookies (/allowAuthorized)

When a user logs in, the browser sends an Authorization header or a session cookie (e.g. login-token).

• Default Behavior: Dispatcher detects authentication credentials and refuses to cache the response to avoid caching user-specific data into a public shared cache.
• The Setting: /allowAuthorized:
  • /allowAuthorized "0" (Default): Any request with an Authorization header bypasses the cache.
  • /allowAuthorized "1": Dispatcher caches responses even if authorization headers are present.
• Production Rule: Set /allowAuthorized "1" only for public, non-personalized content trees, or when using Sling Dynamic Include (SDI) to load user-specific fragments via AJAX or SSI/ESI.

3. Invalidation Scope & statfileslevel

When an author publishes a page, AEM's replication agent sends an HTTP POST flush request to Dispatcher. Dispatcher touches the applicable .stat file. Any cached file with a timestamp older than the .stat file timestamp is treated as expired.

$\mathbf{statfileslevel} \text{ defines the directory depth at which .stat files are created and evaluated. }$

Document Root: /mnt/var/www/html/
├── .stat                               (statfileslevel 0: Touched on ANY publish)
└── content/
    └── wknd/
        ├── .stat                       (statfileslevel 2)
        ├── us/
        │   └── en/
        │       ├── .stat               (statfileslevel 3: Touched on /us/en publish)
        │       └── adventure.html
        └── fr/
            └── fr/
                ├── .stat               (statfileslevel 3: Untouched when /us/en publishes!)
                └── adventure.html

Design Guidance: Choose statfileslevel from the actual cache path hierarchy and desired invalidation scope. Test activations across sibling sites and languages; no single level is correct for every structure.

4. HTTP Cache-Control Headers Emitted by AEM

If a custom Sling Servlet, Filter, or Sling Model outputs Cache-Control: private, no-store, no-cache or Pragma: no-cache, Dispatcher honors these directives and refuses to write the response to disk. Use browser DevTools or curl -I to verify that AEM publish returns Cache-Control: max-age=... or standard public headers.

---

4. Local Validation with AEM as a Cloud Service Dispatcher SDK

In AEM as a Cloud Service, Dispatcher configurations are committed directly to Git and deployed via Cloud Manager pipelines. To prevent deployment pipeline failures, Adobe provides the Dispatcher SDK to validate configurations locally before committing.

SDK Structure & Core Scripts

dispatcher-sdk/
├── bin/
│   ├── validate.sh      <-- Validates Apache syntax, include directives, and file structure
│   └── docker_run.sh    <-- Runs local Apache + Dispatcher in Docker against local AEM
└── src/
    └── conf.dispatcher.d/

Phase 1: Static Validation (bin/validate.sh)

The validate.sh script executes automated linting on the configuration tree:

./bin/validate.sh src/

Validation Checks Performed:

• Apache Syntax: Runs httpd -t to ensure all rewrite rules, virtual hosts, and module directives are syntactically valid.
• Directive Whitelisting: Verifies that no forbidden Apache directives are declared (e.g. AEMaaCS prohibits ServerName, Listen, and certain unauthorized modules).
• Include Structure: Confirms that custom files are included from designated include points (conf.d/enabled_vhosts/*.vhost, conf.dispatcher.d/enabled_farms/*.farm).

Phase 2: Local Docker Execution (bin/docker_run.sh)

Once validation passes, developers execute the local Dispatcher container in Docker, proxying to a local AEM publish instance:

# Syntax: bin/docker_run.sh <config-dir> <aem-publish-host:port> <dispatcher-port>
./bin/docker_run.sh src host.docker.internal:4503 8080

Operational Benefits:

• Spins up an exact replica of the Cloud Service Apache 2.4 + Dispatcher Linux container.
• Directs incoming browser requests at http://localhost:8080 through local Dispatcher caching logic to local AEM publish (4503).
• Generates live container logs directly in the terminal, revealing filter hits, cache creations, and invalidations.

Phase 3: Cloud Manager Pipeline Linting

During Cloud Manager CI/CD pipeline execution, an automated build step runs validate.sh. If any syntax error, broken symlink, or forbidden directive is detected, the pipeline immediately fails at the code quality gate, preventing broken configurations from reaching staging or production.

---

5. Diagnostic Decision Matrix