9.3 Essential Collector Extensions

9.3 Essential Collector Extensions

Quick Answer: Collector extensions provide auxiliary runtime, diagnostic, and security capabilities without processing telemetry directly in the pipeline data path. Key operational extensions include health_check (HTTP port 13133 for Kubernetes liveness/readiness probes), pprof (port 1777 for Go CPU and memory profiling), zpages (port 55679 for live in-process pipeline diagnostics), and authentication extensions (oidc, basicauth, bearertokenauth). Extensions are declared under extensions: and must be explicitly activated in service.extensions: [...]. Note that the legacy memory_ballast extension is deprecated in favor of Go's native GOMEMLIMIT environment variable.

While receivers, processors, and exporters handle the direct ingestion, transformation, and delivery of telemetry data, running an OpenTelemetry Collector reliably in production requires operational infrastructure support. The Collector needs to report its readiness to container orchestrators, expose runtime profiling to engineers troubleshooting memory leaks, and authenticate incoming microservice connections. These auxiliary capabilities are provided by Extensions.

---

What are Collector Extensions?

Extensions are standalone components within the Collector runtime that provide auxiliary services, environment management, and diagnostics. Unlike processors or exporters, extensions never touch telemetry data directly:

• They do not receive pdata instances.
• They do not sit inside service.pipelines.
• They do not mutate or route spans, metrics, or logs.

Instead, extensions hook into the Collector's startup and shutdown lifecycle events to host HTTP/gRPC management servers, background monitors, or authentication providers.

+-------------------------------------------------------------------------+
|                        OpenTelemetry Collector                          |
|                                                                         |
|   +-----------------------------------------------------------------+   |
|   |                     Data Pipelines (In-Path)                    |   |
|   |  [Receivers]  ===>  [Processors]  ===>  [Exporters]             |   |
|   +-----------------------------------------------------------------+   |
|                                ^                                        |
|                                | Uses Auth / Lifecycle Events           |
|   +----------------------------+------------------------------------+   |
|   |                    Extensions (Out-of-Path)                     |   |
|   |  [health_check :13133]  [pprof :1777]  [zpages :55679]  [auth]  |   |
|   +-----------------------------------------------------------------+   |
+-------------------------------------------------------------------------+
        |                         |                  |           |
        v                         v                  v           v
 [K8s Kubelet Probes]     [Go Profiler CLI]    [Web Browser]  [IdP / JWT]

Activating Extensions

Like all Collector components, declaring an extension under the top-level extensions block only configures its parameters. To instantiate and run an extension, it must be explicitly added to the service.extensions array:

extensions:
  health_check:
    endpoint: 0.0.0.0:13133
  pprof:
    endpoint: 0.0.0.0:1777
  zpages:
    endpoint: 0.0.0.0:55679

service:
  extensions: [health_check, pprof, zpages] # Activates and starts extensions
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]

If an extension is omitted from service.extensions, its network listeners will not open, and dependent probes or authenticators will fail.

---

Key Operational Extensions

Four operational extensions matter most day to day. Know their default ports and diagnostic roles:

1. health_check Extension

The health_check extension provides an HTTP endpoint indicating whether the Collector process is operational. It is the primary mechanism used by Kubernetes liveness probes and readiness probes.

• Default Endpoint: localhost:13133 with path /. Because the default binds only to localhost, a Kubernetes probe (which calls the pod IP) needs endpoint: 0.0.0.0:13133 or the pod IP, and you may set path: /healthz.
• HTTP Status Codes:
  • 200 OK: Returned when the Collector has successfully completed initialization and all configured pipelines, receivers, processors, and exporters are operational.
  • 503 Service Unavailable: Returned if the Collector is still booting, if a fatal runtime error has disabled a core component, or if the Collector has entered its graceful shutdown sequence.

extensions:
  health_check:
    endpoint: 0.0.0.0:13133
    path: "/healthz"
# Do not rely on check_collector_pipeline: its README warns that it is not
# working as expected. The newer healthcheckv2 extension reports per-component status.

Kubernetes Integration Example

livenessProbe:
  httpGet:
    path: /healthz
    port: 13133
  initialDelaySeconds: 5
  periodSeconds: 10
readinessProbe:
  httpGet:
    path: /healthz
    port: 13133
  initialDelaySeconds: 3
  periodSeconds: 5

2. pprof Extension (Performance Profiling)

The pprof extension exposes Go runtime profiling data using Go's built-in net/http/pprof package. When high-throughput Collectors experience CPU saturation, thread starvation, or memory growth, pprof allows engineers to inspect runtime internals without restarting the process.

• Default Port: 1777
• Base Endpoint: /debug/pprof/
• Diagnostic Sub-Endpoints:
  • /debug/pprof/profile: Triggers a 30-second CPU sampling profile to identify CPU hotspots (such as aggressive regex matching or JSON serialization).
  • /debug/pprof/heap: Captures heap memory allocations to locate memory leaks and excessive memory allocation churn.
  • /debug/pprof/goroutine: Dumps stack traces of all active goroutines to detect blocked network calls or deadlocked channels.
  • /debug/pprof/allocs: Displays all past memory allocations since process startup.

# Inspecting a live Collector's heap using the Go pprof CLI
go tool pprof http://localhost:1777/debug/pprof/heap

3. zpages Extension (In-Process Web Diagnostics)

Originating from internal diagnostic infrastructure at Google, zpages serves live, in-process HTML diagnostic pages directly from the Collector memory space. It allows operators to debug pipeline issues instantly using a web browser without requiring external Prometheus servers or APM dashboards.

• Default Port: 55679
• Key Diagnostic Pages:
  • http://<host>:55679/debug/servicez: An overview of the Collector build and links to the other pages.
  • http://<host>:55679/debug/pipelinez: Each running pipeline's type, whether it mutates data, and its receivers, processors, and exporters. It shows composition, not throughput counters; use the internal metrics (Section 13.1) for counts.
  • http://<host>:55679/debug/extensionz: The active extensions.
  • http://<host>:55679/debug/featurez: The feature gates and their current status.
  • http://<host>:55679/debug/tracez: Spans created inside the Collector, grouped into latency buckets (such as 0–10 µs, 10–100 µs, up to 1 minute) with sampled error spans.

extensions:
  zpages:
    endpoint: 0.0.0.0:55679

4. memory_ballast (DEPRECATED) vs Modern GOMEMLIMIT

Older configurations still contain the historical memory_ballast extension, so know its modern replacement:

• The Historical Problem: In Go runtimes prior to Go 1.19, the garbage collector (GC) triggered whenever the live heap doubled in size relative to the previous GC baseline. In a Collector with a small live heap (e.g., 50MB), allocating another 50MB would trigger a GC cycle, causing frequent CPU thrashing under high telemetry ingestion rates.
• The Ballast Hack: The memory_ballast extension pre-allocated a large, contiguous virtual memory slice (e.g., 1GB) on startup that was never written to. This artificially raised the GC baseline, delaying GC cycles until an additional 1GB was allocated and drastically reducing CPU usage.
• Modern Deprecation: Go 1.19 introduced the native GOMEMLIMIT environment variable. With GOMEMLIMIT, the Go runtime scheduler monitors the exact container memory boundary and dynamically tunes GC pacing to maximize throughput while preventing out-of-memory crashes. Consequently, the memory_ballast extension was deprecated and then removed in Collector v0.109.0.

---

Authentication Extensions

When Collectors are deployed as shared gateways or multi-tenant proxies, they must authenticate incoming telemetry requests from microservices and authenticate outgoing requests to remote backends. Authentication extensions provide this security capability by attaching directly to receivers and exporters.

[Client Microservice] 
       |
       | (HTTP/gRPC with 'Authorization: Bearer <JWT>')
       v
[otlp receiver]  <==== Validates Token ====> [oidc extension]
       |                                            |
       | (Approved Telemetry)                       v
 [Data Pipeline]                              [Identity Provider (IdP)]
       |
       v
[otlp exporter]  <==== Injects Basic Auth === [basicauth extension]
       |
       v (HTTP with 'Authorization: Basic <Creds>')
[Remote Secure Backend]

1. oidc (OIDC Authenticator)

The oidc extension (from Contrib) validates incoming Bearer JWT tokens against an OpenID Connect (OIDC) identity provider (such as Keycloak, Okta, Microsoft Entra ID, or Google Cloud IAM). It verifies token signatures, expiration timestamps (exp), allowed issuers (iss), and target audience claims (aud).

extensions:
  oidc:
    issuer_url: https://auth.production.example.com
    audience: otel-collector-gateway

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
        auth:
          authenticator: oidc # Binds OIDC validation to OTLP gRPC
      http:
        endpoint: 0.0.0.0:4318
        auth:
          authenticator: oidc # Binds OIDC validation to OTLP HTTP

service:
  extensions: [oidc]

2. basicauth and bearertokenauth

• basicauth: Supports HTTP Basic authentication (username and password). It can function as a server authenticator on receivers (validating incoming credentials) or as a client authenticator on exporters (injecting credentials into outgoing requests).
• bearertokenauth: Injects static or file-based Bearer tokens into outgoing exporter headers, commonly used for authenticating to SaaS observability backends.

---

Top Collector Extensions Summary Table

The following table summarizes the most common OpenTelemetry Collector extensions, their default ports, primary endpoints, and production best practices: