7.3 Standard Exporters & OTLP Wire Protocols

7.3 Standard Exporters & OTLP Wire Protocols

Quick Answer: The OpenTelemetry Protocol (OTLP) is the standardized, vendor-neutral wire protocol designed specifically for transmitting traces, metrics, and logs with maximum efficiency and zero semantic loss. OTLP defines two primary transport bindings: OTLP/gRPC (running binary Protocol Buffers over HTTP/2 on default port 4317) and OTLP/HTTP (running binary Protobuf or JSON over HTTP/1.1 or HTTP/2 on default port 4318). OTLP/HTTP routes signals to explicit endpoint paths: /v1/traces, /v1/metrics, and /v1/logs. In addition to OTLP, the SDK provides standard built-in exporters such as ConsoleExporter (local stdout debugging) and InMemoryExporter (isolated unit test assertions).

Before OpenTelemetry established OTLP, the observability landscape suffered from extreme protocol fragmentation. Distributed tracing relied on Zipkin (JSON/Thrift over HTTP) or Jaeger (Thrift over UDP or gRPC). Metrics relied on Prometheus text exposition or OpenMetrics over HTTP. Logs were scattered across Syslog, fluent forwarders, and proprietary JSON lines. Each protocol had distinct data structures, preventing cross-signal correlation and forcing developers to run multiple agents and translation proxies.

OTLP solved this by providing a unified, high-performance data model built on Protocol Buffers v3 (proto3).

---

OTLP Transport Bindings & Default Ports

Default ports, transports, URL paths, and content types are worth memorizing:

1. OTLP/gRPC (Port 4317)

OTLP/gRPC is widely used for service-to-Collector and Collector-to-Collector traffic. (For SDK exporters, the specification's default protocol is http/protobuf; some SDKs keep grpc as their default for backward compatibility.)

• HTTP/2 Transport: Operates over HTTP/2, enabling connection multiplexing (multiple concurrent trace, metric, and log export streams over a single long-lived TCP connection), binary framing, and header compression (HPACK).
• Performance: Binary Protocol Buffers provide compact serialization and rapid deserialization with minimal CPU overhead.
• RPC Semantics: Exports are defined as standard gRPC unary remote procedure calls:
  • Traces: opentelemetry.proto.collector.trace.v1.TraceService/Export
  • Metrics: opentelemetry.proto.collector.metrics.v1.MetricsService/Export
  • Logs: opentelemetry.proto.collector.logs.v1.LogsService/Export
• Status Codes: Uses standard gRPC status codes (OK, UNAVAILABLE, RESOURCE_EXHAUSTED, DEADLINE_EXCEEDED).

2. OTLP/HTTP (Port 4318)

While gRPC is extraordinarily performant, many corporate environments restrict or block HTTP/2 gRPC traffic at network firewalls, corporate forward proxies, Web Application Firewalls (WAFs), Content Delivery Networks (CDNs), or serverless API gateways. For these environments, OpenTelemetry provides OTLP/HTTP on default port 4318.

Explicit URL Endpoint Paths

Unlike gRPC which invokes service methods, OTLP/HTTP mandates specific URL endpoints based on the telemetry signal:

• Traces: http[s]://<host>:<port>/v1/traces
• Metrics: http[s]://<host>:<port>/v1/metrics
• Logs: http[s]://<host>:<port>/v1/logs

When a developer configures a generic base endpoint (e.g., OTEL_EXPORTER_OTLP_ENDPOINT="http://collector:4318"), the respective signal exporters automatically append the required /v1/traces, /v1/metrics, or /v1/logs paths. If an engineer specifies a signal-specific variable (e.g., OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="http://collector:4318/v1/traces"), the SDK uses that exact path without modification.

Payload Encodings: Protobuf vs. JSON

OTLP/HTTP supports two payload encodings:

1. application/x-protobuf (Binary Protobuf, recommended default): The payload is serialized as raw binary Protocol Buffers. It offers nearly the same wire compression and CPU efficiency as gRPC while traversing standard HTTP/1.1 infrastructure.
2. application/json (Protobuf JSON Mapping): The payload is serialized as JSON formatted according to the canonical Protobuf-to-JSON mapping. While less efficient (noticeably larger on the wire and more expensive to parse), it is human-readable and invaluable for browser-based tracing (OpenTelemetry JavaScript in web clients) and network packet inspection.

---

Standard Non-OTLP Exporters

In addition to OTLP exporters, OpenTelemetry language SDKs package standard built-in exporters for testing, development, and legacy migration:

+-----------------------------------------------------------------------------------+
|                         Standard OpenTelemetry Exporters                          |
+-----------------------------------------------------------------------------------+
|  OTLP Exporters (gRPC & HTTP)    --> Production telemetry transport to Collector   |
|  ConsoleExporter (Stdout)        --> Local development, debugging, and sanity checks|
|  InMemoryExporter                --> Automated CI unit and integration testing     |
|  PrometheusExporter              --> Pull-based metric scraping on /metrics        |
|  Legacy Exporters (Zipkin/Jaeger)--> Migration support (Jaeger natively uses OTLP) |
+-----------------------------------------------------------------------------------+

1. ConsoleExporter (Stdout)

• Prints serialized telemetry records directly to standard output (stdout or stderr) in human-readable text or JSON formatting.
• Use Cases: Rapid local development, verifying that auto-instrumentation is generating spans, tutorial demonstrations.
• Production Anti-Pattern: Never deploy ConsoleExporter to high-throughput production environments. Synchronously printing thousands of spans to stdout causes heavy CPU overhead, disk I/O contention on host log managers, and severe log pollution.

2. InMemoryExporter

• Stores finished spans, metrics, or log records directly in an in-memory thread-safe array or list within the application process runtime.
• Provides programmatic inspection methods (e.g., exporter.get_finished_spans(), exporter.clear()).
• Use Cases: Automated unit and integration test suites. In a CI/CD pipeline, an engineer can write tests asserting that invoking an order checkout API generates a span named process_order with attribute order.id = 1042 and StatusCode.OK, without spinning up an external network collector or mock HTTP server.

3. PrometheusExporter / PrometheusMetricReader

• Implements the pull-based scrape interface described in Section 7.2. It converts internal SDK metric accumulators into Prometheus text or OpenMetrics format upon receiving an HTTP GET scrape request.

4. Legacy Exporters (Zipkin & Jaeger)

• ZipkinExporter: Serializes spans into Zipkin v2 JSON format and posts them to Zipkin collectors (typically on port 9411).
• JaegerExporter: Originally exported spans using Jaeger Thrift over UDP or gRPC. Note for Exam: Modern OpenTelemetry SDKs have deprecated native Jaeger exporters because Jaeger natively accepts OTLP on ports 4317 and 4318. Telemetry should be sent to Jaeger using standard OTLP exporters.

---

Security, Authentication, & Compression in Exporters

Production telemetry pipelines must meet enterprise security standards for encryption, authentication, and bandwidth optimization.

1. Transport Encryption: TLS and mTLS

By default, local development uses insecure plaintext connections (insecure = true or http://). In production environments crossing VPCs or traversing the public internet, TLS is mandatory:

• Standard TLS: The client verifies the collector's certificate authority (CA) certificate to ensure encryption and identity (ca_file or certificate).
• Mutual TLS (mTLS): In zero-trust networks, the collector also validates the application client. The SDK must be configured with a client certificate (cert_file or client_certificate) and private key (key_file or client_key). The standard environment variables are OTEL_EXPORTER_OTLP_CERTIFICATE (trusted CA), OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE, and OTEL_EXPORTER_OTLP_CLIENT_KEY, each with per-signal variants.

2. Authentication Headers

When sending telemetry to a managed SaaS backend or authenticated collector gateway, the exporter must inject authentication tokens into outgoing RPC headers:

• Bearer Tokens / API Keys: Configured in code or via standard environment variables:

export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer eyJhbGciOi...,x-honeycomb-team=secret_key"

Signal-specific headers can also be defined:

export OTEL_EXPORTER_OTLP_TRACES_HEADERS="x-tenant-id=tenant-alpha"

3. Payload Compression

Telemetry data—especially traces and logs—contains highly repetitive strings (attribute keys, resource names, URL paths, error messages). Compressing payloads dramatically reduces cloud egress costs and bandwidth usage:

• Supported Compression: The OpenTelemetry specification standardizes on gzip compression (as well as none).
• Configuration:

export OTEL_EXPORTER_OTLP_COMPRESSION="gzip"

Because telemetry is highly repetitive, gzip usually shrinks payloads substantially for a modest CPU cost; measure the ratio on your own data.

---

Comparison: OTLP/gRPC vs. OTLP/HTTP (Protobuf) vs. OTLP/HTTP (JSON)

---

SDK Environment Variable Reference for Exporters

The OpenTelemetry specification guarantees uniform configuration across all programming languages via standard environment variables:

+-----------------------------------------------------------------------------------+
|                         Core OTLP Environment Variables                           |
+-----------------------------------------------------------------------------------+
|  OTEL_EXPORTER_OTLP_ENDPOINT        --> Base collector URL (e.g., http://otel:4317)|
|  OTEL_EXPORTER_OTLP_PROTOCOL        --> 'grpc', 'http/protobuf', or 'http/json'   |
|  OTEL_EXPORTER_OTLP_HEADERS         --> Global key-value headers (Auth tokens)    |
|  OTEL_EXPORTER_OTLP_TIMEOUT         --> Global timeout in milliseconds (default 10s|
|  OTEL_EXPORTER_OTLP_COMPRESSION     --> Compression algorithm ('gzip' or 'none')  |
+-----------------------------------------------------------------------------------+
|                       Signal-Specific Override Variables                          |
+-----------------------------------------------------------------------------------+
|  OTEL_EXPORTER_OTLP_TRACES_ENDPOINT  --> e.g., http://otel:4318/v1/traces          |
|  OTEL_EXPORTER_OTLP_METRICS_ENDPOINT --> e.g., http://otel:4318/v1/metrics         |
|  OTEL_EXPORTER_OTLP_LOGS_ENDPOINT    --> e.g., http://otel:4318/v1/logs            |
+-----------------------------------------------------------------------------------+