6.1 OpenTelemetry Logs Data Model & Architecture

6.1 OpenTelemetry Logs Data Model & Architecture

Quick Answer: The OpenTelemetry Logs Data Model defines a universal, vendor-neutral structure for representing log records across distributed systems. Unlike legacy logging where logs were isolated strings written to disk or stdout, an OpenTelemetry Log Record standardizes precise event and observed timestamps, distributed trace correlation identifiers (TraceId, SpanId, TraceFlags), a 24-point normalized numerical severity scale (SeverityNumber), extensible key-value attributes, and structured AnyValue body payloads, all contextualized by shared Resource metadata.

Logs are the oldest and most widespread telemetry signal in software engineering. Decades before distributed tracing or standardized metric time-series emerged, developers diagnosed malfunctioning applications using print statements, local text files, and operating system syslog daemons. However, this longevity has made logging the most challenging signal to standardize across modern enterprise environments.

---

The Evolution of Logs in OpenTelemetry

When distributed tracing and metric collection emerged in the cloud-native ecosystem, projects like OpenTracing, OpenMetrics, and OpenCensus established clean, modern specifications unburdened by legacy technical debt. Traces and metrics were adopted in greenfield architectures or standardized through modern client libraries. Logs, by contrast, presented a fundamentally different problem:

1. Decades of Entrenched Ecosystems — Organizations possess hundreds of microservices, third-party libraries, and legacy monolithic applications already integrated with language-specific logging frameworks (such as Log4j, Logback, and SLF4J in Java; Winston and Pino in Node.js; logging and Structlog in Python; Zap and Zerolog in Go; and Serilog in .NET).
2. Incompatible Transport and Storage Standards — Infrastructure components rely on Syslog (RFC 3164 and RFC 5424), Windows Event Logs, journald, or plain text streams routed through heterogeneous log shippers like Fluentd, Fluent Bit, Logstash, Promtail, and Vector.
3. Arbitrary Severity and Field Naming — One framework logs errors as "CRITICAL" with integer level 50, another logs "FATAL", while Syslog defines Emergency (severity 0) and Alert (severity 1). Timestamps vary from Unix epoch milliseconds to microsecond ISO-8601 strings or locale-dependent textual dates.

The OpenTelemetry Strategy for Logs

Recognizing that asking the global software industry to rewrite millions of existing logging statements was impossible, the OpenTelemetry project adopted a pragmatic, non-invasive design strategy:

• Do not replace existing logging APIs — Developers should not abandon their idiomatic logging frameworks.
• Define a unified universal data model — Create a standardized in-memory and on-the-wire representation for log records that bridges existing frameworks into modern observability pipelines.
• Correlate logs with traces and metrics — Transform logs from isolated textual records into fully integrated nodes within the telemetry graph by embedding distributed trace context (TraceId, SpanId) and shared Resource attribution (service.name, k8s.pod.name).

---

The OpenTelemetry Log Record Data Model

The OpenTelemetry specification defines a Log Record as a discrete event at a specific point in time. All of its fields are optional:

Detailed Analysis of Critical Fields

1. Timestamp vs. ObservedTimestamp

A critical distinction is the relationship between Timestamp and ObservedTimestamp:

• Timestamp represents the event generation time recorded by the application runtime. If an application executes logger.info("User logged in"), the runtime captures the system clock at that microsecond.
• ObservedTimestamp represents the moment OpenTelemetry code first observed the event. For a record created through the OpenTelemetry SDK or a log bridge, that is normally the generation time, so it usually equals Timestamp. For a log collected from outside OpenTelemetry, for example a file line read by the Collector's filelog receiver, it is the time the receiver read the line.

The two timestamps diverge when logs are collected late. Suppose devices or legacy hosts write plain log files while offline and the files are only read by a Collector hours later. Timestamp (parsed from each log line) reflects when the event occurred, while ObservedTimestamp reflects when the Collector read it. Distinguishing these two timestamps prevents index corruption, supports accurate out-of-order log analysis, and ensures that pipeline lag can be measured precisely (ObservedTimestamp - Timestamp = Telemetry Lag).

2. Trace and Span Correlation Identifiers

OpenTelemetry logs are not isolated silos. When an application logs a message inside an active distributed span, the logging integration automatically extracts TraceId, SpanId, and TraceFlags from the ambient context. This enables bi-directional telemetry correlation:

• Trace-to-Log Navigation: An engineer viewing a slow span in a distributed tracing UI can click a single button to view all log messages emitted during that exact span's execution lifecycle.
• Log-to-Trace Navigation: An engineer searching error logs in a central log management dashboard can immediately jump from an unhandled exception log directly into the end-to-end distributed trace DAG (Directed Acyclic Graph) to see what upstream caller triggered the failure.

3. The Body as an AnyValue

Legacy logging assumed log bodies were plain text strings (e.g., "Order 4092 failed for customer 108"). Modern logging engines emit deeply nested structured JSON. OpenTelemetry accommodates both paradigms without compromise by defining Body as an AnyValue type:

• stringValue: Simple human-readable string messages.
• int64Value / doubleValue / boolValue: Primitive numeric or boolean payloads.
• bytesValue: Raw binary data payloads.
• arrayValue: Ordered collections of AnyValue items.
• kvlistValue: Nested key-value maps, enabling fully structured, tree-like JSON payloads to be ingested natively without converting them into serialized strings.

---

Categorization of Logs in OpenTelemetry

OpenTelemetry accommodates three distinct categories of logs across enterprise architectures:

1. Legacy Unstructured Logs — Applications write plain, unformatted text strings directly to local disk files, syslog, or container stdout/stderr (e.g., "2026-09-29 10:14:22 [main] ERROR - Failed to connect to DB"). These logs lack explicit field boundaries, requiring downstream collectors to employ regex, grok, or delimiter parsers to extract timestamps, severities, and messages.
2. Structured JSON Logs — Applications utilize modern logging libraries configured to output serialized JSON objects to stdout or local log collectors (e.g., {"time":"2026-09-29T10:14:22Z","level":"error","msg":"Failed to connect to DB","db.name":"users"}). Parsers can readily extract these fields into OpenTelemetry attributes and body structures without fragile regex pattern matching.
3. Direct OpenTelemetry Event Logs — Applications emit native OpenTelemetry Log Records directly through an OpenTelemetry Log Appender or SDK Logger. These records contain native strongly-typed attributes, pre-populated trace context identifiers, normalized SeverityNumber values, and attached Resource metadata before ever leaving the application process.

---

SeverityNumber Mapping and Cross-Language Normalization

In polyglot organizations, microservices written in different languages use disparate severity vocabularies. A Python service may emit CRITICAL, a Go service emits PANIC, a Java service emits FATAL, and an edge router emits Syslog level 1 (Alert). Querying across all services for severe failures traditionally required maintaining complex, error-prone regular expression filters.

OpenTelemetry solves this by introducing a 24-point numerical severity scale (SeverityNumber). The 24 numbers are partitioned into six canonical tiers, with four numerical increments per tier to accommodate framework-specific sub-levels (e.g., DEBUG, DEBUG2, DEBUG3). The mappings below follow the example table in the specification's data-model appendix (Zap's DPanic and Panic map to ERROR2 and ERROR3):

Cross-Framework Querying in Practice

With SeverityNumber normalization, a centralized monitoring alerting rule or SRE dashboard query does not need to enumerate language-specific strings. To capture all critical production failures across a 50-service polyglot cluster, the alerting system executes a single mathematical condition:

SeverityNumber >= 17

This simple numerical filter instantly captures every ERROR from Java, ERROR and CRITICAL from Python, Error and Panic from Go, and Error, Critical, Alert, and Emergency from Syslog appliances. Concurrently, the original framework-level string is preserved in SeverityText (e.g., "CRITICAL"), ensuring that developers inspecting individual log records in search UIs see the exact familiar terms emitted by their source code.

---

Practical Operational Scenario: Resolving Ingestion Lag and Severity Drift

Consider an international e-commerce platform operating an asynchronous payment settlement service on an isolated legacy host. The settlement worker writes plain-text log files locally, and during a three-hour network partition nothing ships those files anywhere.

When network connectivity recovers at 04:00 UTC:

1. A recovery job copies the 500,000-line log file to a central log host, where an OpenTelemetry Collector's filelog receiver reads it.
2. The receiver sets each record's ObservedTimestamp to 04:00:xx UTC, the moment OpenTelemetry code first observed the line.
3. The receiver's timestamp parsing sets Timestamp from the time written in each line, so it correctly shows the execution time between 01:00 UTC and 04:00 UTC.
4. When on-call engineers receive an alert regarding customer transaction failures, querying by Timestamp allows them to reconstruct the exact timeline of database transaction aborts during the 01:00–04:00 window.
5. Simultaneously, comparing ObservedTimestamp against Timestamp alerts the infrastructure team to the 3-hour telemetry ingestion lag, prompting automated checks on network partition recovery.
6. Furthermore, because the payment worker is written in Python (logging CRITICAL for database deadlocks) while the inventory service is written in Java (logging FATAL for stock reservation lockouts), querying for SeverityNumber >= 21 immediately isolates all catastrophic settlement failures across both services in a single correlated dashboard.