16.1 Line Breaking & Event Segmentation

Line Breaking & Event Segmentation

Quick Summary: Line breaking turns a continuous stream of data into separate events. The recommended configuration is SHOULD_LINEMERGE = false with an explicit LINE_BREAKER regex in props.conf: the regex's first capturing group marks the boundary and is discarded. The alternative, SHOULD_LINEMERGE = true (the default), splits data into lines and then merges them back in the merging pipeline using rules such as BREAK_ONLY_BEFORE_DATE, which is slower. TRUNCATE and MAX_EVENTS limit event size.

---

Mechanics of Event Segmentation in the Pipeline

Data passes through the input segment and then the parsing, merging, typing, and index pipelines. Event boundaries are created in two steps:

1. Parsing pipeline (parsingQueue) – line breaking: the LineBreakingProcessor splits the incoming stream into initial events wherever LINE_BREAKER matches. TRUNCATE is enforced here.
2. Merging pipeline (aggQueue) – line merging: if SHOULD_LINEMERGE = true, the aggregator (AggregatorMiningProcessor in splunkd.log) recombines those initial events into multi-line events. It uses rules such as BREAK_ONLY_BEFORE_DATE and MAX_EVENTS. Timestamp extraction also happens in this pipeline.

Where This Happens

• A universal forwarder does not parse data (except for structured-data settings such as INDEXED_EXTRACTIONS). It sends blocks of data, and line breaking happens on the indexer or heavy forwarder that receives them. props.conf line-breaking settings therefore belong on that parsing tier.
• An event can span two blocks of data. LINE_BREAKER_LOOKBEHIND (default 100 bytes) sets how far back from the end of a chunk, with the next chunk appended, Splunk applies LINE_BREAKER. Increase it for very large or multi-line events.

---

Modern Regex Line Breaking: SHOULD_LINEMERGE = false and LINE_BREAKER

Splunk's [default] stanza still uses line merging (SHOULD_LINEMERGE = true), but for any data you onboard deliberately, Splunk recommends letting LINE_BREAKER delimit the events.

The Core Configuration Pattern

Set these two settings together in props.conf:

[my_custom_sourcetype]
SHOULD_LINEMERGE = false
LINE_BREAKER = <regex_with_capturing_group_1>

The Capturing Group 1 Rule

props.conf.spec defines the behavior precisely:

Rule: The regex must contain a capturing group. Wherever it matches, the start of the first capturing group is the end of the previous event, and the end of the first capturing group is the start of the next event. The contents of the first capturing group are discarded and are not in any event. Matched text before the group stays at the end of the previous event, and matched text after it begins the next event.

Syntax Breakdown with Concrete Examples

Example 1: Standard Single-Line Logs

For single-line logs separated by standard line feeds or carriage returns:

LINE_BREAKER = ([\r\n]+)

• The regex matches one or more carriage return (\r) or newline (\n) characters.
• Because ([\r\n]+) is enclosed in parentheses, it constitutes capturing group 1.
• Splunk discards the newline characters and splits the stream into discrete single-line events.

Example 2: Multiline Application Logs with Timestamp Headers

Consider an application log where an event begins with a date, but errors generate 50-line Java stack traces:

2026-09-23 10:14:02.124 INFO Application initialized
2026-09-23 10:14:05.882 ERROR NullPointerException: database pool exhausted
    at com.corp.db.Pool.connect(Pool.java:42)
    at com.corp.app.Main.run(Main.java:118)
2026-09-23 10:14:06.001 INFO Retrying connection...

To break only before lines starting with a new timestamp while keeping the entire stack trace as a single multiline event:

LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}

• Capturing Group 1: ([\r\n]+) matches the preceding newline and is discarded.
• Uncaptured Pattern: \d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2} matches the date and time of the new event.
• Because the timestamp pattern is outside capturing group 1, Splunk does not discard it. The timestamp is preserved at the very beginning of the new event.
• Stack trace lines ( at com.corp.db...) are not followed by a date pattern after the newline, so no break occurs there, and the whole stack trace stays inside the error event.

Why SHOULD_LINEMERGE = false Is Faster

props.conf.spec notes that you get a significant boost to processing speed when LINE_BREAKER delimits multi-line events, compared with using SHOULD_LINEMERGE to reassemble individual lines. With LINE_BREAKER alone, each event is created in one pass. With line merging, the data is first split into lines and each line is then evaluated against the merge rules.

---

Legacy Line Merging in props.conf (SHOULD_LINEMERGE = true)

SHOULD_LINEMERGE = true is the [default] setting, so any source type without its own line-breaking configuration uses line merging.

Legacy Parameters and Evaluation Order

When SHOULD_LINEMERGE = true is set, Splunk uses the following parameters in props.conf to evaluate line boundaries:

Operational Risks of Legacy Merging

1. Oversized events: if the format changes so that the break rule stops matching, lines keep being added to one event until MAX_EVENTS is reached. splunkd.log then shows AggregatorMiningProcessor messages about breaking the event because the limit was exceeded.
2. Queue pressure: line merging runs in the merging pipeline (aggQueue). If it cannot keep up, aggQueue fills and blocks the queues before it, parsingQueue and the input queues, which eventually slows forwarders sending to this instance.

---

Event Sizing and Safety Bounds: TRUNCATE and MAX_EVENTS

Two settings cap how large an event can grow.

1. TRUNCATE = <bytes>

• Default Value: 10000 bytes (approximately 10 KB).
• Function: the default maximum line length in bytes. For multi-byte characters it is rounded down so that it never lands mid-character.
• Mechanics: longer lines are truncated, and splunkd.log records LineBreakingProcessor warnings about truncating lines because the limit was exceeded. When LINE_BREAKER delimits whole events, each event counts as a "line".

The Data Corruption Hazard on Stack Traces and JSON

In enterprise systems, large data structures routinely exceed 10,000 bytes:

• Structured JSON Logs: Cloud service logs (such as AWS CloudTrail or Kubernetes audit events) frequently reach 20 KB to 100 KB in size.
• Diagnostic Stack Traces: Enterprise application crashes involving deep microservice or database call stacks can span 30 KB.

[!WARNING] Silent JSON Ingestion Failure: If TRUNCATE is left at the default 10000 bytes for JSON sources, Splunk cuts the event in the middle of a string or nested object. The trailing closing brackets (}) are severed. Downstream search-time field extractions (KV_MODE = json or spath) will fail completely with syntax errors, rendering the event unsearchable by its structured keys.

Best Practice for TRUNCATE

• Increase TRUNCATE to accommodate the maximum expected payload size for the sourcetype (for example, TRUNCATE = 100000 or TRUNCATE = 500000).
• Setting TRUNCATE = 0 means never truncate. The spec warns that very long lines are often a sign of garbage data, so prefer a generous finite value.

2. MAX_EVENTS = <integer>

• Default Value: 256 lines.
• Function: the maximum number of input lines added to any event during line merging. After that many lines, Splunk breaks the event.
• Mechanics: a 400-line stack trace under line merging becomes two events, the first with 256 lines.
• Interaction with LINE_BREAKER: with SHOULD_LINEMERGE = false, lines are never merged, so MAX_EVENTS has nothing to limit. The event size is set by where LINE_BREAKER matches and by TRUNCATE.