17.2 Data Masking, Redaction & SEDCMD

Data Masking, Redaction & SEDCMD

Quick Summary: SEDCMD in props.conf masks data at index time using sed syntax (s/<regex>/<replacement>/g). It rewrites the event's _raw text in the typing pipeline on a heavy forwarder or indexer, before the event is written. Sensitive values such as card numbers, Social Security numbers, and tokens therefore never reach the rawdata journal or the .tsidx files. This helps organizations meet storage rules such as PCI DSS.

---

Regulatory Imperatives for Ingestion Redaction

Regulated organizations must limit where sensitive personal and financial data is stored. Once such data is indexed in plain text, removing it is painful: it means deleting events (the delete command only hides them from search), cleaning indexes, or letting buckets age out.

Core Regulatory Standards

1. PCI-DSS 4.0 (Payment Card Industry Data Security Standard):
   • Requirement 3: Mandates the protection of stored cardholder data. Primary Account Numbers (PANs) must be masked when displayed (showing at most the first six and last four digits). Furthermore, Sensitive Authentication Data (SAD)—including card verification codes (CVV2, CVC2, CID), full magnetic stripe / chip data, and PINs—must never be stored after authorization, even if encrypted.
2. HIPAA (Health Insurance Portability and Accountability Act):
   • Requires safeguards for electronic protected health information (ePHI). Keeping identifiers such as Social Security numbers and medical record numbers out of operational logs reduces how much of the logging platform must be treated as holding ePHI.
3. GDPR (General Data Protection Regulation):
   • Article 5 (Data Minimization): Personal data must be adequate, relevant, and limited to what is necessary in relation to the purposes for which they are processed.
   • Article 25 (Data Protection by Design and by Default): names pseudonymisation as an example of an appropriate technical measure. Masking at collection is one way to apply it.

Ingestion Redaction vs. Search-Time Masking

---

SEDCMD Syntax and Operational Mechanics

SEDCMD is configured in props.conf under a source type, source, or host stanza, on the heavy forwarder or indexer that parses the data. The spec says it is only used at index time. It needs no transforms.conf stanza.

Syntax Specification

[<stanza_name>]
SEDCMD-<class> = s/<regex>/<replacement>/[flags]

• SEDCMD-<class>: The directive identifier. <class> is an arbitrary string that must be unique within the stanza (e.g., SEDCMD-mask_cc, SEDCMD-mask_ssn).
• s: the replace command. Splunk supports only a subset of sed: replace (s) and character substitution (y/string1/string2/), which replaces each character of string1 with the character at the same position in string2. A sed script can be a space-separated list of commands.
• <regex>: A Perl-Compatible Regular Expression (PCRE) pattern matching the sensitive text.
• <replacement>: the replacement string. It can contain literal text (e.g., XXXX) and back-references written \n, where n is a single digit (\1, \2), to keep non-sensitive captured parts.
• [flags]: either g, which replaces all matches in the event, or a number, which replaces only that match (for example 2 for the second). With no flag, only the first match is replaced.

Where SEDCMD Runs in the Pipeline

1. Parsing pipeline: the data is broken into events (LINE_BREAKER).
2. Merging pipeline: timestamps are extracted, and lines merged if configured. Timestamp extraction therefore sees the original text.
3. Typing pipeline: the regex-replacement processor applies SEDCMD (and TRANSFORMS) to _raw.
4. Index pipeline: the masked _raw is written to the rawdata journal (journal.zst on current versions), and the .tsidx files are built only from the masked text.

Because masking happens after timestamp extraction, a SEDCMD rule cannot break timestamp recognition. However, it does affect every later index-time step, such as TRANSFORMS classes that match on the masked text.

---

Practical SEDCMD Masking Recipes

1. Masking Credit Card Numbers (Preserving Last 4 Digits)

To mask a 16-digit credit card number while retaining the final four digits for account identification:

# props.conf
[payment:gateway:api]
SEDCMD-mask_card = s/(\b\d{4}[- ]?)\d{4}[- ]?\d{4}[- ]?(\d{4}\b)/\1XXXX-XXXX-\2/g

• Input: User tx card=4111-2222-3333-4444 approved
• Output Written to Disk: User tx card=4111-XXXX-XXXX-4444 approved
• Analysis: Group 1 (\1) captures the first 4 digits, while Group 2 (\2) captures the final 4 digits. The middle 8 digits are replaced with XXXX-XXXX-.

2. Redacting Social Security Numbers (SSNs)

To scrub standard 9-digit U.S. Social Security Numbers formatted as ###-##-####:

# props.conf
[hr:employee:onboarding]
SEDCMD-redact_ssn = s/\b\d{3}-\d{2}-\d{4}\b/XXX-XX-XXXX/g

• Input: Employee record SSN=123-45-6789 onboarded successfully
• Output Written to Disk: Employee record SSN=XXX-XX-XXXX onboarded successfully

3. Redacting Bearer Tokens and Authorization Headers

To mask JSON Web Tokens (JWT) or API bearer tokens in HTTP authorization headers:

# props.conf
[nginx:ingress:access]
SEDCMD-mask_bearer = s/(?i)(Authorization:\s*Bearer\s+)[A-Za-z0-9_\-\.]+/\1[REDACTED_TOKEN]/g

• Input: GET /v1/account Authorization: Bearer eyJhbGciOi...
• Output Written to Disk: GET /v1/account Authorization: Bearer [REDACTED_TOKEN]

4. Redacting Password and Secret Fields in Key-Value Pairs

To mask passwords regardless of capitalization or quoting:

# props.conf
[app:auth:trace]
SEDCMD-mask_pw = s/(?i)(password|passwd|secret)\s*[:=]\s*["']?[^"'\s,;]+["']?/\1=****************/g

• Input: login failed user=admin password="SuperSecretP@ss!" host=db01
• Output Written to Disk: login failed user=admin password=**************** host=db01

---

SEDCMD in props.conf vs. Regex Replacement in transforms.conf

Administrators can also rewrite _raw using transforms.conf by setting DEST_KEY = _raw. Understanding the operational differences between these two approaches is essential:

# Alternative approach using transforms.conf:
# props.conf
[app:auth:trace]
TRANSFORMS-mask_pw = redact_pw_transform

# transforms.conf
[redact_pw_transform]
REGEX = ^(.*password=)"[^"]+"(.*)$
FORMAT = $1"********"$2
DEST_KEY = _raw

Architectural Comparison

---

Regular Expression Performance & Catastrophic Backtracking Hazards

Because SEDCMD executes at index time on every incoming event in real time, regex efficiency is paramount. A poorly constructed regular expression can cripple indexer ingestion throughput.

The Mechanism of Catastrophic Backtracking

Splunk uses a Non-Deterministic Finite Automaton (NFA) regex engine. When a regex contains ambiguous, overlapping, or nested quantifiers, the engine must explore every combination of branches before determining that a pattern fails.

Consider an anti-pattern regex intended to find numbers enclosed by words:

(x+x+)+y

If an event contains xxxxxxxxxxxxxxxxxxxxxxxxxxxx (without the trailing y), the engine tests exponential combinations ($2^n$). For a string of just 30 characters, this can require over one billion evaluation steps.

Real-World Catastrophic Regex Anti-Patterns

1. Nested Quantifiers:

   • Bad: s/(\b\d+[- ]?)+/XXX/g
   • Why it fails: The outer quantifier ()+ and the inner quantifier \d+ compete for the same characters, triggering massive backtracking on invalid numbers.
   • Optimized: s/\b(?:\d{4}[- ]?){4}\b/XXXX-XXXX-XXXX-XXXX/g

2. Unanchored Wildcards with Optional Matches:

   • Bad: s/.*credit_card=(.*)/cc=XXXX/g
   • Why it fails: The leading .* scans to the end of the event, then steps backward character-by-character to satisfy the rest of the expression.
   • Optimized: s/credit_card=[0-9]{13,19}/credit_card=XXXX/g

Operational Impact on Splunk Ingestion

When an indexer encounters catastrophic backtracking:

• The pipeline thread running the regex keeps a CPU core busy on each problem event.
• typingQueue fills, then the queues before it (aggQueue, parsingQueue, and the input queues) fill in turn.
• The instance stops accepting data quickly, so forwarders sending to it slow down or switch to other indexers, and ingestion lag grows.