4.3 Field Extraction Management, Permissions & Search-Time Precedence

4.3 Field Extraction Management, Permissions & Search-Time Precedence

Quick Answer: Field extractions are managed centrally in Splunk Web under Settings > Fields > Field extractions. Splunk implements two distinct extraction types: Inline Extractions (EXTRACT-<class> in props.conf), which store regex directly in props.conf, and Transforms-Based Extractions (REPORT-<class> in props.conf pointing to transforms.conf), which allow reusable regex, delimiter parsing, and multi-value extraction. Permissions follow three sharing tiers: Private, This App Only (App), and Global (All Apps). When resolving configuration conflicts at search time, Splunk evaluates user directories (/etc/users/<user>/<app>/) first, followed by app local (/etc/apps/<app>/local/), app default, system local, and system default.

---

1. Managing Field Extractions in Splunk Web

The central administrative hub for field extractions is located in Splunk Web under Settings > Fields > Field extractions.

+---------------------------------------------------------------------------------------------------------+
|                                 SPLUNK WEB: FIELD EXTRACTIONS MANAGEMENT                                |
| Settings  -->  Fields  -->  Field extractions                                                           |
+---------------------------------------------------------------------------------------------------------+
| Filter by App: [ Search & Reporting v ]  Filter by Owner: [ All v ]  Search: [ access_combined        ]|
+---------------------------------------------------------------------------------------------------------+
| Name                         | Sourcetype        | Type           | App    | Owner | Sharing | Status   |
|------------------------------+-------------------+----------------+--------+-------+---------+----------|
| access_combined_status_ip    | access_combined   | Inline         | search | admin | Global  | Enabled  |
| cisco_asa_pipe_delim         | cisco:asa         | Uses transform | search | jdoe  | App     | Enabled  |
| app_debug_trace_user         | log4j_app         | Inline         | app_qa | rchen | Private | Enabled  |
+---------------------------------------------------------------------------------------------------------+
| Actions: [ Edit Permissions ] [ Disable / Enable ] [ Clone ] [ Move ] [ Delete ]                       |
+---------------------------------------------------------------------------------------------------------+

Administrative Capabilities in the Field Extractions View:

1. Filtering & Discovery: Filter extractions by App context, Owner, or search keywords to quickly audit existing schema extractions across large enterprise environments.
2. Type Identification: Displays whether an extraction is Inline (configured purely in props.conf) or Uses transform (paired with a stanza in transforms.conf).
3. Permission Management: The Permissions link allows Power Users to elevate an extraction from Private to App or Global scope and set granular Read/Write access controls for specific user roles.
4. Lifecycle Operations:
   • Disable / Enable: Temporarily deactivates an extraction without deleting its underlying configuration.
   • Clone: Duplicates an existing extraction as a template for new data sources or sourcetypes.
   • Move: Re-assigns the extraction knowledge object to a different application container.
   • Delete: Permanently removes the configuration stanza from the local configuration layer.

2. Inline Extractions (EXTRACT-) vs. Transforms-Based Extractions (REPORT-)

Understanding the architectural distinction between inline and transforms-based extractions is one of the most fundamental competencies evaluated on the Power User exam.

1. Inline Extractions (EXTRACT-<class>)

Inline extractions encapsulate the complete extraction logic—both the trigger and the regular expression—within a single stanza inside props.conf.

# $SPLUNK_HOME/etc/apps/search/local/props.conf
[apache:access]
EXTRACT-client_and_status = ^(?<client_ip>\S+) \S+ \S+ \[[^\]]+\] "\S+ \S+ \S+" (?<status_code>\d{3})

• When to Use Inline Extractions:
  • Simple, one-off regex extractions specific to a single sourcetype, source, or host.
  • Extractions generated directly via the Field Extractor in Regular Expression mode when no advanced transform options are needed.

2. Transforms-Based Extractions (REPORT-<class>)

Transforms-based extractions decouple the sourcetype association in props.conf from the extraction definition in transforms.conf.

# $SPLUNK_HOME/etc/apps/search/local/props.conf
[cisco:asa]
REPORT-cisco_security_fields = asa_regex_transform

[paloalto:firewall]
REPORT-pan_security_fields = asa_regex_transform  <-- Reusing the same transform!

# $SPLUNK_HOME/etc/apps/search/local/transforms.conf
[asa_regex_transform]
REGEX = src_ip=(?<src>[^\s]+)\s+dst_ip=(?<dest>[^\s]+)
MV_ADD = true
CLEAN_KEYS = true

Architectural Comparison Matrix

---

3. Knowledge Object Permission Scopes & Sharing Promotion

Every field extraction exists within a specific permission tier that governs which users and applications can utilize the extracted fields in searches, alerts, and data models.

+-------------------------------------------------------------------------------------+
|                     KNOWLEDGE OBJECT PERMISSION PROMOTION TIERS                     |
+-------------------------------------------------------------------------------------+
| Tier 1: Private (Owner Only)                                                        |
| • Storage: $SPLUNK_HOME/etc/users/<username>/<app_name>/props.conf                 |
| • Access:  Only the creating user account can search using these extracted fields.  |
+-------------------------------------------------------------------------------------+
                                           │
                                           ▼ [Promote to App Scope]
+-------------------------------------------------------------------------------------+
| Tier 2: This App Only (App Scope)                                                   |
| • Storage: $SPLUNK_HOME/etc/apps/<app_name>/local/props.conf                        |
| • Access:  All users working inside <app_name> can use the fields.                  |
| • RBAC:    Requires Power User role (or write permissions to the App).              |
+-------------------------------------------------------------------------------------+
                                           │
                                           ▼ [Promote to Global Scope]
+-------------------------------------------------------------------------------------+
| Tier 3: Global (All Apps Scope)                                                     |
| • Storage: $SPLUNK_HOME/etc/apps/<app_name>/local/props.conf                        |
| • Metadata: $SPLUNK_HOME/etc/apps/<app_name>/metadata/local.meta [export = system] |
| • Access:  Available across EVERY app on the search head (Search, SOC, CIM, etc.).  |
| • RBAC:    Requires write access to the app plus the ability to export objects    |
|            globally; in most deployments this is an admin-level capability.       |
+-------------------------------------------------------------------------------------+

Promotion Workflow via Splunk Web:

1. Navigate to Settings > Fields > Field extractions.
2. Locate the target extraction and click Permissions.
3. In the Permissions dialog:
   • Select Display for: Only read by owner (Private), This app only (App), or All apps (Global).
   • Configure role-based Read/Write permissions (e.g., granting power role Write permissions while granting user role Read permissions).
4. Click Save.

---

4. Configuration File Directory Precedence (Search Time)

When a user executes a search, Splunk evaluates and merges configuration files across multiple filesystem directories. If an identical stanza or extraction class name exists in multiple directories, Splunk resolves conflicts using strict directory precedence rules.

+-------------------------------------------------------------------------------------+
|             SEARCH-TIME CONFIGURATION DIRECTORY PRECEDENCE HIERARCHY                 |
|                               (Highest Priority to Lowest)                          |
+-------------------------------------------------------------------------------------+
| 1. Current User Directory (Highest):                                                |
|    $SPLUNK_HOME/etc/users/<current_user>/<current_app>/                             |
+-------------------------------------------------------------------------------------+
                                           │
                                           ▼
+-------------------------------------------------------------------------------------+
| 2. App Local Directory:                                                             |
|    $SPLUNK_HOME/etc/apps/<current_app>/local/                                       |
+-------------------------------------------------------------------------------------+
                                           │
                                           ▼
+-------------------------------------------------------------------------------------+
| 3. App Default Directory:                                                           |
|    $SPLUNK_HOME/etc/apps/<current_app>/default/                                     |
+-------------------------------------------------------------------------------------+
                                           │
                                           ▼
+-------------------------------------------------------------------------------------+
| 4. System Local Directory:                                                          |
|    $SPLUNK_HOME/etc/system/local/                                                   |
+-------------------------------------------------------------------------------------+
                                           │
                                           ▼
+-------------------------------------------------------------------------------------+
| 5. System Default Directory (Lowest):                                               |
|    $SPLUNK_HOME/etc/system/default/                                                 |
+-------------------------------------------------------------------------------------+

Key Precedence Principles:

1. User Context Always Wins: If user jdoe has a private extraction in /etc/users/jdoe/search/props.conf, it overrides an identically named extraction defined at the app level in /etc/apps/search/local/props.conf whenever jdoe executes a search.
2. local Always Overrides default: Any configuration saved in a local/ directory takes absolute priority over the corresponding default/ directory.
3. Never Edit default/ Directories: Upgrading Splunk or Splunk Technology Add-ons (TAs) completely overwrites all default/ directories. All enterprise customizations must be saved in local/ or user directories.
4. App Tie-Breaking Is Context-Dependent: Search-time props.conf is evaluated in the app/user context. There, the currently running app takes priority over every other app regardless of name, and Splunk only consults other apps for settings exported globally. Among those remaining apps, priority is reverse-lexicographical — app B outranks app A. The opposite rule (plain lexicographical, where A outranks B) applies only in the global context, which is how props.conf is evaluated at index time and how files such as inputs.conf are always evaluated.

---

5. Field Discovery Execution Order in the Search Pipeline

Search-time field extractions operate at the very beginning of the multi-stage search-time field discovery pipeline. Downstream knowledge objects depend on the fields produced by upstream extraction stages:

Stage 0: Default & Indexed Fields  (_time, _raw, host, source, sourcetype, index)
   │
   ▼
Stage 1: Field Extractions         (EXTRACT- inline & REPORT- transforms-based) <--- (WE ARE HERE)
   │
   ▼
Stage 2: Field Aliases             (FIELDALIAS- in props.conf)
   │
   ▼
Stage 3: Calculated Fields         (EVAL- in props.conf)
   │
   ▼
Stage 4: Lookups                   (LOOKUP- in props.conf + transforms.conf)
   │
   ▼
Stage 5: Event Types               (eventtypes.conf)
   │
   ▼
Stage 6: Tags                      (tags.conf applied to field values & event types)

Dependency Rules for Field Extractions:

• Can Field Extractions use Field Aliases? NO. Field extractions (Stage 1) execute before Field Aliases (Stage 2).
• Can Field Aliases reference Extracted Fields? YES. Field Aliases map fields extracted in Stage 1 to canonical names.
• Can Calculated Fields use Extracted Fields? YES. Calculated Fields (Stage 3) evaluate expressions on fields extracted in Stage 1.

---

6. Search-Time vs. Index-Time Field Extractions

---

7. Common Exam Traps & Pitfalls

1. Modifying System Default Files: A question asks where an administrator should manually create a permanent global field extraction. The correct answer is $SPLUNK_HOME/etc/apps/<app_name>/local/props.conf with export = system in local.meta, NEVER /etc/system/default/.
2. Confusing Inline with Transforms Directives: Remember that EXTRACT- defines inline regex directly in props.conf, while REPORT- references a separate stanza in transforms.conf.
3. Pipeline Sequencing Traps: An extraction cannot match against a field created by a field alias or calculated field because extractions execute at Stage 1, while aliases and calculated fields execute at Stages 2 and 3.
4. Permission Scope Traps: Private extractions cannot be viewed or utilized by other users, even if those users have the admin role, unless an administrator explicitly accesses user directories or reassigns ownership.