Section 3.2: Identification and Reconciliation Engine (IRE) Core

Identification and Reconciliation Engine (IRE) Core

1. Introduction to the IRE Engine

The Identification and Reconciliation Engine (IRE) is the foundational metadata engine and API responsible for protecting the integrity of the ServiceNow Configuration Management Database (CMDB). In a multi-source ecosystem, the IRE acts as the central gatekeeper. The core engine is accessed programmatically using the identifyAndReconcile API (or its modern equivalent identifyAndReconcileEnh). This API accepts a nested JSON payload containing CIs, attributes, and relationships. It evaluates this payload against identification, reconciliation, and data source precedence rules, determining whether to create a new CI, update an existing CI, or discard the input attributes.

2. Identification Rules and Identifier Entries

The first step of the IRE workflow is Identification. The engine must determine if the incoming payload matches a CI that already exists in the database. This process is governed by Identification Rules (cmdb_identifier). Each rule is linked to a specific CMDB class (e.g., Computer or Database Instance) and contains one or more ordered Identifier Entries (cmdb_identifier_entry).

When the IRE receives a payload for a class, it evaluates the identifier entries sequentially based on their defined priority:

1. Source Native Key Match: The IRE checks the sys_object_source table for an existing record matching the source name and the incoming source_native_key. If a match is found, the identification succeeds immediately, and subsequent rules are skipped.
2. Identifier Entry 1 (e.g., Serial Number): If no shortcut match exists, the IRE executes the highest-priority identifier entry (e.g., searching for a matching serial number in the cmdb_ci_hardware table).
3. Identifier Entry 2 (e.g., IP Address and MAC Address): If the first entry fails to find a match, the IRE moves to the next entry in the list.
4. New CI Creation: If all identifier entries fail to find a match, the IRE concludes that the CI does not exist and creates a new record.

If any identifier entry matches multiple existing CIs, the IRE throws a "Multi-Match" error, suspends the update, and generates a De-duplication task to prevent data corruption.

3. Independent vs. Dependent Class Architecture

For the ServiceNow CIS-DF exam, you must understand the distinction between independent and dependent classes:

• Independent Classes: These represent CIs that can be uniquely identified in isolation. For example, a Linux Server (cmdb_ci_linux_server) or an IP Router (cmdb_ci_ip_router) can be identified using their own attributes, such as serial number, name, or BIOS UUID.
• Dependent Classes: These represent CIs that cannot be identified without the context of a parent CI. For example, a Network Adapter (cmdb_ci_network_adapter), a Database Instance (cmdb_ci_db_inst), or a Software Installation (cmdb_sam_sw_install) cannot be uniquely identified by their name or version alone; they must be identified in relation to the hardware server that hosts them.

For dependent classes, the identification rule is marked as "Dependent". The IRE resolves these dependencies using Metadata Rules:

• Hosting Rules (cmdb_metadata_hosting): Define what class hosts the dependent CI (e.g., a Computer hosts an Operating System).
• Containment Rules (cmdb_metadata_containment): Define what class physically or logically contains the dependent CI (e.g., a Chassis contains a card).

During ingestion, the JSON payload must include the relationship between the dependent CI and its parent independent CI. The IRE traces this relationship path to perform identification. If the parent CI cannot be found or is not included in the payload, the identification of the dependent CI fails.

4. Reconciliation Rules and Attribute Authority

Once a CI is identified, the IRE moves to the Reconciliation phase. Reconciliation Rules (cmdb_reconciliation_rule) specify which discovery sources are authorized to update a specific class or specific attributes within that class. Reconciliation rules define:

• The target CMDB class.
• The authorized discovery source (e.g., ServiceNow, SG-SCCM, Manual Entry).
• The specific attributes that the source has authority to write. If no attributes are specified, the source has authority over the entire class.

Without reconciliation rules, any source can write to any attribute, which can lead to low-integrity sources overwriting high-integrity data. For example, a script parsing a manual spreadsheet could overwrite the status of a server that is actively monitored by ServiceNow Discovery. By configuring a reconciliation rule, you restrict write permissions to authorized sources.

5. Data Source Precedence and Conflict Resolution

When multiple discovery sources have reconciliation rules allowing them to update the same class or attributes, conflicts are resolved using Data Source Precedence Rules (cmdb_datasource_precedence). A data source precedence rule assigns an integer priority value to each source:

• Lower numbers represent higher priority (e.g., a priority of 100 is higher than 200).
• If Source A (priority 100) has populated the os_version attribute on a CI, and Source B (priority 200) attempts to update os_version with a different value, the IRE blocks the update.
• The IRE silently discards the lower-priority source's value for that specific attribute while continuing to process and update other attributes in the payload that do not have conflicting precedence rules.
• If no precedence rules are defined, the last-writing source wins, and the attribute is overwritten.

6. The Complete IRE Lifecycle Workflow

When a payload is submitted, the IRE executes the following logic:

1. Receive Payload: The nested JSON is parsed and validated.
2. Identification Lookup: The IRE checks sys_object_source.
3. Identifier Entries Evaluation: If no key match, identifier entries are checked.
4. CI State Determination: The CI is determined to be an insert or an update.
5. Apply Reconciliation Rules: The source's write authority for each attribute is checked.
6. Apply Precedence Rules: The source's priority is evaluated against existing attribute metadata.
7. Database Write: The database is updated, and the lookup ledger sys_object_source is updated.