1.6 AD DS Replication: Topology Generation, Convergence & Diagnostics

AD DS Replication: Topology Generation, Convergence & repadmin Diagnostics

Every domain controller holds a writable copy of the directory, so Active Directory is a multi-master database with no single authoritative source for ordinary object changes. Convergence is therefore not a background detail — it is the mechanism that decides whether a password reset in London is honoured in Seattle, whether a disabled account is really disabled everywhere, and whether an object deleted six months ago can be resurrected as a lingering object. This section covers how the replication topology is generated, how changes propagate and converge, and how to diagnose a partnership that has stopped working.

---

1. Intra-Site vs Inter-Site Replication Mechanisms

Active Directory enforces fundamentally different replication engines depending on whether traffic stays within a local site or crosses a WAN site link.

Urgent Replication Triggers

Certain high-security directory modifications bypass all standard intra-site notification delays and inter-site polling schedules, triggering Urgent Replication immediately:

• Account lockouts (to immediately block compromised accounts enterprise-wide).
• Local Security Authority (LSA) secret changes.
• RID Master pool allocation changes.
• Changes to the Domain Password Policy.

[!TIP] Enabling Inter-Site Change Notification: By default, inter-site replication waits for the scheduled replication frequency (e.g., every 15 or 180 minutes). For high-speed fiber WAN links or Azure ExpressRoute connections, administrators can enable near-real-time Change Notification on inter-site links by editing the options attribute on the Site Link object in ADSI Edit or PowerShell, setting bit flag 1 (binary bitmask 0x1).

---

2. KCC, ISTG & Bridgehead Server Generation Mechanics

Replication connection objects are generated dynamically by background directory algorithms to ensure high availability and eliminate single-point-of-failure routing.

+-----------------------------------------------------------------------------+
|                     TOPOLOGY GENERATION ENGINES                             |
|                                                                             |
|   [KNOWLEDGE CONSISTENCY CHECKER (KCC)]                                     |
|   - Runs on EVERY domain controller every 15 minutes                        |
|   - Builds intra-site ring topology (guarantees <= 3 hops between any DCs)  |
|   - Creates dynamic <nTDSSettings> connection objects                       |
|                                                                             |
|   [INTER-SITE TOPOLOGY GENERATOR (ISTG)]                                    |
|   - Exactly ONE domain controller elected per site                          |
|   - Evaluates inter-site links, cost matrices, and site schedules           |
|   - Automatically elects BRIDGEHEAD SERVERS to replicate WAN traffic        |
+-----------------------------------------------------------------------------+

• Knowledge Consistency Checker (KCC): A built-in process running on all DCs every 15 minutes. It evaluates the availability of replication partners and creates unidirectional inbound connection objects (fromServer references). If a DC fails, the KCC automatically recalculates the topology to route around the dead DC.
• Inter-Site Topology Generator (ISTG): In each site, one DC is elected as the ISTG. The ISTG generates the inter-site replication connection objects connecting bridgehead servers across site links.
• Bridgehead Servers: The primary contact DCs that send and receive replicated data across site links. By default, the ISTG automatically elects bridgeheads dynamically. While administrators can manually designate preferred bridgeheads in Active Directory Sites and Services, doing so creates a single point of failure if the designated bridgehead goes offline.

---

3. Replication Diagnostics and Health Auditing with repadmin

The command-line utility repadmin.exe is the primary diagnostic tool for inspecting replication health, identifying replication latency, and resolving synchronization errors.

+-----------------------------------------------------------------------------+
|                        REPADMIN COMMAND CHEAT SHEET                         |
|                                                                             |
|   COMMAND                       DIAGNOSTIC PURPOSE                          |
|   ----------------------------+-------------------------------------------- |
|   repadmin /showrepl * /csv   | Exports complete inbound replication status |
|                               | and error codes for all DCs to CSV          |
|   repadmin /replsum           | Summarizes replication health, latency, and |
|                               | largest replication delta across all DCs    |
|   repadmin /syncall /AePd     | Forces immediate synchronization across all |
|                               | partitions, enterprise-wide, push/pull      |
|   repadmin /kcc *             | Forces immediate KCC topology calculation   |
|   repadmin /showutdvec        | Displays Up-to-Dateness Vector (high-water  |
|                               | USNs) to detect lingering objects           |
+-----------------------------------------------------------------------------+

Essential Diagnostic Syntax Examples:

:: 1. Check replication summary and identify failing DCs
repadmin /replsum

:: 2. Display detailed inbound replication partners and USN status for local DC
repadmin /showrepl

:: 3. Force full cross-partition synchronization using all topology paths
repadmin /syncall /AdePq
:: Switches: /A=All partitions, /d=Distinguished name, /e=Enterprise-wide (across sites), /P=Push, /q=Quiet

:: 4. Force KCC recalculation on all DCs in the current site
repadmin /kcc *

---

4. Convergence Mechanics: USNs, Watermarks and Dampening

Replication is driven by counters, not timestamps. Each domain controller keeps a local Update Sequence Number (USN) that increments on every originating write. Two vectors control what actually crosses the wire:

• The high-watermark vector (up-to-dateness by partner) records the highest USN a domain controller has received from a specific partner, so each replication cycle asks only for "everything after N".
• The up-to-dateness vector (UTDV) records the highest originating USN this domain controller has seen from every domain controller in the forest, identified by invocation ID.

Together these produce propagation dampening: in a ring topology a change that arrives at a domain controller from two directions is applied once and the duplicate copy is suppressed, because the UTDV already shows that originating write as seen.

When two administrators change the same attribute on different domain controllers inside the same replication window, Active Directory resolves the collision deterministically in this order:

1. Higher version number wins — the attribute's version increments on each originating write.
2. If versions tie, the later originating timestamp wins.
3. If timestamps also tie, the higher originating domain controller GUID wins.

That third tiebreaker is why one of two simultaneous edits can vanish with no error logged anywhere: last-writer-wins is by design, not a fault.

5. Tombstones, Lingering Objects and Strict Replication Consistency

Deletions do not remove an object immediately. The object becomes a tombstone — stripped of most attributes, flagged isDeleted, and retained for the forest's tombstone lifetime, which is 180 days in any forest created at Windows Server 2003 SP1 functional level or later. Replication of the deletion is what removes it everywhere.

A domain controller that is offline longer than the tombstone lifetime and is then reconnected never receives those deletions and still holds objects that no longer exist elsewhere. Those are lingering objects, and they cause the classic symptom of a user or group that appears in one site's directory searches and nowhere else.

[!WARNING] The correct response to a domain controller that has been offline longer than the tombstone lifetime is to forcibly demote it and perform metadata cleanup, then repromote. Simply restoring the network connection is what injects lingering objects into a healthy forest.

6. SYSVOL Replication and Whole-Health Diagnostics

The directory database and SYSVOL replicate through different engines. SYSVOL uses DFS Replication (DFS-R) on any modern domain; the legacy File Replication Service (FRS) was deprecated and removed, and a domain still on FRS must complete the dfsrmig migration to the Eliminated state before a Windows Server 2016 or later domain controller can be promoted into it. A domain that replicates the directory correctly but not SYSVOL produces the distinctive symptom of Group Policy changes that never reach some clients while user and group changes do.

:: SYSVOL replication state (0=Start, 1=Prepared, 2=Redirected, 3=Eliminated)
dfsrmig /getglobalstate
dfsrmig /getmigrationstate

:: Whole-domain-controller health, including replication, DNS and SYSVOL checks
dcdiag /v /c /e /test:Replications /test:SysVolCheck /test:NetLogons

:: Attribute-level forensics: who last wrote this attribute, from which DC, and when
repadmin /showobjmeta DC-SEA-01 "CN=Jane Doe,OU=Finance,DC=corp,DC=contoso,DC=com"