9.3 OSPF Diagnostics & Common Failure Scenarios

OSPF Diagnostics & Common Failure Scenarios

Deploying and managing OSPF in enterprise networks requires deep diagnostic competence. When link-state neighbor relationships fail to form or unexpectedly flap, connectivity issues quickly escalate across campus cores and branch links. Resolving these failures requires an engineer to map observed symptoms to the OSPF Finite State Machine (FSM), evaluate mandatory parameter agreements, and leverage Junos OS operational commands and traceoptions while monitoring the operational cost of tracing.

---

1. OSPF Neighbor Adjacency Formation & Finite State Machine

OSPF neighbor formation transitions deterministically through seven sequential FSM states:

$\text{Down} \longrightarrow \text{Init} \longrightarrow \text{2-Way} \longrightarrow \text{ExStart} \longrightarrow \text{Exchange} \longrightarrow \text{Loading} \longrightarrow \text{Full}$

+---------------------------------------------------------------------------------------------------+
| OSPF FINITE STATE MACHINE (FSM) LIFECYCLE & FAILURE VECTORS                                       |
+-----------+-----------------------------------------------+---------------------------------------+
| State     | Description of Exchange Activity              | Common Failure Triggers               |
+-----------+-----------------------------------------------+---------------------------------------+
| Down      | No Hello packets received from neighbor       | Physical down, passive-interface set, |
|           |                                               | Protocol 89 blocked by firewall       |
| Init      | Hello received, but local Router ID not seen  | Unidirectional link, broadcast storm, |
|           | in neighbor's Hello packet neighbor list      | multicast filter on return path       |
| 2-Way     | Bi-directional communication confirmed;       | Expected between DROthers; priority 0 |
|           | DR/BDR election occurs on multi-access links  | on all routers prevents DR election   |
| ExStart   | Master/Slave relationship & initial DBD       | Interface MTU mismatch; DBD packet    |
|           | sequence numbers negotiated                   | dropped; duplicate Router IDs         |
| Exchange  | Routers exchange Database Description (DBD)   | MTU mismatch; corrupt DBD packets     |
|           | packets summarizing LSDB contents             |                                       |
| Loading   | Routers send Link-State Requests (LSR) and    | Corrupt LSA data; memory exhaustion;  |
|           | receive Link-State Updates (LSU)              | high transit packet loss              |
| Full      | LSDBs are fully synchronized; routers are     | Steady-state operational target       |
|           | fully adjacent                                |                                       |
+-----------+-----------------------------------------------+---------------------------------------+

Mandatory Hello Parameter Constraints

For two routers to advance from Down past Init to 2-Way, the following parameters carried in their Hello packets must match identically:

1. Area ID: Both interfaces must belong to the exact same Area ID (e.g., 0.0.0.0 or 10).
2. Subnet and Subnet Mask: On broadcast and NBMA networks, interface IP addresses must reside within the same IP subnet with matching subnet masks.
3. Hello and Dead Intervals: The Hello interval (default 10s on broadcast/P2P) and Dead interval (default 40s) must match.
4. Area Option Flags: The E-bit (Stub capability) and N-bit (NSSA capability) must match across neighbors.
5. Authentication Type & Credentials: The authentication type (Null, Simple, MD5, SHA) and active credentials must be identical.
6. Unique Router IDs: Adjacent routers must have unique 32-bit Router IDs. Duplicate Router IDs cause immediate adjacency collapse and LSDB instability.

---

2. In-Depth Failure Scenarios & State Hangs

A. Neighbors Stuck in Init State

A neighbor stuck in the Init state indicates unidirectional communication. Router A is receiving Hello packets from Router B, but Router B is not receiving Hellos from Router A. Consequently, Router B never lists Router A's Router ID in its transmitted Hello packets.

Primary Root Causes:

• Firewall Filters on Loopback (lo0.0): In Junos, protective routing engine firewall filters applied to lo0.0 must explicitly permit OSPF traffic (IP protocol 89) from neighboring subnets. If a filter blocks protocol 89 inbound, the RE drops OSPF Hellos.
• Layer 2 VLAN / Trunking Mismatch: The return path has an asymmetric native VLAN, or an intermediate switch is dropping OSPF multicast traffic (224.0.0.5).
• Access Control Lists / RPF Check: Unicast or multicast reverse path forwarding checks or switch ACLs blocking OSPF packets in one direction.

B. Neighbors Stuck in 2-Way State

Understanding the 2-Way state requires distinguishing between expected behavior and operational faults:

• Normal Operation on Broadcast Networks: On a broadcast multi-access segment (such as an Ethernet VLAN connecting three or more switches), routers elect a Designated Router (DR) and Backup Designated Router (BDR). Routers that are neither DR nor BDR are designated as DROthers. DROther routers form Full adjacencies exclusively with the DR and BDR. Between two DROther routers, the adjacency relationship intentionally and correctly remains in the 2-Way state. This prevents the $N(N-1)/2$ scaling explosion of full mesh adjacencies.
• Fault Condition: If two routers on a dedicated point-to-point link remain stuck in 2-Way, or if no DR is elected on a broadcast segment because every router is configured with priority 0, the state machine cannot proceed.

C. Neighbors Stuck in ExStart or Exchange State (The MTU Trap)

The most pervasive operational problem on enterprise campus switches running OSPF is neighbors hanging indefinitely in ExStart or Exchange.

The MTU Mismatch Root Cause:

• Hello packets are small (typically 60 to 100 bytes) and easily traverse links regardless of MTU differences.
• In the ExStart state, routers negotiate the master/slave role and initial sequence number using Database Description (DBD) packets. In the Exchange state, routers transmit DBD packets containing LSA headers describing their entire link-state database.
• A DBD packet advertises an Interface MTU field. A receiver can reject a DBD advertising a larger MTU even when the DBD itself is small enough to arrive. The mismatch is therefore not merely a giant-frame drop.
• Large LSUs can additionally expose a real data-path MTU problem. Compare protocol MTU and media MTU on both ends before changing either.
• Check trace/log messages for MTU rejection, duplicate IDs and retransmissions. A state is a useful clue, not a unique diagnosis.

Resolution in Junos OS:

1. Best Practice: Correct the physical interface MTU so both sides match identically:

   [edit interfaces ge-0/0/0]
   user@EX4300# set mtu 9216
   

2. Workaround (mtu-ignore): In multi-vendor environments where changing interface MTU is constrained, Junos allows disabling MTU validation in incoming DBD packets:

   [edit protocols ospf area 0.0.0.0 interface ge-0/0/0.0]
   user@EX4300# set mtu-ignore
   

   Caution: Using mtu-ignore can mask physical MTU mismatches that later cause data-plane transit packet drops when large IP packets traverse the link.

---

3. Junos Traceoptions for OSPF Diagnostics

In legacy network operating systems, running debugging commands on a live production switch sends high-volume output to the console, frequently spiking the CPU and dropping control-plane sessions. In Junos OS, the clean separation of the control plane (Routing Engine) from the forwarding data plane (Packet Forwarding Engine), combined with the traceoptions framework, allows targeted logging to disk; excessive tracing can degrade performance.

Configuring OSPF Traceoptions

[edit protocols ospf]
user@EX4300# set traceoptions file ospf-debug size 10m files 5 no-world-readable
user@EX4300# set traceoptions flag hello detail
user@EX4300# set traceoptions flag error detail
user@EX4300# set traceoptions flag lsa-generation detail
user@EX4300# set traceoptions flag state detail

• file ospf-debug size 10m files 5: Creates a dedicated rotating log buffer maintaining up to five 10 MB files.
• flag hello detail: Captures inbound and outbound Hello exchanges, revealing mismatched timers, subnet masks, area IDs, or missing neighbor IDs.
• flag error detail: Logs protocol anomalies, checksum failures, and authentication key mismatches.

Reading and Monitoring Trace Files

Administrators inspect trace files in real time without entering configuration mode:

user@EX4300> show log ospf-debug | match "timer mismatch\|authentication"
user@EX4300> monitor start ospf-debug

---

4. Comprehensive OSPF Troubleshooting Guide

The following matrix details common failure symptoms, root causes, diagnostic CLI commands, and Junos remediation statements:

+---------------------------------------------------------------------------------------------------+
| JUNOS OSPF ENTERPRISE TROUBLESHOOTING MATRIX                                                      |
+---------------------+-----------------------+-------------------------+---------------------------+
| Observed Symptom    | Probable Root Cause   | Diagnostic Show Command | Junos CLI Remediation     |
+---------------------+-----------------------+-------------------------+---------------------------+
| Neighbor remains in | - Physical link down  | show interfaces terse   | - Check cable/SFP         |
| "Down" state; no    | - Passive interface   | show ospf interface     | - delete protocols ospf   |
| neighbor detected   | - Firewall filter on  | show firewall log       |   interface <if> passive  |
|                     |   lo0.0 blocks proto  |                         | - Permit proto 89 in      |
|                     |   89 (OSPF multicast) |                         |   lo0 filter term         |
+---------------------+-----------------------+-------------------------+---------------------------+
| Neighbor stuck in   | - Unidirectional link | show ospf neighbor      | - Verify L2 VLAN tag      |
| "Init" state        | - Multicast drop      | detail                  | - Check reverse path ACLs |
|                     | - Return Hello not    | show log ospf-debug     | - Verify both ends see    |
|                     |   received by peer    |                         |   each other's RID        |
+---------------------+-----------------------+-------------------------+---------------------------+
| Stuck in "2-Way"    | - All routers on      | show ospf neighbor      | - Expected for DROthers   |
| on broadcast link   |   segment have        | show ospf interface     | - Set priority >= 1 on at |
| (no DR/BDR elected) |   priority set to 0   | detail                  |   least one router        |
+---------------------+-----------------------+-------------------------+---------------------------+
| Neighbor stuck in   | - Interface MTU       | show interfaces <if>    | - Harmonize MTU on link;  |
| "ExStart" or        |   mismatch between    | show ospf neighbor      | - Or configure mtu-ignore |
| "Exchange" state    |   connected neighbors | detail                  |   under ospf interface    |
+---------------------+-----------------------+-------------------------+---------------------------+
| Neighbor flaps      | - Hello/Dead timer    | show ospf interface     | - Match hello/dead timers |
| continuously        |   mismatch            | detail                  | - Check for duplicate RIDs|
|                     | - Duplicate Router ID | show log ospf-debug     | - Fix duplex/CRC errors   |
|                     | - High packet drop    |                         |                           |
+---------------------+-----------------------+-------------------------+---------------------------+
| Routes missing in   | - Area range restrict | show ospf database      | - Remove restrict keyword |
| routing table       | - Export policy drop  | show route protocol ospf| - Fix policy-statement    |
|                     | - Route preference    | show route <prefix>     | - Verify route preference |
|                     |   override (e.g. 5)   | detail                  |   (Static beats OSPF: 5)  |
+---------------------+-----------------------+-------------------------+---------------------------+

---

5. Essential Operational Verification Workflow

When diagnosing an OSPF fault in production, execute the following structured CLI verification sequence:

1. Verify Neighbor Adjacency State:

   user@EX4300> show ospf neighbor
   Address          Interface              State     ID               Pri  Dead
   10.0.0.2         ge-0/0/0.0             Full      10.255.0.2       128    36
   

   Confirm that the state reads Full (or 2-Way for DROther-to-DROther on multi-access links).

2. Inspect Interface Protocol Settings:

   user@EX4300> show ospf interface detail ge-0/0/0.0
   

   Verify interface type (Broadcast vs PtToPt), MTU size, Hello/Dead timers, authentication mode, and DR priority.

3. Inspect the Link-State Database:

   user@EX4300> show ospf database
   

   Confirm that Type 1, 2, and 3 LSAs are present and that advertising router IDs match expected network nodes.

4. Check Route Installation in inet.0:

   user@EX4300> show route protocol ospf
   

   This command inspects the RIB; separately verify show route forwarding-table for forwarding installation. If an OSPF route is active in the LSDB but missing from inet.0, check for competing routes with a lower route preference (such as static routes with preference 5 or direct routes with preference 0).

Output note: Command output shown here is a schematic teaching example, not a captured device session. Field names and feature support depend on platform and Junos release.

Separate IP Reachability from OSPF Exchange

For a directly connected peer 10.0.0.2, test ping 10.0.0.2 source 10.0.0.1 count 5 from the intended local transit address. If it fails, inspect interface state, subnet/VLAN association, ARP and packet filters before interpreting the OSPF database. A successful ping proves an ICMP path, not that IP protocol 89 is allowed or that area/authentication parameters agree. Compare show ospf neighbor detail with both ends’ interface settings.

For a remote learned destination, use traceroute <destination> source <local-address> and compare the observed next hop with show route <destination> detail and show route forwarding-table destination <destination>. A missing traceroute response can reflect filtered ICMP rather than absent forwarding. Correlate the packet test with route selection, interface counters and bounded protocol logs rather than treating one tool as conclusive.