3.2 High Availability Active/Passive Architecture and State Synchronization

3.2 High Availability Active/Passive Architecture and State Synchronization

High Availability (HA) in Palo Alto Networks PAN-OS provides continuous network protection and service availability by pairing two identical Next-Generation Firewalls. In an Active/Passive HA deployment, one firewall actively processes traffic and enforces security policies (Active state), while the second firewall remains in standby (Passive state), mirroring session tables and system state to seamlessly assume traffic forwarding if the primary firewall fails.

---

Active/Passive HA Mechanics

When Active/Passive HA is configured:

• Active Firewall: Owns the Virtual MAC (VMAC) and active IP addresses, processes all data plane traffic, executes security inspections, and logs events.
• Passive Firewall: Maintains physical link connections but does not forward data traffic. It continuously receives real-time session state, ARP tables, forwarding tables, and IPsec Security Association (SA) data from the Active unit via dedicated HA interconnect links.
• Failover Transition: Upon detecting an Active node failure, the Passive unit transitions to Active state, broadcasts Gratuitous ARP (GARP) frames to update adjacent network switches with its VMAC, and begins forwarding traffic within milliseconds without dropping existing active sessions.

---

HA Interconnect Links and Communication Protocols

PAN-OS firewalls communicate over specialized, dedicated high availability links. Proper HA design requires establishing both primary and backup interfaces for control and data synchronization to prevent single points of failure.

1. HA1 Control Link & HA1-Backup

The HA1 link is a point-to-point control plane interface responsible for administrative synchronization and node health monitoring.

• Primary Functions:
  • Configuration Synchronization: Automatically syncs candidate and running configurations when changes are committed on either peer.
  • Heartbeat & Hello Messages: Exchanges continuous health checks to verify node availability.
  • HA State Management: Coordinates role transitions (Active, Passive, Functional, Non-Functional).
• Transport Protocol & Ports:
  • HA1: Uses TCP port 28769 for cleartext/standard control communication or encrypted TLS.
  • HA1-Backup: Uses TCP port 28770 for encrypted control plane fallback.
• Interface Types: Dedicated HA1 physical ports on hardware appliances, or standard data plane interfaces configured in ha mode (in-band) on VM-Series firewalls.

2. HA2 Data Link & HA2-Backup

The HA2 link is a dedicated data plane interface used for real-time state synchronization.

• Primary Functions:
  • Session Table Synchronization: Syncs new session creation, state modifications, and session closures.
  • Forwarding & ARP Tables: Replicates MAC tables, ARP cache entries, and routing state.
  • IPsec SA State: Syncs IPsec keys, sequence numbers, and Phase 1/Phase 2 Security Associations, ensuring IPsec VPN tunnels do not drop during failover.
• Transport Protocols:
  • Layer 2 HA2: Uses raw Ethernet frames with EtherType 0x7261. Recommended when firewalls are directly connected via dedicated patch cables.
  • Layer 3 HA2: Encapsulates data in IP Protocol 99 (or UDP) when HA2 traffic must be routed across intermediate Layer 3 switches.

3. Split-Brain Prevention via Backup Links

A split-brain condition occurs when the Passive firewall loses HA1 communication with the Active firewall, falsely assuming the Active unit has failed. If both firewalls attempt to assume the Active state simultaneously, IP conflicts and MAC flap storms paralyze the network.

• Configuring HA1-Backup and HA2-Backup links creates an out-of-band redundant heart-beat path. If HA1 fails, control traffic instantly failovers to HA1-Backup, preventing split-brain scenarios.

---

Failure Detection and Failover Triggers

PAN-OS monitors multiple indicators to determine when a failover must be initiated:

1. Heartbeat & Hello Polling

• Firewalls exchange Hello packets (default interval: 1000 ms) and Heartbeat pings over the HA1 link.
• If a peer misses heartbeats exceeding the configured loss threshold (default: 3 missed heartbeats / 3000 ms timer), the passive unit initiates failover.

2. Link Monitoring

• Engineers configure Link Groups containing critical physical interface connections (e.g., WAN uplinks, LAN trunk ports).
• Failure Conditions: Configured as any (failover triggers if a single interface goes down) or all (failover triggers only if all interfaces in the group fail).

3. Path Monitoring

• Path Monitoring sends ICMP pings to specified remote IP targets (e.g., upstream ISP gateway, core switch IP).
• If destination ping responses fail continuously beyond the ping interval and retry threshold, PAN-OS marks the path as down and triggers an HA failover.

---

Preemption Settings and Flapping Protection

• Device Priority: Configured on a scale of 1 to 255. The node with the lower numerical value has higher priority (e.g., Priority 50 beats Priority 100).
• Preemption Enabled: When enabled, if the primary firewall (Priority 50) reboots and recovers, it will force the secondary firewall (Priority 100) to yield the Active role and return to Passive state.
• Preemption Hold-Down Timer: Mandatory stability setting (default: 60 seconds, tunable up to 14400 seconds). After recovering from a failure, the primary unit must wait for the hold-down timer to expire before claiming Active state. This prevents failover flapping while physical links, routing protocols (BGP/OSPF), and ARP tables fully converge.

---

HA1 vs HA2 Interconnect Feature Matrix