6.4 Firewall and IDS/IPS Evasion Techniques

6.4 Firewall and IDS/IPS Evasion Techniques

Modern enterprise networks are defended by layered perimeter and internal security controls, including stateful inspection firewalls, Next-Generation Firewalls (NGFWs), Intrusion Detection Systems (IDS), and Intrusion Prevention Systems (IPS). During penetration testing engagements, security analysts must understand how these defensive controls inspect network traffic and how specialized scanning techniques can bypass filters, evade signature detection, and map hidden network architectures.

---

Packet Fragmentation: Bypassing Stateless Packet Filters

Under normal IP transmission, an IP packet encapsulates a complete transport layer header (e.g., a 20-byte TCP header). Many legacy packet filters and basic signature-based IDSs inspect only the first fragment of a fragmented datagram without buffering or reassembling the stream.

                          UNFRAGMENTED TCP SYN PACKET
+-----------------------+---------------------------------------------------+
|  IPv4 Header (20 B)   | TCP Header: SrcPort, DstPort, Seq, ACK, Flags...  |
+-----------------------+---------------------------------------------------+


                 FRAGMENTED PACKET (-f : 8-BYTE FRAGMENTATION)
Fragment 0 (Offset = 0, MF = 1):
+-----------------------+-----------------------------------+
|  IPv4 Header (20 B)   | Source Port (2 B) + Dst Port (2 B)|
|                       | Sequence Number Part 1 (4 B)      |
+-----------------------+-----------------------------------+

Fragment 1 (Offset = 8, MF = 0):
+-----------------------+-----------------------------------+
|  IPv4 Header (20 B)   | Sequence Part 2 + Ack Number (4 B)|
|                       | Data Offset + TCP Flags (SYN)     |
+-----------------------+-----------------------------------+

Mechanics of Fragmentation in Nmap

• -f (8-Byte Fragmentation): Instructs Nmap to split raw probe packets into fragments containing at most 8 bytes of payload after the 20-byte IPv4 header.
• -ff (16-Byte Fragmentation): Splits raw probe packets into 16-byte fragments.
• --mtu [size]: Specifies a custom Maximum Transmission Unit (MTU). The specified size must be a positive integer multiple of 8 (e.g., --mtu 16, --mtu 24, --mtu 32).

Evasion Mechanism

A standard TCP header is 20 bytes in length. When -f is used, the TCP header is split across multiple IP fragments:

1. Fragment 0 (Offset 0): Contains the 20-byte IP header followed by the first 8 bytes of the TCP header: the 16-bit Source Port, 16-bit Destination Port, and 32-bit Sequence Number.
2. Fragment 1 (Offset 8 / 64 bits): Contains the remaining 12 bytes of the TCP header: the Acknowledgment Number, Data Offset, Reserved bits, and the TCP Control Flags (SYN, ACK, RST, FIN).

Defensive Countermeasures

Stateless packet filters that evaluate rule conditions such as "Block inbound TCP packets with SYN flag set targeting port 80" cannot enforce the rule on Fragment 0 because Fragment 0 contains no TCP flags! Unless the firewall maintains state to reassemble fragments, the initial fragment passes through. However, modern stateful inspection firewalls and NGFWs perform Virtual Defragmentation (Virtual Reassembly), holding fragments in memory until the full segment is assembled, rendering basic micro-fragmentation ineffective.

---

Decoy Scanning (-D): Obscuring the Assessment Origin

When conducting external network assessments, an analyst's scanning IP will quickly be logged by the target organization's Security Operations Center (SOC) and SIEM appliances. Decoy scanning (-D) obscures the true scanning source by interspersing genuine scan traffic with spoofed traffic from third-party hosts.

                                  DECOY SCANNING (-D)
                                  
  [Decoy Host 1] (198.51.100.10) -------- (Spoofed SYN) -------->
                                                                 \
  [Attacker / Tester] (198.51.100.50) ---- (Real SYN: ME) ------> [Target Server]
                                                                 /
  [Decoy Host 2] (198.51.100.20) -------- (Spoofed SYN) -------->
  
  Target Logs Show:
  "Simultaneous port scans detected from: 198.51.100.10, 198.51.100.50, 198.51.100.20"

Syntax and Configuration

Testers can specify explicit decoy IP addresses or instruct Nmap to generate random decoys:

# Scan target using two explicit decoy IPs, placing the real IP in the middle
nmap -D 198.51.100.10,198.51.100.20,ME,198.51.100.30 -sS 10.10.10.5

# Scan target generating 10 random decoy addresses
nmap -D RND:10 -sS 10.10.10.5

Operational Rules for Decoy Scanning

• The ME Parameter: Represents the tester's genuine IP address. Placing ME among the decoy list ensures that the real scan traffic is intermixed naturally. If ME is omitted, Nmap randomly inserts the tester's IP into the decoy sequence.
• Decoys Must Be Active Hosts: The designated decoy IP addresses must be active, reachable hosts on the internet. If a tester uses inactive or dead IP addresses as decoys, target SYN-ACK responses will elicit ICMP Destination Unreachable errors back to the target, immediately alerting defenders as to which IP addresses are spoofed.
• Detection by Defenders: While decoy scanning obscures traffic in firewall connection tables, defenders can isolate the true tester through advanced forensic methods: analyzing differences in IP Time to Live (TTL) values across returning packets, or observing which IP address actually completes service handshakes or handles application layer interactions.

---

Source Port Manipulation (--source-port / -g)

In legacy firewall configurations and router Access Control Lists (ACLs), administrators frequently implemented rules designed to permit return traffic from critical infrastructure services without deploying full stateful inspection.

+-----------------------------------------------------------------------------+
|                  LEGACY STATELESS FIREWALL RULESET (FLAWED)                 |
+-----------------------------------------------------------------------------+
| Rule 10: ALLOW INBOUND from ANY IP with Source Port 53 (DNS Return Traffic) |
| Rule 20: ALLOW INBOUND from ANY IP with Source Port 88 (Kerberos Auth)      |
| Rule 30: ALLOW INBOUND from ANY IP with Source Port 20 (FTP Data Channel)   |
| Rule 40: DENY ALL OTHER INBOUND TRAFFIC                                     |
+-----------------------------------------------------------------------------+

Mechanics and Evasion Strategy

By default, Nmap selects a randomized, ephemeral source port (e.g., 49152 to 65535) when transmitting scan probes. Using the --source-port (or -g) flag, the tester forces Nmap to originate all outbound probe packets from a specific, trusted well-known port:

# Force scan traffic to originate from DNS source port 53
nmap --source-port 53 -sS 10.10.10.5

# Short-form syntax forging Kerberos port 88
nmap -g 88 -sS 10.10.10.5

• Port 53 (DNS): Firewalls frequently permit incoming UDP and TCP packets originating from source port 53, assuming the packets represent answers to internal DNS queries or external zone transfers.
• Port 20 (FTP Data): In Active FTP, the server initiates an inbound connection to the client's high port from server port 20. Legacy firewall rules permit inbound traffic originating from port 20.
• Port 88 (Kerberos) & Port 67 (DHCP): Frequently allowed through internal perimeter boundaries to facilitate enterprise domain authentication and IP allocation.

---

Source Address & Interface Spoofing

In scenarios where a penetration tester wishes to verify firewall filtering behavior without receiving responses, or when testing routing configurations:

• -S [IP_Address]: Spoofs the source IP address in outgoing packet headers.
• -e [Interface]: Specifies the network interface (e.g., eth0, tun0) through which the spoofed packets should be injected.

nmap -S 10.10.10.1 -e eth0 -Pn 10.10.10.50

The Asymmetric Routing Reality: When an analyst spoofs the source IP address, the target server directs its responses (SYN-ACK or RST) to the spoofed IP address, not to the tester's machine. Unless the tester is located on the same broadcast segment as the spoofed IP or controls upstream routing to sniff the return traffic, the tester is completely blind to scan results. Source address spoofing is primarily useful for denial-of-service resilience testing, IDS validation, or when paired with an Idle Scan.

---

The Idle (Zombie) Scan (-sI): The Ultimate Blind Reconnaissance

Developed by security researcher Salvatore Sanfilippo (antirez), the TCP Idle Scan (-sI) is a blind port scanning technique that allows an attacker to scan a target without transmitting a single packet to the target from their real IP address.

The Prerequisite: Predictable IP ID Sequences

Every IPv4 packet header contains a 16-bit Identification (IP ID) field used for fragment reassembly. In older operating systems (and many modern IoT devices, network printers, and embedded network appliances), the operating system increments this IP ID counter globally by 1 for every packet it transmits, regardless of destination.

The Three-Stage Idle Scan Mechanism

An idle scan requires three parties: the Scanner, the Zombie Host (an idle third-party system with predictable IP ID increments), and the Target Server.

STAGE 1: Probe Zombie to Obtain Initial IP ID
Scanner                                                  Zombie Host
   |----------------- 1. SYN-ACK (Probe) ------------------>|
   |<---------------- 2. RST (Contains IP ID = 31337) ------| (Base IP ID Recorded)


STAGE 2: Transmit Spoofed SYN to Target Using Zombie's IP
Scanner                               Target Server                              Zombie Host
   |--- Spoofed SYN (Src: Zombie) ------>|                                            |
   |                                     |-- [If Port OPEN: Sends SYN-ACK to Zombie]->|
   |                                     |                                            | (Zombie sees
   |                                     |                                            |  unexpected SYN-ACK)
   |                                     |<-- [Zombie sends RST, IP ID increments!]---|
   |                                     |                                            |
   |                                     |-- [If Port CLOSED: Sends RST to Zombie]--->|
   |                                     |                                            | (Zombie ignores RST;
   |                                     |                                            |  IP ID does NOT inc)


STAGE 3: Re-Probe Zombie to Check IP ID Delta
Scanner                                                  Zombie Host
   |----------------- 3. SYN-ACK (Re-Probe) --------------->|
   |<---------------- 4. RST (Contains New IP ID) ----------|
   |
   | Analysis:
   | - If New IP ID = 31339 (Delta = 2)  ==> Port is OPEN!
   | - If New IP ID = 31338 (Delta = 1)  ==> Port is CLOSED or FILTERED!

Detailed Step-by-Step Analysis:

1. Step 1 (Sample Zombie Baseline): The scanner transmits a SYN-ACK packet to the Zombie host. The Zombie, having no active connection, responds with a RST packet. The scanner records the Zombie's current IP ID value ($IPID_1 = 31337$).
2. Step 2 (Spoofed Probe to Target): The scanner transmits a SYN packet to the Target port, forging the source IP address to be the Zombie host's IP.
   • If the Target Port is OPEN: The target transmits a SYN-ACK to the Zombie. The Zombie has no record of initiating this connection, so its TCP stack automatically returns a RST packet to the target. Emitting this RST packet causes the Zombie to increment its global IP ID counter ($IPID = 31338$).
   • If the Target Port is CLOSED: The target transmits a RST packet to the Zombie. Under RFC 793, receiving a RST requires no response; the Zombie discards it silently. Its IP ID counter is not incremented.
   • If the Target Port is FILTERED: The target firewall drops the packet. No packet reaches the Zombie; its IP ID counter is not incremented.
3. Step 3 (Re-Sample Zombie): The scanner transmits another SYN-ACK to the Zombie host. The Zombie replies with a RST containing its new IP ID ($IPID_2$).
   • Port OPEN Result: $IPID_2 - IPID_1 = 2$. (The Zombie transmitted one RST to the target, plus one RST to the scanner).
   • Port CLOSED / FILTERED Result: $IPID_2 - IPID_1 = 1$. (The Zombie transmitted zero packets to the target, and one RST to the scanner).

Practical Syntax and Limitations

nmap -sI 192.168.1.100:80 10.10.10.50

• The "Idle" Requirement: The Zombie host must be truly idle. If other network traffic hits the Zombie during the scan, its IP ID will increment unpredictably, generating false positives.
• Modern OS Defenses: Modern operating systems (Linux, Windows, macOS) no longer implement global incremental IP ID counters. Instead, they randomize IP ID generation or set it to zero, rendering them unusable as idle zombies.

---

Additional Evasion Flags: Data Length, MAC Spoofing & Bad Checksums

1. Modifying Packet Size (--data-length [number])

By default, Nmap transmits probe packets with minimal headers and empty payloads (e.g., a standard TCP SYN probe has an IP payload length of 0, resulting in a total packet size of 40 or 44 bytes). Many IDS/IPS signatures detect Nmap scans by matching specifically on these abnormally small, static packet lengths. The --data-length [number] option appends random, non-null bytes to the packet payload, altering packet signatures and bypassing length-based inspection rules.

# Append 25 random bytes to all scan probe packets
nmap --data-length 25 -sS 10.10.10.5

2. MAC Address Spoofing (--spoof-mac [MAC/Vendor/0])

When assessing local Ethernet segments or 802.11 wireless networks, Layer 2 Access Control Lists (port security, 802.1X, or wireless MAC filtering) may restrict connectivity to approved hardware vendors. Nmap allows testers to spoof the physical MAC address of the scanning interface:

# Spoof a specific static MAC address
nmap --spoof-mac 00:11:22:33:44:55 192.168.1.1

# Spoof a random MAC address belonging to Apple hardware
nmap --spoof-mac Apple 192.168.1.1

# Generate a completely randomized MAC address
nmap --spoof-mac 0 192.168.1.1

3. Invalid Checksum Transmission (--badsum)

Under standard TCP/IP networking, every IP, TCP, and UDP header contains a mathematical checksum verifying transmission integrity. Operating systems compute this checksum before sending packets. If a packet arrives at an endpoint with an invalid checksum, the receiving operating system's TCP/IP stack silently discards the packet without processing it.

nmap --badsum -sS 10.10.10.5

• The Evasion & Fingerprinting Technique: Some stateful firewalls, proxies, or intrusion detection systems inspect packets at the boundary without validating transport checksums. If a tester scans a host using --badsum and receives any response (such as a RST or SYN-ACK), the response must have originated from an intermediate firewall or proxy rather than the target host itself, exposing the presence and filtering logic of perimeter defenses.

---

Firewall & IDS Evasion Techniques Summary Reference