6.3 OS Fingerprinting, Service Detection & Timing Templates

6.3 OS Fingerprinting, Service Detection & Timing Templates

Once live hosts and open ports have been mapped, the next critical phase of technical reconnaissance involves target identification: determining the exact operating system, kernel version, software daemons, and application patch levels running on target systems. Identifying whether an open port on port 80 is Microsoft IIS 10.0 on Windows Server 2022 or Apache 2.4.52 on Ubuntu 22.04 completely dictates subsequent vulnerability assessment and exploitation strategies.

---

Active Operating System Fingerprinting Mechanics (-O)

Operating system fingerprinting determines the identity and version of a target operating system by analyzing subtle variations in how different vendors implement the TCP/IP protocol suite (RFC 791, 793, 1122).

+-----------------------------------------------------------------------------+
|                     NMAP ACTIVE OS DETECTION ARCHITECTURE                   |
+-----------------------------------------------------------------------------+
| 1. Discovers >= 1 Open TCP Port AND >= 1 Closed TCP Port                     |
| 2. Transmits 16 Specially Crafted Probe Packets:                            |
|    - 6 TCP Sequence Probes (T1 - T6) to Open Port                          |
|    - 1 Explicit Congestion Notification Probe (ECN) to Open Port            |
|    - 6 TCP Flag/Option Probes (T2 - T7) to Open/Closed Ports                |
|    - 2 ICMP Probes (Echo Request with varied flags/TOS)                     |
|    - 1 UDP Probe to Closed Port (PU)                                        |
| 3. Records Bit-Level Header Fields and Generates Fingerprint                |
| 4. Matches Fingerprint against nmap-os-db Signature Database               |
+-----------------------------------------------------------------------------+

The 16 Probe Packets and Analysis Tests

When -O is invoked, Nmap transmits a sequence of 16 specific probe packets to the target host and evaluates the returning responses across dozens of distinct header attributes:

1. TCP Initial Sequence Number (ISN) Sampling: Nmap transmits six consecutive TCP SYN packets to an open port (spaced 100ms apart). It analyzes how the target operating system generates sequence numbers:
   • Linear Increment: Older Windows or embedded devices increment ISNs by constant values.
   • Random Increments: Modern Linux and modern Windows kernels implement cryptographically secure pseudo-random number generators (CSPRNG).
   • Nmap calculates an ISN Predictability Index (scoring difficulty of TCP sequence prediction attacks / blind TCP connection spoofing).
2. IP Identification (IP ID) Sequence Generation: Analyzes how the target increments the 16-bit IP Identification field in the IPv4 header across consecutive responses:
   • Incremental global counter (common in older kernels and network printers).
   • Incremental per-socket counter.
   • Completely randomized values (standard in modern Linux).
   • Constant values (e.g., all zeros, common in FreeBSD).
3. TCP Options Ordering and Negotiation: Different operating systems support different TCP options and order them uniquely within the TCP header. Nmap evaluates:
   • Maximum Segment Size (MSS) negotiation.
   • Window Scale (RFC 1323) support and multiplier factor.
   • Selective Acknowledgment (SACK-Permitted, RFC 2018).
   • TCP Timestamps (TSval and TSecr).
   • NOP (No-Operation) padding alignment. Example: Linux typically orders options as MSS, SACK, Timestamp, NOP, Window Scale, whereas Windows orders them as MSS, NOP, Window Scale, SACK, Timestamp.
4. TCP Initial Window Size: Examines the static or dynamic initial Window size advertised in the target's SYN-ACK response (e.g., 64240 in Windows 10/11 vs 65160 in modern Linux).
5. Response to Bogus and Invalid Flag Combinations: Nmap sends probes with illegal TCP flag combinations (such as SYN+FIN, or PSH+FIN+URG without an established session) to evaluate how the target stack reacts. Some operating systems drop illegal packets, while others reply with RST or echo flags.
6. ICMP Response Formatting: Analyzes the ICMP Echo reply generated by the target: checks whether the IP Total Length was modified, whether the IP Time to Live (TTL) is 64 (Linux/Unix), 128 (Windows), or 255 (Cisco/network gear), and whether the ICMP code field is zeroed.
7. UDP to Closed Port (PU Probe): Transmits a UDP packet to a closed port to elicit an ICMP Port Unreachable message. Nmap checks the quoted IP header within the ICMP payload—specifically whether the original IP ID, Total Length, and UDP checksum were preserved accurately by the target kernel.

The Golden Rule of Nmap OS Fingerprinting

Prerequisite for OS Detection: To achieve a high-confidence OS fingerprint match, Nmap strictly requires finding at least one OPEN TCP port and at least one CLOSED TCP port on the target system. The open port is needed to evaluate connection options, ISN predictability, and window scaling; the closed port is needed to test RST generation, flag quirks, and ICMP port unreachable handling. If all ports are open or all ports are filtered, OS detection confidence drops significantly or fails entirely.

Fingerprint Matching: nmap-os-db

The extracted response metrics are assembled into a structured text fingerprint and compared against the nmap-os-db database, which contains thousands of curated operating system signatures. If an exact match is not found, Nmap uses a weighted algorithm to present the closest matching operating system family and confidence percentage.

---

Service and Version Detection Mechanics (-sV)

Simply knowing that port 80 is open is insufficient; a penetration tester must determine whether the service is Apache, Microsoft IIS, Nginx, or an embedded administrative portal. Nmap's Service and Version Detection engine (-sV) determines software identity and version information.

Scanner                                                  Target Port 22
   |------------------- 1. TCP Handshake ----------------->| (Port Open)
   |<------------------ 2. Wait for Banner ---------------| (Null Probe)
   |                    "SSH-2.0-OpenSSH_8.9p1 Ubuntu"     |
   |                    [Matched in nmap-service-probes]   |

Scanner                                                  Target Port 80
   |------------------- 1. TCP Handshake ----------------->| (Port Open)
   |                    [No Banner Returned]               |
   |------------------- 2. Send Probe: GetRequest --------->|
   |<------------------ 3. HTTP/1.1 200 OK ---------------| 
   |                       Server: Apache/2.4.52 (Ubuntu)  |
   |                    [Regex Match -> Service Identified]|

The nmap-service-probes Database and Probe Sequence

Service detection is driven by the nmap-service-probes file. The process follows a structured sequence:

1. Null Probe (Waiting for Greeting Banners): Nmap establishes a TCP connection and waits a few seconds without transmitting any data. Many daemons (e.g., FTP, SSH, SMTP, POP3, Telnet) immediately transmit an initial welcome banner upon connection. If a banner is received, Nmap matches it against defined regular expressions.
2. Protocol-Specific Probes: If the Null probe elicits no response (as with HTTP, DNS, RPC, or SSL), Nmap steps through a sequence of active probe strings defined in the database:
   • GetRequest: Sends an HTTP GET / HTTP/1.0 request.
   • HTTPOptions: Sends an HTTP OPTIONS / HTTP/1.0 request.
   • RTSPRequest: Sends a streaming protocol probe.
   • SSLSessionReq / TLSSessionReq: Initiates a TLS Client Hello to detect encrypted services.
   • TerminalServer: Probes for Microsoft RDP.
3. Pattern Matching (match and softmatch): Returned data is evaluated against database directives:
   • softmatch: Provides a broad, initial classification (e.g., identifies that the service is an HTTP server), allowing Nmap to prioritize subsequent HTTP-specific probes.
   • match: Employs Perl-Compatible Regular Expressions (PCRE) to extract detailed metadata: vendor product, software version, operating system, and hostname.

Version Intensity Levels

The depth of service detection is governed by the --version-intensity option, which accepts integer values from 0 to 9:

---

Nmap Timing Templates (-T0 to -T5)

Nmap provides six predefined timing templates that control probe parallelization, round-trip time (RTT) timeouts, packet delays, and scan aggressiveness. Calibrating timing templates allows testers to balance assessment speed against operational stealth and network stability.

-T0 (Paranoid)  : Ultra-slow, serial, 5 min delay   --> IDS Evasion
-T1 (Sneaky)    : Very slow, serial, 15 sec delay   --> Network Throttling
-T2 (Polite)    : Slowed, serial, 0.4 sec delay     --> Bandwidth Preservation
-T3 (Normal)    : Dynamic, adaptive parallel probes --> Default Production Mode
-T4 (Aggressive): Fast, max RTT 1.25s, min delay 10ms-> Pen Testing Standard
-T5 (Insane)    : Extreme speed, max RTT 300ms      --> Lab / High-Speed Fabric

Detailed Breakdown of Timing Templates

1. -T0 (Paranoid):
   • Probes are strictly serialized (only one port probed at a time across the entire scan).
   • Enforces a mandatory 5-minute (300-second) delay between consecutive probes.
   • Purpose: Evading Intrusion Detection Systems (IDS) configured with threshold-based alert rules that detect multiple probe packets arriving within short time windows.
   • Practical Note: Scanning the top 1,000 ports on a single host at -T0 takes approximately 3.5 days.
2. -T1 (Sneaky):
   • Serialized probing with a 15-second delay between consecutive packets.
   • Used for stealthy perimeter testing where IDS thresholds are somewhat relaxed.
3. -T2 (Polite):
   • Serialized probing with a 0.4-second delay between probes.
   • Purpose: Preserving network bandwidth and preventing overloaded target hosts (e.g., legacy industrial PLC devices or fragile medical appliances) from crashing.
4. -T3 (Normal):
   • Nmap's default timing mode.
   • Dynamically adapts probe rates based on network responsiveness and observed packet latency.
   • Executes probes in parallel, adjusting timeouts dynamically.
5. -T4 (Aggressive):
   • The industry standard for professional penetration testing engagements on modern, reliable networks.
   • Caps maximum RTT timeout at 1.25 seconds and initial RTT timeout at 500 milliseconds.
   • Caps maximum scan delay at 10 milliseconds.
   • Accelerates scan execution significantly without sacrificing accuracy on low-loss networks.
6. -T5 (Insane):
   • Assumes an extraordinarily fast, dedicated network fabric (gigabit Ethernet or local VM switches).
   • Caps maximum RTT timeout at 300 milliseconds and probe delays at 5 milliseconds.
   • Capped host timeout of 15 minutes.
   • Operational Risk: On standard corporate networks or internet connections, -T5 sends packets faster than intermediate network queues and target socket buffers can handle. Packets are dropped, leading to false negatives (missed open ports).

---

Nmap Scripting Engine (NSE)

The Nmap Scripting Engine (NSE) extends Nmap's core architecture using an embedded Lua interpreter. NSE scripts automate advanced service discovery, configuration auditing, banner harvesting, and vulnerability identification.

The 14 NSE Script Categories

Every script in Nmap belongs to one or more predefined categories:

NSE Command Execution Patterns

# Execute default scripts alongside standard SYN scan and version detection
nmap -sS -sV -sC 10.10.10.50

# Scan for known vulnerabilities while strictly excluding dangerous denial-of-service scripts
nmap --script "vuln and not dos" 10.10.10.50

# Audit SSL/TLS cipher suites and supported protocol versions
nmap -p 443 --script ssl-enum-ciphers 10.10.10.50

# Enumerate Windows SMB shares and Active Directory domain information
nmap -p 445 --script smb-enum-shares,smb-os-discovery 10.10.10.50