4.2 Network Device, Web Server, and Database Logs

Key Takeaways

  • Network telemetry encompasses stateful firewall traffic logs, router/switch flow records (NetFlow/IPFIX vs. sFlow), DNS resolution telemetry, and DHCP lease mappings.
  • NetFlow and IPFIX capture complete 5-tuple session summaries without packet payloads, whereas sFlow relies on statistical packet sampling at fixed intervals.
  • DNS telemetry provides critical indicators of compromise, including high-volume NXDOMAIN spikes indicative of Domain Generation Algorithms (DGAs) and oversized, high-entropy TXT records indicative of DNS tunneling.
  • Web server logs record client interactions in standardized structures like Apache Combined Log Format and IIS W3C Extended Format, where sub-status and Win32 status codes isolate authentication failures and web vulnerabilities.
  • Email investigations correlate gateway verdicts, message trace, SMTP routing, authentication results, mailbox audit events, URL clicks, and attachment hashes; a single header or verdict is not conclusive.
Last updated: September 2026

Network Infrastructure Telemetry Sources

Network devices provide essential visibility into communication traversing internal segmentation boundaries and external perimeters. Because network devices cannot inspect volatile endpoint memory, their logs focus on state transitions, session metadata, protocol headers, and volumetric trends.

1. Firewall Logs

Modern stateful and Next-Generation Firewalls (NGFW) monitor TCP, UDP, and ICMP connection states. Key telemetry fields recorded per connection include:

  • Action: Allowed (Permit) vs. Dropped (Deny/Reset).
  • Network Address Translation (NAT): Pre-NAT and Post-NAT IP addresses and ports (SNAT for outbound client masquerading; DNAT for inbound service publishing).
  • Rule ID / Security Policy: The specific rule permitting or denying the packet.
  • Interface and Zone: Source and destination security zones (e.g., Trust, Untrust, DMZ).
  • Egress Filtering Alerts: Denied outbound connections originating from internal servers toward uncommon foreign destination ports (often indicative of active malware beacons or reverse shells).

2. Routers and Switches: Flow Telemetry (NetFlow, IPFIX, sFlow)

Routers and layer-3 switches generate session-based network accounting records without capturing complete packet payloads:

  • NetFlow (Cisco) / IPFIX (IETF RFC 7011): Aggregates continuous packet streams into flow records characterized by the 5-tuple: Source IP, Destination IP, Source Port, Destination Port, and Layer 4 Protocol. Records also detail byte counts, packet counts, duration, and TCP flags (e.g., SYN, ACK, FIN, RST).
  • sFlow (Sampled Flow - RFC 3176): Rather than tracking every session flow state, sFlow takes statistical packet samples (e.g., 1 out of every 1,000 packets) directly from network interfaces. While computationally lightweight on high-throughput core backbones, sFlow can miss low-and-slow, single-packet threat activities.

Flow Telemetry vs. Full Packet Capture (PCAP)

Telemetry TypeData CapturedStorage OverheadForensic GranularityPrimary SOC Application
NetFlow / IPFIX5-tuple, bytes, packets, duration, TCP flagsMinimal (~1% of network volume)Session-level metadata; no payload contentVolumetric anomaly detection, C2 beaconing analysis, data staging identification
sFlowStatistical packet samples (e.g., 1/1000th)Extremely LowStatistical estimation; packet headers sampledCore backbone routing analysis, volumetric DDoS detection
Full Packet Capture (PCAP)Complete Layer 2–Layer 7 bitstream and payloadsMassive (terabytes per hour)Maximum; exact payloads, commands, and files recoveredDeep protocol dissection, zero-day payload extraction, malware reverse engineering

3. DNS (Domain Name System) Logs

DNS is a primary protocol abused by threat actors for command-and-control (C2) and data exfiltration. Monitored records include query types (A, AAAA, CNAME, TXT, MX), query names, client IPs, and response codes:

  • NXDOMAIN Spikes: A sudden surge in "Non-Existent Domain" (NXDOMAIN) responses indicates malware utilizing Domain Generation Algorithms (DGAs) attempting to locate an active, registered rendezvous domain.
  • DNS Tunneling Indicators: Adversaries encode payloads within subdomains of an attacker-controlled authoritative name server (e.g., exfil.a8f9b2c3d4.attacker.com). Indicators include:
    • Abnormally long Fully Qualified Domain Names (FQDNs > 100 characters).
    • Exceptionally high entropy (randomness) in subdomain strings.
    • High query frequency for TXT or NULL record types containing base64 or hexadecimal data.
  • Fast-Flux DNS Evasion: Threat actors cycle through dozens of IP addresses mapped to a single domain name with very low Time-To-Live (TTL) values (e.g., 60 seconds), rapidly shifting front-end proxy nodes to evade IP blocklists.

4. DHCP Logs (Dynamic Host Configuration Protocol)

DHCP logs map dynamic IP leases over time, providing the critical forensic link between an ephemeral IP address, the endpoint's physical Hardware Media Access Control (MAC) address, and the registered host name. Without DHCP logs, retrospective investigation of historical IP-based alerts is virtually impossible in enterprise LANs.

5. Proxy and Secure Web Gateway (SWG) Logs

Proxies terminate outbound HTTP/HTTPS sessions, providing granular visibility into requested URLs, HTTP methods (GET, POST, PUT), User-Agent headers, MIME types, and referrer headers. When SSL/TLS inspection (decryption) is enabled, proxies log the inner decrypted URI query parameters and inspection certificates, alerting on unauthorized outbound web categories or unclassified domains.


Web Server Access and Error Logs

Web servers record every client transaction. Analysts inspect web logs to detect reconnaissance scans, parameter tampering, and application-layer attacks.

Standard Web Log Formats

1. Apache / Nginx Common Log Format (CLF) and Combined Log Format

The standard Nginx/Apache Combined Log Format structure is: %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i"

# Annotated Apache Combined Log Entry
198.51.100.25 - dbadmin [05/Sep/2026:14:23:18 +0000] "POST /api/v1/auth HTTP/1.1" 200 4521 "https://example.com/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"
  • 198.51.100.25: Client IP address (%h).
  • -: Remote logname (%l, rarely used).
  • dbadmin: Authenticated username (%u).
  • [05/Sep/2026:14:23:18 +0000]: Timestamp and UTC offset (%t).
  • "POST /api/v1/auth HTTP/1.1": HTTP request line (%r).
  • 200: HTTP response status code (%>s).
  • 4521: Transferred payload size in bytes (%b).
  • "https://example.com/login": HTTP Referer header.
  • "Mozilla/5.0...": Client User-Agent string.

2. Microsoft IIS W3C Extended Log Format

IIS logs use space-delimited text files where fields are explicitly defined in the file header (#Fields:):

#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status
2026-09-05 14:35:12 10.0.1.10 GET /admin/dashboard.aspx id=10 443 - 203.0.113.88 Mozilla/5.0 401 1 1326

IIS introduces two critical forensic fields:

  • sc-substatus: Detailed error code (e.g., 401.1 = Logon failed; 401.2 = Logon failed due to server configuration; 403.4 = SSL required).
  • sc-win32-status: Underlying Windows operating system error code (e.g., 0 = Success; 5 = ERROR_ACCESS_DENIED; 1326 = ERROR_LOGON_FAILURE, confirming invalid credentials).

HTTP Status Code SOC Interpretation Matrix

HTTP Status CodeStandard DefinitionSecurity Analyst Interpretation / Attack Indicators
200 OKRequest succeededBaseline traffic. If paired with an exploit query (e.g., id=1' UNION SELECT), indicates potential vulnerability exploitation success.
204 No ContentAction completed, no body returnedAPI operations; potential blind command injection or asynchronous exfiltration channel.
301 / 302Permanent / Temporary RedirectCan indicate open redirect exploitation or routing users to external credential harvesting portals.
304 Not ModifiedClient cached version validNormal caching behavior; used in web shell heartbeat checks.
400 Bad RequestMalformed HTTP request syntaxProtocol fuzzing, HTTP request smuggling, or oversized header attacks.
401 UnauthorizedAuthentication required / failedPassword-spraying, credential stuffing, or brute-force attacks against login endpoints.
403 ForbiddenServer refuses to authorize requestWeb Application Firewall (WAF) blocking, restricted directory access attempts (/etc/passwd, /WEB-INF/).
404 Not FoundResource does not existWeb directory fuzzing, automated discovery (Nikto, Gobuster, Dirbuster), or failed web shell probe.
405 Method Not AllowedMethod not supported for URIProbing for dangerous HTTP methods (e.g., attempting PUT or DELETE on a static asset).
429 Too Many RequestsRate limit exceededHigh-speed automated brute-forcing or API endpoint scraping.
500 Internal ErrorUnhandled server-side exceptionApplication fault; frequently triggered by SQL injection syntax errors, buffer overflows, or deserialization flaws.
502 / 504 Gateway ErrorUpstream server failure / timeoutDoS/DDoS condition exhausting backend application pools or database connection limits.

Common Attack Patterns in Web Access Logs

# 1. SQL Injection (SQLi) - Attempting to extract database version
GET /products.php?id=5%20UNION%20SELECT%20null,version(),null HTTP/1.1" 200 8912

# 2. Path Traversal / Local File Inclusion (LFI) - Targeting sensitive Linux files
GET /view.php?page=../../../../etc/passwd HTTP/1.1" 403 241

# 3. Cross-Site Scripting (XSS) - Reflected script payload in parameter
GET /search.aspx?q=%3Cscript%3Ealert(document.cookie)%3C/script%3E HTTP/1.1" 200 1205

# 4. Automated Vulnerability Scanner Identification
GET /login.action HTTP/1.1" 404 312 "-" "sqlmap/1.7.2#stable (https://sqlmap.org)"

# 5. Web Shell Interaction - Repetitive POSTs with low byte variations
POST /uploads/shell.aspx?cmd=whoami HTTP/1.1" 200 48

Database Activity Telemetry

Databases house the organization's high-value data assets. Database logs provide visibility into authentication, schema modifications, and data extraction.

Database Log Architecture

  • Error Log: Records database service startups, shutdowns, deadlocks, corruption events, and failed authentications.
  • Transaction / Binary Log (WAL / Redo Log): Records all state changes to the database (used for replication and point-in-time recovery; can help reconstruct changed rows when it is enabled, retained, and protected).
  • Slow Query Log: Captures queries exceeding a predefined runtime threshold; useful for detecting unindexed SQL injection payloads causing table locks.
  • Audit Log / Audit Trail: Records targeted security events, capturing exact SQL statements, executing users, client IP addresses, and session timestamps.

DDL vs. DML Telemetry

  • Data Definition Language (DDL): Commands that alter database structures (CREATE, ALTER, DROP, TRUNCATE). DDL logging alerts on adversaries disabling audit triggers, creating backdoor accounts, or dropping entire tables.
  • Data Manipulation Language (DML): Commands that query or alter stored records (SELECT, INSERT, UPDATE, DELETE). Excessive SELECT volume logged by a non-reporting account indicates mass data exfiltration.

Enterprise Database Platforms

  1. Microsoft SQL Server:
    • Logs failed authentications in the ErrorLog as Error 18456 (e.g., Error: 18456, Severity: 14, State: 8 indicates an invalid password, whereas State: 5 indicates an invalid user ID).
    • SQL Server Audit: Configured at the server and database level, capturing privilege escalation commands like EXEC sp_addsrvrolemember 'eviluser', 'sysadmin'.
  2. Oracle Database:
    • Unified Auditing: Consolidates audit records into the UNIFIED_AUDIT_TRAIL view.
    • listener.log: Records client network connections, TNS service requests, and unauthorized connection attempts before authentication occurs.
  3. MySQL / MariaDB / PostgreSQL:
    • MySQL utilizes the General Query Log (records all received queries; resource-intensive) or specialized plugins (e.g., MariaDB Audit Plugin) to capture query statements and failed logins.
    • PostgreSQL deploys the pg_audit extension to record session-level and object-level SQL statements into syslog.

Network vs. Web vs. Database Telemetry Comparison

Telemetry LayerPrimary Log FormatsCore Security Artifacts CapturedPrimary Attack Detection Capabilities
Network LayerNetFlow, IPFIX, sFlow, Firewall logs5-tuple, session duration, byte/packet counts, NAT mappingsC2 beaconing, port scanning, data exfiltration channels, unauthorized lateral movement.
Web LayerApache Combined, IIS W3C ExtendedClient IP, URI query parameters, HTTP status/substatus, User-AgentSQL injection, Cross-Site Scripting (XSS), directory traversal, web shell execution.
Database LayerSQL Audit, ErrorLog, Unified AuditExecuted SQL queries, DB users, DDL/DML actions, client DB driversPrivilege abuse (sysadmin escalation), mass record exfiltration, unauthorized schema drops.

Email Server, Gateway, and Cloud-Mail Logs

Email incident detection requires more than inspecting the visible From: field. Build a timeline from several layers:

Evidence sourceHigh-value fieldsWhat it helps answer
SMTP/MTA logsqueue or message ID, connecting IP, envelope sender/recipient, response code, next hopWas the message accepted, rejected, relayed, deferred, or delivered?
Secure email gatewayverdict, policy action, URL rewrite, attachment hash, sandbox resultWhich control evaluated the message and what action did it take?
Cloud message tracenetwork message ID, sender/recipient, delivery status, connector, timestampsWhich mailboxes received copies and where did routing change?
Authentication resultsSPF result and domain, DKIM result and d= domain, DMARC alignment and dispositionDid authenticated identifiers align with the visible author domain?
Mailbox auditinbox-rule changes, forwarding, delegate access, message deletion, OAuth application activityWas the mailbox itself abused before or after delivery?
Identity and endpoint logssign-in IP/device, token risk, browser process, URL click, downloaded file hashDid a recipient interact, authenticate, or execute content?

Follow Received: headers from the first header added by infrastructure you trust; attacker-supplied lower headers can be forged. Preserve the raw RFC 5322 message, not just a screenshot, and correlate timestamps in UTC. For Business Email Compromise, also hunt for external forwarding, lookalike domains, newly granted mailbox delegates, OAuth consent, payment-thread manipulation, and unusual sign-ins. Delivery proves only that a message reached a system—it does not prove a user clicked, entered credentials, or executed an attachment.

Loading diagram...
End-to-End Web Application Request Flow and Telemetry Generation
Test Your Knowledge

How does NetFlow/IPFIX telemetry fundamentally differ from full packet capture (PCAP) and deep packet inspection proxy logs when monitoring network communications?

A
B
C
D
Test Your Knowledge

When analyzing a Microsoft IIS web server access log formatted in W3C Extended format, an analyst observes the following entry: '2026-09-05 14:22:10 192.168.1.50 GET /admin/login.aspx - 443 - 10.0.4.15 Mozilla/5.0 401 1 1326'. What does the HTTP sub-status and Win32 status combination '401 1 1326' signify?

A
B
C
D
Test Your Knowledge

A SOC analyst inspects internal DNS server telemetry and discovers thousands of queries requesting TXT records with base64-encoded strings exceeding 180 characters prepended to a single external top-level domain. Which attack technique does this telemetry pattern indicate?

A
B
C
D
Test Your Knowledge

In database activity monitoring (DAM), which statement correctly differentiates Data Definition Language (DDL) logging from Data Manipulation Language (DML) logging from a security perspective?

A
B
C
D