4.2 Network Device, Web Server, and Database Logs
Key Takeaways
- Network telemetry encompasses stateful firewall traffic logs, router/switch flow records (NetFlow/IPFIX vs. sFlow), DNS resolution telemetry, and DHCP lease mappings.
- NetFlow and IPFIX capture complete 5-tuple session summaries without packet payloads, whereas sFlow relies on statistical packet sampling at fixed intervals.
- DNS telemetry provides critical indicators of compromise, including high-volume NXDOMAIN spikes indicative of Domain Generation Algorithms (DGAs) and oversized, high-entropy TXT records indicative of DNS tunneling.
- Web server logs record client interactions in standardized structures like Apache Combined Log Format and IIS W3C Extended Format, where sub-status and Win32 status codes isolate authentication failures and web vulnerabilities.
- Email investigations correlate gateway verdicts, message trace, SMTP routing, authentication results, mailbox audit events, URL clicks, and attachment hashes; a single header or verdict is not conclusive.
Network Infrastructure Telemetry Sources
Network devices provide essential visibility into communication traversing internal segmentation boundaries and external perimeters. Because network devices cannot inspect volatile endpoint memory, their logs focus on state transitions, session metadata, protocol headers, and volumetric trends.
1. Firewall Logs
Modern stateful and Next-Generation Firewalls (NGFW) monitor TCP, UDP, and ICMP connection states. Key telemetry fields recorded per connection include:
- Action: Allowed (Permit) vs. Dropped (Deny/Reset).
- Network Address Translation (NAT): Pre-NAT and Post-NAT IP addresses and ports (SNAT for outbound client masquerading; DNAT for inbound service publishing).
- Rule ID / Security Policy: The specific rule permitting or denying the packet.
- Interface and Zone: Source and destination security zones (e.g.,
Trust,Untrust,DMZ). - Egress Filtering Alerts: Denied outbound connections originating from internal servers toward uncommon foreign destination ports (often indicative of active malware beacons or reverse shells).
2. Routers and Switches: Flow Telemetry (NetFlow, IPFIX, sFlow)
Routers and layer-3 switches generate session-based network accounting records without capturing complete packet payloads:
- NetFlow (Cisco) / IPFIX (IETF RFC 7011): Aggregates continuous packet streams into flow records characterized by the 5-tuple: Source IP, Destination IP, Source Port, Destination Port, and Layer 4 Protocol. Records also detail byte counts, packet counts, duration, and TCP flags (e.g., SYN, ACK, FIN, RST).
- sFlow (Sampled Flow - RFC 3176): Rather than tracking every session flow state, sFlow takes statistical packet samples (e.g., 1 out of every 1,000 packets) directly from network interfaces. While computationally lightweight on high-throughput core backbones, sFlow can miss low-and-slow, single-packet threat activities.
Flow Telemetry vs. Full Packet Capture (PCAP)
| Telemetry Type | Data Captured | Storage Overhead | Forensic Granularity | Primary SOC Application |
|---|---|---|---|---|
| NetFlow / IPFIX | 5-tuple, bytes, packets, duration, TCP flags | Minimal (~1% of network volume) | Session-level metadata; no payload content | Volumetric anomaly detection, C2 beaconing analysis, data staging identification |
| sFlow | Statistical packet samples (e.g., 1/1000th) | Extremely Low | Statistical estimation; packet headers sampled | Core backbone routing analysis, volumetric DDoS detection |
| Full Packet Capture (PCAP) | Complete Layer 2–Layer 7 bitstream and payloads | Massive (terabytes per hour) | Maximum; exact payloads, commands, and files recovered | Deep protocol dissection, zero-day payload extraction, malware reverse engineering |
3. DNS (Domain Name System) Logs
DNS is a primary protocol abused by threat actors for command-and-control (C2) and data exfiltration. Monitored records include query types (A, AAAA, CNAME, TXT, MX), query names, client IPs, and response codes:
- NXDOMAIN Spikes: A sudden surge in "Non-Existent Domain" (NXDOMAIN) responses indicates malware utilizing Domain Generation Algorithms (DGAs) attempting to locate an active, registered rendezvous domain.
- DNS Tunneling Indicators: Adversaries encode payloads within subdomains of an attacker-controlled authoritative name server (e.g.,
exfil.a8f9b2c3d4.attacker.com). Indicators include:- Abnormally long Fully Qualified Domain Names (FQDNs > 100 characters).
- Exceptionally high entropy (randomness) in subdomain strings.
- High query frequency for TXT or NULL record types containing base64 or hexadecimal data.
- Fast-Flux DNS Evasion: Threat actors cycle through dozens of IP addresses mapped to a single domain name with very low Time-To-Live (TTL) values (e.g., 60 seconds), rapidly shifting front-end proxy nodes to evade IP blocklists.
4. DHCP Logs (Dynamic Host Configuration Protocol)
DHCP logs map dynamic IP leases over time, providing the critical forensic link between an ephemeral IP address, the endpoint's physical Hardware Media Access Control (MAC) address, and the registered host name. Without DHCP logs, retrospective investigation of historical IP-based alerts is virtually impossible in enterprise LANs.
5. Proxy and Secure Web Gateway (SWG) Logs
Proxies terminate outbound HTTP/HTTPS sessions, providing granular visibility into requested URLs, HTTP methods (GET, POST, PUT), User-Agent headers, MIME types, and referrer headers. When SSL/TLS inspection (decryption) is enabled, proxies log the inner decrypted URI query parameters and inspection certificates, alerting on unauthorized outbound web categories or unclassified domains.
Web Server Access and Error Logs
Web servers record every client transaction. Analysts inspect web logs to detect reconnaissance scans, parameter tampering, and application-layer attacks.
Standard Web Log Formats
1. Apache / Nginx Common Log Format (CLF) and Combined Log Format
The standard Nginx/Apache Combined Log Format structure is:
%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i"
# Annotated Apache Combined Log Entry
198.51.100.25 - dbadmin [05/Sep/2026:14:23:18 +0000] "POST /api/v1/auth HTTP/1.1" 200 4521 "https://example.com/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"
198.51.100.25: Client IP address (%h).-: Remote logname (%l, rarely used).dbadmin: Authenticated username (%u).[05/Sep/2026:14:23:18 +0000]: Timestamp and UTC offset (%t)."POST /api/v1/auth HTTP/1.1": HTTP request line (%r).200: HTTP response status code (%>s).4521: Transferred payload size in bytes (%b)."https://example.com/login": HTTP Referer header."Mozilla/5.0...": Client User-Agent string.
2. Microsoft IIS W3C Extended Log Format
IIS logs use space-delimited text files where fields are explicitly defined in the file header (#Fields:):
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status
2026-09-05 14:35:12 10.0.1.10 GET /admin/dashboard.aspx id=10 443 - 203.0.113.88 Mozilla/5.0 401 1 1326
IIS introduces two critical forensic fields:
sc-substatus: Detailed error code (e.g.,401.1= Logon failed;401.2= Logon failed due to server configuration;403.4= SSL required).sc-win32-status: Underlying Windows operating system error code (e.g.,0= Success;5=ERROR_ACCESS_DENIED;1326=ERROR_LOGON_FAILURE, confirming invalid credentials).
HTTP Status Code SOC Interpretation Matrix
| HTTP Status Code | Standard Definition | Security Analyst Interpretation / Attack Indicators |
|---|---|---|
| 200 OK | Request succeeded | Baseline traffic. If paired with an exploit query (e.g., id=1' UNION SELECT), indicates potential vulnerability exploitation success. |
| 204 No Content | Action completed, no body returned | API operations; potential blind command injection or asynchronous exfiltration channel. |
| 301 / 302 | Permanent / Temporary Redirect | Can indicate open redirect exploitation or routing users to external credential harvesting portals. |
| 304 Not Modified | Client cached version valid | Normal caching behavior; used in web shell heartbeat checks. |
| 400 Bad Request | Malformed HTTP request syntax | Protocol fuzzing, HTTP request smuggling, or oversized header attacks. |
| 401 Unauthorized | Authentication required / failed | Password-spraying, credential stuffing, or brute-force attacks against login endpoints. |
| 403 Forbidden | Server refuses to authorize request | Web Application Firewall (WAF) blocking, restricted directory access attempts (/etc/passwd, /WEB-INF/). |
| 404 Not Found | Resource does not exist | Web directory fuzzing, automated discovery (Nikto, Gobuster, Dirbuster), or failed web shell probe. |
| 405 Method Not Allowed | Method not supported for URI | Probing for dangerous HTTP methods (e.g., attempting PUT or DELETE on a static asset). |
| 429 Too Many Requests | Rate limit exceeded | High-speed automated brute-forcing or API endpoint scraping. |
| 500 Internal Error | Unhandled server-side exception | Application fault; frequently triggered by SQL injection syntax errors, buffer overflows, or deserialization flaws. |
| 502 / 504 Gateway Error | Upstream server failure / timeout | DoS/DDoS condition exhausting backend application pools or database connection limits. |
Common Attack Patterns in Web Access Logs
# 1. SQL Injection (SQLi) - Attempting to extract database version
GET /products.php?id=5%20UNION%20SELECT%20null,version(),null HTTP/1.1" 200 8912
# 2. Path Traversal / Local File Inclusion (LFI) - Targeting sensitive Linux files
GET /view.php?page=../../../../etc/passwd HTTP/1.1" 403 241
# 3. Cross-Site Scripting (XSS) - Reflected script payload in parameter
GET /search.aspx?q=%3Cscript%3Ealert(document.cookie)%3C/script%3E HTTP/1.1" 200 1205
# 4. Automated Vulnerability Scanner Identification
GET /login.action HTTP/1.1" 404 312 "-" "sqlmap/1.7.2#stable (https://sqlmap.org)"
# 5. Web Shell Interaction - Repetitive POSTs with low byte variations
POST /uploads/shell.aspx?cmd=whoami HTTP/1.1" 200 48
Database Activity Telemetry
Databases house the organization's high-value data assets. Database logs provide visibility into authentication, schema modifications, and data extraction.
Database Log Architecture
- Error Log: Records database service startups, shutdowns, deadlocks, corruption events, and failed authentications.
- Transaction / Binary Log (WAL / Redo Log): Records all state changes to the database (used for replication and point-in-time recovery; can help reconstruct changed rows when it is enabled, retained, and protected).
- Slow Query Log: Captures queries exceeding a predefined runtime threshold; useful for detecting unindexed SQL injection payloads causing table locks.
- Audit Log / Audit Trail: Records targeted security events, capturing exact SQL statements, executing users, client IP addresses, and session timestamps.
DDL vs. DML Telemetry
- Data Definition Language (DDL): Commands that alter database structures (
CREATE,ALTER,DROP,TRUNCATE). DDL logging alerts on adversaries disabling audit triggers, creating backdoor accounts, or dropping entire tables. - Data Manipulation Language (DML): Commands that query or alter stored records (
SELECT,INSERT,UPDATE,DELETE). ExcessiveSELECTvolume logged by a non-reporting account indicates mass data exfiltration.
Enterprise Database Platforms
- Microsoft SQL Server:
- Logs failed authentications in the ErrorLog as Error 18456 (e.g.,
Error: 18456, Severity: 14, State: 8indicates an invalid password, whereasState: 5indicates an invalid user ID). - SQL Server Audit: Configured at the server and database level, capturing privilege escalation commands like
EXEC sp_addsrvrolemember 'eviluser', 'sysadmin'.
- Logs failed authentications in the ErrorLog as Error 18456 (e.g.,
- Oracle Database:
- Unified Auditing: Consolidates audit records into the
UNIFIED_AUDIT_TRAILview. listener.log: Records client network connections, TNS service requests, and unauthorized connection attempts before authentication occurs.
- Unified Auditing: Consolidates audit records into the
- MySQL / MariaDB / PostgreSQL:
- MySQL utilizes the General Query Log (records all received queries; resource-intensive) or specialized plugins (e.g., MariaDB Audit Plugin) to capture query statements and failed logins.
- PostgreSQL deploys the
pg_auditextension to record session-level and object-level SQL statements into syslog.
Network vs. Web vs. Database Telemetry Comparison
| Telemetry Layer | Primary Log Formats | Core Security Artifacts Captured | Primary Attack Detection Capabilities |
|---|---|---|---|
| Network Layer | NetFlow, IPFIX, sFlow, Firewall logs | 5-tuple, session duration, byte/packet counts, NAT mappings | C2 beaconing, port scanning, data exfiltration channels, unauthorized lateral movement. |
| Web Layer | Apache Combined, IIS W3C Extended | Client IP, URI query parameters, HTTP status/substatus, User-Agent | SQL injection, Cross-Site Scripting (XSS), directory traversal, web shell execution. |
| Database Layer | SQL Audit, ErrorLog, Unified Audit | Executed SQL queries, DB users, DDL/DML actions, client DB drivers | Privilege abuse (sysadmin escalation), mass record exfiltration, unauthorized schema drops. |
Email Server, Gateway, and Cloud-Mail Logs
Email incident detection requires more than inspecting the visible From: field. Build a timeline from several layers:
| Evidence source | High-value fields | What it helps answer |
|---|---|---|
| SMTP/MTA logs | queue or message ID, connecting IP, envelope sender/recipient, response code, next hop | Was the message accepted, rejected, relayed, deferred, or delivered? |
| Secure email gateway | verdict, policy action, URL rewrite, attachment hash, sandbox result | Which control evaluated the message and what action did it take? |
| Cloud message trace | network message ID, sender/recipient, delivery status, connector, timestamps | Which mailboxes received copies and where did routing change? |
| Authentication results | SPF result and domain, DKIM result and d= domain, DMARC alignment and disposition | Did authenticated identifiers align with the visible author domain? |
| Mailbox audit | inbox-rule changes, forwarding, delegate access, message deletion, OAuth application activity | Was the mailbox itself abused before or after delivery? |
| Identity and endpoint logs | sign-in IP/device, token risk, browser process, URL click, downloaded file hash | Did a recipient interact, authenticate, or execute content? |
Follow Received: headers from the first header added by infrastructure you trust; attacker-supplied lower headers can be forged. Preserve the raw RFC 5322 message, not just a screenshot, and correlate timestamps in UTC. For Business Email Compromise, also hunt for external forwarding, lookalike domains, newly granted mailbox delegates, OAuth consent, payment-thread manipulation, and unusual sign-ins. Delivery proves only that a message reached a system—it does not prove a user clicked, entered credentials, or executed an attachment.
How does NetFlow/IPFIX telemetry fundamentally differ from full packet capture (PCAP) and deep packet inspection proxy logs when monitoring network communications?
When analyzing a Microsoft IIS web server access log formatted in W3C Extended format, an analyst observes the following entry: '2026-09-05 14:22:10 192.168.1.50 GET /admin/login.aspx - 443 - 10.0.4.15 Mozilla/5.0 401 1 1326'. What does the HTTP sub-status and Win32 status combination '401 1 1326' signify?
A SOC analyst inspects internal DNS server telemetry and discovers thousands of queries requesting TXT records with base64-encoded strings exceeding 180 characters prepended to a single external top-level domain. Which attack technique does this telemetry pattern indicate?
In database activity monitoring (DAM), which statement correctly differentiates Data Definition Language (DDL) logging from Data Manipulation Language (DML) logging from a security perspective?