8.2 Web Server and Application Log Analysis (Apache, Nginx, IIS)

Key Takeaways

  • Combined Log Format (CLF) captures client IP, ident, authuser, timestamp, request line, HTTP status, bytes sent, referer, and user-agent across Apache and Nginx web servers.

  • Microsoft IIS leverages the W3C Extended Log Format, recording granular substatus codes (sc-substatus) and Win32 error codes (sc-win32-status) to pinpoint specific authorization, configuration, and execution failures.

  • HTTP status code analysis allows incident handlers to differentiate automated directory scanning (dense clusters of 404/403 errors) from active web shell execution (low-frequency 200 OK responses to POST requests on script files).

  • Rapid command-line log triage utilizing awk, grep, sort, uniq, and sed enables analysts to isolate top requesting IPs, calculate error rates, extract rare user-agent strings, and identify obfuscated exploit payloads.

  • Web Application Firewall (WAF) alerts from ModSecurity, AWS WAF, or Cloudflare must be correlated directly with backend database audit logs to confirm whether injection payloads successfully executed against internal tables.

Last updated: October 2026

Web Server and Application Log Analysis (Apache, Nginx, IIS)

During a web application security incident, web server access and error logs serve as the definitive record of incoming adversary activity. Because web servers sit at the ingress point of enterprise application traffic, their access logs capture every incoming HTTP request, parameters, client metadata, and resulting server responses. Incident handlers must be proficient in parsing standard server log formats, forensically interpreting HTTP status codes, executing rapid command-line triage, and correlating edge Web Application Firewall (WAF) alerts with backend database transaction records.

Web Server Logging Formats: Apache/Nginx CLF and Microsoft IIS W3C

Web platforms record transaction telemetry using standardized schemas across two foundational formats:

NCSA Combined Log Format (Apache and Nginx)

The Combined Log Format (CLF) is the default standard for Apache HTTP Server and Nginx:

198.51.100.47 - admin [03/Oct/2026:14:22:15 +0000] "POST /wp-login.php HTTP/1.1" 200 4812 "https://example.com/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"

Dissecting each field from left to right:

  1. Client IP Address (198.51.100.47): The remote host making the HTTP request. Behind reverse proxies, CDNs, or load balancers, this field records the proxy IP. Responders configure mod_remoteip or real_ip_header to extract true client IPs from the X-Forwarded-For header.
  2. RFC 1413 Ident (-): Identity check of the client; typically disabled and recorded as a hyphen.
  3. Authenticated User (admin): HTTP Basic/Digest authenticated username, or a hyphen if unauthenticated.
  4. Timestamp ([03/Oct/2026:14:22:15 +0000]): Date, time, and UTC offset when the server completed the request.
  5. Request Line ("POST /wp-login.php HTTP/1.1"): HTTP method, requested URI stem, and protocol version.
  6. Status Code (200): Three-digit HTTP response code sent to the client.
  7. Response Size (4812): Body size in bytes, excluding HTTP headers.
  8. Referer ("https://example.com/login"): Source URL linking the client to the requested resource.
  9. User-Agent: Client software, browser engine, operating system, or automated scanner identifier.

W3C Extended Log Format (Microsoft IIS)

Microsoft IIS utilizes the space-delimited W3C Extended Log Format, featuring directional prefixes: c- (client), s- (server), cs- (client-to-server), and sc- (server-to-client):

2026-10-03 14:22:15 10.0.0.5 POST /upload/avatar.aspx - 443 - 198.51.100.47 Mozilla/5.0 - 200 0 0 156

Crucial fields in W3C IIS logs include:

  • cs-method, cs-uri-stem, cs-uri-query: The HTTP verb, file path, and query string parameters.
  • sc-status: Standard HTTP status code (e.g., 200, 401, 403, 404, 500).
  • sc-substatus: Proprietary IIS substatus codes that pinpoint the exact technical root cause. For example, 401.1 indicates invalid logon credentials; 403.4 indicates SSL is required; 404.13 indicates content length was exceeded.
  • sc-win32-status: The underlying Windows OS error code. A value of 0 denotes success, 1326 represents ERROR_LOGON_FAILURE (bad credentials), and 64 indicates the network name is unavailable.
  • time-taken: Processing duration in milliseconds, vital for detecting time-based blind SQL injection.

Forensic Interpretation of HTTP Status Codes

Status codes reveal operational interactions between adversary payloads and application logic:

  • 200 OK: Successful execution. While normal in routine traffic, a 200 OK following a POST request to an unlinked script or uploaded file indicates web shell command execution.
  • 301 / 302 / 304: Redirections and cached responses. Frequent 302 redirects during authentication flows highlight bypasses or open redirect phishing chains.
  • 401 Unauthorized / 403 Forbidden: Access denial. Clustered 401/403 responses indicate brute-force attempts, directory traversal hits, or WAF blocks.
  • 404 Not Found: Missing resource. Automated directory brute-forcing (via Gobuster, Dirbuster, or ffuf) produces dense bursts of hundreds or thousands of 404 errors within seconds as tools fuzz common paths (/admin, /.git, /config.bak).
  • 500 Internal Server Error: Server-side unhandled exception. During attacks, 500 errors are premier indicators of SQL injection syntax faults, template crashes, or deserialization exceptions where payloads broke the backend parser.

Forensic Footprint of Web Shell Execution

Unlike noisy scanners generating thousands of 404s, active web shells display a low-noise signature:

  1. Initial 200 or 302 response to a file upload URI (/upload.php).
  2. Subsequent isolated POST requests to an unexpected script path (e.g., /uploads/temp_img.php).
  3. Each request returns a 200 OK status code.
  4. Response byte sizes (%b) vary dynamically depending on command output (e.g., 120 bytes for whoami, 45,000 bytes for network dumps).

Command-Line Log Triage and Investigation Pipelines

Handlers utilize Linux command-line utilities to rapidly extract actionable intelligence:

  • Top Requesting IP Addresses:
    awk '{print \$1}' access.log | sort | uniq -c | sort -nr | head -n 20
    
  • Automated Directory Scanning Detection:
    awk '(\$9 ~ /404/) {print \$1}' access.log | sort | uniq -c | sort -nr | head -n 15
    
  • Isolating Suspicious Script POST Requests:
    grep "POST" access.log | grep -E "\.(php|jsp|aspx|cgi|pl)" | awk '{print \$1, \$4, \$7, \$9, \$10}'
    
  • Anomalous User-Agents:
    awk -F'"' '{print \$6}' access.log | sort | uniq -c | sort -n | head -n 20
    
  • Extracting SQLi Payloads:
    grep -Ei "(union.*select|waitfor.*delay|order.*by|information_schema|benchmark\()" access.log
    

WAF Telemetry and Correlation with Database Audit Logs

Web Application Firewalls (ModSecurity, AWS WAF, Cloudflare) generate rich diagnostic telemetry:

  • ModSecurity Audit Logs: Record transactional sections: Section A (audit header), Section B (request headers), Section C (request body), Section F (response headers), and Section H (audit trailer containing matched OWASP Core Rule Set / CRS rule ID and anomaly score).
  • Cloud-Native WAF Logs: AWS WAF records JSON detailing the action (BLOCK, ALLOW, COUNT), terminatingRuleId, client IP, and URI.

Cross-Tier Database Log Correlation

Web logs showing SQLi payloads prove attempts, not confirmed breaches. To verify impact, handlers correlate web logs with database audit logs (MySQL general_log, PostgreSQL log_statement = 'all', or Microsoft SQL Server Audit):

  1. Note timestamp, source IP, and session ID from the web log.
  2. Search database transaction logs within that temporal window.
  3. Verify whether the injected SQL executed successfully against backend tables, failed with a syntax error, or was neutralized by parameterization.
Loading diagram...
Anatomy of an HTTP Access Log Entry and Forensic Decoding
Test Your Knowledge

An incident handler investigates an intrusion against a Microsoft Internet Information Services (IIS) web server using W3C Extended Log telemetry. The handler observes a request to /admin/manage.aspx that returned sc-status 401, sc-substatus 1, and sc-win32-status 1326. What does this specific log entry indicate from an evidentiary standpoint?

A

The request succeeded with an administrative redirect to a maintenance portal

B

The server suffered an unhandled ASP.NET runtime exception caused by SQL injection

C

The client was denied access because SSL/TLS encryption was required by IIS configuration

D

The request failed due to invalid logon credentials, specifically corresponding to Windows ERROR_LOGON_FAILURE

Test Your Knowledge

While analyzing an Apache web server access log, an analyst compares two distinct traffic patterns: Pattern X features 12,000 requests within 90 seconds from a single IP address requesting non-existent paths (/admin.php, /.env, /backup.tar.gz, /wp-config.php.bak) with nearly all responses returning HTTP 404. Pattern Y features an isolated series of 15 HTTP POST requests over three hours directed to /media/icons/nav.php, each returning HTTP 200 OK with varying response body byte counts (between 85 and 12,400 bytes). How should the analyst classify these two patterns?

A

Pattern X represents automated web content enumeration/directory scanning, while Pattern Y represents active command execution via a deployed web shell

B

Pattern X represents a distributed denial-of-service volumetric flood, while Pattern Y represents standard search engine spider indexing

C

Pattern X represents successful data exfiltration via boolean-based blind SQL injection, while Pattern Y represents reflected XSS credential theft

D

Pattern X represents server-side session fixation, while Pattern Y represents a CSRF attack on administrative settings

Test Your Knowledge

A security analyst needs to rapidly triage a 10 GB Apache access.log on a compromised Linux host during an active incident. Which command-line pipeline correctly identifies the top 10 external IP addresses generating HTTP 404 Not Found responses to detect automated scanner activity?

A

grep "200" access.log | cut -d' ' -f1 | sort | uniq | head -n 10

B

awk '($9 ~ /404/) {print $1}' access.log | sort | uniq -c | sort -nr | head -n 10

C

sed -n 's/404//p' access.log | awk '{print $7}' | uniq -c | sort -n | tail -n 10

D

cat access.log | cut -d'"' -f2 | grep -v 404 | sort | uniq -c | head -n 10

Sections you finish are checked off in the contents.