13.3 Cloud Logging: Buckets, Log Analytics, Routers & Filtering

Why This Topic Matters on the Exam

Domain 4's monitoring-and-logging cluster is the single densest part of the ACE blueprint, and Cloud Logging alone accounts for four distinct official bullets: exporting logs, configuring buckets/analytics/routers, viewing and filtering logs, and viewing specific log message details. The exam tests whether you understand how a log entry actually flows from being generated to being stored, queryable, and (optionally) exported elsewhere — not just that "Cloud Logging stores logs."

How a Log Entry Flows: The Log Router

Every log entry generated inside a Google Cloud project — audit logs, platform logs (e.g., VPC Flow Logs, GKE logs), and application logs written via the Logging API or Ops Agent — passes through the project's Log Router. The Log Router evaluates the entry against a list of sinks. Each sink has an inclusion filter (which entries it matches) and can have exclusion filters (carve-outs even from a matched inclusion filter), and routes a copy of every matching entry to a destination:

• A Cloud Logging bucket (the default and most common destination)
• A Pub/Sub topic (for streaming to external systems)
• A BigQuery dataset (for large-scale SQL analytics)
• A Cloud Storage bucket (for low-cost, long-term archival)
• Another project's Logging bucket (for centralized, org-wide aggregation)

Every project is created with two built-in sinks you cannot delete:

Log Buckets and Retention

A log bucket is a regional container that stores log entries. Beyond the two built-in buckets above, you can create custom log buckets with retention configurable from 1 day up to 3,650 days (10 years) and route specific log types into them via a custom sink — for example, routing only VPC Flow Logs and firewall logs into a security-team-owned bucket retained for two years, independent of the 30-day _Default retention.

Log Analytics

Log Analytics is an optional upgrade you enable on a log bucket that links it to a BigQuery-like SQL query engine directly inside Logs Explorer. This lets you run ad hoc SQL queries against log data — joins, aggregations, time-series analysis — without first exporting the data to a separate BigQuery dataset (which would mean paying for and maintaining a second copy). This is the key distinction to remember: Log Analytics queries logs in place in a Logging bucket; a BigQuery sink instead creates a separate, exported copy of the data as native BigQuery tables.

Filtering and Viewing Logs (Logs Explorer)

The Logs Explorer query language lets you narrow millions of log entries down to the handful relevant to an investigation. Typical filter components:

resource.type="gce_instance"
AND severity>=ERROR
AND timestamp>="2026-07-01T00:00:00Z"
AND jsonPayload.message=~"connection refused"

• Comparisons on severity use Logging's ordered scale (DEFAULT < DEBUG < INFO < NOTICE < WARNING < ERROR < CRITICAL < ALERT < EMERGENCY).
• =~ performs a regular-expression match; = is exact match.
• Filters can target resource.type, resource.labels.*, logName, severity, timestamp, and any field inside jsonPayload, textPayload, protoPayload, or httpRequest.

To view specific log message details, expanding a single entry in Logs Explorer reveals its full structured payload — the complete JSON body, HTTP request fields (status code, latency, user agent), trace ID (for cross-referencing with Cloud Trace), and resource labels — which is how you go from "an error happened" to "here is exactly what request caused it and on which instance."

Exporting Logs to External Systems

Beyond storing logs in Cloud Logging buckets, the "exporting logs" bullet covers routing copies out of Cloud Logging via a sink:

• To Pub/Sub — for real-time streaming into a third-party SIEM or log-analytics platform (a common pattern is a Dataflow template that reads from Pub/Sub and writes into Splunk).
• To BigQuery — for long-term, large-scale SQL analytics using native BigQuery compute, separate from Log Analytics' in-place querying.
• To Cloud Storage — for the cheapest long-term, compliance-driven archival, typically written as periodic batch files rather than queried directly.

Applying It: A Worked Scenario

"The security team wants VPC Flow Logs and firewall logs retained for two years and wants to run SQL queries against them without duplicating the data into a separate BigQuery dataset."

The correct configuration: create a custom log bucket with retention set to 730 days, enable Log Analytics on that bucket, and create a sink whose inclusion filter matches VPC Flow Logs and firewall rule logs, routing them into the custom bucket. No BigQuery export or Pub/Sub is needed — Log Analytics on a Logging bucket satisfies both the retention and the SQL-query requirement directly.

Section Takeaways

• Every log entry passes through the project's Log Router, which evaluates sinks (inclusion + exclusion filters) and routes matching copies to a destination bucket, Pub/Sub topic, BigQuery dataset, Cloud Storage bucket, or another project's bucket.
• The _Required bucket (Admin Activity, System Event, Policy Denied, Access Transparency) has a fixed, non-configurable 400-day retention; the _Default bucket defaults to 30 days but is configurable.
• Custom log buckets support 1-3,650 days of retention and let you isolate specific log types with their own policy.
• Log Analytics queries log data with SQL in place inside a Logging bucket — it is not the same as exporting to a separate BigQuery dataset.
• Logs Explorer filters use resource/severity/timestamp/payload fields with = for exact match and =~ for regex; expanding an entry reveals the full structured payload for root-cause detail.