13.3 Resource Monitors, Financial Governance & Usage Observability

13.3 Resource Monitors, Financial Governance & Usage Observability

Snowflake's multi-cluster shared data architecture separates storage from compute, allowing organizations to instantly provision, resize, and auto-scale virtual warehouses to match any workload demand. However, this effortless scalability introduces substantial financial governance risk: an unconstrained virtual warehouse, an accidental Cartesian product, or an unmonitored serverless ingestion pipeline can consume thousands of credits in hours.

To maintain enterprise budget predictability, architects implement a defense-in-depth FinOps governance framework consisting of three pillars:

1. Preventative Compute Guardrails: Enforcing hard credit quotas and execution caps via Resource Monitors and statement timeouts.
2. Telemetry & Metering Observability: Auditing historical consumption, storage utilization, and serverless compute using SNOWFLAKE.ACCOUNT_USAGE.
3. Automated Anomaly Detection: Triggering dynamic alerts and notifications when operational thresholds are breached.

For the SnowPro Advanced: Architect exam, you must master resource monitor configuration, understand exact trigger action mechanics, identify serverless monitoring exemptions, and formulate auditing queries to attribute costs across multi-tenant enterprise environments.

Resource Monitor Architecture & Quota Cycles

A Resource Monitor is a first-class Snowflake securable object that monitors the credit consumption of virtual warehouses. When credit usage reaches specified percentage thresholds of a defined quota, the monitor triggers automated alerts or state transitions.

┌─────────────────────────────────────────────────────────────────────────────┐
│                     Resource Monitor Architectural Structure                │
├─────────────────────────────────────────────────────────────────────────────┤
│  CREATE RESOURCE MONITOR rm_enterprise_bi                                   │
│    WITH CREDIT_QUOTA = 2000              ◄── Monthly Budget Allocation      │
│    FREQUENCY = MONTHLY                   ◄── Resets monthly from start date │
│    START_TIMESTAMP = IMMEDIATELY                                            │
│    NOTIFY_USERS = (finops_admin, dba)    ◄── Alert Recipients               │
│    TRIGGERS                                                                 │
│      ON 75 PERCENT DO NOTIFY             ◄── Advisory Warning               │
│      ON 90 PERCENT DO NOTIFY             ◄── Urgent Warning                 │
│      ON 100 PERCENT DO SUSPEND           ◄── Stop New Queries; Let Active Finish
│      ON 110 PERCENT DO SUSPEND_IMMEDIATE;◄── Abruptly Cancel All & Suspend  │
└─────────────────────────────────────────────────────────────────────────────┘

Core Configuration Attributes

1. CREDIT_QUOTA:
   • The total number of Snowflake credits allocated to the monitor for the specified frequency cycle (e.g., 2,000 credits).
2. FREQUENCY (Reset Schedule): DAILY, WEEKLY, MONTHLY, YEARLY, or NEVER. Used credits reset to zero at each interval relative to the monitor's start date; NEVER suits fixed-budget projects.
3. START_TIMESTAMP and END_TIMESTAMP: IMMEDIATELY or a future timestamp starts monitoring and anchors the reset schedule; an optional end timestamp stops it.
4. What counts: The quota includes credits used by the assigned warehouses and the cloud services credits associated with them. Resource monitor limits do not apply the daily 10% cloud services adjustment, so monitor totals can be higher than billed credits.

Monitor Scopes, Trigger Actions & Critical Restrictions

1. Resource Monitor Scopes: Account vs. Warehouse Level

Snowflake supports two distinct scopes of resource monitors:

                                  Account-Level Monitor
                                 [ Quota: 10,000 Credits ]
                                             │
                  ┌──────────────────────────┴──────────────────────────┐
                  ▼                                                     ▼
        Warehouse Monitor A                                   Warehouse Monitor B
       [ Quota: 3,000 Credits ]                              [ Quota: 4,000 Credits ]
                  │                                                     │
         ┌────────┴────────┐                                            │
         ▼                 ▼                                            ▼
   [ ANALYST_WH ]    [ REPORTING_WH ]                             [ ETL_LARGE_WH ]

• Account-Level Monitor:
  • Monitors the cumulative virtual warehouse credit consumption across all virtual warehouses provisioned in the entire Snowflake account.
  • An account can have at most one active account-level monitor.
  • If the account-level monitor triggers a suspension, every virtual warehouse in the account is suspended.
• Warehouse-Level Monitor:
  • Assigned to one or more specific virtual warehouses.
  • A virtual warehouse can be explicitly assigned to at most one warehouse-level monitor.
  • Multiple warehouses can share a single warehouse-level monitor (pooling their shared quota).
• Concurrent Evaluation:
  • If a warehouse is assigned to a warehouse-level monitor AND the account has an account-level monitor, the warehouse is subject to both monitors simultaneously. Whichever monitor reaches an enforcement threshold first dictates the warehouse's action.

2. Trigger Actions: NOTIFY vs. SUSPEND vs. SUSPEND_IMMEDIATE

A critical distinction on the SnowPro Advanced: Architect exam is the exact operational behavior of the three trigger actions:

Trigger Threshold Lifecycle
 0%                      75%                90%              100%                 110%
 ├────────────────────────┼──────────────────┼─────────────────┼────────────────────┤
 │ Normal Execution       │ NOTIFY           │ NOTIFY          │ SUSPEND            │ SUSPEND_IMMEDIATE
 │ Credits accumulating   │ Warning email    │ Escalation      │ In-flight finish;  │ Running queries
 │                        │ to users         │ alert           │ no new queries     │ killed instantly

• NOTIFY:
  • Sends an alert message to users specified in NOTIFY_USERS (and all users with the ACCOUNTADMIN role who have notification preferences enabled).
  • Zero impact on compute: Warehouses continue running; all queries execute normally.
• SUSPEND (Soft Suspend):
  • Triggered typically at 100% of quota.
  • In-flight queries are allowed to finish: Any query actively executing when the threshold is crossed will continue running until normal completion.
  • Blocks new queries: Any newly submitted queries fail immediately or are prevented from starting.
  • Once all running queries finish, the warehouse is suspended. If the queries take 30 minutes to complete, additional credits will be consumed, which is why actual spend can exceed 100%.
• SUSPEND_IMMEDIATE (Hard Suspend):
  • Triggered typically at a safety threshold above 100% (e.g., 105% or 110%).
  • Cancels all running queries immediately: Snowflake abruptly aborts all actively executing queries with an error code.
  • Suspension: The warehouse is suspended right away. Enforcement is not instantaneous to the credit, so a monitor can slightly exceed its threshold; set SUSPEND_IMMEDIATE at a level that leaves headroom.

3. Serverless Compute Exemption & Critical Governance Traps

[!CAUTION] THE #1 EXAM TRAP: Resource Monitors do NOT monitor, throttle, or suspend Serverless Compute Features.

Resource Monitors apply exclusively to Virtual Warehouse compute. The following Snowflake services operate outside of virtual warehouses and are completely exempt from resource monitor enforcement:

• Snowpipe (Continuous streaming and auto-ingest file loading)
• Automatic Clustering Service (ACS) (Background micro-partition re-clustering)
• Search Optimization Service (SOS) (Search access path maintenance)
• Materialized View Maintenance (Background refresh compute)
• Database & Account Replication (Cross-cloud replication compute and data transfer)
• Hybrid Tables (Unistore) and Serverless Tasks

If a team misconfigures search optimization or a clustering key, an account-level resource monitor will not stop that serverless spend. Architects govern it with:

• Budgets — an account budget monitors all credit usage in the account (including serverless features and AI services), and custom budgets monitor groups of objects such as tables, pipes, materialized views, or warehouses. A budget defines a monthly spending limit and sends notifications (email, cloud queue such as SNS/Event Grid/Pub/Sub, or webhooks like Slack) when spending is projected to exceed the limit. Budgets notify; they do not suspend features.
• Feature-level controls — pause pipes, suspend reclustering, drop unneeded search optimization, or adjust task schedules.
• Usage views in SNOWFLAKE.ACCOUNT_USAGE (below).

Snowflake Cost Management & Telemetry in ACCOUNT_USAGE

Enterprise FinOps requires retrospective auditing to allocate spend across business units, detect seasonal cost anomalies, and attribute serverless overhead. The SNOWFLAKE.ACCOUNT_USAGE schema provides institutional telemetry retained for 365 days.

Essential Accounting & Metering Views

ACCOUNT_USAGE vs. INFORMATION_SCHEMA Architectural Comparison

• SNOWFLAKE.ACCOUNT_USAGE:
  • Historical retention: 365 days (1 year).
  • Latency: Telemetry data has an ingestion latency of 45 minutes to 3 hours.
  • Scope: Account-wide, including dropped objects (tables, warehouses, schemas that no longer exist).
  • Security: Accessible only to ACCOUNTADMIN or roles granted SNOWFLAKE.USAGE_VIEWER / imported database privileges.
• INFORMATION_SCHEMA Table Functions:
  • Historical retention: 7 to 14 days.
  • Latency: Near real-time (seconds to minutes).
  • Scope: Current database and active objects only.

Production FinOps Auditing Queries

-- Query 1: Top 5 most expensive virtual warehouses over the past 30 days
SELECT 
    warehouse_name,
    ROUND(SUM(credits_used), 2) AS total_credits,
    ROUND(SUM(credits_used_compute), 2) AS compute_credits,
    ROUND(SUM(credits_used_cloud_services), 2) AS cloud_services_credits,
    ROUND(SUM(credits_used) * 3.00, 2) AS estimated_usd_spend -- Assuming $3/credit contract
FROM snowflake.account_usage.warehouse_metering_history
WHERE start_time >= DATEADD(day, -30, CURRENT_TIMESTAMP())
GROUP BY warehouse_name
ORDER BY total_credits DESC
LIMIT 5;

-- Query 2: Serverless compute attribution (Snowpipe vs ACS vs SOS vs Replication)
SELECT 
    'Automatic Clustering' AS service_type,
    ROUND(SUM(credits_used), 2) AS total_credits
FROM snowflake.account_usage.automatic_clustering_history
WHERE start_time >= DATEADD(day, -30, CURRENT_TIMESTAMP())
UNION ALL
SELECT 
    'Search Optimization' AS service_type,
    ROUND(SUM(credits_used), 2) AS total_credits
FROM snowflake.account_usage.search_optimization_history
WHERE start_time >= DATEADD(day, -30, CURRENT_TIMESTAMP())
UNION ALL
SELECT 
    'Snowpipe Ingestion' AS service_type,
    ROUND(SUM(credits_used), 2) AS total_credits
FROM snowflake.account_usage.pipe_usage_history
WHERE start_time >= DATEADD(day, -30, CURRENT_TIMESTAMP())
UNION ALL
SELECT 
    'Database Replication' AS service_type,
    ROUND(SUM(credits_used), 2) AS total_credits
FROM snowflake.account_usage.database_replication_usage_history
WHERE start_time >= DATEADD(day, -30, CURRENT_TIMESTAMP())
ORDER BY total_credits DESC;

Session & Warehouse Parameters for Runaway Query Control

While Resource Monitors protect against macro-level warehouse credit consumption, micro-level protection requires stopping individual pathological queries—such as accidental Cartesian joins or full-table scans on petabyte tables—before they exhaust warehouse compute slots.

1. Statement Timeout Parameters

Snowflake provides tiered timeout parameters to kill runaway statements automatically:

-- Account Level: Default is 172800 seconds (48 hours! - Severe runaway risk)
ALTER ACCOUNT SET STATEMENT_TIMEOUT_IN_SECONDS = 86400; -- Cap at 24 hours

-- Warehouse Level: Cap ad-hoc analyst warehouse queries at 30 minutes
ALTER WAREHOUSE analyst_adhoc_wh SET STATEMENT_TIMEOUT_IN_SECONDS = 1800;

-- Warehouse Level: Cap queued overload wait time at 5 minutes
ALTER WAREHOUSE analyst_adhoc_wh SET STATEMENT_QUEUED_TIMEOUT_IN_SECONDS = 300;

-- Session Level: a session can LOWER, but not raise, the warehouse limit
ALTER SESSION SET STATEMENT_TIMEOUT_IN_SECONDS = 600;  -- effective limit on analyst_adhoc_wh = 600 s

• STATEMENT_TIMEOUT_IN_SECONDS:
  • Measures the statement's total time, including queuing, compilation, and execution.
  • If the limit is exceeded, Snowflake cancels the statement and rolls back its uncommitted changes.
  • Precedence: resolve ACCOUNT → USER → SESSION (most specific wins), then compare with the warehouse value; the lowest non-zero value applies, so a session cannot raise a warehouse limit.
• STATEMENT_QUEUED_TIMEOUT_IN_SECONDS:
  • Measures the time a query spends waiting in a virtual warehouse queue due to exhausted concurrency slots.
  • Default is 0 (disabled, meaning queries will queue indefinitely until resources free up or the client disconnects).
  • Setting this parameter (e.g., to 300 seconds) prevents queue pileups from creating cascading application timeouts.
• ABORT_DETACHED_QUERY:
  • Default is FALSE. When set to TRUE (account, user, or session), in-progress queries are aborted 5 minutes after connectivity is lost because a session ended abruptly (closed browser, crashed script, dropped network), instead of running to completion.