6.3 Cross-Cloud Auto-Fulfillment & Listing Replication

6.3 Cross-Cloud Auto-Fulfillment & Listing Replication

Snowflake's native Secure Data Sharing is renowned for its zero-copy architecture: a provider grants access to database objects via a share, and consumers instantly query the data without copying files, spinning up extraction jobs, or paying storage fees. However, this zero-copy primitive relies fundamentally on shared cloud storage metadata within a single physical cloud region.

When a provider on AWS US East (N. Virginia) needs to share datasets with a customer on Azure West Europe or GCP US Central (Iowa), direct metadata-only sharing is physically impossible because cloud object storage systems (AWS S3, Azure Blob, GCS) cannot cross cloud network boundaries without data transfer. To solve this challenge at global scale, Snowflake engineered Cross-Cloud Auto-Fulfillment.

Cross-Cloud Sharing Challenge & Auto-Fulfillment Architecture

┌────────────────────────────────────────────────────────────────────────────────────────┐
│                     Snowflake Cross-Cloud Auto-Fulfillment                             │
│                                                                                        │
│   Data Provider Account (AWS us-west-2)                                                │
│   • Publishes Global Marketplace Listing or Private Share                              │
│   • Configures Auto-Fulfillment Refresh (interval, schedule, or trigger)               │
│                                                                                        │
│                                Snowflake Autonomous Control Plane                      │
│                        ┌────────────────────────────────────────────────┐              │
│                        │ • Monitors Consumer Listing Requests           │              │
│                        │ • Provisions Secure Share Areas (SSA)          │              │
│                        │ • Manages Autonomous Serverless Replication    │              │
│                        └────────────────────────────────────────────────┘              │
│                                   │                           │                        │
│            Automated Egress       ▼                           ▼    Automated Egress    │
│  ┌──────────────────────────────────────────┐    ┌──────────────────────────────────┐  │
│  │ Azure West Europe Secure Share Area      │    │ GCP us-central1 Secure Share Area│  │
│  │ • Local Read-Only Database Replica       │    │ • Local Read-Only Replica        │  │
│  │ • Local Secure Share                    │    │ • Local Secure Share             │  │
│  └──────────────────────────────────────────┘    └──────────────────────────────────┘  │
│                        │                                               │               │
│                        ▼ Local Zero-Copy Share                         ▼ Local Share   │
│  ┌──────────────────────────────────────────┐    ┌──────────────────────────────────┐  │
│  │ Consumer Account A (Azure West Europe)   │    │ Consumer Account B (GCP)         │  │
│  │ • Queries data via LOCAL warehouse       │    │ • Queries data via LOCAL wh      │  │
│  │ • Zero cross-region network latency      │    │ • Zero cross-region latency      │  │
│  └──────────────────────────────────────────┘    └──────────────────────────────────┘  │
└────────────────────────────────────────────────────────────────────────────────────────┘

The Fundamental Barrier: Intra-Region vs. Cross-Cloud Sharing

1. Intra-Region Sharing (Standard Secure Data Sharing):
   • Provider and consumer reside in the same cloud provider and same geographical region (e.g., both in AWS us-east-1).
   • Zero data movement: the consumer's virtual warehouse accesses the provider's underlying immutable S3 micro-partitions directly via cloud services metadata pointers.
   • No storage duplication, no data transfer charges.
2. Cross-Region / Cross-Cloud Sharing (The Requirement for Auto-Fulfillment):
   • Provider and consumer reside in different regions (e.g., AWS us-east-1 to AWS eu-west-1) or different cloud providers (e.g., AWS to Azure or GCP).
   • Object storage perimeters prevent direct cross-cloud micro-partition queries.
   • Without Auto-Fulfillment: The provider would have to create accounts in every consumer region, replicate databases with replication groups, keep them refreshed, and create local shares in each account.
   • With Auto-Fulfillment: Snowflake provisions and manages a secure share area (SSA) in each needed region and keeps the data product refreshed.

How Auto-Fulfillment Works

• The provider creates a Listing (either publicly on the Snowflake Marketplace or privately for specific consumer accounts).
• When a consumer in a remote region gets the listing (or, for private listings, once the specified consumers get it), Snowflake transfers the data product to a secure share area (SSA) that Snowflake manages in that region. If the product already exists there, access is immediate.
• The data product can be a share or a Native App application package; a database used by several listings is fulfilled only once per region.
• Later changes are synchronized according to the refresh configuration. For paid listings delivered to other regions, and for private listings to consumers in other regions, auto-fulfillment is the required mechanism.

Configuration, Fulfillment Groups & Refresh Schedules

Data providers configure Cross-Cloud Auto-Fulfillment through the Snowsight UI under Data Products > Provider Studio, or programmatically when managing listing objects.

Refresh Frequency & Data Freshness

The provider defines the replication schedule that governs how frequently updates in the primary database synchronize to target fulfillment regions:

• Interval-based refresh: a cadence from one minute to eight days, set in Provider Studio or SQL; each completed refresh starts the next interval.
• Schedule-based refresh: a cron-style schedule so consumers get updates at predictable times (interval and schedule cannot both apply).
• Trigger-based refresh: SYSTEM$TRIGGER_LISTING_REFRESH (SQL only) refreshes on demand — ideal right after an upstream ETL pipeline finishes.
• Managed compute: fulfillment runs on Snowflake-managed resources, not a provider warehouse.

When Fulfillment Happens

Telemetry & Auditing Auto-Fulfillment

Providers track fulfillment cost and activity with views such as:

• SNOWFLAKE.DATA_SHARING_USAGE.LISTING_AUTO_FULFILLMENT_REFRESH_DAILY — refresh activity and credits by region.
• SNOWFLAKE.DATA_SHARING_USAGE.LISTING_AUTO_FULFILLMENT_DATABASE_STORAGE_DAILY — storage used in remote regions.
• SNOWFLAKE.ORGANIZATION_USAGE.LISTING_AUTO_FULFILLMENT_USAGE_HISTORY — organization-wide fulfillment usage (about 72 hours latency).
• SNOWFLAKE.DATA_SHARING_USAGE.LISTING_ACCESS_HISTORY and LISTING_CONSUMPTION_DAILY — which consumers use the listing.

-- Review auto-fulfillment refresh activity for the last 30 days
SELECT *
FROM snowflake.data_sharing_usage.listing_auto_fulfillment_refresh_daily
WHERE usage_date >= DATEADD('day', -30, CURRENT_DATE())
ORDER BY usage_date DESC;

Cost Attribution & Economic Model

A paramount concept tested on the SnowPro Advanced: Architect exam is the strict demarcation of costs between the data provider and the data consumer in Cross-Cloud Auto-Fulfillment.

Cost Allocation Breakdown

┌────────────────────────────────────────┐       ┌────────────────────────────────────────┐
│         PROVIDER EXPENSES              │       │          CONSUMER EXPENSES             │
│                                        │       │                                        │
│ • Cloud Provider Data Egress           │       │ • Virtual Warehouse Compute Credits    │
│   (Network transfer from source cloud) │       │   (Running queries against local share)│
│ • Serverless Replication Credits       │       │                                        │
│   (Snapshot deltas & transport)        │       │   CONSUMER PAYS ZERO STORAGE FEES      │
│ • Target Cloud Storage Fees            │       │   CONSUMER PAYS ZERO EGRESS FEES       │
│   (Replicated micro-partitions in      │       │   CONSUMER PAYS ZERO REPLICATION FEES  │
│    each remote fulfillment region)     │       │                                        │
└────────────────────────────────────────┘       └────────────────────────────────────────┘

Detailed Cost Responsibilities

Architectural Tradeoff: Remote Storage vs. Cross-Region Query Latency

Why does Snowflake duplicate micro-partitions to remote regions instead of executing federated cross-region queries over the network?

1. Local Query Performance: Because the data product is stored in the consumer's region, the consumer's warehouse reads local storage and gets the same pruning and caching behavior as for local tables.
2. Elimination of Per-Query Egress: If queries were federated across regions dynamically, every single consumer query would incur high network latency and trigger unpredictable data egress charges. Auto-Fulfillment incurs egress once during scheduled batch syncs.

Architectural Comparison: Sharing & Replication Mechanisms