10.3 Data Sharing and Analytics Hub

10.3 Data Sharing and Analytics Hub

[!NOTE] Cross-organizational data collaboration is a core pillar of modern enterprise data mesh architectures. On the Google Cloud Professional Data Engineer exam, understand how Analytics Hub replaces legacy file exports with zero-copy linked datasets, and how fine-grained security policies and Data Clean Rooms protect sensitive data during multi-party analytics.

In traditional enterprise architectures, sharing data across distinct departments, subsidiary companies, or external corporate partners required fragile, expensive data movement workflows. Data engineers typically authored scheduled batch pipelines that extracted table dumps into Cloud Storage buckets as CSV or Parquet files, transferred files via SFTP or cross-project transfer jobs, and re-ingested the records into the destination warehouse.

This legacy paradigm introduced severe operational challenges: duplicate storage costs, stale data copies, synchronization delays, fragile pipeline maintenance, and severe security risks once sensitive customer data left the perimeter of the primary enterprise warehouse.

To solve these challenges, Google Cloud introduced BigQuery Analytics Hub, a fully managed data exchange platform built natively into BigQuery that enables secure, zero-copy data sharing across organizational boundaries.

---

Analytics Hub Architecture and Core Abstractions

Analytics Hub establishes a centralized governance framework that enables organizations to publish, discover, and subscribe to shared datasets while keeping the underlying data securely anchored in place.

+-------------------------------------------------------------------------+
|                    Publisher Project: finance-prod                      |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Colossus Storage: fact_market_ticks (Encrypted, Governed)          |  |
|  | • Publisher pays 100% of Storage Costs                            |  |
|  | • Row-Level Security & Column Masking Enforced                    |  |
|  +-------------------------------------------------------------------+  |
|                                  |                                      |
|                                  v                                      |
|  +-------------------------------------------------------------------+  |
|  | Analytics Hub Data Exchange                                       |  |
|  | • Listing: "Global Market Ticks 2026" (Private / Shared)          |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+
                                   |
        +--------------------------+--------------------------+
        | Zero-Copy Reference                                 | Zero-Copy Reference
        | (No physical data duplication)                       | (No physical data duplication)
        v                                                     v
+------------------------------------+   +------------------------------------+
| Subscriber Project A: risk-analytics|   | Subscriber Project B: external-corp|
|                                    |   |                                    |
| +--------------------------------+ |   | +--------------------------------+ |
| | Linked Dataset: market_ticks_ro| |   | | Linked Dataset: market_ticks_ro| |
| | • Read-Only Reference Pointer  | |   | | • Read-Only Reference Pointer  | |
| | • Subscriber Slot Compute      | |   | | • Subscriber Slot Compute      | |
| +--------------------------------+ |   | +--------------------------------+ |
|                 |                  |   |                 |                  |
|                 v                  |   |                 v                  |
|   SELECT * FROM market_ticks_ro    |   |   SELECT * FROM market_ticks_ro    |
|   [Billed to: risk-analytics]      |   |   [Billed to: external-corp]       |
+------------------------------------+   +------------------------------------+

Analytics Hub is built around four fundamental concepts:

1. Data Exchange

A Data Exchange is a secured, centrally managed container created by an organization to curate and group related data listings. Data Exchanges can be scoped to different trust boundaries:

• Private Exchanges: Restricted to specific internal Google Cloud projects, corporate teams, or authorized external partner organizations. Private exchanges are common in enterprise data mesh implementations to facilitate cross-domain data product consumption.
• Public Exchanges: Globally accessible marketplaces (such as Google Cloud Marketplace datasets) containing public data assets, economic benchmarks, geospatial repositories, and commercial third-party datasets.

2. Listing

A Listing is a discoverable package published inside a Data Exchange that references a specific BigQuery dataset. A listing contains rich operational metadata, including title, descriptive documentation, category tags, terms of service, and access controls. Listings can be marked as free, commercial (subscription-based), or restricted.

3. Publisher

The Publisher is the organization or project that owns the original BigQuery dataset and controls its distribution. Publishers define Data Exchanges, create listings, review subscription requests, and manage access permissions via Cloud IAM roles (such as roles/analyticshub.publisher and roles/analyticshub.admin).

4. Subscriber

The Subscriber is an internal or external consumer who browses a Data Exchange, discovers a listing, and requests access. Once authorized, the subscriber creates a Linked Dataset inside their own Google Cloud project (requiring roles/analyticshub.subscriber).

---

Linked Datasets: Zero-Copy Sharing Mechanics

The foundational technical breakthrough of Analytics Hub is the Linked Dataset.

When a subscriber subscribes to an Analytics Hub listing, BigQuery does not copy, duplicate, or export any physical storage blocks from the publisher's project. Instead, BigQuery provisions a read-only, logical reference pointer inside the subscriber's project that links directly to the publisher's underlying dataset in Colossus.

Key Architectural Properties of Linked Datasets

1. Zero-Copy Architecture: The physical data resides solely in the publisher's Colossus storage blocks. A publisher sharing a 500-terabyte table with fifty separate subscriber organizations consumes exactly 500 terabytes of total physical storage across Google Cloud, completely eliminating storage replication overhead.
2. Real-Time Data Synchronization: Because the linked dataset points directly to the live publisher storage blocks, any updates, inserts, deletions, or streaming appends committed by the publisher are immediately visible to subscribers the instant the write commits. There is zero replication latency, zero batch synchronization lag, and zero ETL failure points.
3. Read-Only Enforcement: Linked datasets are strictly read-only for subscribers. Subscribers cannot perform DML modifications (INSERT, UPDATE, DELETE), alter table schemas, or drop tables within the linked dataset.
4. Regionality Constraints: To query a linked dataset, the subscriber's Google Cloud project and the linked dataset must reside in the same geographic region or multi-region as the publisher's source dataset. A dataset hosted in europe-west1 cannot be queried directly from a linked dataset provisioned in us-central1 without cross-region replication.

---

Decoupled Billing Model

One of the most appealing features of Analytics Hub for enterprise governance is its strict, unambiguous separation of billing responsibilities.

+-----------------------------------------------------------------------------+
|                   Analytics Hub Cost Allocation Model                       |
+-----------------------------------------------------------------------------+
|  Cost Dimension        | Responsible Party | Billing Mechanism              |
+------------------------+-------------------+--------------------------------+
|  Colossus Storage      | Publisher         | Billed for Active/Long-Term    |
|  (Physical Table Data) |                   | Storage in Publisher Project   |
+------------------------+-------------------+--------------------------------+
|  Query Compute         | Subscriber        | Billed to Subscriber Project   |
|  (Analysis & Slots)    |                   | (On-demand TB or Slot Reserva) |
+------------------------+-------------------+--------------------------------+
|  Network Egress        | None / Subscriber | $0.00 within same region;      |
|                        |                   | Egress billed if data exported |
+-----------------------------------------------------------------------------+

1. Storage Billing (Publisher Responsibility)

The publisher incurs 100% of the storage costs for the primary dataset hosted in their Google Cloud project. Storage is billed according to standard BigQuery active and long-term storage rates. Even if a thousand subscribers query the listing, the publisher never pays an extra penny for storage.

2. Compute Billing (Subscriber Responsibility)

When a subscriber executes a query against a linked dataset:

• The compute processing is powered entirely by the subscriber's compute resources.
• Under on-demand analysis billing, the subscriber's billing account is charged for the bytes scanned by their query.
• Under capacity-based commitments (BigQuery Editions), the query consumes slots from the subscriber's assigned slot reservations.
• The publisher incurs zero compute charges when subscribers run heavy analytical queries against the shared data.

---

Security, Governance, and Data Clean Rooms

Sharing enterprise data externally requires strict governance to ensure regulatory compliance (e.g., GDPR, HIPAA, CCPA) and prevent unauthorized access.

1. Preserving Fine-Grained Security Controls

When a publisher shares a dataset via Analytics Hub, BigQuery preserves all underlying security and access policies:

• Row-Level Security (RLS): If the publisher defines row-level security filters based on user identity or corporate domain, these filtering predicates apply dynamically when the subscriber executes queries against the linked dataset.
• Column-Level Security and Data Masking: If the publisher applies policy tags with data masking rules (e.g., masking PII fields like Social Security Numbers, phone numbers, or credit cards using SHA-256 hashes or null masking), the subscriber sees only the masked representations. The raw plaintext values are never exposed across the listing.
• Authorized Datasets and Views: Publishers can share an authorized view via a listing. The subscriber queries the view without ever gaining access to the underlying raw tables from which the view is derived.

2. Data Clean Rooms: Multi-Party Privacy-Preserving Collaboration

A critical frontier in enterprise data collaboration is the Data Clean Room. A Data Clean Room enables multiple organizations—such as a retail merchant and a media publisher, or two financial institutions combating fraud—to join sensitive datasets on common identifiers (such as hashed email addresses) to perform aggregate analysis without either party exposing raw data to the other.

+----------------------------+             +----------------------------+
| Organization A (Retailer)  |             | Organization B (Media Pub) |
| Customer Purchase Logs     |             | Ad Impressions & Clicks    |
+----------------------------+             +----------------------------+
              \                                           /
               \  Analytics Hub Clean Room Listing       /
                v                                       v
+-----------------------------------------------------------------------+
|                   Privacy-Preserving Clean Room Perimeter             |
|                                                                       |
|  • Join Condition: SHA256(a.email) = SHA256(b.email)                  |
|  • Privacy Policy: Aggregation-Only (Minimum Threshold = 100 users)   |
|  • Prohibited: SELECT * or raw row-level export                       |
+-----------------------------------------------------------------------+
                                   |
                                   v
+-----------------------------------------------------------------------+
| Clean Room Output: Campaign Conversion Rate (Aggregated KPI Only)     |
| Result: 12.4% conversion across overlapping cohort                    |
| [Zero PII or individual customer transactions exposed]                |
+-----------------------------------------------------------------------+

Clean Room Governance Features in Analytics Hub

• Analysis Rules (Aggregation Constraints): Publishers can enforce analysis rules on clean room listings requiring that all queries include GROUP BY aggregations and return a minimum threshold of rows (e.g., a query must aggregate across at least 100 distinct individuals). Queries attempting to inspect individual rows are rejected by the query engine.
• Join Constraints: Clean room policies restrict queries to join solely on specific encrypted or hashed join keys (e.g., hashed_customer_id), preventing unapproved joins with external data.
• Differential Privacy Integration: By applying BigQuery's native differential privacy functions, noise is calibrated and added to query results, mathematically guaranteeing that an attacker cannot infer the presence or absence of any single individual in the underlying dataset.

---

Traditional Data Copying vs Analytics Hub Linked Datasets

---

Summary of Key Exam Takeaways

• Linked Datasets are Zero-Copy: No storage is replicated; queries run against publisher Colossus storage blocks.
• Zero-Latency Freshness: Streaming appends and DML modifications are visible immediately to subscribers.
• Decoupled Billing Boundary: Publisher pays storage; subscriber pays query compute slots or scanned bytes.
• Strict Regional Isolation: Publisher and subscriber datasets must be located in the exact same region or multi-region.
• Governance Propagation: Row-Level Security and Column-Level Policy Tags remain active on shared listings.
• Data Clean Rooms: Enforce aggregation rules and privacy budgets to allow multi-party analysis without PII exposure.