11.2 Streaming Insights (SI) Architecture, Windowing (Tumbling/Sliding) & Real-Time Signals

Streaming Insights (SI) Architecture, Windowing (Tumbling/Sliding) & Real-Time Signals

In modern digital customer experiences, waiting for an hourly or nightly batch calculation is often too late. When an e-commerce shopper adds high-value merchandise to their digital shopping cart and encounters a payment gateway failure, or when a banking customer attempts multiple consecutive unauthorized funds transfers, organizations must detect and respond to these behavioral signals in seconds, not hours.

Salesforce Data Cloud delivers this real-time capability through Streaming Insights (SIs). Streaming Insights continuously process unbounded streams of high-velocity event data as records land in the platform. By calculating real-time metrics across precise temporal windows, Streaming Insights identify critical customer signals and immediately dispatch automated interventions via Data Actions.

---

Core Streaming Architecture & Stream Processing

Unlike Calculated Insights—which run batch Spark queries against static, at-rest Lakehouse tables—Streaming Insights run on an active, stateful streaming query engine.

┌────────────────────────────────────────────────────────────────────────┐
│                     STREAMING DATA INGESTION PIPELINE                  │
│  - Web & Mobile SDK: Page views, button clicks, cart adds              │
│  - Ingestion API (Streaming): POS swipes, IoT telemetry, call starts   │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │ Continuous Event Ingestion (Seconds)
                                    ▼
┌────────────────────────────────────────────────────────────────────────┐
│                   STREAMING INSIGHT ENGINE (STATEFUL)                  │
│  - Temporal Windowing: Tumbling or Sliding Windows                     │
│  - Continuous State Buffer & Metric Accumulation                       │
│  - Real-Time SQL Aggregation (COUNT, SUM, AVG)                         │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │ Emits Metric on Window Evaluation
                                    ▼
┌────────────────────────────────────────────────────────────────────────┐
│                         DATA ACTION TRIGGER                            │
│  - Evaluates Threshold: (e.g., CartItems > 0 AND Checkout = 0)         │
│  - Target 1: Salesforce Platform Event ──► Real-Time Flow in CRM       │
│  - Target 2: Webhook ──► External Marketing API / Kafka / AWS Lambda   │
└────────────────────────────────────────────────────────────────────────┘

Key Architectural Characteristics:

1. Continuous Processing: Records are evaluated immediately upon arrival from streaming ingestion sources (Salesforce Interactions Web/Mobile SDK or Streaming Ingestion API).
2. Unbounded Streams: Because streaming event data has no defined beginning or end, the engine must divide the unbounded stream into finite temporal buckets known as Windows.
3. Event-Time vs. Ingestion-Time: SIs evaluate aggregations based on event timestamps, ensuring that late-arriving events within defined watermark thresholds are accurately allocated to their appropriate temporal windows.
4. Downstream Decoupling: Streaming Insights do not write to traditional batch segment tables. Instead, their primary architectural output is the immediate triggering of Data Actions (Platform Events and Webhooks).

---

Temporal Windowing Mechanics: Tumbling vs. Sliding Windows

The most heavily tested technical concept regarding Streaming Insights on the Data Cloud Consultant exam is the distinction between Tumbling Windows and Sliding Windows.

1. Tumbling Windows (Fixed, Non-Overlapping)

A Tumbling Window is a fixed-duration, non-overlapping, contiguous time interval. Each incoming event belongs to one and only one window.

Time:        12:00         12:10         12:20         12:30
             ├─────────────┼─────────────┼─────────────┤
Window 1:    [12:00 - 12:10)
Window 2:                  [12:10 - 12:20)
Window 3:                                [12:20 - 12:30)
Events:         ★    ★            ★          ★   ★   ★
Window 1 Count: 2 events
Window 2 Count: 1 event
Window 3 Count: 3 events

• Mechanics: When the window reaches its specified duration (e.g., 10 minutes), the window "closes," the aggregation is computed, the metric is emitted, and a brand-new window opens immediately.
• Overlap: Zero overlap. No event is ever counted in more than one window.
• Best Suited For: Periodic, discrete operational rollups—such as counting the number of API errors per 15-minute block, calculating total transactions per hour, or monitoring server health status in fixed 5-minute intervals.

2. Sliding Windows (Overlapping, Continuous Lookback)

A Sliding Window is an overlapping time interval defined by two distinct parameters: the Window Duration (lookback length) and the Slide Increment (evaluation frequency).

Time:        12:00   12:05   12:10   12:15   12:20   12:25
             ├───────┼───────┼───────┼───────┼───────┤
Window 1:    [12:00 ─────────────── 12:15)
Window 2:            [12:05 ─────────────── 12:20)
Window 3:                    [12:10 ─────────────── 12:25)
Events:         ★        ★       ★       ★       ★

• Mechanics: The window maintains a rolling lookback period (e.g., 15 minutes) and advances forward at a regular, smaller step size (e.g., every 5 minutes).
• Overlap: Significant overlap. Because the slide increment is smaller than the window duration, an event can fall into multiple successive windows. In the example above, an event occurring at 12:08 falls into Window 1, Window 2, and Window 3.
• Mathematical Constraint: The Window Duration must always be greater than or equal to the Slide Increment (Duration ≥ Slide). If Duration = Slide, the sliding window degrades into a tumbling window.
• Best Suited For: Velocity detection and in-flight behavioral triggers—such as detecting cart abandonment ("Customer added items in the last 15 minutes, evaluated every 2 minutes"), rapid security breaches ("5 failed PIN attempts in the last 10 minutes, evaluated every 1 minute"), or urgent customer distress signals.

Tumbling vs. Sliding Windowing Comparison Matrix

---

Windowing SQL Syntax & Query Structure

Streaming Insights are authored using a specialized streaming dialect of ANSI SQL that incorporates the WINDOW function in the GROUP BY clause.

Tumbling Window SQL Syntax

SELECT
    DeviceApplication__dlm.PartyId__c AS CustomerId__c,
    COUNT(DeviceApplication__dlm.Id__c) AS ActionCount__c,
    WINDOW.start AS WindowStartTime__c,
    WINDOW.end AS WindowEndTime__c
FROM
    DeviceApplication__dlm
GROUP BY
    CustomerId__c,
    WINDOW(TUMBLING, 'MINUTE', 10)

Sliding Window SQL Syntax (Cart Abandonment)

SELECT
    WebEngagement__dlm.PartyId__c AS CustomerId__c,
    SUM(WebEngagement__dlm.CartAddQuantity__c) AS TotalCartItems__c,
    COUNT(WebEngagement__dlm.OrderConfirmationId__c) AS PurchaseCount__c,
    WINDOW.start AS WindowStartTime__c,
    WINDOW.end AS WindowEndTime__c
FROM
    WebEngagement__dlm
GROUP BY
    CustomerId__c,
    WINDOW(SLIDING, 'MINUTE', 15, 'MINUTE', 5)

Key Syntax Nuances for the Exam:

1. WINDOW in GROUP BY: The WINDOW(...) expression must appear inside the GROUP BY clause alongside any entity grouping dimensions.
2. Pseudo-Columns WINDOW.start and WINDOW.end: Data Cloud provides built-in pseudo-columns (WINDOW.start and WINDOW.end) that capture the exact UTC timestamps defining the start and end boundaries of each evaluated window.
3. Restricted Join Topology: Unlike Calculated Insights, which support multi-table relational joins across deep DMO graphs, Streaming Insights are optimized for streaming throughput and restrict joins to lightweight lookups or single streaming DMO sources.

---

Streaming Insight Trigger Targets: Real-Time Data Actions

When a Streaming Insight finishes calculating a window and produces an aggregated record, how does Data Cloud act on that intelligence? The answer is Data Actions.

A Data Action subscribes directly to a Streaming Insight and evaluates filtering conditions against the emitted metrics. When an insight row satisfies the criteria, the Data Action fires immediately.

┌────────────────────────────────────────────────────────────────────────┐
│                     DATA ACTION ORCHESTRATION                          │
│                                                                        │
│  Streaming Insight Output:                                             │
│  - CustomerId: 003xx00001AbCd                                          │
│  - TotalCartItems: 4                                                   │
│  - PurchaseCount: 0                                                    │
│  - WindowEnd: 2026-09-21T14:15:00Z                                     │
│                                                                        │
│  Data Action Filter Condition:                                         │
│  TotalCartItems__c > 0 AND PurchaseCount__c == 0                       │
│                                                                        │
│  Dispatches Payload to Target:                                         │
│  ┌───────────────────────────────┐   ┌───────────────────────────────┐ │
│  │  Salesforce Platform Event    │   │       External Webhook        │ │
│  │  - Triggers Flow in CRM       │   │  - JSON Post to External API  │ │
│  │  - Opens High-Priority Task   │   │  - Triggers Marketing Journey │ │
│  └───────────────────────────────┘   └───────────────────────────────┘ │
└────────────────────────────────────────────────────────────────────────┘

Data Action Targets:

1. Salesforce Platform Events:
   • Dispatches a native Salesforce Platform Event into the connected Salesforce CRM core org.
   • Immediately triggers a Platform Event-Triggered Flow or Apex trigger in Sales, Service, or Financial Services Cloud.
   • Example: Automatically creates a High-Priority Case and notifies an Omni-Channel agent in Service Cloud.
2. Webhooks:
   • Transmits a secure, real-time HTTP POST request with a standardized JSON payload to an external endpoint.
   • Connects to external systems such as AWS Lambda, Apache Kafka, MuleSoft, or third-party marketing automation platforms.
   • Example: Signals an external fraud mitigation microservice to temporarily freeze a transaction.

---

Real-World Enterprise Use Cases

1. E-Commerce Cart Abandonment (15-Minute SLA)

• Business Requirement: A global retailer wants to send an abandoned cart SMS notification within 20 minutes of a shopper leaving the site without completing their purchase.
• Implementation: A Streaming Insight evaluates WebEngagement__dlm using a 15-minute sliding window evaluated every 2 minutes (WINDOW(SLIDING, 'MINUTE', 15, 'MINUTE', 2)). It aggregates SUM(CartAddCount__c) and COUNT(CheckoutComplete__c). When cart items $>0$ and checkouts $=0$, a Data Action triggers a Webhook to Marketing Cloud Journey Builder to inject the subscriber into an immediate SMS recovery journey.

2. Rapid VIP Customer Distress & Service Escalation

• Business Requirement: If a high-net-worth banking customer experiences 3 or more mobile deposit errors within 10 minutes, the bank must proactively intervene.
• Implementation: A Streaming Insight monitors mobile app error telemetry with a 10-minute sliding window evaluated every 1 minute. If error count ≥ 3, a Data Action fires a Platform Event to Service Cloud, routing an instant priority callback task to the customer's dedicated private banker.

3. Payment Gateway Fraud Velocity Monitoring

• Business Requirement: Detect rapid-fire micro-transactions indicative of card testing attacks across point-of-sale terminals.
• Implementation: A Streaming Insight evaluates transaction frequency per credit card token using a 5-minute tumbling window (WINDOW(TUMBLING, 'MINUTE', 5)). If transaction count exceeds 8 within the 5-minute slice, a Data Action notifies the fraud prevention firewall via Webhook to block further authorizations.