11.3 CI vs. SI Architectural Decision Matrix & Performance Optimization

CI vs. SI Architectural Decision Matrix & Performance Optimization

When designing enterprise customer data solutions in Salesforce Data Cloud, one of the most critical architectural responsibilities of a certified consultant is selecting the right computational paradigm for analytical and real-time requirements. Implementing the wrong tool—such as attempting to use batch Calculated Insights for sub-minute cart abandonment triggers, or attempting to use Streaming Insights for multi-year customer lifetime value calculations—results in project failure, query timeouts, or runaway consumption of Data Cloud credits.

This section provides an exhaustive decision framework comparing Calculated Insights (CI), Streaming Insights (SI), Segmentation Canvas Formulas, and Batch Data Transforms, alongside deep-dive performance tuning rules and credit governance strategies.

Comprehensive Architectural Comparison Matrix

The following matrix details the fundamental technical differences between Calculated Insights and Streaming Insights across all evaluation axes:

The 4-Way Compute Decision Framework

In addition to CIs and SIs, Data Cloud architects have two other data manipulation tools at their disposal: Segmentation Canvas Formulas and Batch Data Transforms. Understanding when to deploy each tool is essential for the exam.

                                  START: Analytical Requirement
                                                │
                       Is the requirement real-time (sub-minute latency)?
                                    ├─── YES ──► STREAMING INSIGHT (SI)
                                    │            (Windowing + Data Actions)
                                    └─── NO
                                         │
                  Does it require multi-object data cleansing, flattening,
                   pivoting, or restructuring before canonical DMO mapping?
                                    ├─── YES ──► BATCH DATA TRANSFORM
                                    │            (SQL or Data Prep Recipe)
                                    └─── NO
                                         │
                 Is it a complex multi-row aggregation (SUM, AVG, COUNT)
                  reused across multiple segments, activations, or CRM UI?
                                    ├─── YES ──► CALCULATED INSIGHT (CI)
                                    │            (Pre-aggregated __cio)
                                    └─── NO
                                         │
                                    Ad-hoc, single-segment direct attribute filter?
                                    └──────────► SEGMENTATION CANVAS FORMULA

1. Calculated Insights (CI)

• When to Use: Reusable historical aggregations across multiple source records (e.g., Customer Lifetime Value, RFM scores, Total Orders, Preferred Category). When multiple marketing segments or CRM pages need the same pre-computed number without recalculating it repeatedly.
• Exam Indicator: "Calculate total spend over the last 3 years and make it available across multiple marketing segments."

2. Streaming Insights (SI)

• When to Use: Event-driven behavioral detection requiring action within seconds or minutes. Windowed aggregations over unbounded telemetry.
• Exam Indicator: "Detect when a customer experiences 3 payment failures in 10 minutes and immediately open a high-priority case in Service Cloud."

3. Segmentation Canvas Formulas & Direct Filters

• When to Use: Ad-hoc, one-off filtering logic specific to a single audience segment. Evaluating simple attributes (e.g., City = 'Chicago' or LastLoginDate >= LAST_30_DAYS) on single objects without pre-aggregation.
• Exam Indicator: "A marketer wants to create a one-time promotional segment of customers living in California who have logged in within the past 7 days."

4. Batch Data Transforms (SQL / Recipe Transforms)

• When to Use: Heavy ETL/ELT transformations, joining disparate raw DLOs before DMO mapping, data cleansing, address standardization, or flattening nested JSON structures into tabular formats.
• Exam Indicator: "Combine and clean raw POS and online order staging tables to create a unified transaction dataset prior to canonical mapping."

Refresh Schedules, Performance Tuning & Optimization

Calculated Insight Optimization Rules

Calculated Insights run against distributed storage. Poorly authored SQL can lead to long execution runtimes and excessive credit consumption.

1. Right-Size the Dimension Grain:
   • Anti-Pattern: Grouping by high-cardinality fields such as TransactionTimestamp__c or SessionId__c in a customer-level insight.
   • Best Practice: Group strictly by the necessary business dimensions (e.g., UnifiedIndividual__dlm.Id__c and ProductCategory__dlm.Name__c). Every additional dimension multiplies output row volume.
2. Filter Early with WHERE Clauses:
   • Anti-Pattern: Aggregating all 10 years of historical transactions and filtering the output later.
   • Best Practice: Restrict input rows at the earliest possible stage using WHERE SalesOrder__dlm.OrderedDate__c >= DATE_ADD('year', -2, CURRENT_DATE()).
3. Align Refresh Cadence with Upstream Ingestion:
   • If e-commerce orders are ingested once daily at midnight, scheduling a Calculated Insight to refresh every 4 hours wastes compute credits without providing fresh data. Schedule the CI refresh to run shortly after upstream batch ingestion completes.
4. Prune Unused Insights:
   • Inactive or deprecated CIs continue to consume scheduled compute resources unless explicitly disabled or deleted.

Streaming Insight Optimization Rules

Streaming Insights maintain an in-memory state store for the duration of open windows. Misconfigured window parameters can crash the streaming pipeline.

1. The Window Duration vs. Slide Increment Ratio:
   • The memory footprint of a sliding window is directly proportional to (Duration / Slide).
   • Catastrophic Configuration: A 24-hour window sliding every 10 seconds creates 8,640 concurrent overlapping windows in memory for every active customer!
   • Best Practice: Keep slide increments reasonable (e.g., 15-minute window sliding every 2 to 5 minutes).
2. Avoid Joining Static Historical Tables in SIs:
   • Attempting to perform deep relational joins between a streaming engagement DMO and massive historical batch DMOs in a Streaming Insight introduces state bloat and latency violations.

Credit & Resource Governance

Data Cloud billing is metered through Data Cloud Credits. Understanding credit consumption drivers is an executive-level consulting skill tested on the exam:

Calculated Insight Credit Consumption

• Credits are consumed based on the volume of data scanned (Gigabytes/Terabytes processed) during each scheduled refresh and the compute complexity (number of joins, sort operations, and group-by cardinality).
• Cost-Reduction Tactics:
  • Use incremental refresh capabilities where available.
  • Avoid full table scans by applying date partitions in WHERE clauses.
  • Reduce refresh frequency from hourly to daily for non-urgent metrics.

Streaming Insight Credit Consumption

• Credits are consumed based on streaming event throughput (millions of ingested events evaluated) and the continuous compute capacity allocated to stateful sliding window evaluation.
• Cost-Reduction Tactics:
  • Filter streaming events at the ingestion source (e.g., configure the Web SDK to only stream relevant e-commerce events rather than every mouse hover).
  • Increase slide increment intervals to reduce evaluation frequency.

Common Exam Traps & Anti-Patterns

Consultants frequently encounter challenging, trap-laden scenarios on the certification exam. Memorize these critical anti-patterns:

[!WARNING] Trap 1: Attempting to Use Calculated Insights for Sub-Minute Real-Time Actioning The Scenario: A question asks how to trigger an immediate SMS alert when a customer walks past a beacon or abandons a digital shopping cart within 5 minutes. The Trap: Selecting a Calculated Insight with an hourly schedule. The Reality: CIs cannot achieve sub-minute execution latencies. Any scenario requiring immediate closed-loop automation within minutes mandates a Streaming Insight with a Data Action.

[!WARNING] Trap 2: Expecting Streaming Insights to Appear on the Segment Canvas The Scenario: A marketer wants to drag a Streaming Insight metric onto the Segment Canvas to build an audience of shoppers. The Trap: Looking for SIs under the Segment Canvas attribute library. The Reality: Streaming Insights cannot be used directly in the Segment Canvas. SIs output exclusively to Data Actions (Platform Events and Webhooks). If an aggregation is needed for audience segmentation, it must be authored as a Calculated Insight.

[!WARNING] Trap 3: Multi-Table Historical Relational Joins in Streaming Insights The Scenario: An architect attempts to write an SI that joins incoming mobile clickstream data across Sales Order, Order Line Item, Master Product, and Customer Loyalty Ledger. The Trap: Assuming SQL capabilities are identical between CIs and SIs. The Reality: Streaming Insights do not support complex multi-hop historical joins. They are designed for fast aggregations on streaming feeds with minimal lookup data.

[!WARNING] Trap 4: Missing Mandatory Dimension Keys in CIs The Scenario: A data engineer writes a Calculated Insight to compute overall total company revenue and omits the GROUP BY clause. The Reality: The platform rejects the insight. CIs strictly require at least one dimension to define the aggregation grain and primary key.