4.4 Common Traps in Monitor and optimize an analytics solution

Capacity traps

The biggest trap is conflating capacity throttling with item-level errors. The exam often shows utilization spiking and asks what is happening. Remember: utilization over 100% does not automatically mean throttling. Fabric first applies smoothing (spreading CU over future timepoints) and overage protection (10 minutes of future capacity), so a spike can exceed 100% without any user-visible delay. Throttling only begins after the 10-minute window fills, escalating from 20-second interactive delays to interactive rejection to full background rejection.

Use the Throttling chart in the Capacity Metrics app — not the raw Utilization chart — to confirm actual throttling.

The second capacity trap is reaching for a bigger SKU to fix everything. Increasing the SKU relieves throttling (and burns down carryforward faster), but it does nothing for a pipeline that fails on bad data, a dataflow broken by a schema change, or a notebook with a logic error. Match the remedy to the cause.

Maintenance and error traps

VACUUM retention. Setting retention below seven days is rejected by default for a reason: VACUUM permanently deletes files, and a short window can remove files still needed by time travel or by concurrent readers mid-query, causing failures. Never lower it just to reclaim storage faster.

OPTIMIZE is not a cure-all. It fixes the small-file problem but does not change a query that filters on an unindexed column — that calls for Z-Order on the filter columns. And re-running OPTIMIZE constantly wastes CU; schedule it after meaningful write volume.

Error triage by symptom. Many stems hide the root cause in a timing cue:

• A Dataflow Gen2 refresh that breaks right after the source removed or renamed a column → investigate the removed/renamed column in the transformation, not the gateway.
• A OneLake shortcut that worked yesterday and now errors after an upstream engineer reorganized folders → the source path/folder structure changed; re-point the shortcut.
• A semantic-model refresh that errors when triggered from a pipeline → a refresh may already be in progress; the API rejects overlapping refreshes.

Choosing the right monitoring surface

A subtle trap is using the wrong tool for the time horizon. Real-time, automated notification on a failure is Fabric Activator, not a person watching the Monitoring hub. Historical, queryable analysis of trends across many runs is workspace monitoring in the KQL Eventhouse, not the Monitoring hub's recent-activities list (which only holds ~100 activities per item for 30 days). CU cost and throttling questions belong to the Capacity Metrics app, never the item-level run detail.

Finally, watch the eventstream exception: eventstreams are not throttled like other operations. Because a stream can run for years, Fabric instead reduces the CU allocated to keeping the stream open when the capacity is overloaded, rather than rejecting the stream outright. Real-Time Intelligence similarly skips the 20-second delay stage and only throttles at the rejection phase, preserving real-time performance.

More traps to recognize

Several additional distractors recur often enough to memorize.

• Over-partitioning. Partitioning by a high-cardinality column (such as a customer ID or timestamp to the second) shatters a table into many tiny partitions and files, hurting performance — the opposite of the intended speed-up. Partition on low-cardinality, frequently-filtered columns only.
• VACUUM cannot speed up queries. VACUUM reclaims storage and prunes old versions; it does not compact active files (that is OPTIMIZE) and is not a query-performance tool. Choosing VACUUM to fix slow reads is a trap.
• Statistics vs. file layout. Updating warehouse statistics fixes the SQL optimizer's row estimates; it does nothing for lakehouse Parquet file layout, where OPTIMIZE/V-Order/Z-Order apply. Match the layer to the lever.
• Wrong monitoring tool for time horizon. Using the Monitoring hub for historical trend analysis (it only holds ~100 recent activities per item) or watching it manually instead of using Activator for alerts.
• Bigger pool for a logic error. Adding Spark nodes will not fix a notebook that throws an exception on bad data or a wrong path; read the executor logs and fix the code or input.

The through-line of every trap: match the remedy to the actual cause and the correct layer, and read the timing cue in the stem before reaching for the most powerful-sounding option.

Timing cues that reveal the cause

Many trap questions embed a 'right after' clause that names the true cause. 'Failures started right after a source column was renamed' points to the transformation, not the gateway. 'A shortcut broke right after folders were reorganized' points to the source path. 'A refresh errored when triggered from a pipeline' points to an overlapping in-progress refresh. Train yourself to find that temporal cue first; it usually eliminates the capacity-scaling distractor and the generic outage answer in a single step, leaving the precise, layer-correct remediation as the obvious choice.