8.1 Problem Lifecycle, Severity Categories & Root Cause Cards
Key Takeaways
- Davis AI replaces static threshold-based alert storms with deterministic root cause analysis, evaluating topological dependencies across Smartscape rather than relying on statistical correlation.
- Dynatrace categorizes all detected anomalies into five distinct severity categories: AVAILABILITY, ERROR, PERFORMANCE, RESOURCE_CONTENTION, and CUSTOM_ALERT.
- The problem lifecycle encompasses four sequential stages: event anomaly detection, causal correlation and problem opening, dynamic evolution (blast radius and severity updates), and automatic resolution.
- A Problem Card segregates symptoms (business and user impact) from the single originating root cause, providing direct causal evidence including PurePath traces and metric baselines.
- If an incident originates from an unmonitored component or external third-party API outside OneAgent visibility, Davis flags the affected service while transparently designating 'No root cause found'.
8.1 Problem Lifecycle, Severity Categories & Root Cause Cards
Modern enterprise architectures encompass thousands of virtualized hosts, containerized microservices, serverless functions, and third-party APIs. In such distributed environments, a single infrastructure component failure—such as a saturated database disk or an exhausted connection pool—can trigger cascading errors and latency spikes across dozens of upstream applications.
Legacy application performance monitoring (APM) tools rely on static, metric-by-metric threshold alerts. When a failure occurs, these legacy systems trigger hundreds of disconnected alerts simultaneously—a phenomenon known as an alert storm. Site Reliability Engineers (SREs) and operations teams are forced to manually sift through disparate alarms to guess which event represents the true culprit.
Dynatrace eliminates alert storms through Davis, its purpose-built, deterministic AI causation engine. Rather than using machine learning statistical correlations or heuristic guesswork, Davis evaluates anomalies in real time against the live topological dependency graph maintained by Smartscape. Understanding the mechanics of Davis AI problem detection, the five severity categories, the complete problem lifecycle, and the anatomy of the Root Cause Card is fundamental to passing the Dynatrace Certified Associate examination.
Davis AI Anomaly Detection vs. Legacy Rule-Based Alerting
To appreciate how Dynatrace handles incidents, one must understand the core architectural differences between traditional monitoring alerting and the Davis causation engine:
- Deterministic Causation vs. Statistical Guesswork: Traditional APM and AIOps tools use statistical time-series correlation, flagging events that occur within the same arbitrary time window as "related." Davis does not guess based on temporal proximity. Instead, Davis navigates the deterministic topological links discovered automatically by OneAgent. Davis knows that Service A calls Service B, which executes on Process C, which runs on Host D, because Smartscape continuously maps these physical and logical relationships.
- Symptom Consolidation: When an infrastructure fault occurs, upstream services inevitably experience slow response times or HTTP 5xx errors. While legacy tools generate an alert for every affected service, Davis opens a single Problem Card. It groups all related symptom events under one incident and pinpoints the originating dependency as the root cause.
- Dynamic Baselining: Instead of requiring administrators to manually configure static upper and lower thresholds for thousands of metrics, Dynatrace continuously calculates dynamic baselines. These automated multi-dimensional baselines evaluate the 15-minute moving percentiles of response time, error rate, and throughput across both time of day and day of week, adapting automatically to seasonal traffic shifts and business cycles.
+---------------------------------------------------------------------------------------------------+
| DAVIS CAUSAL CORRELATION ARCHITECTURE |
+---------------------------------------------------------------------------------------------------+
| [SYMPTOM: Web Application] --> High User Action Duration / JavaScript Error Surge |
| │ (Upstream) |
| ▼ |
| [SYMPTOM: Order Service] --> Response Time Degradation (PurePath latency surge) |
| │ (Downstream Call) |
| ▼ |
| [SYMPTOM: Payment Service] --> Connection Timeout / HTTP 504 Gateway Timeouts |
| │ (Runs on) |
| ▼ |
| [ROOT CAUSE: Database Host] --> CPU Saturation (99%) & Disk I/O Queue Bottleneck |
+---------------------------------------------------------------------------------------------------+
| RESULT: Davis raises exactly 1 Problem Card, identifying Database Host as the single Root Cause |
+---------------------------------------------------------------------------------------------------+
The Five Problem Severity Categories
Whenever Davis detects an anomaly that violates dynamic baselines or static thresholds, it classifies the event into one of five distinct severity categories. These categories reflect the operational urgency of the event and govern how Alerting Profiles evaluate and route notifications to external IT service management (ITSM) systems.
| Severity Category | Core Triggers & Conditions | Relative Priority | Typical Baseline / Threshold Mechanism | Production Scenarios |
|---|---|---|---|---|
| AVAILABILITY | Complete outage, process crash, host offline, synthetic monitor step failure, web application unresponsive. | Highest (Priority 1) | Immediate detection; binary state (healthy vs. down) or synthetic failure confirmation across locations. | JVM process crash, Linux kernel panic, AWS EC2 instance termination, synthetic transaction failure from 3 of 3 private locations. |
| ERROR | Surge in unhandled runtime exceptions, abnormal spike in HTTP 5xx responses, transaction failure rate breach. | High (Priority 2) | Dynamic baseline deviation (e.g., failure rate exceeds 5% threshold above baseline for 3+ minutes). | Payment gateway returning HTTP 500 Internal Server Errors, database driver throwing connection refused exceptions. |
| PERFORMANCE | Significant degradation in response time, transaction latency, database query duration, or user action duration. | Medium (Priority 3) | Multi-dimensional dynamic baselines (evaluating median response time and 90th percentile degradation). | Checkout service response time degrading from 200 ms to 4.5 seconds due to unindexed database query locks. |
| RESOURCE_CONTENTION | Physical or virtual infrastructure resource saturation impacting or threatening software execution. | Lower-Medium (Priority 4) | Static thresholds (e.g., host CPU > 90%, memory > 95%, disk space < 10%) and dynamic anomaly thresholds. | Linux host memory exhaustion triggering OOM killer risk, storage volume running out of disk space, high JVM GC pause time. |
| CUSTOM_ALERT | Violation of user-defined metric events, custom alert rules, log metric extraction rules, or third-party metric events. | Configurable (Priority 5) | Static user-configured threshold or custom dynamic threshold defined in custom metric event settings. | Custom metric monitoring message queue depth exceeding 10,000 messages for 10 minutes, or log pattern alert on security authentication failures. |
Exam Key Point: The DCA examination strictly tests the five severity categories in order of precedence: AVAILABILITY, ERROR, PERFORMANCE, RESOURCE_CONTENTION, and CUSTOM_ALERT. Availability is the highest severity and represents an immediate operational disruption, whereas Custom Alerts depend entirely on user-defined metric event criteria.
The Four Stages of the Dynatrace Problem Lifecycle
A Dynatrace problem is not a static alert; it is a living, evolving entity. The lifecycle of a problem progresses through four well-defined architectural phases:
+---------------------------------------------------------------------------------------------------+
| DYNATRACE PROBLEM LIFECYCLE |
+---------------------------------------------------------------------------------------------------+
| 1. ANOMALY & EVENT DETECTION |
| • OneAgent / ActiveGate telemetry streams to cluster in real time. |
| • Metric deviations evaluated against dynamic baselines or static thresholds. |
| • Sliding observation window must be satisfied (filtering transient micro-spikes). |
| │ |
| ▼ |
| 2. CAUSAL CORRELATION & PROBLEM OPENING |
| • Davis traverses Smartscape topological dependency model (vertical & horizontal). |
| • Related events merged into a single Problem Card with a unique Persistent ID (PID). |
| • Davis identifies the earliest originating fault as the Root Cause. |
| │ |
| ▼ |
| 3. DYNAMIC EVOLUTION (BLAST RADIUS EXPANSION) |
| • As failure cascades, newly affected upstream services and user sessions are linked. |
| • Overall problem severity dynamically upgrades if conditions worsen (e.g., Error -> Avail). |
| • External notification channels receive UPDATE webhooks if configured. |
| │ |
| ▼ |
| 4. AUTOMATIC RESOLUTION & ARCHIVAL |
| • Underlying metrics return to dynamic baseline for the required sliding evaluation window. |
| • Davis automatically marks the problem RESOLVED (no manual close required). |
| • RESOLVED webhook dispatched to ITSM tools; problem preserved for historical post-mortem. |
+---------------------------------------------------------------------------------------------------+
Stage 1: Anomaly Ingestion and Event Detection
When a metric deviates from normal behavior, Dynatrace does not instantly declare a full problem. To prevent false positives caused by temporary network blips or momentary CPU spikes lasting a few seconds, Davis enforces a sliding observation window:
- For service response times and failure rates, the anomaly must typically persist across multiple consecutive evaluation intervals (such as 3 out of 5 minutes, or 5 consecutive minutes depending on sensitivity configurations).
- If an anomaly resolves before meeting the duration threshold, the event is logged in the entity's event ledger but no Problem Card is created, sparing on-call engineers from alert fatigue.
Stage 2: Causal Correlation and Problem Opening
Once the duration threshold is satisfied, Davis activates the causation engine. Rather than searching the entire data center, Davis performs a targeted search across the Smartscape topological dependency tree:
- Vertical Traversal: Davis inspects the physical stack supporting the affected service—moving from the Service down to the Process Group Instance, Host, Hypervisor, and Cloud Data Center.
- Horizontal Traversal: Davis traces the PurePath call graph upstream and downstream—checking all client services calling the affected component and all backend dependencies called by it.
All correlated events are bound into a single Problem Card assigned a unique Persistent Identifier (PID). Davis marks the earliest originating failure as the Root Cause.
Stage 3: Dynamic Evolution (Blast Radius Expansion)
In complex distributed architectures, an outage often spreads over several minutes. A database deadlock may initially cause slow queries on a single service. Three minutes later, upstream worker processes become starved of threads. Five minutes later, the public e-commerce web application begins displaying HTTP 504 errors to thousands of shoppers.
Davis dynamically accommodates this cascade:
- It does not generate secondary or tertiary problem cards for the newly degraded components.
- Instead, it appends the new symptoms to the existing open problem, dynamically updating the Blast Radius (impacted user sessions, affected synthetic monitors, and impacted services).
- If an incident starts as a Performance issue and subsequently causes an unhandled crash, Davis dynamically escalates the overall problem severity to Availability.
Stage 4: Automatic Resolution and Archival
When remediation takes place—whether through automated self-healing scripts, auto-scaling, or engineering intervention—the underlying metrics begin to recover.
Davis monitors the recovery telemetry against the dynamic baseline. Once all contributing events have normalized for a continuous sliding observation window, Davis automatically closes the problem and sets its state to RESOLVED:
- A resolution notification is dispatched to all integrated external systems (ServiceNow, Jira, Slack, PagerDuty).
- The resolved problem is permanently archived in Dynatrace with full diagnostic replay capabilities, allowing teams to conduct blameless post-mortems with complete historical PurePath call trees and infrastructure charts.
Exam Key Point: Davis automatically resolves problems when metrics return to normal. While administrators can manually close a problem in the Dynatrace console (providing a mandatory comment), manual closure does not stop Davis from reopening a new problem if the underlying anomalous conditions persist.
A critical e-commerce checkout application experiences a sudden slowdown. Davis AI detects elevated response time on the customer-facing web service, an error spike on a middleware payment processing service, and high disk I/O queue wait on a backend PostgreSQL database host. Instead of opening three separate incident tickets, Davis consolidates all three anomalies into a single problem card. How does Davis determine this consolidation, and which entity is designated the root cause?
An application service experiences an intermittent latency spike where average response time surges from 80 milliseconds to 1,200 milliseconds. The spike lasts for exactly 45 seconds before returning to normal baseline performance. An engineer checks the Dynatrace console and notices that no problem card was generated. What platform mechanism explains why a problem was not opened?
A bank's mobile banking application experiences an outage where thousands of users encounter HTTP 500 errors. When the operations lead opens the corresponding Dynatrace problem card, the impact section lists 4,500 affected user sessions and two impacted web services, but the Root Cause Card explicitly states 'No root cause found'. What is the architectural reason for this status?