17.4 Diagnostics: Trace Parser, Async & Sandbox Frameworks

17.4 Diagnostics: Trace Parser, Async & Sandbox Frameworks

Quick Answer: Performance diagnostics in Dynamics 365 Finance and Operations centers on capturing ETW traces using the browser-based Trace Cockpit and analyzing them in Microsoft Dynamics Trace Parser. Trace Parser isolates X++ execution time from SQL statement duration, identifies missing index recommendations, and diagnoses client-server chattiness (excessive RPC round-trips). To eliminate interactive UI freezes, long-running operations are offloaded using the runAsync framework (which requires static methods and parameter passing via primitive X++ containers) or isolated using the Sandbox framework (SysDictClass::invokeObjectMethodSandbox), which executes code in a separate worker session to achieve complete fault isolation.

---

1. Capturing Performance Traces with Trace Cockpit

Developers and administrators can capture detailed Event Tracing for Windows (ETW) logs directly within the Dynamics 365 web client without requiring Remote Desktop or administrative Azure portal access.

Trace Capture & Analysis Workflow

┌─────────────────────────────────────────────────────────────┐
│                     Dynamics 365 Web Client                 │
│  (System administration > Inquiries > Trace > Trace cockpit)│
├─────────────────────────────────────────────────────────────┤
│ 1. Open Trace Cockpit                                       │
│ 2. Set Trace Name & Enable 'Capture SQL Statement Events'   │
│ 3. Click 'Start Trace'                                      │
│ 4. Reproduce exact user scenario in new browser tab         │
│ 5. Return to Trace Cockpit and click 'Stop Trace'          │
│ 6. Click 'Download Trace' (.etl.zip package)                │
└──────────────────────────────┬──────────────────────────────┘
                               │ Download .etl.zip
                               ▼
┌─────────────────────────────────────────────────────────────┐
│               Microsoft Dynamics Trace Parser               │
│  • Import Trace (.etl)                                      │
│  • Analyze Call Tree, X++ vs. SQL time, RPC counts          │
│  • Inspect Missing Indexes & Slow Statements                │
└─────────────────────────────────────────────────────────────┘

Trace Capture Best Practices

• Keep Scenarios Narrow: Start the trace immediately before clicking the problematic button and stop it immediately after the operation completes. Capturing general navigation or idle time inflates trace sizes with noise.
• Include SQL Events: Always ensure the Capture SQL Statement Events toggle is enabled to record exact T-SQL text, execution plans, and duration metrics.
• Target User Session: Traces can be bound to the current user session or configured to trace background batch processes by specifying the target worker thread.

---

2. Microsoft Dynamics Trace Parser Deep Dive

Microsoft Dynamics Trace Parser is the primary diagnostic desktop tool for analyzing captured .etl trace files.

The Call Tree & Execution Metrics

Trace Parser structures trace events into a hierarchical Call Tree that visualizes the exact execution stack from UI event down to database statement:

• Inclusive Time (Total Time): The total elapsed time spent inside a method, including all child methods and database calls it invoked.
• Exclusive Time (Self Time): The time spent purely inside that specific method's own instructions, excluding any calls to child methods.
• Call Count: The total number of times a method was executed. High call counts (e.g., 50,000 executions of a helper method) immediately signal loop inefficiency.

Trace Parser Diagnostic Breakdown

Scenario A: Database Bottleneck (I/O Bound)
┌─────────────────────────────────────────────────────────────┐
│ Total Elapsed Time: 120 seconds                             │
│ ├─ X++ Execution Time:          8 seconds  ( 6.7%)          │
│ └─ SQL Statement Duration:    112 seconds  (93.3%)  <── BOTTLENECK
│    ↳ Inspect: Missing indexes, table scans, lock waits      │
└─────────────────────────────────────────────────────────────┘

Scenario B: AOS Logic Bottleneck (CPU Bound)
┌─────────────────────────────────────────────────────────────┐
│ Total Elapsed Time: 120 seconds                             │
│ ├─ X++ Execution Time:        114 seconds  (95.0%)  <── BOTTLENECK
│ └─ SQL Statement Duration:      6 seconds  ( 5.0%)          │
│    ↳ Inspect: Algorithmic complexity, container loops       │
└─────────────────────────────────────────────────────────────┘

Diagnosing Client-Server Chattiness (RPC Round-Trips)

In the Trace Parser Summary tab, examine the RPC Round-Trip Count:

• If opening a form or scrolling a grid produces thousands of RPC calls, the system is suffering from client-server chattiness.
• The root cause is almost always an uncached display method, unjoined table datasources, or redundant calls to reread() and research().

Missing Index Suggestions & Top SQL Statements

The SQL Statements tab in Trace Parser ranks all queries by cumulative duration:

• Missing Indexes: Trace Parser highlights queries where SQL Server suggested an index during execution plan evaluation, displaying the recommended inequality columns and included columns.
• Row-by-Row Detection: If an update_recordset degraded into row-by-row fallback, Trace Parser will show thousands of individual UPDATE statements matching the row count rather than a single set-based statement.

---

3. Asynchronous Processing Patterns: The runAsync Framework

Executing long-running operations synchronously in interactive user sessions causes browser "Page Unresponsive" dialogs and HTTP gateway timeouts. The runAsync framework executes intensive tasks asynchronously on a background worker thread.

runAsync Architecture & Thread Separation

Interactive User Session (Thread A)         Background Worker Thread (Thread B)
┌────────────────────────────────────┐      ┌────────────────────────────────────┐
│ User clicks 'Recalculate'
│ 1. Pack inputs into container      │      │                                    │
│    params = [AccountNum, Date];    │      │                                    │
│ 2. runAsync(class, method, params, │      │                                    │
│             successCb, failCb);    │─────>│ 3. Unpack container                │
│                                    │      │ 4. Execute heavy calculation       │
│ UI remains fully responsive!       │      │ 5. Pack results into container     │
│ User continues working...          │<─────│ 6. Dispatch result container       │
│                                    │      └────────────────────────────────────┘
│ 7. asyncSuccessCallback(result)    │
│    (Updates UI / Displays message) │
└────────────────────────────────────┘

Rules for Implementing runAsync

1. Static Method Requirement: The asynchronous method to run must be a public static method.
2. Container Serialization: Because Thread A and Thread B have completely separate memory stacks, object pointers and class instances cannot cross thread boundaries. All input parameters and output results must be serialized into an X++ container.
3. Callback Methods: The developer defines static callback methods to handle completion or failure.

Complete runAsync Implementation Example

/// <summary>
/// Demonstrates asynchronous processing using the runAsync framework.
/// </summary>
public final class FinancialCalculationAsyncService
{
    /// <summary>
    /// Initiates the background calculation from a form or action button.
    /// </summary>
    public static void startCalculation(CustAccount _accountNum, TransDate _cutOffDate)
    {
        // Pack parameters into a container
        container asyncParams = [_accountNum, _cutOffDate];

        // Invoke runAsync with worker method, parameters, and callbacks
        runAsync(
            classNum(FinancialCalculationAsyncService),
            staticMethodStr(FinancialCalculationAsyncService, calculateWorker),
            asyncParams,
            classNum(FinancialCalculationAsyncService),
            staticMethodStr(FinancialCalculationAsyncService, calculationSuccessCallback),
            classNum(FinancialCalculationAsyncService),
            staticMethodStr(FinancialCalculationAsyncService, calculationFailureCallback)
        );
    }

    /// <summary>
    /// Asynchronous worker method executing on background thread.
    /// </summary>
    public static container calculateWorker(container _params)
    {
        CustAccount accountNum;
        TransDate   cutOffDate;
        [accountNum, cutOffDate] = _params;

        // Execute heavy business calculation
        AmountMST calculatedTotal = CustTrans::calcTotalOutstanding(accountNum, cutOffDate);

        // Return result packed in a container
        return [accountNum, calculatedTotal];
    }

    /// <summary>
    /// Success callback executed upon completion.
    /// </summary>
    public static void calculationSuccessCallback(container _result)
    {
        CustAccount accountNum;
        AmountMST   total;
        [accountNum, total] = _result;

        Info(strFmt("@ApplicationPlatform:AsyncCalculationSuccess", accountNum, total));
    }

    /// <summary>
    /// Failure callback executed if an unhandled exception occurs in the worker.
    /// </summary>
    public static void calculationFailureCallback(container _exceptionInfo)
    {
        Error("@ApplicationPlatform:AsyncCalculationFailed");
    }
}

---

4. The Sandbox Framework (SysDictClass::invokeObjectMethodSandbox)

While runAsync offloads work asynchronously to keep the UI interactive, the Sandbox Framework is designed for execution isolation and fault containment.

Architectural Purpose of the Sandbox

When integrating with third-party components, executing untrusted document parsers, or evaluating complex pricing matrices, a runtime crash (such as a stack overflow, fatal memory exception, or unhandled CLR abort) would normally terminate the entire user session, destroying all unsaved form data.

Executing logic through SysDictClass::invokeObjectMethodSandbox spins up a secondary, isolated worker session:

• Fault Isolation: If the sandboxed code throws an unhandled exception or crashes, only the secondary sandbox session is terminated. The user's primary interactive session remains healthy and intact.
• Clean Memory Scope: All temporary allocations and memory buffers created during the sandboxed call are automatically reclaimed when the secondary session closes.

Sandbox Implementation Example

/// <summary>
/// Executes complex CAD parsing inside a protected sandbox worker session.
/// </summary>
public static container parseCadDocumentSandbox(CADModelParser _parserInstance, Filename _filePath)
{
    container inputParams = [_filePath];
    container resultContainer;

    SysDictClass dictClass = new SysDictClass(classNum(CADModelParser));

    // Executes the method inside an isolated secondary worker session
    resultContainer = dictClass.invokeObjectMethodSandbox(
        _parserInstance,
        methodStr(CADModelParser, executeParse),
        inputParams
    );

    return resultContainer;
}

---

5. Diagnostic & Asynchronous Framework Decision Matrix

---

6. Realistic Enterprise Scenario: Diagnosing & Remediating a Frozen Warehouse Scanning Form

Business Problem

In a 24/7 logistics hub, warehouse operators use custom mobile tablets running an interactive warehouse dispatch scanning form (WHSDispatchScanForm). During peak shift changeovers, scanning a pallet barcode causes the form UI to freeze for 35 to 45 seconds. Operators repeatedly click the scan button, generating additional thread contention and locking terminal sessions. The operations manager escalates the incident as a critical operational blocker.

Architecture & Implementation Walkthrough

1. Capturing Telemetry via Trace Cockpit: A developer navigates to System administration > Inquiries > Trace > Trace cockpit, starts a trace, replicates scanning a single pallet on the mobile form, and immediately stops and downloads the .etl.zip trace.
2. Isolating Root Causes in Trace Parser:
   • RPC Chattiness: The Summary tab shows an astounding 3,400 RPC round-trips during the 40-second freeze.
   • Uncached Display Method: Expanding the Call Tree reveals that WHSDispatchScanForm includes a grid displaying pallet items with a display method calcTotalPalletWeight(). The method recalculates physical weight by querying InventDim and InventTrans on every redraw, firing 3,400 times as records render.
   • Synchronous Carrier Rate Call: The Call Tree further reveals an 11-second synchronous call to an external legacy C++ DLL (CarrierRateEngine.dll) executing inside the UI thread to calculate real-time freight surcharges.
3. Applying Remediation:
   • Display Method Caching: The developer decorates calcTotalPalletWeight() with [SysClientCacheDataMethodAttribute(true)]. This eliminates 3,380 RPC round-trips, satisfying subsequent row views from browser memory.
   • Offloading Calculation via runAsync: The freight calculation logic is refactored into a public static worker method, dispatched asynchronously via runAsync using container parameters ([palletId, warehouseCode]). The form remains fully responsive and immediately displays a loading spinner.
   • Isolating C++ DLL Crashes via Sandbox: Because the third-party carrier DLL occasionally throws memory segmentation faults on corrupt barcode payloads, the execution is wrapped inside SysDictClass::invokeObjectMethodSandbox. If the DLL crashes, only the secondary sandbox session terminates, and a user-friendly error message is displayed without crashing the warehouse operator's active form.

Measurable Outcomes

• Form scan response time dropped from 42 seconds to 320 milliseconds (a 99.2% improvement).
• RPC round-trips per scan dropped from 3,400 down to 4.
• Terminal app crashes dropped to zero, eliminating warehouse dispatch bottlenecks.

---

7. Real-World MB-500 Exam Traps

[!WARNING] Exam Trap 1: Attempting to Pass Living Objects Across runAsync Threads A favorite MB-500 trick question presents code attempting to pass a table buffer (CustTable) or a class instance directly as an argument into runAsync. In .NET Core / Dynamics 365, asynchronous worker threads execute on completely isolated thread stacks. Living object references and pointers cannot cross thread boundaries. All input parameters and return values must be packed and unpacked using primitive X++ container objects.

[!WARNING] Exam Trap 2: Using Instance Methods for runAsync Workers The target worker method and both callback methods passed to runAsync must be public static. Attempting to pass an instance method or private method causes a runtime reflection exception when the AOS thread pool attempts invocation.

[!WARNING] Exam Trap 3: Confusing the Sandbox Framework with runAsync While both frameworks execute code away from standard execution paths, their objectives differ fundamentally. runAsync is designed for asynchronous responsiveness on background worker threads (non-blocking UI). The Sandbox framework (invokeObjectMethodSandbox) is designed for fault isolation and session containment (shielding the primary user session from fatal crashes in risky or unmanaged code). Sandbox execution is typically synchronous within an isolated secondary session.

[!WARNING] Exam Trap 4: Misinterpreting Inclusive vs. Exclusive Time in Trace Parser When analyzing a slow trace, candidates often jump to the method with the highest Inclusive Time. However, top-level methods (like FormRun.init() or Class.run()) almost always have the highest Inclusive Time because they enclose all child calls. To pinpoint the actual line of slow code, developers must sort by Exclusive Time (Self Time) to see where CPU time is actually being spent, or inspect the SQL Statements tab for long-running database queries.

[!WARNING] Exam Trap 5: Forgetting to Enable SQL Statement Events in Trace Cockpit If an ETW trace is captured without checking the Capture SQL Statement Events toggle in the Trace Cockpit, the trace will capture X++ call stacks and RPC events, but the SQL Statements tab in Trace Parser will be completely empty. For database diagnostics, capturing SQL events is mandatory.