14.3 Composite Entities & Batch OData API

14.3 Composite Entities & Batch OData API

Quick Answer: Composite Data Entities allow developers to model deeply nested, multi-tier document hierarchies—such as Purchase Order Header -> Lines -> Line Schedules—into a single structured schema authored in Visual Studio. However, composite entities have a critical platform limitation: they are supported strictly in Data Management Framework (DMF) data packages using XML format, and are completely unsupported over OData, BYOD, or Excel integration. Conversely, for high-throughput, synchronous transactional integrations that require real-time execution and immediate feedback, developers leverage the Batch OData API (/data/$batch). The Batch OData API bundles multiple requests into a single HTTP payload formatted as multipart/mixed. Write operations grouped inside a Changeset execute as an atomic database transaction (ttsbegin/ttscommit); if any single operation fails, the entire changeset rolls back automatically.

---

1. Composite Data Entities: Concept & Architecture

Standard data entities flatten normalized relational tables into a single row representation. While effective for simple master entities (such as Customers or Vendors), flattening complex transactional documents creates severe structural inefficiencies. For example, in a Purchase Order document with 1 header and 500 lines, a flat entity must duplicate all header columns 500 times in a flat CSV or Excel file.

A Composite Data Entity solves this by representing business documents as a hierarchical tree of individual data entities.

Composite Data Entity Hierarchy (Purchase Order Example)

┌─────────────────────────────────────────────────────────────┐
│         Root Entity: PurchPurchaseOrderHeaderEntity         │
│  • PurchaseOrderNumber, VendorAccount, CurrencyCode, Date   │
└──────────────────────────────┬──────────────────────────────┘
                               │ 1 : N Relation (PurchId)
                               ▼
┌─────────────────────────────────────────────────────────────┐
│         Child Entity: PurchPurchaseOrderLineEntity          │
│  • LineNumber, ItemId, Quantity, UnitPrice, DeliveryDate    │
└──────────────────────────────┬──────────────────────────────┘
                               │ 1 : N Relation (LineRefRecId)
                               ▼
┌─────────────────────────────────────────────────────────────┐
│     Grandchild Entity: PurchPurchaseOrderScheduleEntity     │
│  • ScheduleLineNumber, ScheduledDeliveryDate, ScheduledQty  │
└─────────────────────────────────────────────────────────────┘

Architectural Characteristics

• Entity Tree Composition: A composite entity does not bind directly to physical tables; it encapsulates other existing data entities. A root entity hosts one or more child entities, which in turn can host grandchild entities.
• XML Schema Definition (XSD): The visual hierarchy generates an XML schema representing the nested structure. Each business document is serialized into a single coherent XML document.
• Single Document Transactional Integrity: When a composite document imports through DMF, the header and its associated child lines are evaluated together, preventing orphaned line records.

---

2. Visual Studio Modeling of Composite Entities

In Visual Studio Application Explorer, developers model composite entities under Data Model > Composite Data Entities.

Authoring Steps

1. Prerequisite Single Entities: Ensure all participating entities exist as compiled, valid single data entities (e.g., SalesOrderHeaderEntity and SalesOrderLineEntity).
2. Create Composite Entity Artifact: Right-click your project > Add > New Item > Data Model > Composite Data Entity.
3. Define Root Node: Drag the root entity (e.g., SalesOrderHeaderEntity) onto the designer.
4. Add Child Nodes: Right-click the root node > New Embedded Data Entity, and select the child entity (e.g., SalesOrderLineEntity).
5. Configure Entity Relations: In the child entity properties, define the relation mapping child fields to parent fields (e.g., SalesOrderLineEntity.SalesOrderNumber == SalesOrderHeaderEntity.SalesOrderNumber).
6. Generate Schema: Right-click the composite entity node and select Generate Schema to export the structural .xsd file used by external systems for message validation.

---

3. Critical Platform Constraints & Supported Scenarios

Understanding where Composite Data Entities can and cannot be used is one of the most heavily tested topics on the MB-500 exam.

Composite Data Entity Platform Compatibility Matrix

┌──────────────────────────────────────┬─────────────┬──────────────────────────────────────┐
│ Feature / Technology Channel         │ Supported?  │ Reason / Restriction Details         │
├──────────────────────────────────────┼─────────────┼──────────────────────────────────────┤
│ Data Management (DMF) Data Packages  │    YES      │ Strictly with XML format in packages │
│ Recurring Integrations (Async Queue) │    YES      │ XML data package queues              │
│ OData v4 REST Endpoints              │   NO (!)    │ IsPublic property not supported      │
│ Bring Your Own Database (BYOD)       │   NO (!)    │ Staging tables must be flat SQL rows │
│ Open in Excel (Excel Add-in)         │   NO (!)    │ Requires Public OData Entities       │
│ Flat File Formats (CSV / TSV)        │   NO (!)    │ Delimited files cannot model trees   │
└──────────────────────────────────────┴─────────────┴──────────────────────────────────────┘

[!CAUTION] Core Exam Rule: Composite Entities Strictly Prohibited on OData A composite data entity cannot be marked with IsPublic = Yes. It cannot be exposed as an OData v4 endpoint. Any architectural proposal suggesting that external systems make live OData CRUD calls directly against a composite entity is technically invalid.

---

4. The Batch OData API (/data/$batch) Architecture

When external systems require synchronous, real-time integration (e.g., an e-commerce checkout committing a sales order header and 5 lines with immediate confirmation), using individual OData calls produces severe network latency:

• 1 POST for Header + 5 POST requests for Lines = 6 independent HTTP round trips.
• Each HTTP request incurs SSL/TLS handshaking, authentication evaluation, and a separate database transaction.
• If Line 4 fails validation, Lines 1 through 3 are already committed, leaving corrupted, partial data in the database.

To solve this, Dynamics 365 Finance and Operations implements the OData v4 Batch Specification ($batch).

Batch OData API Architecture

External Client (Single HTTP POST to /data/$batch)
 │
 ├── Request Header: Content-Type: multipart/mixed; boundary=batch_12345
 │
 ├── Part 1: Independent Query (GET /data/CustomersV3?$top=2)
 │
 └── Part 2: Transactional Changeset (boundary=changeset_abcde)
      ├── Operation 1: POST /data/SalesOrderHeadersV2
      ├── Operation 2: POST /data/SalesOrderLines
      └── Operation 3: POST /data/SalesOrderLines
              │
              ▼ (AOS Processing Engine)
     ┌──────────────────────────────────────────────┐
     │  Changeset Atomic Transaction Boundary       │
     │  ttsbegin;                                   │
     │    Operation 1 -> validateWrite() & insert   │
     │    Operation 2 -> validateWrite() & insert   │
     │    Operation 3 -> validateWrite() -> FAILS!  │
     │  ttsabort; // FULL ROLLBACK OF ALL 3 OPS     │
     └──────────────────────────────────────────────┘
              │
              ▼
 Client Receives 400 Bad Request; Database remains completely untouched

---

5. Multipart/Mixed MIME Formatting & Changeset Atomicity

A $batch request is submitted as a single HTTP POST to https://<environment>.operations.dynamics.com/data/$batch.

MIME Structure & Boundary Identifiers

1. Batch Boundary: Defined in the HTTP header: Content-Type: multipart/mixed; boundary=batch_boundary_id.
2. Query Operations: Standalone GET requests placed directly between batch boundaries. These execute outside any changeset and do not initiate write transactions.
3. Changesets: Delimited by a secondary boundary: Content-Type: multipart/mixed; boundary=changeset_boundary_id. A Changeset encapsulates write operations (POST, PATCH, PUT, DELETE).

Changeset Atomicity & Transaction Rollback

The fundamental rule of OData Changesets is Atomicity (All-or-Nothing):

• The AOS wraps all operations within an individual Changeset inside a single X++ database transaction (ttsbegin ... ttscommit).
• If all operations pass server-side validations (validateWrite()) and table constraints, the entire changeset commits permanently to SQL Server.
• If any single operation within the changeset fails (e.g., an invalid financial dimension, duplicate key, or missing mandatory field), the entire changeset is aborted (ttsabort).
• Every change made by prior operations within that same changeset is rolled back. No partial or orphaned records are ever committed.

Inter-Operation Referencing with Content-ID

When inserting a parent header and child lines in the same changeset, the client does not yet know the generated primary key of the header. OData allows operations within the same changeset to reference newly created resources using the Content-ID header:

• Header operation assigns Content-ID: 1.
• Line operations reference the header using $1 in the URI path or property binding: POST $1/SalesOrderLines or setting SalesOrderNumber: "$1".

Sample Multipart/Mixed Request Payload

POST /data/$batch HTTP/1.1
Host: contoso.operations.dynamics.com
Authorization: Bearer <OAuth_Token>
Content-Type: multipart/mixed; boundary=batch_mybatch_01

--batch_mybatch_01
Content-Type: multipart/mixed; boundary=changeset_mychange_01

--changeset_mychange_01
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 1

POST /data/SalesOrderHeadersV2 HTTP/1.1
Content-Type: application/json;odata.metadata=minimal

{
    "dataAreaId": "usmf",
    "SalesOrderNumber": "SO-00991",
    "OrderingCustomerAccountNumber": "US-001"
}

--changeset_mychange_01
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 2

POST /data/SalesOrderLines HTTP/1.1
Content-Type: application/json;odata.metadata=minimal

{
    "dataAreaId": "usmf",
    "SalesOrderNumber": "SO-00991",
    "ItemNumber": "D0001",
    "OrderedSalesQuantity": 10
}

--changeset_mychange_01--
--batch_mybatch_01--

---

6. High-Throughput Batch Integration Guidelines & Sizing

To achieve optimal performance when integrating transactional systems using the Batch OData API:

• Batch Sizing Guidelines: Microsoft recommends bundling 50 to 100 operations per $batch request. Exceeding 100 operations per request increases memory pressure on the AOS, risks HTTP gateway timeouts (ARR 504 Gateway Timeout), and extends database lock durations.
• Multiple Changesets vs. Single Changeset: A single $batch envelope can contain multiple independent changesets. Each changeset represents its own isolated transaction. If Changeset 1 succeeds and Changeset 2 fails, Changeset 1 remains committed while Changeset 2 rolls back. If all operations must succeed or fail together, they must reside within the same changeset.
• Throttling & Retries: High-frequency OData batch requests are subject to Priority-Based Throttling. Middleware must inspect response headers for Retry-After and implement exponential backoff upon receiving HTTP 429 Too Many Requests.

Architectural Decision Framework

---

7. Scenario Walk-Through: POS Real-Time Checkout vs. Warehouse ASN Import

Scenario A: Point-of-Sale (POS) Real-Time Order Insertion

A retail client operates 50 physical stores. When a cashier completes a transaction, the POS system must immediately register the sales order header, payment lines, and order items in Dynamics 365 Finance. If payment validation fails or an item code is invalid, the entire transaction must be rejected immediately so the cashier can correct the issue.

• Solution: The POS middleware sends an HTTP POST to /data/$batch. All line and header operations are enclosed in a single Changeset. If any line fails server validation (validateWrite()), the AOS executes ttsabort, returns HTTP 400 Bad Request, and leaves the ERP database completely untouched.

Scenario B: Automated Advance Shipping Notice (ASN) Import

A 3PL logistics provider delivers a nightly batch of 25,000 Advance Shipping Notices (ASNs), each containing header, container, pallet, and item lines. Real-time synchronous confirmation is not required, but hierarchical document relationships must be maintained without duplicating header data across hundreds of thousands of lines.

• Solution: A Composite Data Entity (PurchASNCompositeEntity) is authored in Visual Studio. The 3PL generates a DMF data package containing the XML ASN payload. Middleware pushes the package via /api/connector/enqueue. The recurring batch job processes the package asynchronously in background batch threads.

---

8. Real-World Exam Traps: Composite Entities & Batch OData

[!WARNING] Exam Trap 1: Attempting to Expose Composite Entities via OData An exam question may propose setting IsPublic = Yes on a composite entity to allow Power Apps or logic apps to query nested purchase orders over OData. This is impossible. Composite entities do not support the IsPublic property and cannot be consumed by OData.

[!WARNING] Exam Trap 2: Using CSV / Flat Formats with Composite Entities When configuring a data project for a composite entity, selecting Delimited (CSV) or Excel as the source data format causes immediate validation failures. Composite entities strictly require XML data packages.

[!WARNING] Exam Trap 3: Expecting Multi-Changeset Cross-Atomicity If an exam question describes a $batch request containing two changesets (Changeset A and Changeset B), and an operation in Changeset B fails, Changeset A does not roll back. Atomicity applies strictly within an individual Changeset, not across multiple changesets in the batch envelope.

[!WARNING] Exam Trap 4: Exceeding Batch Sizing Limits Proposals to submit thousands of operations in a single $batch request to maximize throughput will lead to gateway timeouts (HTTP 504) and memory exhaustion. The recommended batch size is 50 to 100 operations per $batch call.