14.4 Monitoring Entity Status, Availability & Job Execution

14.4 Monitoring Entity Status, Availability & Job Execution

Quick Answer: The skills outline lists "Monitor the status and availability of entities" as a distinct objective, and it covers two things candidates routinely conflate. Availability is whether an entity is present and usable at all, which is governed by the entity list refresh and by configuration keys. Status is what happened when a job actually ran, which lives in job history — execution details and the execution log. The refresh path to memorise is Data management > Framework parameters > Entity settings > Refresh entity list. It runs automatically when a new environment is deployed, but you must start it manually after deploying a code package and manually after changing configuration keys, and you must let it finish before running any job that uses the affected entities.

---

1. Entity List Refresh: The Source of Entity Availability

The Data management framework does not read entity metadata live. It builds a runtime metadata catalogue — including the configuration-key picture described below — and every job, template, and mapping operation reads from that catalogue. Refreshing the entity list is what rebuilds it.

While the refresh is running, the entity list page displays a message saying so. The rule is explicit: wait for the entity list refresh to complete before using jobs and entities. If you do not, the configuration-key metadata may be out of date and the job produces unexpected outcomes — a failure mode that looks like a data defect and is actually a stale catalogue.

[!IMPORTANT] The duplicate-label trap. Every entity, including every ISV entity, must use a unique label for its Label property. If two entities share a label, entities can go missing after an entity list refresh, and existing data management projects can silently bind to a different entity. This cannot be repaired in production: you must fix the duplicate label, redeploy the package, and test thoroughly in pre-production first. A custom entity that "disappeared after deployment" is the exam's way of describing this.

---

2. Configuration Keys Decide Whether an Entity Is Available At All

Configuration keys can be assigned at four levels — the data entity, the tables used as data sources, the table fields, and the data entity fields — and the framework evaluates them top-down. Only the levels that remain reachable are evaluated at all.

When an entity uses another entity as a data source, or is a composite entity, the framework applies these same rules recursively.

The Four Monitoring Surfaces

• Data entity list page (Data management workspace). Start here. It shows a Configuration key column naming the key associated with each entity — blank means no key is associated — and a Configuration key status column showing a checkmark when the key is enabled and blank when it is disabled or absent.
• Target fields form. Drill into an entity to see the configuration key and status for its related tables and fields. If the entity's own key is disabled, the form warns that nothing listed is available regardless of individual statuses.
• Child entities form. For entities that use other entities as data sources, or for composite entities, this form shows the same key information one level down.
• Validate, available on the entity list, forces a run-time validation on demand rather than waiting for a job to fail.

When Run-Time Validation Fires

The framework re-checks the configuration-key metadata automatically at ten points, which is worth knowing because it explains where a stale catalogue surfaces: when an entity is added to a job; when the user selects Validate on the entity list; when a data package is loaded into a data project; when a template is loaded into a data project; when an existing data project is loaded; before any export or import job runs, whether batch, non-batch, recurring, or OData; when the user generates a mapping; when the user maps fields in the mapping UI; and when the user adds only importable fields.

After any configuration-key change at entity, table, or field level, the entity list shows a warning until you refresh. The changes take effect immediately once the refresh completes, so validate existing projects and jobs afterwards to confirm they still behave as intended.

---

3. Job Status: Job History, Execution Details & the Execution Log

Availability answers "can this entity run?" Job history answers "what happened when it did?" It is reached from the Data management workspace and organises historical runs by time range.

Each job run exposes two artefacts:

From execution details you can also download the staging data — as a file for export jobs, or as a package for both import and export jobs — which is the supported way to inspect exactly what landed in staging versus what reached the target.

[!TIP] Refresh the form, not the browser. To see the latest progress on the Data import/export form, use the form refresh icon. A browser-level refresh is explicitly discouraged because it interrupts any import or export job that is not running in batch. This is a favourite scenario detail: a user reports that a large interactive import "stopped for no reason" after they pressed F5.

Interactive, Batch, or Immediate

How a job is launched changes how you monitor it. Import and Export schedule a batch job that runs once, so execution may be delayed if the batch service is throttling under load. Import now and Export now start the work immediately and synchronously, which is the workaround when throttling prevents a batch from starting. Run in batch schedules for later and allows a specific batch group, which gives the most load-balancing control and is the recommended option for large volumes.

---

4. History Retention: Why Yesterday's Evidence May Be Gone

Monitoring has a shelf life, and the exam tests the defaults.

• The system automatically deletes job history entries and their related staging table data older than 90 days, whether or not you configure anything.
• Job history cleanup (Data management > Job history cleanup) lets you purge on a shorter retention. Its two scheduling parameters are Number of days to retain history, which defaults to seven days and behaves as a continuously moving window when scheduled as a recurring batch, and Number of hours to execute the job, whose default and minimum are two hours. The job processes one execution ID at a time from oldest to newest and stops picking up new ones once the remaining time is inside the final 10 percent of the window.
• The cleanup removes rows from all staging tables plus the DMF staging validation log, staging execution errors, staging log detail, staging log, definition group execution history, and execution tables. This functionality replaces the deprecated staging table cleanup.
• In the newer system-batch model the cleanup is scheduled for you — it runs hourly from midnight, retains the most recent seven days, and archives the purged history to the DIXF blob storage as a downloadable package kept for seven days by default and configurable up to 90 via Days to retain package in blob.

The practical consequence: if a monthly reconciliation depends on being able to inspect an import that ran three weeks ago, a seven-day retention setting will already have removed it. Retrieve the archive package, or lengthen retention before you need it.

---

5. Scenario Walk-Through: An Entity That Vanished After a Release

Scenario Background

A partner deploys a release into a Tier 2 sandbox. The next morning the integration team reports three problems: a custom ABC_VendorOnboarding entity is missing from the entity list entirely; a standard entity that previously exported 12 columns now exports 9; and last week's import run that finance wanted to audit cannot be found.

Step-by-Step Diagnosis

1. Start the entity list refresh. Because the environment already used data management, deploying a code package does not trigger the refresh automatically. The team opens Framework parameters > Entity settings > Refresh entity list and waits for the in-progress message on the entity list page to clear before touching any job.
2. Investigate the still-missing entity. After the refresh, ABC_VendorOnboarding is still absent. Comparing metadata shows its Label property duplicates the label on an ISV entity shipped in the same release. Duplicate labels cause entities to go missing after a refresh and can rebind existing projects to the wrong entity. The label is made unique, the package is redeployed, and the refresh is run again — and the team notes that had this reached production, it could not have been fixed there.
3. Explain the three lost columns. The release also disabled a configuration key. On the Data entity list page the entity's Configuration key status column is still checked, so the entity itself is fine. Drilling into Target fields shows three fields whose table-level key is now disabled — and a disabled table field key removes the field even though the corresponding entity field key is still enabled. The export is behaving correctly; the release note was wrong.
4. Confirm the change took. The team selects Validate on the entity list to force run-time validation immediately rather than waiting for the next scheduled job to fail, then reloads the affected data projects so the framework re-validates them.
5. Recover the missing history. Job history no longer holds last week's run because job history cleanup is scheduled with the default seven-day retention. The team opens Job history cleanup > Package backup history, locates the archived DIXF package, and downloads it — the archive holds the staging table file together with the execution, staging log, and staging error tables. They then raise Days to retain package in blob toward the 90-day maximum so the next month-end audit is not a race.

---

6. Real-World Exam Traps: Entity Status & Availability

[!WARNING] Exam Trap 1: Expecting the Entity List to Refresh Itself After a Deployment The refresh is automatic only when a new environment is deployed. After a code package deployment into an existing environment, and after any configuration key change, it must be started manually. An option claiming the framework picks up new entities automatically post-deployment is wrong.

[!WARNING] Exam Trap 2: Running Jobs During the Refresh Starting a job while the entity list is still refreshing means the configuration-key metadata may be stale, producing wrong results that look like data defects. The correct action is always to wait for completion, then validate the affected projects.

[!WARNING] Exam Trap 3: Blaming Security When a Configuration Key Is the Cause "A field disappeared from the entity" and "the entity isn't functional" are configuration-key symptoms, diagnosed on the Data entity list page and the Target fields form. Field-level security produces an access error for a user; a configuration key removes the artefact for everyone.

[!WARNING] Exam Trap 4: Pressing F5 During an Interactive Import A browser-level refresh interrupts any import or export job not running in batch. The form refresh icon is the supported way to see progress. When a stem describes a long interactive import that stopped without an error, look for the browser refresh.

[!WARNING] Exam Trap 5: Assuming Job History Is Kept Indefinitely Execution history and its staging data older than 90 days are deleted automatically, and the job history cleanup feature defaults to retaining only seven days. Evidence for a long-running investigation must come from the archived package, not from job history.