3.1 Create and Manage Data Assets

Create and Manage Data Assets

Quick Answer: An Azure Machine Learning data asset is a versioned bookmark to data that already lives in storage. The three SDK/CLI v2 types are uri_file (one file), uri_folder (a folder), and mltable (a table blueprint). A datastore is the connection; a data asset is the named version you pass to jobs. Create with az ml data create -f data.yml. Versions are immutable — archive mistakes, do not expect deletion.

Domain 1 of Exam AI-300 asks you to create and manage data assets in an Azure Machine Learning workspace. That bullet is not “upload a CSV.” It is about giving Machine Learning Operations (MLOps) a durable identity for the data a job consumed so you can reproduce training, audit lineage, and stop embedding storage secrets in scripts.

Data asset versus datastore

Keep these two workspace objects separate. You studied datastores in Chapter 2; this section consumes them.

A datastore can use credential-based authentication (account key, shared access signature, service principal) or identity-based authentication (Microsoft Entra ID or a managed identity). Users with Reader on the workspace can retrieve credential-based secrets stored on the datastore, which is why identity-based access is the safer default for production. A data asset does not replace that connection. It uses a path — often a datastore URI — and records which version of “claims training images” a pipeline pinned.

Microsoft’s own analogy is a browser bookmark. You stop memorizing a 200-character wasbs:// or abfss:// string and instead ask for azureml:claims-images:2026-08-15. Creating the asset writes a reference plus metadata. The bytes stay where they are, so you do not pay a second storage bill and you do not fork the source of truth.

Exception that the exam loves: if path is on your local computer, Azure Machine Learning uploads the file or folder into the workspace default datastore (workspaceblobstore) and then registers the asset against that cloud path. Local create is the one case where “registering” also copies bytes.

You are not required to create a data asset before a notebook or job can read data. Datastore URIs work immediately. Assets become valuable the moment a second person, a pipeline, or a later calendar quarter needs the same identity.

The three v2 types

A Uniform Resource Identifier (URI) can point at a file, a folder, or a table. Job inputs and outputs must pick one of these three data types:

1. uri_file — exactly one file, any format (CSV, Parquet, PNG, pickle, JSON). In v2 the job maps that file, not a parent folder you then os.path.join.
2. uri_folder — one directory. Canonical uses: a folder of Parquet or CSV files you load with pandas or Spark, or unstructured collections (images, audio, text, video) for deep learning.
3. mltable — Azure Machine Learning Tables. Use when the schema is complex or changes often, you need a subset of a large table, you run automated machine learning (AutoML) on tables, or files are spread across multiple storage locations. The materialization blueprint (which files, which read_delimited options, which filter / drop_columns) lives in your storage as an MLTable file, so you can even use it disconnected from the workspace. Do not rename that file to MLTable.yaml; the platform expects the name MLTable.

Embedded newlines inside quoted CSV fields are a classic misalignment bug. Register that file as mltable and set support_multi_line on read_delimited. A plain uri_file of CSV does not apply that transform.

You cannot change an asset’s type across versions. A wrong type means: archive the name, then create a new asset name with the correct type.

URIs, modes, and materialization versus reference

Supported path locations include a local folder, a datastore URI, a public https:// file, Blob (wasbs://), Azure Data Lake Storage Gen2 (abfss://), and Gen1 (adl://). A job maps the URI onto the compute target filesystem using a mode:

• Inputs: ro_mount (read-only mount), download, or direct (pass the URI string; your code talks to storage itself).
• Outputs: rw_mount (read-write mount) or upload.

Reference is the default story for a data asset: metadata in the workspace, bytes in the original account. Materialization is what happens at job time — mount, download, or, for mltable, evaluate the blueprint (eval_mount / eval_download) so only the globbed files appear on the node. Azure Machine Learning’s data runtime (a lightweight Rust stack, no Java Virtual Machine) performs those mounts and table materializations with parallel load and background prefetch so GPUs are not stalled on disk.

Mount versus download is an operations choice: mounts stream; downloads pay copy time and local disk. direct is for code that already understands abfss:// and should not pretend the data is a local path.

Create, version, tag, and consume

A minimal data.yml for CLI v2:

$schema: https://azuremlschemas.azureedge.net/latest/data.schema.json
type: uri_folder
name: claims-images
version: "2026-08-15"
description: Weekly claims photos through 15 Aug 2026
tags:
  medallion: silver
  sensitivity: nonPII
path: azureml://datastores/adls_claims/paths/images/year=2026/week=33/

Submit with az ml data create -f data.yml. Python SDK v2 uses azure.ai.ml.entities.Data with type=AssetTypes.URI_FOLDER (or URI_FILE / MLTABLE) and ml_client.data.create_or_update. Studio can create uri_file and uri_folder; mltable authoring is still stronger in the SDK.

A finished command job can register its output by setting name on the output. That is how a preprocessing step publishes job_output_titanic_asset without a separate az ml data create.

Versioning practice that survives ETL: store data in time-partitioned folders (year=2026/week=33) and, for tables or multi-folder image sets, freeze the included partitions inside an MLTable path list. Version 20260815 keeps last week’s glob even after week 34 lands. If you instead point uri_folder at a directory your overnight job overwrites, the asset identity is immutable but the bytes are not — reproducibility is a lie.

Tags are mutable extra metadata (az ml data update --set tags.medallion=gold). Use them for lakehouse medallion stage, sensitivity:PII, or RAI_audit:approved. Lineage in Studio lists jobs that consumed the asset, which is the first place you look when a pipeline regresses.

Lifecycle: why you cannot delete

Data asset deletion is not supported. Microsoft’s reasons are operational, not pedantic: production jobs would start failing, experiments would not reproduce, lineage graphs would have holes, and audit trails would lie. If the name is ugly or the team moved on, archive (az ml data archive --name ...). Archived assets hide from default lists but still run. Restore with az ml data restore. If the path is wrong, create a new version with the same name. If the type is wrong, archive and start a new name.

Exam scenario

A claims team drops JPEG folders into Azure Data Lake Storage Gen2 every Sunday. A data scientist trained from a raw abfss:// URI pasted into a notebook, using a personal shared access signature in the script. Six weeks later the signature expired, the production pipeline failed, and nobody could prove which week of images trained model claims-seg-12. The MLOps fix is: create an identity-based datastore to that filesystem, register uri_folder (or mltable-of-paths) data assets named by week (claims-images:2026-08-15), and pass azureml:claims-images:2026-08-15 into the training component. Credentials leave the script; lineage in Studio shows the asset on the job.

Common trap

Do not treat “I created a data asset” as “I snapshot-copied the lake.” Unless the create used a local path, you stored a pointer. Pair immutable asset versions with immutable paths (or an MLTable path list). Also do not confuse this with the workspace model registry or with Azure Machine Learning registries in section 3.4 — those share assets across workspaces; a workspace data asset is local until you promote it.