6.1 Unity Catalog Governance & Security

Governing GenAI Assets with Unity Catalog

Unity Catalog (UC) is the single governance layer for every asset a generative-AI application touches: tables, files, registered models, UC functions (used as agent tools), and Vector Search indexes. Because one system controls all of them, you grant, audit, and trace access to an entire RAG or agent pipeline as a single governed asset graph rather than stitching together per-service permissions. On the exam, when a scenario says tables, functions, and models must all follow centrally managed permissions, the answer is Unity Catalog. Its two headline benefits are centralized access control and lineage for AI assets.

The three-level namespace

UC organizes objects as catalog.schema.object. The catalog is the top logical container (often a data domain or environment); the schema (database) groups related objects; and individual objects - tables, models, functions, and vector indexes - live at the schema level. A registered model is therefore addressed as catalog.schema.model_name, and a UC function an agent calls is catalog.schema.function_name. Knowing that models and functions sit under a schema (not directly under a catalog) is a common recall question.

Privileges and least privilege

Access uses inherited privileges. To read a table an agent needs USE CATALOG on the catalog, USE SCHEMA on the schema, and SELECT on the table; to invoke a UC function it also needs EXECUTE. Model Serving endpoints use their own permission tiers, and choosing the least-privileged tier that still works is a frequent scenario:

An app that only sends requests should get Can Query, never Can Manage; one that only reads metadata gets Can View. A common mistake is over-granting Can Manage "to be safe" - grant the minimum that satisfies the task.

Lineage, audit, and workspace isolation

UC automatically records lineage (which notebook, model, or query touched which table) and audit logs of access. For a high-stakes release you should retain the approved model and prompt versions, the dataset snapshot, and the scoring method used for approval - UC lineage plus MLflow gives you that trail. To keep a production catalog completely unreachable from a development workspace regardless of object-level grants, use workspace-catalog binding, which restricts a catalog to named workspaces. Also note that UC access requires a supported cluster access mode (standard or dedicated); a misconfigured access mode is the usual reason a cluster cannot read UC tables in a GenAI pipeline.

Secrets and identities (never hardcode credentials)

External model-provider keys and SaaS tokens must live in Databricks secret management, never in notebooks, prompts, or source control. Reference a secret inside a serving-endpoint environment variable with the syntax {{secrets/scope/key}}, and the endpoint creator must hold READ on that secret. Create a separate secret scope per app, because secret permissions apply at the scope level, so separate scopes limit the blast radius of a leak. For automation such as CI/CD that rotates credentials, a service principal using OAuth machine-to-machine (M2M) is the correct identity. Inside a Databricks App, the platform injects DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET so the app authenticates as its own service principal, and the SDK Config() object discovers those injected credentials automatically through unified authentication.

Securing RAG source data and entitlements

A subtle but heavily tested idea: filtering the answer after generation is not enough. If a multi-tenant RAG app retrieves governed documents a user is not entitled to, the generated answer may already reflect that content - the model saw it. The fix is to enforce entitlements at retrieval. For a Databricks App that must preserve each signed-in user's row-level permissions, use user authorization (on-behalf-of-user) so Databricks applies the current user's UC permissions to the query instead of the app's broad service-principal access. Governance belongs at the data layer, before generation.

Masking, PII, and guardrails

Protect sensitive data with layered controls. UC supports column masks and row filters so restricted users see redacted values. At the edge, AI Gateway PII detection with masking redacts SSNs, credit-card numbers, and similar tokens from both requests and responses - choose masking (not mere flagging) when the requirement is to redact. Content-safety guardrails filter unsafe input and output for a consumer chatbot, and the AI Gateway valid topics guardrail keeps an internal assistant answering only approved subjects. Because monitoring traces can contain PII, restrict log access, mask sensitive fields, and enforce retention controls rather than logging raw payloads.

Legal, licensing, and compliance

Governance also covers legal risk. Track model licensing so you ship only models whose terms permit your use, and vet the licensing of source documents you index. When a workload needs a broader compliance profile - for example beyond HIPAA - prefer provisioned throughput Foundation Model endpoints, which carry stronger compliance guarantees than pay-per-token modes. For auditability on a regulated release, keep the approved model and prompt versions, the dataset snapshot, and the scoring method together so an auditor can reproduce exactly what was shipped. And even when automated safety checks and judge scores look clean, keep a human-reviewed sample: humans catch nuanced failures and detect judge miscalibration that fully automated pipelines quietly miss. Governance stack mnemonic: UC (govern) -> Masking (PII) -> Guardrails (filter) -> AI Gateway (track).