1.6 Custom Model Deployment & Lifecycle Management

1.6 Custom Model Deployment & Lifecycle Management

Deployment options are model-specific

Avoid the obsolete rule that every custom Bedrock model must use Provisioned Throughput. Current Amazon Bedrock documentation describes both Provisioned Throughput and custom model deployments for on-demand inference for supported custom models. Imported models can also support on-demand inference. Availability depends on the model, customization type, Region, and current feature-support table.

Provisioned Throughput allocates model units for predictable sustained capacity and is billed while the allocation exists. On-demand custom model deployment charges for use and avoids a fixed provisioned allocation, but supported models and operational characteristics differ. A SageMaker AI endpoint remains appropriate when the team needs container-level control, specialized accelerators, deployment variants, or hosting features not available through the Bedrock path.

The deployment decision must therefore be discovered, not memorized:

1. Identify the exact custom or imported model artifact.
2. Check supported inference options and Regions.
3. Benchmark cold behavior, steady-state latency, quotas, and cost.
4. Register the version and its evaluation evidence.
5. Deploy behind a stable application interface.
6. Canary the new version and retain the previous target for rollback.

Lifecycle management and retirement

Model lifecycle control includes versioning, approvals, rollout, monitoring, rollback, and retirement. Store the base model ID, training data snapshot, hyperparameters or adapter configuration, evaluation report, intended uses, limitations, owner, and expiration review date. SageMaker Model Registry and SageMaker Model Cards can support version and governance records; Bedrock resources and CloudTrail events provide service-specific operational evidence.

Monitor production quality and input drift. A model can remain technically available while becoming unsuitable because user language, policies, or downstream schemas changed. Trigger reevaluation when the base model changes, the retrieval corpus changes materially, a safety incident occurs, or business acceptance metrics deteriorate.

Retirement is an engineering step, not merely deletion. Stop new traffic, preserve required audit evidence, revoke permissions, remove unused endpoints or throughput allocations, update dependent configuration, and verify that rollback or archive obligations are met. The strongest exam answer is the one that treats customization as a governed lifecycle and chooses prompting or RAG when weight changes are unnecessary.

Release record and rollback drill

For each release, create a record containing the immutable artifact identifier, base model, training-data snapshot, code commit, hyperparameters or adapter settings, supported inference path, Region, IAM role, KMS key, evaluation report, and approver. Link the application prompt, Guardrail, retrieval, and tool versions because those dependencies can change observed behavior without changing weights.

Deploy the candidate beside the active version. Shadow traffic can reveal latency and schema behavior without showing outputs to users. A canary then sends a small eligible percentage of live requests and compares quality signals, errors, token use, and business outcomes. Stop automatically on a safety or policy threshold; do not let an average quality gain offset a critical regression.

Practice rollback before the release. Confirm that the prior target remains invokable, configuration propagation is fast enough, state formats remain compatible, and queued jobs retain the intended version. Rollback should restore the complete release configuration, not only the model ARN.

For retirement, drain traffic and jobs, revoke unused roles and keys, remove paid deployments or throughput, update catalogs, and preserve evidence for the required period. A deleted endpoint with an active broad IAM policy is not a complete retirement.

Operational monitoring

Track deployment availability, cold restoration or startup behavior where applicable, latency percentiles, token use, error classes, schema validity, safety interventions, and task quality. Compare these by model version. Training metrics are historical evidence and cannot show production drift.

Set a review trigger for new source data, a model-provider update, a changed business process, an incident, or an expired approval. If reevaluation fails, freeze expansion and route to the approved version while the owner investigates. Version permissions and alarms with the release so a rollback restores observability and access, not only inference.

Ownership and cost cleanup

Assign an owner for the model artifact and every paid inference resource. Tag environments and releases, alarm on unused but billable capacity, and review access after staff or application changes. Lifecycle governance fails when an obsolete deployment remains reachable or a no-commitment allocation continues billing after the experiment.

Before closing the release, verify that documentation names the tested Region and inference option. Support can differ across Regions, and a general product announcement is not evidence that the target deployment path is available in the production account.