10.1 On-Demand vs Provisioned Throughput & Capacity Planning
Key Takeaways
- Supported custom Bedrock models may use on-demand custom model deployments instead of mandatory Provisioned Throughput.
- Size model units against both input and output throughput; the larger requirement is binding.
- A quota is an upper bound, not a guarantee that every on-demand request succeeds during high demand.
10.1 On-Demand vs Provisioned Throughput & Capacity Planning
Amazon Bedrock offers multiple inference paths. On-demand inference charges for use and is governed by model- and Region-specific capacity and quotas. Provisioned Throughput purchases model units for a fixed hourly cost and a selected commitment term. Supported custom models may also be deployed for on-demand inference through a custom model deployment. The available choice depends on the exact model, customization type, Region, endpoint, and current support table.
Compare capacity options without obsolete absolutes
| Option | Cost behavior | Appropriate workload | Important caveat |
|---|---|---|---|
| On-demand | Usage-based model pricing | Variable, bursty, or early-stage traffic | A quota is an upper bound, not a promise that every request succeeds during high demand |
| Cross-Region inference | On-demand requests routed within the profile's supported Regions | Traffic that can use the listed Regions and benefits from a larger capacity pool | Validate residency, IAM, logging, and model support |
| Provisioned Throughput | Hourly model-unit charge while allocated | Sustained load needing reserved capacity and consistent performance | Model-unit capacity and purchase options are model-specific |
| Custom model deployment | Usage-based on-demand inference for supported custom models | Customized models with variable traffic | Not every custom model or Region supports this path |
| Batch inference | Asynchronous job pricing and quotas | Large offline workloads without interactive latency requirements | Input format, minimum job size, and model support apply |
Provisioned Throughput documentation describes no-commitment, one-month, and six-month choices for supported purchases. Billing continues until a no-commitment allocation is deleted or until the relevant commitment lifecycle permits termination. Never insert a made-up discount percentage into an architecture decision; use the current pricing page and the quote for the selected model.
A model unit represents input- and output-token processing capacity over time for a particular model. AWS advises customers to obtain current model-unit details from the relevant documentation or account team. One model's unit numbers cannot be reused for another model. Provisioned capacity reduces exposure to shared on-demand capacity variation, but applications still need timeouts, retries, monitoring, and admission control.
Eligibility before economics
Create an eligibility matrix before comparing prices. A path is eligible only if the exact model supports the required Region, API, modality, Guardrail or tool behavior, streaming mode, and deployment option. An inexpensive batch price is irrelevant to a live chat request. A cross-Region profile is ineligible when policy requires one Region. A custom on-demand deployment is ineligible when the selected custom model does not support it.
Then estimate economics with measured distributions, not a single average. Include peak and quiet hours, input and output tokens, retries, cache writes and reads, failed calls, evaluation traffic, and idle provisioned time. Model a low, expected, and high case. Sensitivity analysis often shows whether token growth or utilization is the dominant risk.
Provisioned Throughput has a fixed hourly component and model-specific units. It can fit a sustained known workload, but buying for a one-hour peak may create large idle cost. On-demand fits variable traffic but requires queue and retry behavior. Cross-Region inference can improve available capacity for eligible workloads. Batch moves offline work out of the interactive path. The decision is a portfolio, not necessarily one mode for the whole application.
Record the evidence and review it after launch. A workload that begins as a low-volume prototype can later justify reserved capacity, while a seasonal product may move the other direction.
Endpoint and quota inventory
Before testing, record the endpoint surface, model or inference-profile ID, Region, service tier, quota names, and requested limits. Token accounting can differ across surfaces, and some models have output-token burndown factors. The application should report actual input, output, cache, and retry usage from responses where available rather than estimating all demand from character counts.
Separate business traffic, evaluation, load testing, and background processing with tags, accounts, queues, or schedules appropriate to the organization. A benchmark that consumes the same on-demand quota as production can create the outage it is intended to prevent. Conversely, testing an isolated path may not reveal the real production quota.
Review supported-model tables immediately before purchase or launch. Bedrock evolves quickly; a valid option for one model version can be unavailable for another.
Finally, verify regional availability and account quotas in the environment that will run the workload. A feature visible in documentation or another account is not proof that the intended account can purchase or invoke it immediately.
A supported custom Bedrock model has sporadic traffic. Which inference choice should be evaluated before assuming fixed model units are mandatory?