1.3 Foundation Model Landscape & Model Selection on Amazon Bedrock
Key Takeaways
- Choose models from representative quality, safety, latency, token, cost, modality, and residency evidence—not size or catalog position.
- Externalize model targets for canary rollout and rollback while preserving model-specific schemas and feature constraints.
- Cross-Region inference can improve capacity only when every destination Region satisfies policy and residency requirements.
1.3 Foundation Model Landscape & Model Selection on Amazon Bedrock
Foundation model selection is a requirements exercise, not a contest to choose the largest model. The current AIP-C01 blueprint expects a developer to compare capabilities, limitations, performance, cost, deployment options, and failure behavior. Amazon Bedrock exposes models through a managed API surface, but model availability and feature support vary by model, Region, endpoint, and release. A sound design records those dependencies instead of assuming that every model supports every Bedrock feature.
Translate the workload into measurable requirements
Begin with the business task and its acceptance criteria. A summarization service might prioritize factual coverage and predictable structure. An interactive assistant might prioritize time to first token, tool use, and conversational quality. A document-understanding workflow may require images or files, while a routing classifier may need only short text inputs. Capture at least these dimensions:
| Decision dimension | Questions to answer | Evidence to collect |
|---|---|---|
| Quality | What errors are unacceptable? Is reasoning, extraction, coding, or multilingual performance central? | A representative evaluation set with task-specific metrics and human review |
| Modality | Does the request contain text, images, audio, video, or documents? | Supported input and output modalities for the exact model version |
| Context | How large are system instructions, retrieved passages, tool schemas, and conversation history? | Token distributions at p50, p95, and maximum |
| Integration | Are Converse, streaming, tool use, Guardrails, batch inference, prompt caching, or Provisioned Throughput required? | The current Bedrock model-support tables for the target Region |
| Operations | What latency, throughput, residency, availability, and rollback objectives apply? | Load tests, quota checks, inference-profile support, and a failure-mode review |
| Economics | What are input, output, cache, batch, and fixed-capacity costs at expected traffic? | A cost model based on measured tokens and request volume |
Use the Bedrock model catalog and the control-plane APIs to discover candidates, then consult the model card and feature-support documentation for each candidate. A model identifier is a deployment dependency. Pin it in configuration, test it in every required Region, and treat a model upgrade like an application change.
Shortlist, benchmark, and validate
Do not select a model from marketing claims or a single public benchmark. Build a small evaluation corpus from real prompts, difficult edge cases, unsafe inputs, and known failure examples. Compare candidates using the same inference settings and the same scoring method. Measure answer quality, groundedness, refusal behavior, structured-output validity, latency percentiles, token use, and cost per successful task.
A useful metric is cost per accepted result rather than cost per token. A cheaper model that needs repeated retries, human correction, or a second model call can cost more at the workflow level. Likewise, a powerful model may be wasteful for classification or routing. Model cascading can send routine requests to a smaller candidate and escalate low-confidence or complex cases, but the routing rule itself must be evaluated.
For production selection, test the complete path: prompt rendering, retrieval, Guardrails, tools, networking, and client retry behavior. A playground result does not reveal API timeouts, quota behavior, or schema failures. Store the evaluation dataset, configuration, model ID, Region, and results so that a later model version can be compared reproducibly.
Worked selection exercise
Suppose an insurer needs to classify short claims, explain the classification to an adjuster, and process scanned damage images. Build two workload paths instead of forcing one model to do everything. A low-cost text model can classify normalized text if it passes the labeled confusion-matrix threshold. A multimodal model is eligible only for records that contain images. Both paths must produce the same validated application schema and satisfy the insurer's data-location policy.
Test false positives, rare classes, blurry images, non-English notes, malicious instructions embedded in documents, and requests near token limits. Record precision and recall by class, structured-output validity, p95 latency, average input and output tokens, and cost per accepted claim. If the smaller model meets the classification gate but produces weak explanations, a cascade can generate the explanation only for uncertain or reviewed cases. This is more defensible than selecting a single frontier model for every request.
The proof also checks feature compatibility. If the workflow requires tool use, streaming, prompt caching, or Guardrails, confirm each feature for the exact model and API. A model that wins a quality benchmark but lacks a mandatory feature is not an eligible candidate.
A team must select a foundation model for a regulated document assistant. Which approach produces the most defensible selection?