11.2 Select Foundation Models for Specific Use Cases
Key Takeaways
- Select a foundation model by task first (chat completion, embeddings, vision, reasoning, image, audio), then by context window, latency, cost, region, content-filter support, and multimodal needs. Do not default to the largest model in the catalog.
- The Foundry model card is the source for version, supported data types, deployment options, region, benchmarks, license, and existing deployments. Catalog filters include collection, region, deployment option, SKU, lifecycle, industry, supported features, and inference task.
- Foundry Models sold by Azure carry Microsoft support and Azure SLAs. Partner and community models are supported by their providers, with Azure providing integration guidance.
- Global Standard is usually cheapest and gets new models first but is not a residency control. Use Data Zone or Regional types when processing must stay in US, EU, APAC, or a single region.
- Embedding models have no built-in content filters and cannot be upgraded in place to another embedding family. Moving from text-embedding-ada-002 to text-embedding-3-large means regenerating vectors.
Select Foundation Models for Specific Use Cases
Quick Answer: Choose a catalog model by task (chat, embeddings, vision, reasoning, image, audio), then by context window, latency, cost, region and data residency, content-filter support, and multimodal needs. Read the model card. Do not deploy the largest model by default.
Domain 3 asks you to select appropriate models for specific use cases. Section 11.1 decided Serverless API versus managed compute. This section decides which model to put on that path. A wrong family wastes tokens, misses a modality, or violates residency even if the deployment type is correct.
The catalog is organized for search, not for prestige. Filters include collection (provider), region, deployment options (Serverless API or managed compute), deployment SKU, lifecycle (Preview, Generally available, Deprecated), industry, supported features (reasoning, tool calling), and inference task (chat completion, embeddings, audio generation). The model performance leaderboard and Compare models views expose benchmark metrics for selected models. None of those scores replace an evaluation on your prompts and data (Domain 4).
Read the model card before you deploy
Open the card and treat it as the contract:
- Quick facts. Task, context window, max output tokens, training-data cutoff, and whether the model is Preview or GA.
- Details. Description, version string (often a dated snapshot such as
2025-04-14forgpt-4.1), and supported data types (text, image, audio). - Deployments. Existing deployments of that model in the current resource, so you do not create a duplicate SKU by accident.
- Benchmarks. Public quality and performance numbers for selected models. Use them to shortlist, not to skip evaluation.
- License. Provider terms. Partner and community models are Non-Microsoft Products under Microsoft Product Terms.
Microsoft still expects you to read provider documentation and to run your own safety evaluation. Internal Responsible AI reviews of models sold by Azure do not guarantee that residual risk is gone.
Match the task, then the size
Start with the inference task. A larger chat model does not become an embedding model, and a reasoning model is the wrong default for a 50-millisecond classifier.
| Use case | Typical catalog direction | Why not the largest chat model |
|---|---|---|
| Multi-turn assistant, tools, structured output | Chat or Responses API models in the current GPT family (gpt-4.1, gpt-5.2, mini or nano variants) or a sold-by-Azure partner chat model | Full-size reasoning SKUs add latency and output-token cost you may not need |
| Retrieval-augmented generation (RAG) vectors | Embedding models such as text-embedding-3-large or text-embedding-3-small | Chat models do not emit a stable embedding space; you cannot upgrade ada-002 vectors in place |
| Rerank retrieved chunks | A rerank model (for example Cohere rerank) | A chat model used as a pairwise judge is slower and more expensive than a dedicated reranker |
| Image understanding in the same call as text | Multimodal chat (gpt-4.1, gpt-4o, later GPT-5.x vision-capable SKUs) | Text-only models reject image parts |
| Image or video generation | Image models (GPT image family, FLUX) or Sora video models | Chat completions do not draw pixels |
| Speech in or speech out | Audio and realtime models (whisper, gpt-4o-transcribe, gpt-realtime) | Text chat models do not consume waveforms |
| Hard math, code, or multi-step plans | Reasoning models (o-series, GPT-5.x reasoning SKUs) | Fast chat models under-think; reasoning models overspend tokens if you only need a greeting |
Do not pick the largest model by default. Mini and nano variants exist because most production turns are short. A gpt-4.1-nano or gpt-5-mini deployment often meets groundedness and latency targets at a fraction of the token price of the flagship SKU. Promote to a larger model only after evaluation shows a quality gap you cannot close with prompt or retrieval changes (Chapters 12 and 15).
Context window is not a brag number. The card's context window is the total budget. When a row lists separate input and output limits, those limits share one budget: more prompt leaves fewer tokens for generation. An API max_output_tokens value does not reserve room if the prompt already filled the window. Provisioned GPT-4.1 deployments also cap long context (under 128,000 tokens); longer prompts return HTTP 400 unless spillover to Standard is enabled (section 11.3).
Latency and cost travel together. Output tokens are more expensive to generate than input tokens on both Standard pricing and PTU utilization. Reasoning models spend hidden reasoning tokens inside the same budget. If the application is a high-QPS classifier, a small chat or a specialized model beats a frontier reasoning SKU.
Region, residency, filters, and support
After the task fits, constrain the map:
- Region availability. Not every model and version exists in every region. Specialized audio, image, and video models often appear only as Data Zone or Global types. Successive versions may land in different regions. Check the card and the region-availability tables, not tribal knowledge from last quarter.
- Data residency. Global Standard is the default for newest models, lowest price, and broadest quota, but inferencing may run in any Azure region. Data Zone Standard or Provisioned keeps processing in US, EU, or APAC. Regional Standard or Provisioned pins a single Azure geography. Section 11.1.
- Content filter support. Serverless language models get built-in Azure AI Content Safety. Embedding models and time-series models do not. If the workload is vectors only, plan a separate safety layer on the text that produced those vectors, not on the embedding API.
- Multimodal needs. Confirm input and output modalities on the card.
gpt-4oaccepts text and images. Some audio models are speech-in/speech-out. A text-only partner model will not suddenly grow vision because you attached a JPEG in the SDK. - Support and SLA. Models sold by Azure: Microsoft support, Azure SLAs, first-party billing. Partner and community: provider support, provider license, often Marketplace billing. Anthropic Claude in Foundry is a partner serverless path with Microsoft Support for operational issues and Anthropic terms for data processing. Hugging Face models typically ride managed compute with Hugging Face support.
- Lifecycle. Preview models can change weights, APIs, and retirement dates; they are not a production baseline. Deprecated models stay available to existing customers on that subscription only. Retired models return
410 Gone.
Sold-by-Azure versus partner is a procurement and operations choice as much as a quality choice. A regulated workload that needs a Microsoft SLA should not land on a community checkpoint just because the leaderboard looks exciting.
A practical selection sequence
Work this list in order so the exam scenario has a method, not a vibe:
- Name the inference task and required modalities.
- Filter the catalog to models that advertise that task and those modalities.
- Drop Preview models for production; prefer GA.
- Apply region and deployment-type filters that match residency.
- Compare context window, max output, tool calling, and content-filter notes on the cards.
- Shortlist a small SKU and a larger SKU in the same family.
- Estimate token cost and latency from the card and from a prototype on Global Standard (or Data Zone Standard if residency requires it).
- Run a Domain 4 evaluation on representative prompts before you freeze the model ID and version.
RAG designs often need two models: an embedding model for the index and a chat model for generation. Changing the embedding family later means re-embedding the corpus. Changing the chat model does not. Select them independently.
Exam scenario
A claims team needs (a) vectors for a policy-document index and (b) a grounded chat answer with citations. Documents are English PDFs with occasional scanned diagrams. Residency requires US processing. Select text-embedding-3-large or text-embedding-3-small for the index, a vision-capable GA chat model such as gpt-4.1 for generation so diagrams are not dropped, and Data Zone Standard in a US data-zone region. Do not use one giant reasoning model for both jobs. Do not pick Global Standard if the legal review forbids processing outside the United States.
Common trap
Defaulting to the flagship chat model for every Foundry workload. Embeddings, rerank, image generation, transcription, and reasoning are different tasks. Another trap: treating context window as extra headroom you can always spend on output. A third: choosing Global Standard for an EU-only processing requirement because the resource itself lives in West Europe. A fourth: planning an in-place upgrade from text-embedding-ada-002 to text-embedding-3-large — you must generate new embeddings.
A support bot answers short, grounded questions from a 400-token retrieval pack. Evaluation already meets quality on gpt-4.1-nano. Product wants to switch to the largest reasoning model in the catalog because it scores highest on public benchmarks. What should you recommend?
An EU insurer must keep prompt and completion processing inside the Azure EU Data Boundary. The copilot needs a GA multimodal chat model. Which combination matches the requirement?
A search team is moving a RAG index from text-embedding-ada-002 to text-embedding-3-large and asks whether they can set versionUpgradeOption on the existing embedding deployment so vectors upgrade in place. What is true?