15.2 Select and Fine-Tune Embedding Models

Select and Fine-Tune Embedding Models

Quick Answer: An embedding is a dense vector that represents meaning. In Microsoft Foundry RAG, the same embedding model, version, and dimensions must vectorize documents and queries. text-embedding-3-large is the strongest Azure OpenAI embedder (up to 3,072 dimensions, 8,192-token input). You cannot mix models in one vector field, and you must re-index after any model change. Fine-tune a custom embedder only when domain jargon is far from general web text.

Domain 5 continues with select and fine-tune embedding models for domain-specific use cases. Exam AI-300 expects you to treat embeddings as production infrastructure: a Foundry deployment, an Azure AI Search vectorizer, and an index field whose dimensions match the model output.

What an embedding is in this stack

You send text to an embeddings API (Azure OpenAI in Foundry, or another catalog embedder). The service returns an array of floats. Nearby vectors mean similar meaning, which is why HNSW can find “waiting period” near “elimination period” even when the words differ. Integrated vectorization in Azure AI Search calls a configured vectorizer at index time (skillset) and at query time (vectorQueries.kind = text), so application code does not have to call the embeddings API itself.

Hard constraints from current Microsoft docs:

• Current Azure OpenAI embedding models accept up to 8,192 tokens per input.
• A single embeddings request can send at most 2,048 input strings, and the sum of tokens across the batch must stay under 300,000 or the call returns HTTP 400.
• The model value is your deployment name, not necessarily the base model ID.
• Microsoft Entra ID with Cognitive Services OpenAI User is the recommended auth path; keys are supported but not the exam’s preferred production pattern.

Catalog selection: dimensions, language, domain

Foundry Models sold by Azure include three Azure OpenAI embedders. Microsoft states that text-embedding-3-large is the latest and most capable, and that you cannot upgrade between embedding models—moving from ada-002 to 3-large means generating new embeddings.

OpenAI’s published MTEB (English) and MIRACL (multilingual retrieval) averages improve from ada-002 → 3-small → 3-large (MIRACL 31.4 / 44.0 / 54.9; MTEB 61.0 / 62.3 / 64.6). If the corpus is mixed-language benefits content, 3-large’s MIRACL lift is the reason to pay for more dimensions—not because “larger is always required for English FAQs.”

Third-generation models accept a dimensions parameter so you can store fewer than 3,072 or 1,536 floats. Microsoft notes that even reduced 3-large vectors can still beat ada-002’s 1,536-d MTEB score. Smaller dimensions cut Azure AI Search storage and HNSW memory. The vector field’s dimensions in the index schema must equal the vectors you write. You cannot store 3,072-d and 1,536-d rows in the same field.

The Foundry catalog also lists partner embedding models (for example Cohere embed families) and multimodal embedders. Use a partner model when you need that vendor’s language coverage or image+text vectors. The rule does not change: one model per vector field, query-side identical to index-side.

Selection checklist:

• Language: English-only FAQs can start on 3-small; cross-lingual retrieval should compare 3-large (or a multilingual partner model) on a labeled set.
• Domain: Product SKUs and error codes still need keyword/hybrid even with a strong embedder (section 15.3).
• Cost and latency: 3-large at 3,072-d is the quality default; shrink dimensions before you jump to a custom model.
• Similarity metric: Azure OpenAI embeddings are trained for cosine. Set the Azure AI Search vector profile to cosine, not Euclidean, unless you have a non-OpenAI model that specifies otherwise.

Same model for query and index — no mixing

Azure AI Search can query up to 10 vector fields in one vectorQueries item, but Microsoft requires that each field contain embeddings from the same embedding model, and that the query vector is generated from that same model. Mixing ada-002 document vectors with a 3-large query is not “close enough”: the spaces are incompatible, ranks are noise, and you may even fail dimension checks.

Operational implications:

1. Record the embedding deployment name, model ID, version, and dimensions in the index metadata and in Git.
2. Point the index vectorizer at that deployment so query-time kind: text cannot silently drift.
3. When you change any of those four facts, re-embed the entire corpus and rebuild or reload the vector field. There is no rolling mixed-model index.
4. Do not “A/B two embedding models” inside one field. Stand up a second index (or a second vector field populated entirely by the candidate model), run the same query set, and compare nDCG / recall@k (section 15.4).

Exam scenario: A team upgrades a Foundry embedding deployment from text-embedding-ada-002 to text-embedding-3-large and only re-embeds new SharePoint files. Old vectors stay 1,536-d ada space; new vectors are 3,072-d. Queries encoded with 3-large cannot be compared to the old rows, and the field dimensions no longer match. The correct change control is: create or update the vector field for 3,072 (or a chosen reduced dimension), batch-embed all documents, reset the indexer, and point both the skillset and the query vectorizer at the new deployment.

When to fine-tune — and what that actually means

Foundry fine-tuning UIs and SFT/DPO/RFT jobs target chat and reasoning models, not the Azure OpenAI embedding trio. Do not tell the exam that you “fine-tune ada-002 like GPT-4o-mini” in the same wizard.

Fine-tune or customize an embedding model when a general embedder systematically fails on domain vocabulary that never appeared in web-scale training: internal SKU syntax, clinical codes, legal citations, or a private language mix. Typical paths:

• Stay with a catalog model first. Better chunking, hybrid search, and 3-large often close the gap without a custom embedder.
• Shrink or grow dimensions on 3-small/3-large as a cheaper experiment than training.
• Train a domain embedder (contrastive query–passage pairs, or a community sentence-transformer) and host it on managed compute or a supported catalog/partner endpoint. Then treat it as the new source of truth for that vector field.
• Partner embedders (Cohere and others in the catalog) when their domain or language fit is documented and you accept a second bill.

If you do train or swap, the blast radius is the whole index. Query-time embedding must switch on the same day as the re-index. Keep the old index until the new one beats it on a frozen eval set. Fine-tuning the generator (LoRA on a chat model, Chapter 16) will not fix neighbors that were never close in vector space.

Common trap: Deploying text-embedding-3-large for queries while the indexer skill still calls ada-002 “until we have time to rebuild.” Every query then searches the wrong space. A second trap is stuffing two models’ vectors into one field by padding 1,536-d arrays with zeros to look like 3,072-d — the geometry is meaningless.

Lock the embedding identity in Bicep alongside the Search index: model name, version, dimensions, cosine metric, and vectorizer resource ID. That is the GenAIOps control the exam is after.