15.3 Hybrid Search: Semantic and Keyword Retrieval

Hybrid Search: Semantic and Keyword Retrieval

Quick Answer: Hybrid search in Azure AI Search is one query that sets both search (keyword BM25) and vectorQueries. Both run in parallel. Reciprocal Rank Fusion (RRF) merges the ranked lists. Optional semantic ranker (queryType: semantic) is a premium L2 reranker over the top 50 fused hits. Microsoft’s published benchmarks often rank hybrid + semantic first for RAG grounding.

Domain 5 asks you to implement and optimize hybrid search (semantic + keyword). On the exam, “semantic” in this bullet means two related ideas: vector / embedding retrieval (meaning search) and the optional semantic ranker feature. Do not confuse them.

Why hybrid exists

Vector search finds conceptually similar passages (“termination for convenience” near “the buyer may end the contract without cause”). Keyword / full-text search finds exact tokens: SKU P-4418-B, error 0x80070005, a person’s name, an ICD-10 code, a date. Hybrid is the union: one HTTP POST to indexes/{index}/docs/search with both search and vectorQueries.

The index must contain searchable text fields and vector fields. You can still use filters, facets, and security trimming. Do not set orderby if you want relevance ranking; an explicit sort overrides BM25/RRF order.

Reciprocal Rank Fusion

BM25 scores have no upper bound. Cosine vector scores for Azure OpenAI embeddings typically sit in about 0.333–1.00. Those scales are not comparable, so Azure AI Search does not add the raw scores. It uses RRF:

1. Run each subquery (one BM25 list, one list per vector field/query).
2. For each document in each list, assign 1 / (rank + k) where k is a constant (Microsoft cites 60). This k is not the vector nearest-neighbor k.
3. Sum a document’s reciprocal ranks across lists.
4. Sort by the sum. The fused value is @search.score.

A document that is #1 in both BM25 and vector lists beats a document that is #1 in only one list. Parallel executions add up: one text query plus one vector field is two executions; one text query plus two vector fields is three, and RRF fuses all of them. Vector weight (default 1.0) multiplies a vector list’s contribution before fusion; maxTextRecallSize (preview hybridSearch object, default 1,000, max 10,000) controls how many BM25 candidates enter RRF.

RRF scores look tiny. A cosine hit of 0.84 might become an RRF score near 0.03. That is normal, not a relevance failure.

Semantic ranker as L2

Semantic ranker is a premium Azure AI Search feature billed by usage (a monthly free allowance exists, then pay-as-you-go). It is not the same as vector search. After BM25 or RRF produces an L1 list, the service sends up to 50 documents to Microsoft language models (Bing-derived) and writes @search.rerankerScore on a 0.00–4.00 scale (4 = highly relevant and complete; 0 = irrelevant).

When you enable it on a hybrid query:

• Set queryType to semantic and name a semanticConfiguration (title, keywords, content fields in priority order; title and keywords get 128 tokens each, content gets the rest of a 2,048-token summary).
• Set vector k to 50 so the ranker is fed a full candidate set. Fewer than 50 inputs “deprives the semantic ranking models of necessary inputs.”
• Optional captions and answers are extractive (verbatim from your index), not generated by GPT.
• Empty search=* with semantic queryType is not billed; a non-empty search string is.

Semantic ranker cannot scan the whole corpus. It only reranks the L1 top 50. If hybrid never retrieved the right chunk, L2 cannot invent it. Over-aggressive preFilters can starve the 50-slot window; postFilter after k = 50 can also leave fewer than 50 documents for L2.

Microsoft’s hybrid-query documentation states that in benchmark tests, hybrid queries with semantic ranking consistently produced the most relevant results. That sentence is exam gold—but you still measure it on your qrels (section 15.4). Keyword-only remains better for pure identifier lookup; semantic ranker helps most on prose-rich knowledge bases.

searchMode any versus all

The text side of hybrid still honors searchMode:

Microsoft’s Lucene guidance: when the query includes operators, prefer searchMode=all so every criterion is actually required. In a hybrid RAG bot, default any plus vectors often recovers paraphrases; switch to all for “must include policy number and year” tools.

Balanced hybrid starting points from Microsoft’s how-to: vector k in the 30–50 range, top 10–20 for the response (then you may send fewer chunks to the LLM), add semantic ranker only when it improves measured nDCG. Overloaded queries (huge k, huge maxTextRecallSize, semantic L2, aggressive efSearch) cause latency and HTTP 429. Tune efSearch / maxConnections before adding Search replicas.

Exam scenario: A parts-ordering copilot misses SKU BRG-2200-ZZ on vector-only search because the embedding treated it like a random token, but BM25 finds it in the sku field. You issue one hybrid request: search = the user text (and maybe a quoted SKU), vectorQueries against contentVector with k = 50, queryType = semantic. RRF surfaces the BM25 SKU hit alongside conceptual “bearing replacement interval” passages; semantic ranker promotes the passage that actually answers the question.

Common trap: Calling the optional semantic ranker “free vector search” or assuming queryType: semantic embeds the query. Semantic ranker reranks text. Another trap: comparing @search.score across a vector-only query and a hybrid query (different algorithms and magnitudes), or using searchMode=any while believing every keyword was required.

Production checklist:

• One request, both search and vectorQueries.
• Same embedding model as the index (section 15.2).
• select human-readable fields, not the float arrays, for the LLM.
• k = 50 if semantic ranker is on; top controls what you actually ship to the prompt.
• Filter with preFilter for row-level security; test whether filters starve L2.
• Keep Git-tracked query templates (searchMode, k, semantic config name, vectorizer) next to the index Bicep.