3.1 Hybrid Search, Metadata Filtering & Semantic Reranking

3.1 Hybrid Search, Metadata Filtering & Semantic Reranking

Retrieval-Augmented Generation (RAG) systems in enterprise environments require far greater retrieval precision than basic nearest-neighbor semantic search can provide. When generative AI applications transition from prototype to production, developers encounter query patterns where pure dense embeddings struggle—most notably queries involving exact part numbers, error codes, proprietary acronyms, and strict multi-attribute access control filters. This section examines advanced retrieval architectures within Amazon Bedrock Knowledge Bases and Amazon OpenSearch Serverless, focusing on hybrid search, metadata pre-filtering, and two-stage semantic reranking.

---

The Limitations of Pure Dense Vector Retrieval

Dense vector retrieval maps text chunks into continuous, high-dimensional vector spaces (such as 1,024 or 1,536 dimensions) using embedding models like Amazon Titan Text Embeddings v2. Retrieval operates by calculating mathematical proximity—typically cosine similarity, dot product, or Euclidean distance (L2 norm)—between the query vector and indexed document vectors.

While dense vector search excels at capturing conceptual nuance, thematic intent, and cross-lingual meaning, it suffers from well-documented failure modes in production applications:

1. Exact Alphanumeric Identifiers and Serial Numbers: When a user searches for an exact part code such as SKU-8492-X or a vehicle identification number (VIN), dense embedding models project the subword tokens into an abstract semantic cluster. A document containing SKU-8492-Y may have an almost identical cosine similarity score (e.g., 0.985 vs. 0.987), leading the vector database to retrieve the incorrect product manual.
2. Software Stack Traces and Error Codes: Strings like NullPointerException, ECONNRESET_403, or ERR_DB_CLIENT_TIMEOUT carry critical operational meaning that subword tokenizers (like Byte-Pair Encoding) split into generic subwords (ECONN, RESET, 403), losing the exact technical signature.
3. Out-of-Vocabulary (OOV) Terms and Project Names: Highly specific internal codenames, novel chemical compounds, or brand-new regulatory acronyms not represented in the embedding model's pre-training corpus produce dispersed, uninformative vector representations.
4. Short Queries with High Precision Intent: Queries consisting of one or two keywords (e.g., "Form 1099-MISC box 7") often map to broad semantic spaces, retrieving generalized tax overviews rather than the specific filing instructions for that exact box.

---

Sparse Lexical Search and BM25 Mechanics

To overcome these limitations, production architectures leverage sparse lexical search, primarily powered by the BM25 (Best Matching 25) ranking function within OpenSearch Serverless. BM25 is an advanced probabilistic information retrieval algorithm that improves upon classic TF-IDF (Term Frequency-Inverse Document Frequency).

BM25 evaluates relevance based on three core components:

• Term Frequency (TF): How frequently the query terms appear in a given document chunk, governed by a saturation parameter ($k_1$) that prevents a term repeated 50 times from artificially dominating the score.
• Inverse Document Frequency (IDF): How rare the query term is across the entire corpus, giving immense weight to rare tokens (like SKU-8492-X) while suppressing ubiquitous words (like system or manual).
• Document Length Normalization ($b$): Penalizes excessively long chunks to prevent documents with higher word counts from scoring artificially high simply by chance.

While BM25 is virtually immune to the exact-token matching failures of dense vectors, it suffers from the vocabulary mismatch problem: if a user searches for "automobile warranty coverage" and a document chunk discusses "car guarantee protection," BM25 returns zero relevance because no exact terms overlap.

---

Hybrid Search Mechanics: Score Fusion and RRF

Hybrid search combines the semantic generalization of dense vector retrieval with the exact-token precision of BM25 sparse search. In Amazon Bedrock Knowledge Bases backed by Amazon OpenSearch Serverless, a single query triggers parallel execution of both search mechanisms.

                         ┌─────────────────────────────┐
                         │         User Query          │
                         └──────────────┬──────────────┘
                                        │
                ┌───────────────────────┴───────────────────────┐
                ▼                                               ▼
   ┌──────────────────────────┐                   ┌──────────────────────────┐
   │  Dense Vector Retrieval  │                   │   Sparse Lexical Search  │
   │  (k-NN Cosine Similarity)│                   │       (BM25 Score)       │
   └────────────┬─────────────┘                   └─────────────┬────────────┘
                │                                               │
                │ Ranked Candidates (1..N)                      │ Ranked Candidates (1..N)
                └───────────────────────┬───────────────────────┘
                                        ▼
                         ┌─────────────────────────────┐
                         │   Reciprocal Rank Fusion    │
                         │            (RRF)            │
                         └──────────────┬──────────────┘
                                        ▼
                         ┌─────────────────────────────┐
                         │   Fused Ranked Results      │
                         └─────────────────────────────┘

The Challenge of Direct Score Combination

A critical challenge in hybrid search is score calibration:

• Dense vector cosine similarity produces bounded float values between 0.0 and 1.0 (or -1.0 to 1.0).
• BM25 produces unbounded positive real numbers between 0.0 and +infinity (often ranging from 2.5 to 35.0+ depending on corpus size and term rarity).

Directly adding raw scores ($Score_{vector} + Score_{BM25}$) causes the unbounded BM25 score to overwhelmingly dictate the final ranking, rendering the vector similarity score virtually irrelevant.

Reciprocal Rank Fusion (RRF)

To resolve disparate scoring scales, Amazon Bedrock and OpenSearch utilize Reciprocal Rank Fusion (RRF). RRF is an algorithmic rank-merging technique that operates on the ordinal ranking positions of documents across multiple retrieval lists, completely disregarding raw numerical scores.

The RRF score of document $d$ across a set of retrieval methods $M$ is defined as:

$RRF\_Score(d) = \sum_{m \in M} \frac{1}{k + r_m(d)}$

Where:

• $M$ is the set of retrieval systems (Dense Vector search and Sparse BM25 search).
• $r_m(d)$ is the 1-based rank position of document $d$ in the results of system $m$.
• $k$ is a constant smoothing parameter (by default $k = 60$ in information retrieval standards).

If a document does not appear in the top results of a specific retrieval system, its rank term for that system is omitted (or treated as infinity).

Why $k = 60$ Matters

The constant $k$ dampens the penalty difference between adjacent ranks. For example, when $k = 60$:

• Rank 1 yields a score component of $\frac{1}{60 + 1} = 0.01639$
• Rank 2 yields a score component of $\frac{1}{60 + 2} = 0.01613$
• Rank 10 yields a score component of $\frac{1}{60 + 10} = 0.01428$

Because the drop-off is smooth, a document that ranks moderately well in both dense search (e.g., rank 3) and sparse search (e.g., rank 4) will achieve a higher cumulative RRF score ($0.01587 + 0.01562 = 0.03149$) than a document that ranks 1st in dense search ($0.01639$) but fails to appear in the sparse search list entirely.

---