3.2 Metadata Filtering, Reranking & Query Transformation

3.2 Metadata Filtering, Reranking & Query Transformation

Metadata Filtering with Amazon S3 Companion Files

Enterprise RAG pipelines cannot expose the entire vector index to every query. Workloads require deterministic boundary enforcement—such as filtering documents by department, geographic region, publication year, document classification, or user authorization level.

In Amazon Bedrock Knowledge Bases, metadata attributes are ingested using companion JSON files stored in Amazon S3 alongside the raw source files.

Companion File Naming and Schema Rules

When an Amazon S3 data source needs custom filter attributes, a source document can use a companion metadata file that follows the documented naming and schema rules:

• Exact File Suffix: The metadata file must match the complete file name of the source document, immediately appended with .metadata.json.
• Location: The metadata file must be placed in the identical S3 bucket prefix (folder) as the source document.

For example, if the source file is located at: s3://corp-legal-knowledge-base/contracts/2026/master_service_agreement_acme.pdf

The companion metadata file must be placed at: s3://corp-legal-knowledge-base/contracts/2026/master_service_agreement_acme.pdf.metadata.json

{
  "metadataAttributes": {
    "department": "legal",
    "contract_type": "master_service_agreement",
    "client_id": "CLI-9824",
    "effective_year": 2026,
    "access_tier": 3,
    "is_active": true
  }
}

Supported Metadata Data Types

Bedrock Knowledge Bases support the following attribute primitive types:

• String: Alphanumeric text (e.g., "legal", "draft")
• Number: Integers or floating-point numbers (e.g., 2026, 4.5)
• Boolean: true or false
• String List: Arrays of strings (e.g., ["nda", "confidential", "emea"])

Query-Time Metadata Filtering API

When calling the Retrieve or RetrieveAndGenerate API in the Amazon Bedrock Agent Runtime, developers define boolean filters inside vectorSearchConfiguration.filter:

{
  "knowledgeBaseId": "KB10EXAMPLE",
  "retrievalQuery": {
    "text": "What are the termination liability clauses for enterprise clients?"
  },
  "retrievalConfiguration": {
    "vectorSearchConfiguration": {
      "numberOfResults": 5,
      "overrideSearchType": "HYBRID",
      "filter": {
        "andAll": [
          {
            "equals": {
              "key": "department",
              "value": "legal"
            }
          },
          {
            "greaterThanOrEquals": {
              "key": "effective_year",
              "value": 2024
            }
          },
          {
            "in": {
              "key": "access_tier",
              "value": [1, 2, 3]
            }
          }
        ]
      }
    }
  }
}

Pre-Filtering vs. Post-Filtering

Bedrock Knowledge Bases execute pre-filtering within OpenSearch Serverless. Pre-filtering applies metadata constraints to the vector index graph before performing nearest-neighbor traversal or BM25 scoring. Pre-filtering restricts candidates before ranking and usually avoids wasting the result set on ineligible records. It does not replace application authorization, source permissions, or negative tests for missing and malformed metadata.

---

Semantic Reranking with Cross-Encoders

Even after hybrid search and metadata pre-filtering, the top-k candidate chunks may contain passages that match keywords or broad themes but fail to answer the user's specific informational need. This is a consequence of the bi-encoder architecture used by embedding models.

Bi-Encoders vs. Cross-Encoders

Implementing Two-Stage Retrieval

A production two-stage retrieval pipeline operates as follows:

1. First-Stage Candidate Generation: Hybrid search (dense k-NN + sparse BM25) combined with metadata pre-filtering queries OpenSearch Serverless to retrieve a broad candidate pool—typically $N = 50$ to $100$ candidate chunks.
2. Second-Stage Reranking: A cross-encoder model—such as the Cohere Rerank v3.5 model or Amazon Bedrock's native reranking capability—processes all 50 candidate pairs, calculating a unified semantic relevance score for each.
3. Context Truncation: Only the top $K$ (e.g., $K = 3$ to $5$) highest-scoring reranked chunks are extracted and assembled into the final prompt sent to the LLM (such as Claude 3.5 Sonnet or Amazon Titan Text Premier).

Operational Advantages of Reranking

• Drastic Reduction in Hallucination: Prevents the generator model from being distracted by marginally relevant chunks (the "needle-in-the-haystack" distraction effect).
• Cost and Latency Optimization: Feeding 3 highly focused chunks instead of 20 sprawling chunks saves thousands of input tokens per inference call, accelerating Time-To-First-Token (TTFT).

---

Comparison of Retrieval Paradigms

---

Exam Scenarios & Common Traps

Real-World Exam Scenario

An enterprise engineering organization deploys an internal troubleshooting assistant using Amazon Bedrock Knowledge Bases. Field engineers search for specific equipment issues using inputs like: "Overheating alert on turbine unit SN-7740-B error code E-9042".

Under pure vector search, the assistant frequently returns troubleshooting guides for SN-7740-A or generic overheating turbine summaries, causing engineers to follow incorrect diagnostic steps. Furthermore, field engineers must only view documentation applicable to active models released after 2022.

Architecture Solution:

1. Ingest equipment documentation into S3 accompanied by .metadata.json files storing {"metadataAttributes": {"unit_series": "7740", "release_year": 2023, "status": "active"}}.
2. Configure the Knowledge Base retrieval mode to HYBRID in OpenSearch Serverless, enabling BM25 to lock onto exact tokens (SN-7740-B and E-9042) while dense vectors match "Overheating alert".
3. Pass a pre-filter specifying status == "active" and release_year >= 2022.
4. Apply a second-stage Cohere Rerank model to select the top 3 highest-scoring chunks from the top 50 retrieved candidates prior to prompting Amazon Bedrock.

Common Architectural Traps

• Trap 1: Incorrect Metadata Suffix: Creating companion files named document.json or document.pdf.meta will result in Bedrock ignoring the metadata silently during ingestion. The filename must be <original_filename>.metadata.json.
• Trap 2: Post-Retrieval Application Filtering: Querying 5 items from the vector index and filtering by department in Python code causes empty responses if all 5 returned items belong to other departments. Use retrieval filtering when the selected store and operator support the required eligibility rule, and enforce security in trusted application and storage controls as well.
• Trap 3: Adding Raw BM25 and Vector Scores: Attempting to implement custom hybrid scoring by adding unnormalized BM25 scores directly to cosine similarity scores skews all results toward lexical matches, neutralizing the semantic index.

Query transformation and standardized retrieval access

Complex questions often need query expansion or decomposition before vector search. Expansion adds synonyms and domain terms; decomposition turns a multi-part request into focused subqueries whose evidence can be merged. Guard these transformations against scope drift: retain the original intent, cap fan-out, deduplicate evidence, and log which transformed query produced each result.

A standardized API or Model Context Protocol (MCP) client can expose retrieval as a tool to agents and other foundation-model applications. The contract should define typed inputs, bounded result counts, tenant context, timeout and error behavior, and source metadata. Validate every tool argument server-side and enforce authorization in the retrieval service. MCP standardizes discovery and invocation; it does not grant trust or replace IAM, OAuth, network, or data controls.