7.2 Vector Search, Semantic Retrieval & RAG with Metadata Filters

7.2 Vector Search, Semantic Retrieval & RAG with Metadata Filters

Quick Answer: Implement RAG on Azure Database for PostgreSQL by storing chunk text + embeddings + metadata, retrieving top-k neighbors with vector similarity search, applying WHERE metadata filters for tenancy and business constraints, then sending only those chunks to the generative model. This retrieve-then-generate flow is the core pattern for grounded AI answers.

This section maps directly to the skill: run vector similarity search, including storing embeddings, semantic retrieval, and implementing retrieval-augmented generation (RAG) patterns by using metadata filters.

Storing Embeddings in PostgreSQL

A typical RAG table design keeps retrieval artifacts together:

Enable the vector extension on Azure Database for PostgreSQL Flexible Server, choose a dimension that matches your embedding model, and insert embeddings in the same transaction path that persists chunk text. Keeping metadata beside the vector lets one query enforce security and relevance without a second store round-trip.

Embedding Lifecycle Notes

• Model upgrades change dimensions or geometry—plan a parallel column or table and cut over after backfill.
• Chunking strategy (size, overlap) affects both storage volume and retrieval quality.
• Normalization may be required depending on distance metric (for example, cosine versus inner product workflows).
• Idempotent re-embed jobs should update vectors without duplicating chunks.

Vector Similarity Search Mechanics

Semantic retrieval finds chunks whose embeddings are nearest to a query embedding produced from the user question with the same embedding model.

Common pgvector distance styles include:

Queries generally ORDER BY distance to the query vector and LIMIT to top-k (for example, 3–20 chunks). ANN indexes make this practical at scale:

• IVFFlat — partitions vectors into lists; good build characteristics; tune lists/probes for recall vs latency.
• HNSW — graph-based ANN; strong query latency/recall trade-offs for many production RAG systems; higher build cost and memory.

Exact scans are fine for tiny labs; production corpora need ANN indexes plus monitoring of recall quality.

Semantic Retrieval Versus Keyword Search

Semantic retrieval matches meaning: “reset MFA” can retrieve a chunk titled “multi-factor authentication recovery” even without shared keywords. Keyword or full-text search matches tokens. Many Azure solutions combine both (hybrid), but AI-200’s PostgreSQL vector emphasis is the embedding path: embed query → similarity search → return chunks.

Semantic retrieval quality depends on chunk boundaries, embedding model choice, top-k, and—critically—filters that remove irrelevant or unauthorized rows before generation.

RAG Pattern: Retrieve-Then-Generate

Retrieval-augmented generation (RAG) reduces hallucination by grounding the model on private or domain data:

1. Accept the user question and optional application filters (tenant, locale, product).
2. Embed the question with the same model used at index time.
3. Retrieve top-k chunks via vector similarity search, constrained by metadata predicates.
4. Assemble a prompt that includes instructions plus retrieved evidence (and citations).
5. Generate the answer with Azure OpenAI or another chat/completions model.
6. Return the answer with source references from source_uri / document metadata.

The database’s job is step 3 (and storing the corpus for step 3). The generative model never needs write access to the vector table for ordinary Q&A.

User question
    → embedding model
    → SQL: WHERE metadata filters + ORDER BY embedding <=> :query LIMIT k
    → prompt packer
    → generative model
    → grounded answer + citations

Metadata Filters: The Production Differentiator

Unfiltered ANN search over a multi-tenant corpus is a security and quality bug. Metadata filters encode business rules in SQL WHERE clauses so retrieval cannot cross tenants or pull expired policy docs.

Examples of filter dimensions:

• Tenancy / ACL — tenant_id = @currentTenant
• Product scope — product_sku = ANY(@skus)
• Language — language = 'en'
• Freshness — effective_on <= now() AND (expires_on IS NULL OR expires_on > now())
• Sensitivity — classification <= @userClearance
• Collection — kb_id = @selectedKnowledgeBase

Pattern: Filter + Vector Rank

A canonical shape looks like:

SELECT id, content, source_uri,
       embedding <=> @query_embedding AS distance
FROM kb_chunks
WHERE tenant_id = @tenant
  AND language = @lang
  AND product_sku = @sku
ORDER BY embedding <=> @query_embedding
LIMIT @k;

The WHERE clause narrows candidates; the ORDER BY distance clause ranks remaining rows semantically; LIMIT caps context sent to the LLM. This is the RAG pattern “by using metadata filters” that the exam describes.

Why Filters Belong in the Database Layer

Filtering only in application memory after fetching global top-k is fragile: you may retrieve k neighbors from the wrong tenant and end with zero usable rows, or worse, leak content if the app filter is buggy. Pushing predicates into PostgreSQL enforces isolation next to the vectors and lets the planner use metadata indexes (B-tree/GIN) together with the vector index strategy your extension supports.

Designing Filters for Recall and Safety

Tune k as a product decision: too small misses evidence; too large wastes tokens and can dilute the prompt. Filters often improve answer quality more than simply increasing k.

End-to-End Example Scenario

An Azure AI chatbot for Contoso support embeds each knowledge article chunk into PostgreSQL. When a Contoso Canada agent asks about refund windows for SKU Contoso-Bike-2026 in French:

1. The app embeds the question.
2. SQL applies tenant_id = contoso, region = ca, language = fr, product_sku = Contoso-Bike-2026.
3. Vector search ranks matching chunks.
4. Top chunks ground Azure OpenAI.
5. The reply cites article URLs from source_uri.

Without metadata filters, the same similarity search might surface US English policy chunks or another tenant’s data—semantically “close” but operationally wrong.

Operational Practices for RAG Corpora

• Re-embed when the embedding model changes; keep dimension metadata documented.
• Monitor empty-result rates after filters (over-filtering) versus cross-collection contamination (under-filtering).
• Use transactions for chunk+embedding writes; avoid orphan vectors without content.
• Version prompts separately from the vector schema.
• Load-test filtered ANN queries, not only unfiltered lab queries.

Exam Focus

When a scenario says answers are ungrounded, look for missing retrieval. When answers leak across customers, look for missing WHERE tenancy filters. When results are irrelevant despite embeddings, inspect chunking, model mismatch, top-k, or missing business metadata predicates. The winning architecture is consistently store embeddings → filtered vector similarity search → generate.