1.2 Grounding Data Evaluation: Accuracy, Relevance, Timeliness, Cleanliness & Availability

Grounding Data Evaluation: Accuracy, Relevance, Timeliness, Cleanliness & Availability

Quick Answer: Grounding data is the bedrock of enterprise agent reliability. The AB-100 exam evaluates your ability to audit and score grounding data across five dimensions: Accuracy (golden record authority), Relevance (semantic alignment and context budget), Timeliness (freshness SLAs and cache strategies), Cleanliness (structural normalization and noise reduction), and Availability (rate limits and uptime SLAs). Quantitative evaluation relies on the RAG Triad framework.

An agent is only as intelligent as the data grounding its reasoning. In enterprise architectures, the primary root cause of model hallucinations, non-deterministic errors, and regulatory compliance breaches is not model capability, but ungoverned, stale, noisy, or unauthenticated grounding data.

As a Microsoft Solutions Architect, you must systematically evaluate, score, and remediate enterprise knowledge sources before connecting them to Microsoft Copilot Studio or Azure AI Foundry agents.

---

1. The 5 Core Dimensions of Grounding Data Readiness

                    +---------------------------------------+
                    |       GROUNDING DATA READINESS        |
                    +---------------------------------------+
                                        |
       +----------------+---------------+---------------+----------------+
       |                |               |               |                |
       v                v               v               v                v
  [ ACCURACY ]    [ RELEVANCE ]   [ TIMELINESS ]  [ CLEANLINESS ]  [ AVAILABILITY ]
  - Golden records - Vector match  - Freshness SLA - Markdown norm  - 99.99% Uptime
  - Conflict rules - Token budget  - Real-time vs  - Noise removal  - TPM / RPM limits
  - Source of truth- Metadata filter  cached index - Parent-child   - VNet isolation

1.1 Accuracy: Authoritative Truth & Conflict Resolution

Accuracy assesses whether the grounding data represents verifiable, authorized factual truth.

• Single Source of Truth (SSOT): Enterprise data estates frequently hold duplicate documents with conflicting data (e.g., an outdated 2024 Travel Policy PDF on a legacy SharePoint site versus the official 2026 Expense Policy in Dataverse). Agents will retrieve both unless authoritative sources are explicitly designated.
• Precedence Hierarchies: When grounding spans multiple stores, architects must establish deterministic precedence rules: $\text{Core ERP (Dynamics 365 F\&O)} \succ \text{Operational CRM (Dataverse)} \succ \text{Published Intranet (SharePoint)} \succ \text{Informal Teams Chats}$
• Source Verification & Provenance: Every retrieved grounding chunk must carry cryptographically verifiable or system-generated metadata (source_uri, document_version, hash_id, last_modified_by). Agents must cite these source attributes in final responses to enable human verification.

1.2 Relevance: Semantic Precision & Token Budget Optimization

Relevance measures how closely the retrieved grounding context addresses the user's specific business query without polluting the context window with distracting information.

• The "Lost in the Middle" Phenomenon: Large Language Models exhibit degraded reasoning accuracy when key factual data is buried inside thousands of tokens of irrelevant background text. Pumping 50,000 tokens of raw policy documentation into context degrades reasoning performance.
• Pre-Filtering vs. Post-Filtering:
  • Pre-Filtering (OData Filters in Azure AI Search): Filters the document index based on metadata attributes (e.g., tenant_id eq 'US', department eq 'Finance', security_clearance ge 3) before executing vector similarity calculations. This reduces the search space and guarantees tenant isolation.
  • Post-Filtering (Semantic Reranking): Evaluates the top $K$ candidate chunks returned by dense vector search using a cross-encoder model to score semantic relevance against the exact query intent.
• Context Pruning: Extracting only the relevant sub-paragraphs or tables from a 100-page document before injecting them into the agent's prompt context.

1.3 Timeliness: Freshness Latency, Caching & Real-Time Connectors

Timeliness evaluates the latency between an enterprise data update in the system of record and its availability to the agent's reasoning engine.

• Time-to-Live (TTL) & Cache Invalidation: When caching grounding responses or embeddings, architects must configure explicit TTL policies. When a master document in SharePoint or Dataverse is updated, a webhook or Azure Event Grid event must trigger immediate cache eviction.

1.4 Cleanliness: Normalization, Noise Removal & Structural Chunking

Cleanliness evaluates how free the source data is from navigational artifacts, syntactic garbage, and formatting anomalies that impair vector embedding quality.

• Boilerplate & Noise Stripping: Raw enterprise files contain headers, footers, copyright notices, cookie disclaimers, and HTML CSS/script tags. If indexed, these repeated boilerplates artificially inflate vector similarity scores across unrelated queries.
• Markdown Normalization: Transforming raw HTML, Word documents, and PDFs into clean, structured Markdown. Markdown preserves structural semantics—such as heading hierarchies (#, ##, ###), bulleted lists, and tables (| Col1 | Col2 |)—which language models understand natively.
• Structural & Semantic Chunking:
  • Fixed-size chunking (e.g., 500 characters with 50-character overlap): Fractures tables, splits sentences across chunks, and severs contextual links between headers and body paragraphs.
  • Layout-Aware / Hierarchical Chunking (Azure AI Document Intelligence): Segments documents along natural semantic boundaries (sections, headings, tables). Utilizes Parent-Child Chunking, where small child chunks (e.g., 200 tokens) are used for vector similarity matching, but the surrounding parent section (e.g., 1,000 tokens) is passed to the LLM for grounded context.

1.5 Availability: Uptime SLAs, Rate Limiting & Enterprise Isolation

Availability measures whether the grounding service can reliably meet runtime concurrency demands and latency SLAs without dropping requests.

• Throughput & Throttling Limits: Azure OpenAI models enforce strict limits on Tokens Per Minute (TPM) and Requests Per Minute (RPM). Azure AI Search enforces Queries Per Second (QPS) limits per search unit. If 5,000 enterprise users concurrently query an agent, unmitigated spikes trigger HTTP 429 (Too Many Requests) errors.
• Resiliency Patterns: Implement Provisioned Throughput Units (PTU) for mission-critical production workloads to eliminate noisy-neighbor latency spikes, paired with exponential backoff retry policies in Azure API Management.
• Network Security & Isolation: Enterprise grounding data must never transit the public internet. Architects must enforce Azure Private Endpoints, Virtual Network (VNet) integration, and Managed Identities, ensuring that Copilot Studio and Azure AI Foundry communicate with vector stores across private IP backbones.

---

2. Quantitative Scoring and Auditing Data Readiness

Prior to deploying an agent into production, solution architects must audit grounding data using objective quality benchmarks. The industry standard framework tested on the AB-100 is the RAG Triad:

                       [ User Query ]
                         /          \
      Context Relevance /            \ Answer Relevance
                       v              v
             [ Retrieved Context ] ----> [ Agent Response ]
                        Groundedness (Faithfulness)

The RAG Triad Metric Definitions

1. Context Relevance: Evaluates whether the retrieved grounding chunks are pertinent and focused on the user query, with minimal irrelevant noise. $\text{Context Relevance} = \frac{\text{Number of Relevant Sentences in Context}}{\text{Total Sentences in Retrieved Context}}$
2. Groundedness (Faithfulness): Measures whether every factual claim in the agent's generated response can be directly inferred from the retrieved grounding context. A response containing claims absent from the context represents a hallucination. $\text{Groundedness} = \frac{\text{Number of Output Claims Directly Supported by Grounding Context}}{\text{Total Factual Claims Made in Output}}$
3. Answer Relevance: Evaluates whether the agent's generated response directly addresses the user's initial question, regardless of whether the context was complete.

Data Readiness Audit Checklist

---

3. Real-World Architectural Case Scenario: Stale Grounding Data Remediation

The Incident

During an annual Open Enrollment period, an enterprise HR Benefits Agent deployed in Microsoft Teams provided contradictory and obsolete healthcare deductibles to over 3,000 employees. Employees were informed that the in-network deductible was $500 (the 2025 rate), whereas the ratified 2026 plan required $750. Multiple employees made binding medical elections based on the agent's erroneous output, creating significant union grievance exposure.

Root Cause Analysis (RCA)

1. Vector Index Contamination: The Azure AI Search index contained both the legacy 2025_Benefits_Handbook.pdf (300 pages) and the newly uploaded 2026_Benefits_Addendum.pdf (12 pages).
2. Semantic Similarity Distortion: Because the 300-page 2025 handbook contained dozens of mentions of deductible terms with extensive supporting narrative, standard cosine vector search scored the 2025 document chunks higher than the concise 2-page table in the 2026 addendum.
3. Absence of Temporal Filtering: The search query did not include any date metadata constraints, allowing historical documents to compete equally with current policies.
4. Batch Indexing Lag: The index synchronization pipeline operated on a weekly cron job; several recent policy amendment memos uploaded to SharePoint 48 hours prior had not yet been processed into vectors.

[User: What is my 2026 deductible?]
            |
            v
  [Standard Vector Search]
       /                \
      v                  v
[2025 Handbook Chunk]    [2026 Addendum Chunk]
Cosine Score: 0.89       Cosine Score: 0.81  <-- Outdated 2025 chunk wins!
      | 
      v
[Agent Generates Erroneous $500 Response]

The Architectural Remediation Pattern

To permanently resolve this vulnerability, the solution architect implements four architectural fixes:

1. Index Schema Enrichment with Temporal Metadata: Update the index schema to include mandatory OData filterable fields:
   • effectiveStartDate (Edm.DateTimeOffset)
   • effectiveEndDate (Edm.DateTimeOffset)
   • isSuperseded (Edm.Boolean)
   • documentVersion (Edm.String)
2. Mandatory OData Pre-Filtering: Configure the agent's knowledge retrieval query to execute an automated pre-filter enforcing active validity:

   isSuperseded eq false and effectiveStartDate le now() and (effectiveEndDate ge now() or effectiveEndDate eq null)
   

3. Freshness Scoring Profile with Date-Decay Boosting: Configure an Azure AI Search scoring profile that applies a mathematical decay function to document scores based on their lastModifiedDate:
   • Documents modified within the past 30 days receive an automatic score boost, ensuring newly ratified addenda outrank older base manuals.
4. Event-Driven Synchronization Pipeline: Replace the weekly batch crawler with an Azure Event Grid subscription on the SharePoint document library. When a new benefits document is uploaded or marked as approved, an event immediately triggers an Azure Function to extract, chunk, embed, and upsert the document within 90 seconds.