13.1 Generative Answers with Azure AI Search & Foundry

13.1 Generative Answers with Azure AI Search & Foundry

Quick Answer: For AB-620 “Azure integration,” Azure AI Search retrieves enterprise chunks and Microsoft Foundry (Azure OpenAI / Foundry models) generates the natural-language answer. Wire that RAG stack into a generative answers path in Copilot Studio so replies are grounded, citable, and policy-aligned—not freeform model recall.

Chapter 9 taught connecting Azure AI Search as knowledge: formal data connection, one vector index, hybrid/semantic readiness, and citations. This section is the next exam leaf: configure generative answers by using Azure AI Search with Foundry. Same retrieval substrate, different emphasis—Foundry-backed generation configuration, RAG architecture decisions, and grounding quality when Search + models work as one solution.

Why this skill sits under “Integrate agents with Azure”

Microsoft’s AB-620 outline groups three Azure skills: generative answers with Search and Foundry, custom prompts against the Foundry model catalog, and Application Insights monitoring. Together they describe a production pattern: retrieve enterprise truth, generate with a managed model estate, observe runtime behavior in Azure Monitor.

Without Foundry-class generation, Search is only a knowledge feed into Copilot Studio’s default generative stack. Without Search, Foundry models answer from pretrained knowledge or whatever prompt context you hand them. The exam wants you to design the combined path when enterprises already invested in Azure AI Search indexes and Foundry model deployments.

Retrieval-Augmented Generation (RAG) in this architecture

RAG means: do not rely solely on the model’s training cutoff or general knowledge. At runtime:

1. The user asks a question in a Copilot Studio channel or test canvas.
2. Copilot Studio invokes the generative answers path (topic node and/or conversational boosting).
3. Azure AI Search queries the prepared index (keyword, vector, hybrid; optional semantic ranker).
4. Relevant chunks (passages, not necessarily whole PDFs) return with metadata.
5. Those passages are passed as grounding context to a model hosted through Azure AI Foundry / Azure OpenAI.
6. The model produces a grounded reply that should stick to retrieved content.
7. When citation fields exist (for example a document URL or metadata_storage_path), the agent can surface references users can open.

Exam trap: Azure AI Search does not “write the answer.” Foundry-hosted models do. Search retrieves. Confusing those roles is a common distractor.

How this differs from Chapter 9 knowledge-only Search

Both skills require a healthy index. Chapter 13 expects you to reason about the full loop—including model connection, grounding priority, and quality of retrieved context—not only the knowledge connector wizard.

Microsoft Learn configuration pattern (generative answers + Azure OpenAI / Foundry data)

Microsoft documents connecting data through Azure OpenAI Service so agents use Azure resources from a generative answers node. Conceptually this is “Azure OpenAI on your data”: models plus an Azure AI Search index that keeps answers inside organizational content and policies.

High-level maker steps:

1. In Microsoft Foundry / Azure OpenAI, connect enterprise data so models can use a Search-backed index (Azure OpenAI on your data pattern).
2. Optionally Deploy to a new Microsoft Copilot Studio agent so an agent is created with Azure resources linked; a Conversational boosting system topic may be generated automatically.
3. In an existing agent topic, add a generative answers node (Create generative answers).
4. Open Data source (Edit under Data sources, or node … → Properties → Data source).
5. Under Classic data, Add connection to the Azure OpenAI Service resource you prepared.
6. Open Connection properties to review model and data parameters.
7. Save the topic and test with questions that only corporate documents can answer.

Priority rule (exam-ready): Knowledge sources defined on a generative answers node take priority over agent-level knowledge. Agent-level sources function as fallback. If a scenario says “this topic must answer only from the Contoso legal index,” fix the node data sources—not only the agent Knowledge page.

Index connection and retrieval quality (grounding quality)

Grounding fails more often from bad retrieval than from a weak model. AB-620 expects fluency with the same index building blocks as Chapter 9, now judged as inputs to generation:

Scenario — Contoso HR agent: Contoso indexes 400 policy PDFs. Employees ask, “How much annual leave do I get after five years?” Keyword-only retrieval misses a section titled “Paid time off accrual.” After enabling vector + hybrid retrieval and re-chunking long PDFs into section-sized segments, the generative answers node returns a grounded summary with a citation to the benefits handbook. The model did not “learn HR”; Search supplied the passage Foundry used.

Scenario — SKU lookup vs narrative Q&A: Product SKUs often work better when the Azure-side “on your data” pipeline is tuned for identifiers, while pure natural-language product questions need strong vector fields on descriptions. Exam stems that mention wrong answers for codes but good answers for prose point at retrieval field design, not “switch to a larger GPT.”

Designing for responsible grounded answers

• Prefer grounded generative answers over unrestricted general knowledge for regulated domains (HR, finance, safety).
• Treat ungrounded model knowledge as an explicit product decision, not an accident.
• Keep sensitive content out of indexes that broad audiences can query—or enforce identity-aware access patterns.
• Test permission gaps: citation points to SharePoint the end user cannot open → “helpful” answer with a dead link.
• After major content updates, validate incremental indexing rather than hoping the model remembers yesterday’s PDF.

Configuration checklist for makers

1. Confirm Azure AI Search service, vector index, and (if needed) semantic ranker are production-ready.
2. Confirm Foundry / Azure OpenAI deployment and on your data linkage to that index.
3. In Copilot Studio, attach generative answers data sources via supported connections—not ad-hoc undocumented key pastes outside the formal connection UX.
4. Set node-level sources when a topic must override agent-wide knowledge.
5. Test with known-answer, synonym, out-of-corpus, and restricted-user cases.
6. Inspect citations and empty-result behavior (fallback message vs hallucination).
7. Hand off latency and error diagnostics to Application Insights (Section 13.3).

Pitfalls

1. Treating Search knowledge connection (Ch. 9) as already satisfying the Foundry generative answers skill.
2. Believing the LLM indexes documents by itself—indexing is a Search pipeline job.
3. Skipping hybrid/vector setup then blaming the model for synonym failures.
4. Omitting citation URL fields and failing auditability.
5. Expecting agent-level knowledge to override a restrictive generative answers node configuration (priority is the opposite).
6. Using generative answers for transactions (create ticket, update record)—that is tools/flows territory.
7. Private Search endpoints without Power Platform VNet alignment (connection “works in Azure portal,” fails from the agent).

Success checklist

• Explain RAG roles: Copilot Studio orchestrates, Search retrieves, Foundry generates.
• Contrast knowledge connection vs Foundry-backed generative answers.
• Walk generative answers Data source / Classic data / Azure OpenAI connection configuration at a conceptual level.
• Name chunking, embeddings, hybrid, semantic ranker, citations as grounding quality controls.
• Apply node-over-agent knowledge priority.
• Choose RAG over static topics when content is large, changing, or multi-repository.

Master this leaf and you can defend why Contoso’s agent quotes the current handbook paragraph instead of inventing benefits—and which Azure pieces make that possible.