6.6 RAG After On Your Data, and AI Agents

Quick Answer: Azure OpenAI On Your Data (the data_sources extension that attached an index to chat completions) was retired on October 14, 2026. For an Azure AI Search-backed app, Microsoft's migration path is a Foundry IQ knowledge base that reuses the existing index; classic RAG with app-managed answer generation is a generally available alternative. Agents add planning + tool calling to do multi-step work. Foundry Agent Service is the managed agent platform: create a versioned agent → conversation → response → return function_call_output items → read the result.

Azure OpenAI On Your Data (Retired)

The final AI-102 outline asks you to "implement a RAG pattern by grounding a model in your data". Until its retirement, Azure OpenAI On Your Data did this without retrieval code: a chat completions request carried a data_sources block (legacy dataSources, or the /extensions/chat/completions path) pointing at Azure AI Search, Blob Storage, uploaded files, URLs, Azure Cosmos DB for MongoDB, Elasticsearch, or MongoDB Atlas, and the service returned grounded answers with citations. Microsoft retired it on October 14, 2026. Ordinary chat completions requests without that configuration are not affected.

Grounding Now: Foundry IQ or Classic RAG

Migration isn't automatic: you map fields and filters, enforce per-user authorization in your backend on every request, validate citation IDs, and keep conversation history in your application.

On the Exam: A 2026 design should not call On Your Data. For a standard "answer questions over my documents" requirement, ground the model through Azure AI Search with hybrid search and the semantic ranker (classic RAG) or a Foundry IQ knowledge base, and instruct the model in the system message to answer only from the retrieved content.

AI Agents (Agentic AI)

An agent wraps an LLM with the ability to plan, call tools, run code, retrieve documents, and remember state — turning a one-shot chat into an autonomous multi-step worker.

Microsoft Foundry Agent Service Lifecycle (azure-ai-projects 2.x)

import json
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition, FunctionTool, CodeInterpreterTool
from azure.identity import DefaultAzureCredential

project = AIProjectClient(credential=DefaultAzureCredential(),
    endpoint="https://my-resource.services.ai.azure.com/api/projects/my-project")

agent = project.agents.create_version(                   # 1. versioned agent
    agent_name="product-assistant",
    definition=PromptAgentDefinition(
        model="gpt-4o",
        instructions="Help customers find and compare products.",
        tools=[FunctionTool(name="search_products", parameters={...},
                            description="Search the product catalog"),
               CodeInterpreterTool()]))

openai = project.get_openai_client(agent_name=agent.name)
conversation = openai.conversations.create()              # 2. conversation (was: thread)
response = openai.responses.create(                       # 3. response (was: run)
    conversation=conversation.id,
    input="Find a laptop under $1000 with 16GB RAM")

while any(item.type == "function_call" for item in response.output):   # 4. tool loop
    outputs = [{"type": "function_call_output", "call_id": item.call_id,
                "output": json.dumps(run_tool(item.name, json.loads(item.arguments)))}
               for item in response.output if item.type == "function_call"]
    response = openai.responses.create(conversation=conversation.id, input=outputs)

print(response.output_text)                               # 5. final answer

The function call is the heart of the pattern: when the agent decides to use one of your functions, the response contains a function_call item instead of a final answer. Your code runs the function and sends a function_call_output item with the same call_id in the next response, and this can repeat several times before the agent answers. The classic threads-and-runs API showed the same pause as a run in requires_action until you called submit_tool_outputs; Microsoft has deprecated those classic agents and retires them on March 31, 2027.

Choosing a RAG/Agent Approach

Managed retrieval vs. building it yourself

A Foundry IQ knowledge base handles retrieval and citations for you; a hand-built pipeline (section 6.3) gives full control of chunking, ranking, filtering, and post-processing. Choose the managed path for standard "answer questions over my documents" requirements, and choose custom RAG when a scenario demands bespoke ranking, filtering, or transformation of retrieved content before it reaches the model.

Why agents differ from plain RAG

The distinction the exam draws is autonomy and steps. A RAG call retrieves once and answers once. An agent can decide to search, then call an API, then run code, then answer — looping through function calls as many times as the task needs, carrying state in its conversation. That power costs complexity: more tokens, harder debugging, and the need to secure every tool the agent can invoke. Built-in tool types — code_interpreter for sandboxed code and file_search for managed RAG over uploaded files — let you add common capabilities without writing your own functions.

On the Exam: The 2026 AI-102 leans into agents. Memorize the current lifecycle (create agent version → conversation → response → return function_call_output → read result) and recognize the classic one (agent → thread → message → run → requires_action → submit_tool_outputs), where requires_action = waiting for your tool output, not an error or completion. Prefer the simplest option that satisfies the requirement — don't reach for an agent when plain RAG or a single function call suffices.