Azure OpenAI On Your Data and AI Agents

Quick Answer: "On Your Data" connects Azure AI Search to chat completions for managed RAG without custom code. AI agents autonomously perform complex tasks using function calling and tool use. Azure AI Foundry Agent Service provides a managed platform for building and deploying agents.

Azure OpenAI On Your Data

Supported Data Sources

Configuration via API

response = client.chat.completions.create(
    model="gpt4o-deployment",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What are the key features of our product?"}
    ],
    extra_body={
        "data_sources": [{
            "type": "azure_search",
            "parameters": {
                "endpoint": "https://my-search.search.windows.net",
                "index_name": "product-docs",
                "authentication": {
                    "type": "api_key",
                    "key": "<search-api-key>"
                },
                "query_type": "vector_semantic_hybrid",
                "embedding_dependency": {
                    "type": "deployment_name",
                    "deployment_name": "text-embedding-3-large-deployment"
                },
                "in_scope": True,  # Only answer from provided data
                "strictness": 3,  # 1-5, higher = stricter relevance filtering
                "top_n_documents": 5
            }
        }]
    }
)

# Response includes citations
message = response.choices[0].message
print(message.content)

# Access citations
if hasattr(message, 'context') and message.context:
    for citation in message.context.get('citations', []):
        print(f"Source: {citation['title']}")
        print(f"Content: {citation['content'][:200]}")

Key Parameters for On Your Data

AI Agents (Agentic AI)

AI agents are autonomous systems that use LLMs to plan, reason, and execute multi-step tasks:

Agent Capabilities

Azure AI Foundry Agent Service

The Agent Service provides a managed platform for building agents:

# Create an agent with tools
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

client = AIProjectClient(
    credential=DefaultAzureCredential(),
    endpoint="https://my-project.services.ai.azure.com/"
)

# Define tools the agent can use
tools = [
    {
        "type": "function",
        "function": {
            "name": "search_products",
            "description": "Search the product catalog",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string"},
                    "category": {"type": "string"}
                },
                "required": ["query"]
            }
        }
    },
    {
        "type": "code_interpreter"  # Built-in code execution
    },
    {
        "type": "file_search"  # Built-in RAG over uploaded files
    }
]

# Create the agent
agent = client.agents.create_agent(
    model="gpt-4o",
    name="Product Assistant",
    instructions="You help customers find and compare products.",
    tools=tools
)

Agent Conversation Loop

# Create a conversation thread
thread = client.agents.create_thread()

# Add a user message
client.agents.create_message(
    thread_id=thread.id,
    role="user",
    content="Find me a laptop under \$1000 with at least 16GB RAM"
)

# Run the agent
run = client.agents.create_run(
    thread_id=thread.id,
    assistant_id=agent.id
)

# Wait for completion and handle tool calls
while run.status in ["queued", "in_progress", "requires_action"]:
    run = client.agents.get_run(thread_id=thread.id, run_id=run.id)

    if run.status == "requires_action":
        # Agent wants to call a tool
        tool_calls = run.required_action.submit_tool_outputs.tool_calls
        tool_outputs = []

        for call in tool_calls:
            if call.function.name == "search_products":
                # Call your actual product search API
                result = search_products(**json.loads(call.function.arguments))
                tool_outputs.append({
                    "tool_call_id": call.id,
                    "output": json.dumps(result)
                })

        # Submit tool outputs back to the agent
        client.agents.submit_tool_outputs(
            thread_id=thread.id,
            run_id=run.id,
            tool_outputs=tool_outputs
        )

# Get the final response
messages = client.agents.list_messages(thread_id=thread.id)
print(messages.data[0].content[0].text.value)

On the Exam: The 2026 AI-102 exam includes questions about agentic AI patterns. Know the agent lifecycle: create agent with tools → create thread → add message → run agent → handle tool calls → get response. Function calling is the mechanism that allows agents to interact with external systems.

Comparing RAG Approaches