Question Answering and Conversational AI

Quick Answer: Custom Question Answering (replacing QnA Maker) creates knowledge bases from FAQs and documents. It supports multi-turn conversations, active learning, and integration with CLU for orchestration. Deploy as a chatbot via Azure Bot Service.

Custom Question Answering vs. QnA Maker

Creating a Knowledge Base

Sources for Knowledge Base Content

Adding Sources via REST API

# Create a project
project_url = f"{endpoint}/language/authoring/question-answering/projects/{project_name}?api-version=2023-04-01"

project_body = {
    "description": "Customer support knowledge base",
    "language": "en",
    "multilingualResource": True,
    "settings": {
        "defaultAnswer": "I'm sorry, I don't have information about that. Please contact our support team."
    }
}

requests.patch(project_url, headers=headers, json=project_body)

# Add a URL source
sources_url = f"{endpoint}/language/authoring/question-answering/projects/{project_name}/sources?api-version=2023-04-01"

source_body = [{
    "op": "add",
    "value": {
        "displayName": "Company FAQ",
        "sourceUri": "https://www.example.com/faq",
        "sourceKind": "url"
    }
}]

requests.patch(sources_url, headers=headers, json=source_body)

Querying the Knowledge Base

query_url = f"{endpoint}/language/:query-knowledgebases?api-version=2023-04-01"

query_body = {
    "question": "What are your return policy terms?",
    "top": 3,
    "projectName": project_name,
    "deploymentName": "production",
    "confidenceScoreThreshold": 0.5,
    "includeUnstructuredSources": True
}

response = requests.post(query_url, headers=headers, json=query_body)
result = response.json()

for answer in result["answers"]:
    print(f"Answer: {answer['answer']}")
    print(f"Confidence: {answer['confidenceScore']:.2f}")
    print(f"Source: {answer['source']}")

Multi-Turn Conversations

Multi-turn conversations use follow-up prompts to guide users through complex topics:

User: "How do I return a product?"
Bot: "To return a product, you need to initiate a return within 30 days.
      Would you like to know about:
      → In-store returns
      → Online returns
      → International returns"

User: "Online returns"
Bot: "For online returns:
      1. Log into your account
      2. Go to Order History
      3. Select the item and click 'Return'
      4. Print the shipping label
      Would you like to know about:
      → Refund timeline
      → Shipping costs"

Follow-Up Prompt Structure

Each Q&A pair can have child prompts that link to other Q&A pairs, creating a conversation tree.

Active Learning

Active learning improves the knowledge base based on real user interactions:

1. User asks a question that partially matches multiple Q&A pairs
2. System suggests alternative questions to the knowledge base editor
3. Editor approves/rejects suggestions (adding them as alternate questions)
4. Model improves over time as more alternate phrasings are added

Enabling Active Learning

# Accept an active learning suggestion
suggestions_url = f"{endpoint}/language/authoring/question-answering/projects/{project_name}/feedback?api-version=2023-04-01"

feedback_body = {
    "records": [{
        "userId": "user123",
        "userQuestion": "Can I get my money back?",
        "qnaId": 42  # Map this question to an existing Q&A pair
    }]
}

requests.post(suggestions_url, headers=headers, json=feedback_body)

Orchestration with CLU

The Orchestration workflow in CLU routes user utterances to the appropriate backend service:

[User Input] → [Orchestration Model (CLU)]
                    ├── Intent: BookFlight → [CLU Flight Booking Model]
                    ├── Intent: FAQ → [Custom Question Answering KB]
                    └── Intent: None → [Default Response]

On the Exam: Orchestration workflow is the pattern for combining CLU (intent recognition) with Question Answering (FAQ responses) in a single conversational system. The orchestrator determines whether the user is asking a question (routes to Q&A) or performing an action (routes to CLU).