4.3 Data Type, Quality, and Representative Datasets

Data decides what the model can honestly say

Microsoft's business-leader training for generative AI is direct: models are only as good as the data they learn from and the data they retrieve. AB-731 tests three lenses: type, quality, and representative datasets. Those lenses apply to training corpora, fine-tuning sets, and grounding indexes. A biased or thin grounding corpus produces biased or thin answers even when retrieval-augmented generation is technically in place.

Grounding data is the information provided to a language model at inference time so it can generate a more accurate response to this user's question. Source data is what already exists in production: structured tables, semi-structured logs, or unstructured documents, images, and audio. User interaction data — prompts and answers in a chat turn — is a third pile, with its own retention rules. Leaders who collapse those piles into "we will just add our data" lose the ability to classify, refresh, and retire the right objects.

Data type

Microsoft 365 Copilot work experiences lean on unstructured text in Graph. Foundry Tools and Azure AI Search can extend to vector and multimodal pipelines, including image-aware retrieval where the organization has prepared it. If the business question lives in a warehouse table, a text-only Copilot prompt over PDFs will not magically compute a balance. Type mismatch is a scoping error. It looks like a model failure in the demo.

Data quality: accuracy, completeness, and labeling

Quality is accuracy (the fact is correct), completeness (required fields and documents are present), and labeling (metadata, sensitivity labels, and topic tags that make retrieval and governance work). Microsoft Purview data-quality language also uses completeness, consistency, conformity, accuracy, freshness, and uniqueness. For a leader, the operating questions are practical. Who certifies that this PDF is the current policy? What share of tickets lack a resolution code? Are drafts sitting next to signed versions with no status field?

Poor quality shows up as confident wrong answers, not as a red error banner. An incomplete benefits corpus will omit the plant that uses a different carrier. An inaccurate price list will generate a discount the company cannot honor. Missing labels will either hide a document from retrieval or, worse, place a highly confidential file in a general index. Labeling is therefore both a search problem and a confidentiality problem.

Data readiness work that belongs on the leader's plan includes inventory, classification, secure accessibility, metadata, stewardship, and a feedback loop so users can flag bad answers. Those practices are how an organization keeps grounding data reliable after the first demo.

Representative datasets: who is in the data, whose outcomes change

A representative dataset reflects the people, languages, regions, products, and edge cases the system will actually serve. "Who is in the data?" and "Whose outcomes will the model affect?" must be answered together. If a claims-triage corpus is ninety percent one product line, one geography, or one language, retrieval will over-serve that slice. If a hiring screen is grounded only on resumes from one university feeder, the assistant will sound like that feeder.

Bias is not only a training-data problem. An unrepresentative grounding corpus — missing accessibility policies, omitting a newly acquired brand, excluding Spanish-language standard operating procedures — will systematically under-help those customers and employees. Fairness in Microsoft's responsible AI standards starts with representative data. Reliability starts with high-quality data. Accountability starts with being able to explain which corpus was used and who was missing from it.

Leaders should treat representativeness as a sampling problem, not a volume problem. More rows from the same headquarters ticket system do not make the dataset more representative of frontline plants. A small, current, deliberately sampled set of policies and cases from each served group beats a huge dump from one office.

Work through this checklist before you call a corpus ready:

• Inventory and classify before you index, including personal data hiding in tickets and email.
• Measure quality on the source, not only on a handful of demo questions.
• Sample grounding corpora the way you would sample a financial population: by region, product, language, and role.
• Keep a feedback loop so users can flag answers that reveal a missing slice.
• Do not treat "we have a lot of data" as proof of representativeness.
• Re-check the corpus when the served population changes — a merger, a new market, a new union contract.

Classification and retention as leader concerns

Data classification groups content by sensitivity and handling rules: public, general, confidential, highly confidential, regulated. Classification drives who may ground on the content, whether it belongs in a broad Copilot index, and whether it needs a separate Azure AI Search index with stricter filters. Microsoft Copilot honors sensitivity labels and Information Rights Management usage rights. Custom RAG that copies files into an index without carrying those labels forward is a classification failure.

Retention decides how long source files, indexes, and Copilot interaction history live. When a user interacts with Microsoft Copilot in Microsoft 365 apps, Microsoft stores the prompt, the response, and citations used to ground the response. Admins can manage that history with Microsoft Purview, including retention policies. Users can delete their Copilot activity history from the My Account portal. Leaders should not retain chat traces "forever to improve the model." Microsoft states that prompts, responses, and Graph-accessed data are not used to train the foundation LLMs used by Copilot. Retain for audit and operations, then delete.

Stale retained content that remains searchable is a freshness failure wearing a compliance badge. Azure Well-Architected grounding guidance is to avoid storing personal data in an index unless it is required to answer the query, to respect source classification when populating indexes, and to discard noisy or irrelevant content that reduces retrieval quality.

In practice: An HR director wants an internal Copilot for benefits questions. The corpus is five years of English-only tickets from headquarters, plus a 2019 PDF. Frontline plants in two other countries will be the primary users. The director's first task is not picking a model. It is commissioning a representative, current, labeled benefits corpus, classifying personal data out of tickets, and setting retention so old plans do not keep answering.