3.4 Azure AI Document Intelligence
Key Takeaways
- Azure AI Document Intelligence (formerly Form Recognizer) extracts structured data from documents — text, tables, key-value pairs, and layout information.
- Pre-built models handle common document types: invoices, receipts, ID documents, W-2 tax forms, and business cards without training.
- Custom models can be trained to extract data from your organization's specific document formats (purchase orders, contracts, custom forms).
- The Layout API extracts text, tables, selection marks, and document structure (paragraphs, sections, headers) from any document.
- Document Intelligence goes beyond OCR by understanding document STRUCTURE — it knows a value belongs to a specific field, not just what text is present.
Quick Answer: Azure AI Document Intelligence extracts structured data from documents including text, tables, key-value pairs, and layout information. It provides pre-built models for invoices, receipts, ID documents, and tax forms. Custom models can be trained for your specific document types. It goes beyond basic OCR by understanding document structure.
What Is Document Intelligence?
Azure AI Document Intelligence (formerly Azure Form Recognizer) is a service that uses machine learning to extract text, key-value pairs, tables, and structures from documents. Unlike basic OCR which just reads text, Document Intelligence understands the structure of documents.
OCR vs. Document Intelligence
| Capability | Basic OCR (Azure AI Vision) | Document Intelligence |
|---|---|---|
| Extract text from image | Yes | Yes |
| Identify text position | Yes | Yes |
| Extract key-value pairs | No | Yes ("Invoice Number: 12345") |
| Extract tables | No | Yes (rows, columns, cells) |
| Understand document structure | No | Yes (headers, paragraphs, sections) |
| Pre-built document models | No | Yes (invoices, receipts, IDs) |
| Custom document models | No | Yes (train on your documents) |
Pre-Built Models
Document Intelligence provides pre-trained models for common document types:
| Pre-Built Model | What It Extracts | Example Fields |
|---|---|---|
| Invoice | Invoice data | Invoice number, date, total, line items, vendor |
| Receipt | Receipt data | Merchant name, date, total, items, tax |
| ID Document | Identity card data | Name, date of birth, address, document number |
| W-2 | Tax form data | Employee info, wages, tax withholdings |
| Business Card | Contact info | Name, company, phone, email, address |
| Health Insurance Card | Insurance data | Insurer, member ID, group number, plan |
How Pre-Built Models Work
- Submit a document (image or PDF) to the pre-built model endpoint
- The model analyzes the document structure and content
- Receive extracted fields with confidence scores
No training is needed — the models are pre-trained by Microsoft on millions of documents.
Custom Models
When pre-built models don't cover your document types, you can train custom models:
Custom Template Models
- Trained on a specific document template/layout
- Works best when documents have a consistent format
- Requires 5+ sample documents per template
- Ideal for: standardized forms, purchase orders, specific contracts
Custom Neural Models
- Uses deep learning for more flexible extraction
- Handles documents with varying layouts
- Requires more training data (10+ documents)
- Ideal for: documents with varied formats, mixed layouts
The Layout API
The Layout API extracts document structure without requiring a specific document model:
| Extracted Element | Description |
|---|---|
| Text | All text content with positions |
| Tables | Table structure with rows, columns, and cells |
| Selection marks | Checkboxes and radio buttons (selected/unselected) |
| Paragraphs | Text grouped into paragraphs |
| Sections | Document sections and subsections |
| Headers/Footers | Page headers and footers |
| Page numbers | Page number identification |
| Barcodes | 1D and 2D barcodes (QR codes, Code 128, etc.) |
On the Exam: The Layout API is the most general Document Intelligence capability — it works on any document without training. Pre-built models are for specific document types. Custom models are for your organization's unique document formats. Know which to recommend for each scenario.
How does Azure AI Document Intelligence differ from basic OCR?
A company processes thousands of invoices from different vendors. They want to automatically extract invoice numbers, dates, totals, and line items. Which Document Intelligence approach should they use?
When should you train a custom Document Intelligence model instead of using a pre-built model?
Naming History: Form Recognizer Is Now Document Intelligence
For the exam, fix the current name: the service once called Azure Form Recognizer is now Azure AI Document Intelligence, part of the Azure AI services family (the former Cognitive Services umbrella). The capabilities are the same — pre-built models, custom models, and the Layout API — but every current AI-900 question uses the new name. If a practice question or older guide says "Form Recognizer," read it as Document Intelligence.
Three Ways to Extract Data — and When to Use Each
The single most testable idea in this section is choosing among the three Document Intelligence approaches. Map the scenario to the right one:
| Approach | Training Needed | Use When |
|---|---|---|
| Pre-built model | None | The document is a common type Microsoft already supports — invoice, receipt, ID, W-2, business card, health insurance card |
| Layout API | None | You need raw structure (text, tables, selection marks, headers) from any document, regardless of type |
| Custom model | Yes (your samples) | Your document is a unique/proprietary format the pre-built models do not cover — purchase orders, internal forms, specialized contracts |
A worked decision: "Extract totals and line items from supplier invoices" → pre-built Invoice model (no training, understands invoice fields). "Pull every table and checkbox out of an arbitrary scanned report" → Layout API. "Read our company's custom intake form with a layout Microsoft has never seen" → custom model.
Custom: Template vs. Neural Models
When you do train a custom model, there are two flavors:
| Model | Strength | Best Fit |
|---|---|---|
| Custom template | Fast, accurate on a fixed layout; needs as few as 5 samples | Documents that always look the same |
| Custom neural | Handles varied layouts using deep learning; needs more samples | Documents whose structure varies between sources |
Why It Beats Plain OCR
OCR answers "what characters are on the page?" Document Intelligence answers "what does each value mean?" It returns key-value pairs (so it knows $1,240 is the Total, not just a number), tables with rows and columns preserved, selection marks (checkbox state), and confidence scores per field. That structural understanding is why it powers straight-through automation of accounts-payable, onboarding, and claims processing, where a bare text dump would still need a human to interpret it.
Document Intelligence Studio and Output
Like Custom Vision, Document Intelligence has a no-code Studio portal where you can test pre-built models, label sample documents, train custom models, and review extraction quality before wiring the model into an application via API. Results are returned as structured JSON containing fields, tables, and the bounding regions of every extracted element.
On the Exam: Three-way rule — known common document → pre-built model; any document, just need structure → Layout API; your unique format → custom model. And remember the headline distinction from Azure AI Vision's OCR: Document Intelligence understands fields, key-value pairs, and tables, while plain OCR only returns text and its position.