16.2 AI Builder Prebuilt & Custom Models in Apps and Flows

AI Builder Prebuilt & Custom Models in Apps and Flows

Modern enterprise workflows demand automated data extraction from unstructured files, cognitive sentiment analysis, intelligent routing, and predictive business analytics. AI Builder is the native Power Platform capability that packages advanced Azure Cognitive Services and Azure Machine Learning models into accessible low-code components. Functional consultants can incorporate prebuilt turnkey models or train custom machine learning models directly within Power Apps and Power Automate. On the PL-200 exam, you must master the complete taxonomy of AI Builder models, custom model training data requirements, Canvas app UI controls, Power Automate connector actions, and tenant capacity governance.

---

1. AI Builder Architecture & Model Taxonomy

AI Builder divides its capabilities into two distinct architectural classes: Prebuilt Models (instant consumption with zero training) and Custom Models (trained and tailored using organization-specific data).

+-----------------------------------------------------------------------------------+
|                         AI BUILDER MODEL TAXONOMY                                 |
|                                                                                   |
|  +-------------------------------------+  +------------------------------------+  |
|  |          PREBUILT MODELS            |  |           CUSTOM MODELS            |  |
|  |    (Ready Out-of-the-Box)           |  |    (Trained on Customer Data)      |  |
|  |                                     |  |                                    |
|  |  - Invoice Processing               |  |  - Document Processing             |  |
|  |  - Receipt Processing               |  |    * Structured (Fixed Forms)      |  |
|  |  - Business Card Reader             |  |    * Unstructured (Contracts, Docs)|  |
|  |  - Text Recognition (OCR)           |  |    * Invoices / Receipts Tuning    |  |
|  |  - Sentiment Analysis               |  |  - Object Detection                |  |
|  |  - Language Detection               |  |  - Text Classification             |  |
|  |  - Key Phrase Extraction            |  |  - Prediction (Binary / Multi)     |  |
|  |  - Identity Document Reader         |  |                                    |
|  |  - Text Translation                 |  |                                    |
|  +------------------+------------------+  +------------------+-----------------+  |
|                     |                                        |                    |
|                     v                                        v                    |
|  +-----------------------------------------------------------------------------+  |
|  |                             CONSUMPTION LAYER                               |
|  |                                                                             |  |
|  |  POWER APPS (UI CONTROLS):              POWER AUTOMATE (ACTIONS):           |  |
|  - Document Processor Control              - 'Extract information from...'     |
|  - Business Card Reader Control            - 'Process and save...'             |
|  - Receipt Processor Control               - 'Predict' Action (Custom Models)  |
|  - Object Detector Control                 - Dynamic Tokens & JSON Parsing     |
|  +-----------------------------------------------------------------------------+  |
+-----------------------------------------------------------------------------------+

Prebuilt vs. Custom Models Comparison

---

2. Deep Dive: Prebuilt AI Builder Models

Prebuilt models allow functional consultants to deliver immediate business value without the overhead of collecting training data or evaluating statistical model fit.

1. Invoice Processing (Prebuilt)

• Automatically extracts key header fields: Vendor Name, Vendor Address, Customer Name, Invoice ID, Invoice Date, Due Date, Purchase Order Number, Subtotal, Total Tax, and Total Invoice Amount.
• Extracts line-item tables including Description, Quantity, Unit Price, and Amount.

2. Receipt Processing (Prebuilt)

• Optimized for printed and handwritten thermal and paper retail receipts.
• Extracts Merchant Name, Merchant Address, Transaction Date, Transaction Time, Subtotal, Tax, Tip, and Total Amount.

3. Business Card Reader (Prebuilt)

• Scans physical business cards captured via mobile camera or image upload.
• Extracts Full Name, First Name, Last Name, Job Title, Company Name, Email Address, Phone Number, Mobile Number, and Address.

4. Text Recognition (OCR) (Prebuilt)

• Extracts all printed and handwritten text lines from images (.jpg, .png) and document files (.pdf).
• Returns raw text strings, bounding box coordinates [X, Y, Width, Height], and word-level confidence scores.

5. Sentiment Analysis & Text Analytics (Prebuilt)

• Sentiment Analysis: Classifies text into Positive, Negative, Neutral, or Mixed, providing individual sentiment scores per sentence and for the document as a whole.
• Language Detection: Identifies the language of incoming text and outputs the standard ISO language code (e.g., en, es, fr, de) with confidence percentage.
• Key Phrase Extraction: Identifies the main talking points, key noun phrases, and thematic concepts from customer feedback, emails, or case descriptions.
• Identity Document Reader: Parses worldwide passports, US driver's licenses, and state IDs, extracting names, birthdates, expiration dates, and document numbers.

---

3. Deep Dive: Custom AI Builder Models & Training Lifecycle

When standard prebuilt schemas do not match proprietary corporate documents, products, or historical data schemas, consultants must build and train custom models.

+-----------------------------------------------------------------------------+
|                   CUSTOM MODEL TRAINING & ALM LIFECYCLE                     |
|                                                                             |
|  [1. MODEL TYPE]         [2. SAMPLE DATA]          [3. TAGGING & LABELS]    |
|  Choose Document         Upload minimum 5          Define fields/tables and |
|  Processing, Object      sample documents per      tag bounding boxes on    |
|  Detection, Text, etc.   collection / layout.      training documents.      |
|         |                         |                         |               |
|         v                         v                         v               |
|  [6. ALM & DEPLOY]       [5. PUBLISH MODEL]        [4. TRAIN & EVALUATE]    |
|  Add published model     Publish model to make     Train model; inspect     |
|  to Solution; export as  available to Apps and     Model Performance Score  |
|  Managed to Prod.        Cloud Flows.              (0-100 / Grade A-D).     |
+-----------------------------------------------------------------------------+

1. Document Processing (Form Processing)

Extracts structured data, tables, and checkboxes from standard company documents.

• Structured Documents: Fixed layouts where form elements always appear in the exact same physical coordinates (e.g., standard tax forms, standardized loan applications).
• Unstructured Documents: Free-form documents where data appears anywhere in varying layouts and paragraphs (e.g., contracts, letters, resumes, statements of work).
• Training Requirement:
  • Strict Minimum: 5 sample documents per collection / layout.
  • A collection represents a distinct document layout. If an organization receives invoices in 3 distinct layout formats, the model requires 3 collections with at least 5 sample documents in each collection (minimum 15 total documents).
  • Best practice recommendation: 10 to 20 realistic samples per collection for production accuracy.

2. Object Detection

Identifies and counts specific physical objects in images, drawing bounding boxes around each occurrence.

• Training Requirement: Minimum 15 sample images per object class / label (e.g., 15 images of "Damaged Valve", 15 images of "Intact Valve").
• Used in field service and inventory management to inspect stock levels or identify equipment defects.

3. Text Classification (Category Classification)

Categorizes unstructured text into one or more user-defined categories (e.g., routing incoming support emails into "Billing", "Hardware", "Software", or "Account Access").

• Training Requirement: Minimum 10 sample text entries per category/tag (50+ recommended).

4. Prediction Models (Binary & Multi-Outcome Classification)

Analyzes historical Dataverse tabular records to predict binary outcomes (e.g., Will lead qualify? Yes/No, Will customer churn? Yes/No) or numerical values.

• Evaluates relationships across historical columns and generates a Model Performance Score (0 to 100) and feature influence chart showing which fields drive the prediction.

[!IMPORTANT] Custom Model Lifecycle Rules:

1. A newly trained model cannot be used in Power Apps or Power Automate until it is explicitly Published.
2. Custom models are Solution-Aware components. When deploying across environments (Dev -> Test -> Prod), models must be added to a Dataverse Solution and exported as Managed Solutions.

---

4. Consuming AI Builder in Power Apps

Power Apps Studio provides dedicated Canvas App UI controls designed for real-time document capture, processing, and visual inspection.

+-----------------------------------------------------------------------------+
|                   CANVAS APP: AI BUILDER UI CONTROLS                        |
|                                                                             |
|  +---------------------------------+  +----------------------------------+  |
|  |   DOCUMENT PROCESSOR CONTROL    |  |   BUSINESS CARD READER CONTROL   |  |
|  |  - Drag & drop / upload file    |  |  - Mobile camera scan / file     |  |
|  |  - Bound to Custom/Prebuilt     |  |  - Extracts Contact info         |  |
|  |  - `DocProcessor.Fields`        |  |  - `BusinessCardReader.FirstName`|  |
|  |  - `DocProcessor.Tables`        |  |  - `BusinessCardReader.Company`  |  |
|  +---------------------------------+  +----------------------------------+  |
|                                                                             |
|  +---------------------------------+  +----------------------------------+  |
|  |    RECEIPT PROCESSOR CONTROL    |  |     OBJECT DETECTOR CONTROL      |  |
|  |  - Real-time mobile receipt OCR |  |  - Live camera view with boxes   |  |
|  |  - `ReceiptProcessor.Total`     |  |  - `ObjectDetector.GroupResults` |  |
|  |  - `ReceiptProcessor.Items`     |  |  - Counts items per class label  |  |
|  +---------------------------------+  +----------------------------------+  |
+-----------------------------------------------------------------------------+

Binding Extracted Properties in Power Fx

When a user scans or uploads a document in a Canvas App, the AI Builder control exposes extracted data as strongly-typed Power Fx record and table objects:

// Populating a form field from the Business Card Reader control:
Txt_FirstName.Default = BusinessCardReader1.FirstName
Txt_LastName.Default  = BusinessCardReader1.LastName
Txt_Email.Default     = BusinessCardReader1.Email
Txt_Company.Default   = BusinessCardReader1.CompanyName

// Populating a Gallery from Document Processor Line Items:
Gallery_InvoiceLines.Items = DocumentProcessor1.Tables.InvoiceLines

// Saving an invoice record to Dataverse using Patch:
Patch(
    Invoices,
    Defaults(Invoices),
    {
        'Invoice Number': DocumentProcessor1.Fields.InvoiceId.Value,
        'Vendor Name': DocumentProcessor1.Fields.VendorName.Value,
        'Total Amount': Value(DocumentProcessor1.Fields.TotalAmount.Value),
        'Confidence Score': DocumentProcessor1.Fields.TotalAmount.Confidence
    }
)

---

5. Consuming AI Builder in Power Automate Cloud Flows

Automated document processing pipelines execute AI Builder models silently in the background when emails arrive, files are uploaded to SharePoint, or records are created in Dataverse.

+-----------------------------------------------------------------------------+
|                 AUTOMATED INVOICE PROCESSING FLOW PIPELINE                  |
|                                                                             |
|  [TRIGGER]       -> When an email arrives with an attachment (PDF)          |
|         |                                                                   |
|         v                                                                   |
|  [ACTION 1]      -> 'Extract information from invoices' (AI Builder)        |
|  - Document: Attachment Content                                             |
|         |                                                                   |
|         v                                                                   |
|  [ACTION 2]      -> 'Condition': Confidence Score > 0.80                    |
|  ├─ IF YES:                                                                 |
|  │   ├─ Action: 'Add a new row' (Dataverse Invoices table)                  |
|  │   └─ Action: 'Apply to each' (Line Items) -> Add rows to Invoice Lines   |
|  └─ IF NO:                                                                  |
|      └─ Action: Send approval email to AP Team for manual review            |
+-----------------------------------------------------------------------------+

Power Automate Action Types

• Dedicated Prebuilt Actions: Direct actions such as Extract information from invoices, Extract information from receipts, Recognize text in an image, Classify text with AI Builder, and Detect sentiment in text. Dynamic content tokens for all standard fields are surfaced directly in the flow designer.
• The Generic Predict Action: Used to call Custom Trained Models. The maker selects the Model from a dropdown (or passes the Model GUID) and supplies the input payload (e.g., document payload or tabular parameters). Output properties are parsed using standard dynamic tokens or JSON parsing.

---

6. AI Builder Capacity & Credits Governance

AI Builder does not bill per API call or per user seat. Instead, it utilizes an enterprise resource pooling model called AI Builder Capacity Credits.

+-----------------------------------------------------------------------------+
|                     AI BUILDER CAPACITY ALLOCATION MODEL                    |
|                                                                             |
|  [TENANT POOL] ───> Purchased AI Builder Add-on Units (e.g., 1,000,000 / mo)|
|       │                                                                     |
|       ├── Allocates 500,000 credits ──> [PRODUCTION ENVIRONMENT]            |
|       │                                 - Runs Invoice Processing Flow      |
|       │                                 - Runs Canvas App Receipt Scans     |
|       │                                                                     |
|       ├── Allocates 100,000 credits ──> [UAT / TEST ENVIRONMENT]            |
|       │                                 - Model Testing & Validation        |
|       │                                                                     |
|       └── Unallocated Pool (400,000) ──> Available for on-demand allocation |
+-----------------------------------------------------------------------------+

Capacity Management Rules

• Tenant-Level Add-On: AI Builder credits are purchased as tenant-level add-ons (or bundled with Power Apps Premium licenses) and pooled across the organization.
• Environment Allocation: In the Power Platform Admin Center (PPAC) under Resources -> Capacity -> Add-ons, administrators explicitly allocate specific quantities of monthly credits to designated production and test environments.
• Consumption Rate: Different model operations consume varying amounts of credits (e.g., processing an invoice document consumes more credits than running sentiment analysis on a short sentence).
• Monitoring & Over-Quota: Administrators monitor consumption in the PPAC AI Builder capacity reporting dashboard. If an environment exhausts its allocated credits, AI processing may be throttled or blocked until additional capacity is assigned.