3.2 Building with Document AI, Vision, Translation & Other AI APIs
Key Takeaways
- Document AI processors fall into three categories: digitize (OCR), extract (custom extractor, Form Parser, Layout Parser, pretrained parsers), and classify (custom classifier and custom splitter).
- Document AI Layout Parser extracts text, tables, and lists and creates context-aware chunks for retrieval in gen AI applications.
- Vision API DOCUMENT_TEXT_DETECTION handles dense text and handwriting and takes precedence over TEXT_DETECTION when both are requested.
- Cloud Translation Advanced adds glossaries, batch and document translation, the Translation LLM, and Adaptive Translation beyond the Basic API's standard NMT model.
- Google deprecated the Video Intelligence API on September 14, 2026 and plans to shut it down on September 14, 2027.
Section 1.2 of the exam guide expects you to build applications with industry-specific APIs such as the Document AI API, Vision API, and Translation API. These APIs are the lowest-effort way to add AI: no training data, no model hosting, and results through a REST call or client library. Your job on the exam is to spot when a task matches one of these APIs, and when it needs customization or a foundation model instead.
Document AI
Document AI turns unstructured documents (PDFs, scans, images) into structured data. You create a processor in your project and send documents to it.
| Category | Processor | Use it for |
|---|---|---|
| Digitize | Enterprise Document OCR | Text and layout from scans, with image-quality detection and deskewing |
| Extract | Form Parser | Key-value pairs and tables from generic forms |
| Extract | Layout Parser | Text, tables, and lists with context-aware chunks for RAG and search |
| Extract | Custom Extractor | Your own fields, using generative AI or custom models. It can be fine-tuned on your documents |
| Extract | Pretrained parsers | Invoice, Expense, Bank Statement, Pay Slip, W2, US Driver License, Identity Document Proofing |
| Classify | Custom Classifier | Routing documents by type (claim form vs. medical record) |
| Classify | Custom Splitter | Splitting one PDF that holds several documents |
Pretrained parsers return normalized fields, such as dates and amounts in standard formats, which you can load straight into BigQuery. Document AI also supports dataset management, auto-labeling, and review workflows to improve custom processors.
Scenario: An insurer receives 50,000 mixed PDFs a day, each bundling a claim form, an invoice, and photos of damage. A good design uses Custom Splitter → Custom Classifier → Invoice Parser for invoices and Custom Extractor for claim fields, then writes to BigQuery. Documents with low confidence go to a human review step.
Vision API
The Vision API analyzes images with pre-trained models:
| Feature | Output |
|---|---|
LABEL_DETECTION | General labels ("dog", "beach") with confidence scores |
OBJECT_LOCALIZATION | Multiple objects with bounding boxes |
TEXT_DETECTION | OCR for sparse text in photos (signs, packaging) |
DOCUMENT_TEXT_DETECTION | OCR for dense text, handwriting, and PDF/TIFF files. Wins if both OCR features are requested |
FACE_DETECTION | Faces and attributes (not identity recognition) |
LANDMARK_DETECTION, LOGO_DETECTION | Known landmarks and brand logos |
SAFE_SEARCH_DETECTION | Explicit content likelihoods for moderation |
IMAGE_PROPERTIES, CROP_HINTS, WEB_DETECTION | Dominant colors, crop suggestions, web matches |
If your labels are specific to your business (for example, "cracked weld" vs. "acceptable weld"), general Vision labels won't be enough. Train an AutoML image model, or use a Gemini multimodal model with examples. For scanned business documents, Google recommends Document AI over Vision OCR because it adds structure and entity extraction.
Cloud Translation
| Edition | Capabilities |
|---|---|
| Basic | Plug-and-play Neural Machine Translation (NMT) for text |
| Advanced | NMT plus the Translation LLM (TLLM), glossaries (fixed translations for brand and domain terms), batch and document translation (preserving formatting), custom models, and Adaptive Translation, which matches your style from a small set of example translation pairs |
Pick glossaries when product names or regulated terms must always translate the same way. Pick Adaptive Translation or model customization when tone and domain style matter. Use document translation for DOCX/PDF files whose formatting must be preserved.
Speech and Language APIs
- Speech-to-Text transcribes audio with Chirp-family models, including telephony-tuned variants for call-center audio.
- Text-to-Speech generates natural voices for IVR and accessibility.
- Natural Language API provides sentiment, entity, and entity-sentiment analysis, syntax analysis, and content classification.
Video: A Deprecation to Watch
Google deprecated the Video Intelligence API on September 14, 2026, with shutdown planned for September 14, 2027. For new video labeling, summarization, or moderation designs, plan on Gemini video understanding on Agent Platform.
API vs. Foundation Model vs. Custom Model
| Question to ask | If yes... |
|---|---|
| Does a pre-trained API already produce exactly the needed output (invoice fields, OCR, translation)? | Use the API. It's the fastest, cheapest, and lowest-overhead choice |
| Is the task open-ended, like summarizing, reasoning across a document, or answering questions? | Use Gemini through Agent Platform, optionally with Layout Parser chunks for grounding |
| Are the labels unique to your business and do you have labeled examples? | AutoML (image or tabular), a Document AI custom processor, or Gemini tuning |
| Do you need a custom architecture or strict control over the model? | Custom training |
Worked Comparison: Three Ways to Read a Receipt
An expense app needs merchant, date, total, and tax from photographed receipts.
| Approach | What you get | Trade-off |
|---|---|---|
Vision API DOCUMENT_TEXT_DETECTION | Raw text in a page, block, paragraph, word hierarchy | You still write and maintain parsing rules for every receipt layout |
| Document AI Expense Parser | Labeled, normalized fields (supplier, date, total, tax) with confidence scores | Limited to the fields the parser defines. Use a Custom Extractor for extra fields |
| Gemini with a response schema | Flexible structured JSON, including fields no parser defines | You own prompt design, evaluation, and monitoring for extraction accuracy |
For a standard document type, the pretrained parser is usually the best exam answer: it needs the least engineering and returns structured, normalized output. Gemini is a better fit when the documents are too varied for fixed processors or when the task goes beyond extraction, such as explaining a policy exception.
Production Design Considerations
- Batch vs. online: Document AI and Translation both offer batch methods for large volumes from Cloud Storage. Use online calls for interactive apps.
- Quotas: retry with exponential backoff when you exceed quota. For high-volume pipelines, spread requests over time and check quota limits before you launch.
- Data governance: keep processors in regions that meet residency rules, and restrict access with IAM and VPC Service Controls (Chapter 18).
- Human in the loop: send low-confidence extractions to reviewers rather than writing them straight to systems of record.
- Composition: real solutions chain APIs. For example, Speech-to-Text transcribes calls, Translation normalizes the language, and Gemini summarizes and categorizes the result.
A logistics company receives scanned PDFs, each containing a bill of lading, a commercial invoice, and a packing list stapled together. They need each document type routed and invoice totals extracted with minimal model development. Which design fits best?
A pharmaceutical company translates product leaflets into 15 languages and requires that drug names and regulated phrases always translate identically. Which Cloud Translation capability addresses this requirement directly?
A media company is starting a new project to label and summarize video clips on Google Cloud. What should the architect consider first?