Image and Video Generation, Editing, and Content Understanding Pipelines
Key Takeaways
- The computer vision domain has a dedicated sub-skill for generating and editing both images and videos from text prompts and reference media, not just analyzing existing media.
- Image editing uses inpainting and mask-based edits; video editing means producing and modifying generated video clips, both governed by the platform's generation and editing controls.
- Multimodal understanding includes video analysis: processing and interpreting video segments, captioning, visual question answering, and identifying objects, components, or regions.
- Azure AI Content Understanding runs in single-task and pro mode; single-task is one analyzer for one field set, while pro mode reasons across multiple inputs and reference data for harder extraction.
- Visual responsible AI is its own sub-skill: classify unsafe visual content, detect indirect prompt injection from text embedded in images, apply watermarks, flag prohibited symbols, and enforce brand-usage rules.
Image and Video Generation, Editing, and Content Understanding Pipelines
Quick Answer: The AI-103 computer vision domain (10-15%) is split into three sub-skills: generate and edit images and videos, multimodal understanding workflows (including video analysis), and responsible AI for multimodal content. Many candidates over-prepare for image analysis and under-prepare for generation, video, and Content Understanding pipeline modes. This section targets exactly those blueprint bullets.
The blueprint lists "Design and implement image- and video-generation solutions" as its own functional group. That means the exam tests producing new media, not just describing existing media. Generation questions add inputs the analysis questions never use: a text prompt, reference media, a mask, an aspect ratio, a resolution, and a duration. Read the verb in the stem: describe / read / detect points to analysis, while generate / create / edit / modify / inpaint points to a generation or editing workflow.
Image Generation and Editing
Image generation turns a text prompt (and optionally a reference image) into a new image. The blueprint also names editing workflows explicitly, and these have precise vocabulary you must separate:
| Technique | What it does | Exam clue |
|---|---|---|
| Text-to-image generation | Creates a brand-new image from a prompt | "Generate product mockups from a description" |
| Image variation / reference media | Produces new images guided by a sample image | "Match the style of this existing asset" |
| Inpainting | Regenerates only a masked region, keeping the rest | "Replace just the background, keep the product" |
| Mask-based edit | Restricts edits to the mask the user supplies | "Edit only the area the user painted over" |
| Outpainting / extend | Expands the canvas beyond the original frame | "Extend the banner to a wider aspect ratio" |
The single most-missed point is that inpainting and mask-based edits modify an existing image inside a mask, while plain generation creates from scratch. If a scenario says "keep everything except the logo" or "change only the sky," the answer is masked inpainting, not a fresh generation prompt. Editing controls the platform provides — strength, mask, seed, reference weight — are the levers you tune, and a strong answer names the control plus the safety policy in one breath.
Video Generation and Editing
New to AI-103 versus older vision exams: generating videos from text prompts and reference media, plus workflows to edit generated videos. A video-generation request adds parameters that images do not: duration, frame/aspect settings, resolution, and sometimes a reference clip or starting frame. Editing a generated video means iterating on a clip — trimming, regenerating a segment, or applying prompt-driven modifications — rather than re-rendering from zero each time.
The exam trap is conflating video generation (create or edit a clip) with video analysis (interpret an existing clip). They are opposite directions: generation is an output workflow under the generation sub-skill; analysis is an input workflow under multimodal understanding. A clue like "produce a 10-second promo from this script" is generation; "summarize what happens in this 10-minute recording" is analysis.
Multimodal Understanding and Video Analysis
The second sub-skill, multimodal understanding workflows, is where vision input lives. The blueprint lists building solutions that analyze visual context with multimodal models, producing concise or detailed captions for single or multiple images, grounded visual question answering, alt-text and extended descriptions for accessibility, and — importantly — video analysis workflows to process and interpret video segments and identifying objects, components, or regions within images or video.
Use this decision flow for an understanding scenario:
- Caption or answer about what is visible → multimodal model (reasons over pixels plus any text).
- Locate items: object, component, region, zone, bounding box → object/region detection or spatial analysis.
- Interpret a video over time: segments, scenes, events → a video-analysis workflow that processes segments.
- Extract named visual characteristics into a schema → Content Understanding configured for visual extraction.
A worked example: a manufacturer wants to flag which component in an assembly photo is missing, then caption the photo for an audit log. "Which component and where" is object/region identification; "describe it" is multimodal captioning. One scenario, two sub-skills — score each part, then pick one architecture (a multimodal model with region grounding) that satisfies both.
Single-Task versus Pro-Mode Content Understanding
The blueprint explicitly says: "Configure single-task and pro-mode Content Understanding pipelines." This is a high-value, frequently-missed distinction.
| Mode | What it is | When to choose |
|---|---|---|
| Single-task analyzer | One analyzer extracts one schema/field set from one input type | Predictable, repeatable extraction from a known document or image type |
| Pro mode | Reasons across multiple inputs and reference data, performing multi-step analysis and validation | Complex cases needing comparison, cross-referencing, or knowledge beyond one file |
Single-task is the lean default: define an analyzer with the fields you want, run it against one input, and get structured or markdown output. Pro mode is for harder problems where the analyzer must consider multiple documents at once or consult reference data (such as a policy library or prior records) to reason, link, and validate before producing output.
The exam clue for pro mode is language like "cross-reference against company policy," "reconcile several documents," or "multi-step reasoning with reference data"; the clue for single-task is "extract these fields from each invoice the same way every time." Choosing pro mode for a simple one-form extraction is the over-engineering trap; choosing single-task when the task needs cross-document reasoning is the under-powered trap.
Content Understanding also produces markdown or structured JSON outputs designed for downstream reasoning, which is why it appears under both the vision domain (visual characteristics, video segments) and the information-extraction domain (grounded representations for agents and RAG). One service, two blueprint homes.
Responsible AI for Multimodal Content
The third sub-skill is governance specific to visual media. It is not optional decoration — the blueprint lists it as its own group:
| Control | Purpose | Scenario phrasing |
|---|---|---|
| Visual content filters | Classify unsafe or disallowed images | "Block adult, violent, or hateful imagery" |
| Indirect prompt-injection defense | Detect instructions hidden in text inside an image | "A screenshot contains text telling the agent to ignore its rules" |
| Watermarking / provenance | Mark generated media as synthetic | "Label AI-generated images as synthetic" |
| Prohibited-symbol flagging | Detect banned logos or symbols | "Reject generations containing a competitor or hate symbol" |
| Brand-usage enforcement | Keep output within brand rules | "Only approved colors, logos, and layouts" |
The most exam-relevant trap is indirect prompt injection through embedded image text: OCR-readable instructions inside a screenshot or photo are untrusted input that can try to hijack the agent. The answer pairs prompt shields with treating extracted image text as data, never as system instructions. For generation specifically, remember provenance and watermarks plus human-review paths for sensitive use cases, because visual safety is broader than just blocking explicit content.
A marketing app must change only the background of an existing product photo while leaving the product untouched, and the team also wants to label the result as AI-edited. Which combination best fits the AI-103 computer vision blueprint?
An insurer must extract the same handful of fields from each incoming invoice identically, while a separate workflow must reconcile a claim against multiple supporting documents and the company policy library before producing a decision summary. Which Content Understanding modes fit, in that order?