Computer Vision Concepts and Tasks

Quick Answer: Computer vision enables machines to interpret images and videos. The key tasks are image classification (label an entire image), object detection (find and locate objects with bounding boxes), semantic segmentation (classify every pixel), OCR (extract text from images), and facial analysis (detect and analyze faces). Each task answers a different question about the visual content.

What Is Computer Vision?

Computer vision is a field of AI that enables computers to interpret and extract meaningful information from visual data — images and videos. It is how machines "see" and understand the visual world.

Computer vision powers applications you use every day:

• Unlocking your phone with your face
• Depositing checks by photographing them
• Self-driving cars navigating roads
• Security cameras detecting intruders
• Medical imaging detecting tumors
• Barcode and QR code scanning

Core Computer Vision Tasks

Understanding the differences between these tasks is critical for the AI-900:

1. Image Classification

What it does: Assigns a single label to an entire image.

Question it answers: "What is in this image?"

Output: A class label with a confidence score.

Use cases:

• Medical image screening (disease vs. healthy)
• Quality inspection in manufacturing (defective vs. acceptable)
• Content moderation (appropriate vs. inappropriate)
• Wildlife monitoring (species identification)
• Plant disease detection (healthy vs. infected)

2. Object Detection

What it does: Identifies AND locates multiple objects within an image using bounding boxes.

Question it answers: "What objects are in this image, and where are they?"

Output: A list of detected objects, each with a class label, confidence score, and bounding box coordinates (x, y, width, height).

Use cases:

• Autonomous vehicles (detect pedestrians, cars, signs)
• Retail analytics (count products on shelves)
• Security and surveillance (detect people, vehicles)
• Medical imaging (locate tumors, lesions)
• Wildlife monitoring (count and locate animals)

3. Semantic Segmentation

What it does: Classifies every pixel in an image, creating a detailed map of the scene.

Question it answers: "What class does every pixel in this image belong to?"

Output: A pixel-by-pixel classification map where each pixel is assigned a category.

Use cases:

• Autonomous driving (road, sidewalk, car, pedestrian, building — for every pixel)
• Medical imaging (precise tumor boundaries)
• Land use mapping (forest, water, urban, agricultural — from satellite images)
• Augmented reality (separate foreground from background)

4. OCR (Optical Character Recognition)

What it does: Extracts text from images, converting visual text into machine-readable text.

Question it answers: "What text is in this image?"

Output: Extracted text with position information.

Use cases:

• Reading documents (contracts, forms, letters)
• License plate recognition
• Receipt and invoice digitization
• Sign and label reading
• Handwriting recognition
• Business card scanning

5. Facial Detection and Analysis

What it does: Detects human faces in images and optionally analyzes attributes.

Question it answers: "Are there faces in this image, and what can you tell about them?"

Output: Face locations and optional attributes.

Important: Microsoft has retired facial recognition capabilities that infer emotional state, gender, and age from the Face API due to responsible AI concerns. The current Face API focuses on face detection, verification, and identification — not emotion or demographic inference.

Comparison Table: Key Computer Vision Tasks

On the Exam: The most common question type is: "A company needs to [scenario]. Which computer vision task should they use?" Focus on the output: single label = classification, located objects = detection, pixel map = segmentation, extracted text = OCR, face analysis = facial detection.