2.5 Deep Learning and Neural Networks

Quick Answer: Deep learning uses neural networks with multiple layers to learn complex patterns. Neural networks have input layers (features), hidden layers (processing), and output layers (predictions). The Transformer architecture powers modern large language models like GPT. Deep learning excels at image recognition, speech, and language tasks.

What Is Deep Learning?

Deep learning is a subset of machine learning that uses artificial neural networks with multiple layers (hence "deep") to learn complex patterns from large amounts of data. While traditional ML algorithms work well for structured data with clear features, deep learning shines on unstructured data like images, audio, and text.

When to Use Deep Learning vs. Traditional ML

How Neural Networks Work (Conceptual)

A neural network is inspired by the human brain — it consists of interconnected nodes (neurons) organized in layers:

Network Structure

Input Layer          Hidden Layers          Output Layer
(features)           (processing)           (prediction)
   [x₁] ─────┐
              ├──→ [h₁] ──→ [h₄] ──┐
   [x₂] ─────┤                     ├──→ [y]
              ├──→ [h₂] ──→ [h₅] ──┤
   [x₃] ─────┤                     │
              └──→ [h₃] ──→ [h₆] ──┘

The Three Layer Types

How Learning Happens

1. Forward pass: Input data flows through the network, each neuron applies a mathematical transformation
2. Loss calculation: The network's prediction is compared to the actual answer — the difference is the "loss"
3. Backpropagation: The network adjusts its internal weights (connections between neurons) to reduce the loss
4. Repeat: This process repeats thousands or millions of times with different training examples
5. Convergence: Eventually, the network's predictions become accurate

On the Exam: You do NOT need to understand the mathematics of neural networks. Know that deep learning uses layers of neurons that process data, learn from errors (backpropagation), and improve over time with more data and training.

What Makes Deep Learning "Deep"

The "deep" in deep learning refers to having multiple hidden layers. Each layer learns different levels of abstraction:

Example: Image recognition of a face

• Layer 1: Detects edges and simple shapes
• Layer 2: Combines edges into facial features (eyes, nose, mouth)
• Layer 3: Combines features into facial structures
• Layer 4: Recognizes specific faces

More layers allow the network to learn more complex patterns — this is why deep learning outperforms traditional ML on complex tasks like image and speech recognition.

The Transformer Architecture

The Transformer is a neural network architecture introduced in 2017 that revolutionized NLP and is the foundation of modern AI:

Key Features of Transformers

Transformers in Modern AI

On the Exam: You need to know that Transformers are the architecture behind modern language models (GPT, BERT) and that they use self-attention to understand context. You do NOT need to understand the mathematical details of attention mechanisms.

Deep Learning Applications on Azure

Activation Functions and Weights (Conceptual)

Two terms appear in beginner-level neural-network questions. Weights are the adjustable strengths of the connections between neurons — training is the process of nudging these weights to reduce error. An activation function decides whether and how strongly a neuron "fires" based on its inputs; it introduces non-linearity so the network can learn curved, complex relationships rather than just straight lines. You do not need the math, but recognizing that "the network learns by adjusting weights" and "activation functions let it model complex patterns" covers the exam's depth.

A Quick Map of Neural-Network Types

On the Exam: Connect architecture to workload — CNN → vision, Transformer → language/generative AI. The Transformer is explicitly named in the skills measured ("identify features of the Transformer architecture"), so expect at least one question on it.

Why the Transformer Changed Everything

Before Transformers, language models processed text word-by-word in sequence, which was slow and struggled to connect words that were far apart in a sentence. The Transformer's self-attention mechanism lets the model weigh the relationship between every word and every other word at the same time, capturing long-range context. Because it processes the whole input in parallel rather than sequentially, it also trains far faster on modern GPUs, which is what made it possible to scale to billions of parameters and produce models like GPT-4o.

Two more terms worth recognizing: tokens (the chunks of text — roughly word-pieces — that a language model reads and produces) and embeddings (numeric vector representations of tokens that capture meaning, so that similar words sit near each other in vector space). Embeddings are also what power vector search in retrieval-augmented generation.

On the Exam: Self-attention = the mechanism that lets a Transformer understand context across the whole input. Parallel processing = why Transformers train faster and scale larger than older sequential models. You will not be asked to explain the attention formula.

When Deep Learning Is Overkill

Deep learning is not always the right tool. For small, structured, tabular datasets — say predicting loan default from 12 columns and 5,000 rows — a simpler model like logistic regression or a boosted tree is often more accurate, faster, cheaper, and easier to explain. Reserve deep learning for unstructured data at scale: images, audio, video, and natural language. The exam may present a small tabular scenario where deep learning is the wrong (over-engineered) choice.