5.3 Foundation Models, JumpStart, and Amazon Bedrock

5.3 Foundation Models, JumpStart, and Amazon Bedrock

The rapid evolution of Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) has fundamentally transformed enterprise machine learning architectures. Rather than training deep neural networks from scratch over months with millions of dollars in compute, ML engineers leverage pre-trained Foundation Models (FMs) and customize them using prompt engineering, Retrieval-Augmented Generation (RAG), parameter-efficient fine-tuning (PEFT/LoRA), or full domain adaptation.

For the AWS Certified Machine Learning Engineer — Associate (MLA-C01) exam, you must master the three tiers of the AWS Generative AI stack, evaluate the architectural trade-offs between Amazon Bedrock (serverless managed APIs) and Amazon SageMaker JumpStart (dedicated infrastructure hosting), understand model fine-tuning methodologies, and architect managed RAG solutions using Amazon Bedrock Knowledge Bases.

+------------------------------------------------------------------------------------------------+
|                            THE THREE-TIER AWS GENERATIVE AI STACK                              |
|                                                                                                |
|   [TIER 3: APPLICATIONS]                                                                       |
|   - Amazon Q Developer / Amazon Q Business                                                     |
|   - Pre-built enterprise conversational AI assistants                                          |
|                                                                                                |
|   [TIER 2: TOOLS & FOUNDATION MODEL PLATFORMS]  <--- CORE MLA-C01 FOCUS AREA                   |
|   - Amazon Bedrock: Serverless API access to leading FMs + Knowledge Bases (RAG) + Agents       |
|   - SageMaker JumpStart: Model catalog, 1-click deploy to dedicated instances, PEFT/LoRA      |
|                                                                                                |
|   [TIER 1: INFRASTRUCTURE ACCELERATORS & ML PLATFORM]                                          |
|   - AWS Trainium (Trn1/Trn2) & AWS Inferentia2 (Inf2) custom silicon                           |
|   - Amazon EC2 GPU Clusters (NVIDIA H100/A100 instances: p5, p4d, g5)                          |
|   - SageMaker HyperPod & SageMaker Distributed Training Libraries                              |
+------------------------------------------------------------------------------------------------+

---

1. Amazon Bedrock: Managed Serverless Foundation Models

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models from leading AI companies (Anthropic, Meta, Mistral AI, Cohere, AI21 Labs, Stability AI) alongside Amazon's proprietary Titan models via a single, unified API. Bedrock is serverless, meaning you do not manage EC2 instances, GPU memory allocation, or endpoint scaling.

+------------------------------------------------------------------------------------------------+
|                       AMAZON BEDROCK FOUNDATION MODEL ECOSYSTEM                                |
|                                                                                                |
|   Provider        Model Families             Primary Strengths & Specializations               |
|   -------------   ------------------------   -----------------------------------------------   |
|   Anthropic       Claude 3.5 Sonnet,         Advanced reasoning, complex coding, vision,       |
|                   Claude 3 Opus, Haiku       nuanced comprehension, 200k token context window  |
|                                                                                                |
|   Meta            Llama 3 / 3.1              Open-weights versatility, multilingual dialogue,  |
|                   (8B, 70B, 405B)            instruction following, high throughput            |
|                                                                                                |
|   Amazon          Titan Text, Titan Image,   Cost-effective text generation, enterprise search |
|                   Titan Embeddings V2        embeddings, image generation, watermarking        |
|                                                                                                |
|   Cohere          Command R / R+,            Enterprise RAG optimization, multilingual query   |
|                   Cohere Embed Multilingual  grounding, specialized reranking models           |
|                                                                                                |
|   Mistral AI      Mistral Large,             High-efficiency reasoning, low-latency code       |
|                   Mixtral 8x7B (MoE)         generation, Mixture-of-Experts architecture       |
+------------------------------------------------------------------------------------------------+

1.1 Model Inference Parameters

When invoking foundation models via the Bedrock API (InvokeModel or InvokeModelWithResponseStream), engineers calibrate response generation using key decoding parameters:

1. Temperature (0.0 to 1.0):
   • Controls the randomness of token probability distribution.
   • Low Temperature (0.0–0.2): Deterministic, focused, and precise. Ideal for factual Q&A, structured JSON extraction, and code generation.
   • High Temperature (0.7–1.0): Diverse, creative, and exploratory. Ideal for creative writing and brainstorming.
2. Top P (Nucleus Sampling, 0.0 to 1.0):
   • Dynamically selects tokens from the smallest cumulative probability pool exceeding threshold $P$.
   • Setting top_p=0.9 considers only the subset of tokens comprising the top 90% probability mass, filtering out the long tail of improbable tokens.
3. Top K (1 to 500):
   • Restricts token selection to strictly the $K$ most probable candidate tokens at each generation step.
4. Stop Sequences:
   • An array of character strings that explicitly signal the model to cease token generation immediately (e.g., ["\n\nUser:", "### End"]).

1.2 Invoking Amazon Bedrock via Python Boto3 SDK

import boto3
import json

bedrock_runtime = boto3.client(service_name="bedrock-runtime", region_name="us-east-1")

# Construct Claude 3 Messages API payload
prompt_payload = {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 1024,
    "temperature": 0.1,
    "top_p": 0.9,
    "messages": [
        {
            "role": "user",
            "content": "Analyze the following system log for security anomalies and output JSON: [AUTH_FAIL user=admin ip=192.168.1.100]"
        }
    ]
}

response = bedrock_runtime.invoke_model(
    modelId="anthropic.claude-3-5-sonnet-20240620-v1:0",
    contentType="application/json",
    accept="application/json",
    body=json.dumps(prompt_payload)
)

response_body = json.loads(response["body"].read())
print(response_body["content"][0]["text"])

---

2. Customizing Models in Amazon Bedrock

When prompt engineering alone cannot achieve desired accuracy or domain adherence, Bedrock provides managed customization without requiring infrastructure provisioning.

+------------------------------------------------------------------------------------------------+
|                       BEDROCK MODEL CUSTOMIZATION COMPARISON                                   |
|                                                                                                |
|   Customization Type       Input Data Format          Primary Objective                        |
|   ----------------------   ------------------------   --------------------------------------   |
|   Supervised Fine-Tuning   JSONL with labeled pairs   Teach specific style, tone, structured   |
|                            `{"prompt":..,"completion":..}` output schemas, or specialized tasks|
|                                                                                                |
|   Continued Pre-Training   JSONL with raw text        Domain Adaptation: teach model private   |
|   (Domain Adaptation)      `{"input": "..."}`          jargon (medical, legal, proprietary docs)|
|                                                                                                |
|   Direct Preference        JSONL with preferences     Align model outputs with human preference|
|   Optimization (DPO)       `{"prompt":..,"chosen":..}` reducing harmful or sub-optimal answers |
+------------------------------------------------------------------------------------------------+

• Data Storage: Customization datasets must be uploaded to Amazon S3 in JSONLines (.jsonl) format.
• Security & Privacy: Custom models created in Bedrock are private to your AWS account. Your proprietary data is never used to train base foundation models or shared across accounts.
• Provisioned Throughput: Custom models in Bedrock must be deployed using Provisioned Throughput (purchasing Model Units / PMUs with 1-month or 6-month commitments or no-commitment options) to guarantee consistent throughput.

---

3. Managed RAG with Amazon Bedrock Knowledge Bases

Retrieval-Augmented Generation (RAG) augments an FM prompt with relevant context retrieved from authoritative corporate document repositories. Amazon Bedrock Knowledge Bases is a fully managed service that implements the entire end-to-end RAG workflow without requiring custom vector pipelines.

+------------------------------------------------------------------------------------------------+
|                       BEDROCK KNOWLEDGE BASES MANAGED RAG PIPELINE                             |
|                                                                                                |
|   [DATA INGESTION & SYNC PHASE]                                                                |
|   1. Documents in Amazon S3 (PDF, DOCX, TXT, CSV, HTML)                                        |
|         |                                                                                      |
|         v                                                                                      |
|   2. Automated Chunking (Fixed-size, Hierarchical, Semantic, or Custom via Lambda)             |
|         |                                                                                      |
|         v                                                                                      |
|   3. Embedding Generation (e.g., Amazon Titan Text Embeddings V2, Cohere Embed)                |
|         |                                                                                      |
|         v                                                                                      |
|   4. Vector Index Storage (OpenSearch Serverless / Pinecone / Aurora pgvector / Neptune)       |
|                                                                                                |
|   ------------------------------------------------------------------------------------------   |
|   [RUNTIME QUERY & RETRIEVAL PHASE]                                                            |
|   Client Query ---> [Bedrock Knowledge Base] ---> Hybrid Search (Vector + BM25 Keyword)        |
|                                          |                                                     |
|                                          v (Top K Chunks with Citations)                       |
|                         [Foundation Model: Claude 3.5 / Llama 3]                               |
|                                          |                                                     |
|                                          v                                                     |
|                         Grounded Response with Source Attribution                              |
+------------------------------------------------------------------------------------------------+

3.1 Knowledge Base Chunking Strategies

1. Fixed-Size Chunking: Splits text into uniform token counts (e.g., 300 tokens) with a configurable overlap percentage (e.g., 20% / 60 tokens) to prevent context fragmentation across chunk boundaries.
2. Hierarchical (Parent-Child) Chunking: Ingests documents into large parent chunks (e.g., 1500 tokens) and subdivides them into smaller child chunks (e.g., 300 tokens). Vector search matches the precise child chunk, but provides the surrounding parent chunk to the FM for richer semantic context.
3. Semantic Chunking: Uses natural linguistic breaks (paragraphs, headings) to preserve semantic coherence.
4. Custom Chunking: Uses an AWS Lambda function for custom document parsing and domain-specific tokenization rules.

3.2 Supported Vector Store Backends in Bedrock

• Amazon OpenSearch Serverless (Default): Purpose-built vector search collection created automatically by Bedrock with zero cluster management.
• Amazon Aurora PostgreSQL: Configured with the open-source pgvector extension.
• Pinecone: Serverless managed vector database.
• Amazon Neptune Analytics: Graph-based vector search for complex relational knowledge graphs.

3.3 Bedrock Agents: Autonomous Multi-Step Tool Execution

While Knowledge Bases handles information retrieval, Amazon Bedrock Agents autonomously orchestrate complex multi-step business tasks:

• Uses ReAct (Reason + Act) prompting to break down user requests into discrete action steps.
• Interacts with enterprise systems by parsing OpenAPI schemas and invoking AWS Lambda functions to execute transactions (e.g., querying databases, booking flights, creating support tickets).

---

4. Amazon SageMaker JumpStart

Amazon SageMaker JumpStart is a machine learning hub within SageMaker that provides access to hundreds of pre-trained open-source and proprietary foundation models (Meta Llama 3, Mistral, Falcon, Stable Diffusion, Flan-T5, BGE Embeddings).

+------------------------------------------------------------------------------------------------+
|                          SAGEMAKER JUMPSTART ARCHITECTURE                                      |
|                                                                                                |
|   JumpStart Model Catalog (HuggingFace, Meta, Stability AI)                                    |
|         |                                                                                      |
|         +---> Option A: 1-Click / SDK Deployment to Dedicated Endpoint                         |
|         |     (SageMaker Real-Time Endpoint on ml.g5.12xlarge / ml.p4d.24xlarge)               |
|         |                                                                                      |
|         +---> Option B: Fine-Tuning with Custom S3 Dataset (PEFT / LoRA)                       |
|               (Executes managed SageMaker Training Job, outputs private weights to S3)         |
+------------------------------------------------------------------------------------------------+

4.1 Parameter-Efficient Fine-Tuning (PEFT) and LoRA

Full fine-tuning of a 70-billion parameter LLM requires modifying all 70B weights, consuming hundreds of gigabytes of GPU VRAM across multi-node clusters (ml.p4de.24xlarge).

Low-Rank Adaptation (LoRA) is a PEFT technique that freezes the pre-trained model weights $W_0 \in \mathbb{R}^{d \times k}$ and injects trainable rank-decomposition matrices $B \in \mathbb{R}^{d \times r}$ and $A \in \mathbb{R}^{r \times k}$ (where rank $r \ll \min(d, k)$):

$W = W_0 + \Delta W = W_0 + B \times A$

+------------------------------------------------------------------------------------------------+
|                                 LoRA ARCHITECTURE MECHANISM                                    |
|                                                                                                |
|               Input Vector x (Dimension d)                                                     |
|                     /              \                                                           |
|                    /                \                                                          |
|                   v                  v                                                         |
|       [Frozen Pre-trained W_0]      [Down-projection A (r x d)]                                |
|          (Dimension d x k)                   |                                                 |
|         (ZERO GRADIENT UPDATE)               v                                                 |
|                   |                 [Up-projection B (k x r)]                                  |
|                   |                 (TRAINABLE PARAMETERS: <1% of W_0)                         |
|                   \                  /                                                         |
|                    v                v                                                          |
|                     +-------------> (+)  ---> Output Vector h = W_0(x) + BA(x)                 |
+------------------------------------------------------------------------------------------------+

• Benefits of LoRA:
  • Reduces trainable parameters by up to 99%.
  • Enables fine-tuning large 8B–70B models on a single GPU instance (e.g., ml.g5.2xlarge or ml.g5.12xlarge).
  • Eliminates catastrophic forgetting of general world knowledge.
  • Base model weights remain shared; multiple task-specific LoRA adapter weights (few megabytes each) can be swapped dynamically at inference.

4.2 Fine-Tuning via SageMaker JumpStart Python SDK

from sagemaker.jumpstart.estimator import JumpStartEstimator

# Initialize JumpStart Estimator for Llama 3 8B
estimator = JumpStartEstimator(
    model_id="meta-textgeneration-llama-3-8b",
    instance_type="ml.g5.12xlarge",
    instance_count=1
)

# Set LoRA Hyperparameters
estimator.set_hyperparameters(
    instruction_tuned="True",
    epoch="3",
    learning_rate="0.0002",
    lora_r="16",           # LoRA rank dimension
    lora_alpha="32",       # LoRA scaling factor
    lora_dropout="0.05"
)

# Execute fine-tuning with training data in S3
estimator.fit({"training": "s3://my-ml-bucket/llama3-fine-tune-data/"})

# Deploy fine-tuned model to dedicated endpoint
predictor = estimator.deploy(instance_type="ml.g5.2xlarge")

---

5. AWS AI Services: Task-Specific Pre-Trained APIs

Not every ML problem requires training a model. AWS AI services are fully managed, pre-trained APIs that solve common business tasks through a single HTTPS call — no data preparation, training jobs, or endpoint hosting required. Task Statement 2.1 expects you to recognize when a managed AI service is the correct answer versus a custom SageMaker model or a Bedrock foundation model.

[!IMPORTANT] Selection hierarchy for the exam: (1) If a managed AI service performs the task natively, choose it first — it carries the least operational overhead. (2) If the task needs a generative foundation model or RAG, choose Amazon Bedrock. (3) If you need full control over model architecture, custom training data, or specialized evaluation metrics, build on SageMaker (built-in algorithm, Script Mode, BYOC, or JumpStart).

6. Architectural Decision Matrix: Bedrock vs. JumpStart vs. Custom Models

+------------------------------------------------------------------------------------------------+
|                 GENAI ARCHITECTURE DECISION MATRIX FOR MLA-C01                                 |
|                                                                                                |
|   Evaluation Dimension       Amazon Bedrock                  SageMaker JumpStart               |
|   ------------------------   -----------------------------   -------------------------------   |
|   Infrastructure Model       Fully Serverless (API-based)    Dedicated EC2 Endpoints           |
|   Operational Overhead       Lowest (No cluster management)  Medium (Instance right-sizing)    |
|   Model Selection            Claude, Llama, Titan, Mistral   Open-weights (Llama, Falcon, Mistral) |
|   Customization Method       Supervised FT, Pre-training     PEFT/LoRA, Full Fine-Tuning       |
|   RAG Implementation         Managed Knowledge Bases         Self-managed LangChain / LlamaIndex|
|   Billing Model              Per-token / Provisioned Units   Hourly instance billing ($/hr)    |
|   VPC / Network Isolation    PrivateLink endpoints           Full VPC ENI native isolation     |
|   Inference Latency Guarantee Provisioned Throughput (PMUs)  Dedicated GPU instance throughput |
+------------------------------------------------------------------------------------------------+

[!IMPORTANT] MLA-C01 Exam Decision Framework:

1. Choose Amazon Bedrock when the requirement specifies serverless operation, minimal operational overhead, accessing proprietary frontier models like Anthropic Claude, or requiring a managed RAG pipeline (Knowledge Bases) with OpenSearch Serverless.
2. Choose SageMaker JumpStart when the requirement requires dedicated GPU instances, deploying open-weights models (e.g., Llama 3) in a fully private VPC subnet with network isolation, or running custom LoRA fine-tuning using SageMaker Training jobs.