5.1 SageMaker Built-in Algorithms & Selection Criteria

5.1 SageMaker Built-in Algorithms & Selection Criteria

Amazon SageMaker provides a suite of pre-packaged, highly optimized machine learning algorithms designed to scale seamlessly across multi-core CPUs and multi-GPU distributed clusters. These built-in algorithms are pre-compiled into optimized Docker container images managed entirely by AWS, allowing ML engineers to train and deploy production models without writing custom model architectures or low-level framework code.

On the AWS Certified Machine Learning Engineer — Associate (MLA-C01) exam, you must demonstrate a deep understanding of algorithm selection criteria based on data modality (tabular, text, image, time series), problem formulation (supervised, unsupervised, anomaly detection), dataset sparsity, required input formats, and hardware acceleration profiles.

+------------------------------------------------------------------------------------------------+
|                       SAGEMAKER BUILT-IN ALGORITHMS TAXONOMY                                   |
|                                                                                                |
|   +-----------------------+   +-----------------------+   +--------------------------------+   |
|   |  SUPERVISED TABULAR   |   |   SUPERVISED TEXT     |   |       SUPERVISED VISION        |   |
|   |  - XGBoost            |   |  - BlazingText        |   |  - Image Classification        |   |
|   |  - Linear Learner     |   |    (Word2Vec/Classif) |   |    (ResNet backbone)           |   |
|   |  - Factorization Mach |   |  - Seq2Seq            |   |  - Object Detection (SSD)      |   |
|   |  - K-Nearest Neighbors|   |    (Attention RNN)    |   |  - Semantic Segmentation       |   |
|   +-----------------------+   +-----------------------+   +--------------------------------+   |
|                                                                                                |
|   +-----------------------+   +-----------------------+   +--------------------------------+   |
|   |  UNSUPERVISED/CLUST   |   |   ANOMALY DETECTION   |   |      TIME-SERIES FORECAST      |   |
|   |  - K-Means Clustering |   |  - Random Cut Forest  |   |  - DeepAR Forecasting          |   |
|   |  - PCA (Dim Reduction)|   |    (RCF Anomaly Tree) |   |    (Autoregressive Prob RNN)   |   |
|   |  - LDA / NTM (Topics) |   |  - IP Insights (IPv4) |   |                                |   |
|   +-----------------------+   +-----------------------+   +--------------------------------+   |
+------------------------------------------------------------------------------------------------+

---

1. Supervised Tabular Algorithms

Tabular datasets represent the vast majority of enterprise machine learning workloads. SageMaker provides four specialized built-in algorithms for tabular data, each optimized for distinct data characteristics and structural complexities.

+------------------------------------------------------------------------------------------------+
|                       SUPERVISED TABULAR ALGORITHMS COMPARISON                                 |
|                                                                                                |
|   Algorithm         Best Suited For                   Key Strengths           Input Formats    |
|   ---------------   -------------------------------   ---------------------   --------------   |
|   XGBoost           Non-linear tabular classification Gradient boosted trees, CSV, Parquet,    |
|                     & regression; tabular default     handles missing data    RecordIO-protobuf|
|                                                                                                |
|   Linear Learner    High-speed linear regression,     L1/L2 Elastic Net, auto CSV,             |
|                     binary/multi-class classification class-weight balancing  RecordIO-protobuf|
|                                                                                                |
|   Factorization     Sparse high-dimensional data,     Pairwise 2nd-order      RecordIO-protobuf|
|   Machines (FM)     click-through rate, recommenders  feature interactions    (Float32 only)   |
|                                                                                                |
|   K-Nearest         Non-parametric instance-based     Fast index search,      CSV,             |
|   Neighbors (KNN)   classification & regression       Euclidean/Cosine dist   RecordIO-protobuf|
+------------------------------------------------------------------------------------------------+

1.1 XGBoost (Extreme Gradient Boosting)

• Underlying Mechanism: An optimized distributed gradient boosting library implementing tree-based ensemble learning. New decision trees are sequentially added to predict and minimize the residual errors of prior trees using a second-order Taylor approximation of the loss function.
• Problem Modalities:
  • Binary Classification (binary:logistic, binary:hinge)
  • Multi-class Classification (multi:softmax, multi:softprob)
  • Regression (reg:squarederror, reg:squaredlogerror)
  • Ranking (rank:pairwise, rank:ndcg)
• Key Features:
  • Missing Value Handling: Natively accommodates missing values in feature columns without requiring explicit imputation; learns optimal default branch routing during split training.
  • Feature Importance: Generates Gain, Weight, and Coverage metrics for model interpretability.
  • Data Formats: Supports CSV (first column must be the target label, no header row), Apache Parquet (supported in SageMaker XGBoost versions 1.2+), and RecordIO-protobuf.
• Critical Hyperparameters:
  • max_depth: Maximum depth of a tree (controls model complexity; default: 6). Higher values lead to overfitting.
  • eta (learning rate): Step size shrinkage applied to update weights to prevent overfitting (range: 0.0–1.0; default: 0.3).
  • gamma (min_split_loss): Minimum loss reduction required to make a further partition on a leaf node.
  • min_child_weight: Minimum sum of instance weight (hessian) needed in a child node.
  • subsample & colsample_bytree: Subsampling ratio of training instances and feature columns per tree to prevent co-adaptation.
  • scale_pos_weight: Controls the balance of positive and negative weights for imbalanced binary classification.

1.2 Linear Learner

• Underlying Mechanism: Trains linear models using distributed Stochastic Gradient Descent (SGD) with automated hyperparameter optimization across multiple loss functions and learning rates simultaneously.
• Problem Modalities: Binary classification (logistic regression), multi-class classification (softmax regression), and continuous regression (linear regression, absolute loss, Huber loss).
• Key Features:
  • Automatic Loss & Optimizer Tuning: Trains multiple candidate models in parallel with varying learning rates and regularization penalties, automatically selecting the optimal checkpoint on validation data.
  • Class Imbalance Mitigation: Provides positive_example_weight_mult to weight positive samples higher, or optimizes directly for binary classification metrics via binary_classifier_model_selection_criteria (e.g., precision_at_target_recall, accuracy, f1).
  • Regularization: Built-in L1 (Lasso) and L2 (Ridge) penalties, combining into Elastic Net regularization to handle multicollinearity and drive sparse feature selection.
• Data Formats: CSV (label in column 0) and RecordIO-protobuf (optimized for Pipe Mode streaming).

1.3 Factorization Machines (FM)

• Underlying Mechanism: An extension of linear models that captures all pairwise second-order feature interactions using factorized vector dot products ($v_i \cdot v_j$), maintaining linear computational complexity $\mathcal{O}(k \cdot d)$ even in massive feature spaces.
• Problem Modalities: Binary classification and regression only. Does NOT support multi-class classification.
• Primary Use Cases:
  • Click-Through Rate (CTR) Prediction: Modeling user-ad click probabilities where categorical IDs generate high-cardinality one-hot encodings.
  • Recommender Systems: Handling extremely sparse user-item interaction matrices (e.g., millions of users $\times$ hundreds of thousands of catalog items).
• Data Requirements: RecordIO-protobuf with Float32 tensors only. CSV format is not supported for training Factorization Machines in SageMaker.

1.4 K-Nearest Neighbors (KNN)

• Underlying Mechanism: A non-parametric, distance-based supervised algorithm that stores training instances and predicts labels based on the $k$ closest data points in the vector space.
• Problem Modalities: Classification (majority voting) and Regression (average of $k$ nearest neighbors).
• Scaling & Dimensionality Reduction:
  • SageMaker KNN scales to large datasets by constructing an internal index structure (index_type options: INDEX_FLAT for exact brute-force search, INDEX_EQUALITY for quantized search).
  • Supports dimension reduction via random projection before index construction (dimension_reduction_type = sign or matrix).
• Distance Metrics: Euclidean ($L_2$), Cosine similarity, and Inner Product.

---

2. Supervised Text and Computer Vision Algorithms

+------------------------------------------------------------------------------------------------+
|                     SUPERVISED TEXT & VISION BUILT-IN ALGORITHMS                               |
|                                                                                                |
|   Algorithm         Modality   Backbone / Architecture    Primary Application                  |
|   ---------------   --------   -----------------------    ---------------------------------    |
|   BlazingText       Text/NLP   FastText / Word2Vec        Supervised text classification,      |
|                                (Skip-gram, CBOW)          word embeddings at 10x FastText speed|
|   Seq2Seq           Text/NLP   Encoder-Decoder RNN        Machine translation, text            |
|                                with Attention             summarization, sequence mapping      |
|   Image Classif.    Vision     ResNet-50 / 101 / 152      Multi-label & multi-class image tag  |
|   Object Detection  Vision     SSD (Single Shot Detector) Bounding box localization & class    |
|   Semantic Seg.     Vision     FCN / PSPNet / DeepLabV3   Pixel-level segmentation masks       |
+------------------------------------------------------------------------------------------------+

2.1 BlazingText

• Underlying Mechanism: Highly optimized GPU/CPU implementation of fastText and Word2Vec.
• Operational Modes:
  1. Supervised Text Classification: Classifies sentences, customer reviews, or support tickets into categorical labels. Achieves GPU acceleration with custom CUDA kernels, training up to 10x faster than standard fastText.
  2. Unsupervised Word Embeddings (Word2Vec): Generates dense vector representations of vocabulary tokens via:
     • skip_gram: Predicts context words given a target word (effective on smaller datasets with rare words).
     • cbow (Continuous Bag of Words): Predicts target word from surrounding context (faster, smooths over frequent words).
     • batch_skipgram: Distributed multi-GPU implementation of Skip-gram.
• Input Format: Newline-delimited UTF-8 text files. For supervised classification, each line must begin with the label prefix: __label__<label_name> <text_content>.

2.2 Sequence-to-Sequence (Seq2Seq)

• Underlying Mechanism: Supervised neural network utilizing a Recurrent Neural Network (RNN / LSTM / GRU) Encoder-Decoder architecture with attention mechanisms.
• Use Cases: Neural machine translation (e.g., English to German), text summarization, speech-to-text token transcription.
• Data Format: Tokenized integer sequences packaged into RecordIO-protobuf format.

2.3 Computer Vision Algorithms

1. Image Classification:

   • Predicts discrete category labels for whole images using a deep ResNet (Residual Network: ResNet-50, ResNet-101, ResNet-152) convolutional backbone.
   • Training Modes: Full training from scratch or Transfer Learning (loading weights pre-trained on ImageNet and fine-tuning top classification layers with smaller domain-specific datasets).
   • Data Formats: RecordIO (.rec format generated by im2rec.py) or raw image directory structure with .lst metadata annotation files.

2. Object Detection:

   • Detects, classifies, and draws bounding boxes around multiple target objects within an image using the Single Shot MultiBox Detector (SSD) framework with VGG or ResNet base feature extractors.
   • Data Formats: RecordIO or JSON format specifying bounding box coordinates [class_id, xmin, ymin, xmax, ymax].

3. Semantic Segmentation:

   • Assigns a semantic class label to every individual pixel in an input image (creating dense segmentation masks).
   • Architectures: Fully Convolutional Networks (FCN), Pyramid Scene Parsing Network (PSPNet), and DeepLabV3 with ResNet backbones.
   • Data Formats: RecordIO or raw PNG images paired with 8-bit PNG segmentation label masks.

---

3. Unsupervised, Topic Modeling & Anomaly Detection Algorithms

+------------------------------------------------------------------------------------------------+
|                 UNSUPERVISED, TOPIC MODELING & ANOMALY ALGORITHMS                              |
|                                                                                                |
|   Algorithm         Type               Mechanism                  Key Hyperparameters          |
|   ---------------   ----------------   ------------------------   ---------------------------  |
|   Random Cut        Anomaly            Ensemble of random trees   `num_trees`,                 |
|   Forest (RCF)      Detection          partitioning metric space  `num_samples_per_tree`,      |
|                                        (complexity spike = error) `shingle_size` (time series) |
|   IP Insights       Network Anomaly    Dual neural embeddings of  `vector_dim`,                |
|                     Detection          IPv4 subnets & entity IDs  `epochs`, `learning_rate`    |
|   K-Means           Clustering         Lloyd's algorithm /        `k`, `init_method`,          |
|                                        Mini-batch K-Means         `extra_center_factor`        |
|   PCA               Dimensionality     Linear Singular Value      `num_components`,            |
|                     Reduction          Decomposition (SVD)        `mode` (regular / randomized)|
|   NTM / LDA         Topic Modeling     NTM: Neural Autoencoder    `num_topics`,                |
|                                        LDA: Dirichlet Multinomial `feature_dim`                |
+------------------------------------------------------------------------------------------------+

3.1 Random Cut Forest (RCF)

• Underlying Mechanism: An unsupervised anomaly detection algorithm that constructs a collection (forest) of binary trees. Each tree is built by taking a random sample of training points and recursively cutting bounding boxes along a randomly selected dimension.
• Anomaly Scoring:
  • Anomaly score is proportional to the inverse depth of the point in the tree.
  • An anomalous point (outlier) falls in a sparse region of feature space; isolating it requires very few random cuts, placing it near the root of the tree.
  • Normal points reside in dense clusters, requiring many random cuts (deep in the tree).
  • The algorithm outputs an anomaly score where higher values (e.g., scores > 3 standard deviations above mean) indicate an anomaly.
• Sequential & Time-Series Anomaly Detection (shingle_size):
  • By setting the shingle_size hyperparameter (e.g., shingle_size=10), RCF transforms consecutive 1D time-series data points into a 10-dimensional vector (shingle).
  • This enables RCF to detect shape-based and frequency-based anomalies (such as sudden plateauing, periodicity breaks, or phase shifts) rather than simple numerical threshold spikes.

3.2 IP Insights

• Underlying Mechanism: An unsupervised anomaly detection algorithm specifically designed for network security and fraud prevention. It learns latent vector representations (embeddings) of IPv4 addresses and entity identifiers (e.g., user_id, account_number, device_id).
• IPv4 Subnet Awareness: Natively understands IPv4 hierarchical subnet structures (/24, /16 CIDR blocks), clustering adjacent IP blocks together.
• Inference Output: Given a pair of (entity_id, ip_address), IP Insights outputs an anomaly score indicating how atypical or suspicious the IP address is for that particular user entity.
• MLA-C01 Use Case: Detecting account takeovers, unauthorized VPN access, geo-location spoofing, or compromised API tokens based on historical login telemetry.

3.3 Principal Component Analysis (PCA) & K-Means

• PCA (Dimensionality Reduction): Computes orthogonal principal components capturing maximal dataset variance.
  • mode='regular': Full SVD, suitable for datasets with moderate feature counts.
  • mode='randomized': Approximation algorithm using randomized SVD, scalable to millions of rows and high feature dimensions.
• K-Means Clustering: Partitions observations into $K$ distinct clusters minimizing within-cluster sum of squared Euclidean distances.
  • Supports init_method='kmeans++' to space initial centroids apart, speeding up convergence and avoiding sub-optimal local minima.

3.4 Topic Modeling: LDA vs. NTM

• Latent Dirichlet Allocation (LDA): Traditional statistical, non-neural probabilistic model representing documents as mixtures of topics, and topics as distributions over words. Accepts Bag-of-Words (RecordIO or CSV) inputs. CPU-bound.
• Neural Topic Model (NTM): Deep learning-based topic modeling using a Variational Autoencoder (VAE). Accommodates both unsupervised topic discovery and supervised topic modeling with auxiliary metadata. Scalable across multi-GPU instances.

---

4. Time-Series Forecasting with DeepAR

Amazon SageMaker DeepAR is a supervised learning algorithm for forecasting scalar time series using autoregressive recurrent neural networks (RNNs).

+------------------------------------------------------------------------------------------------+
|                                DEEPAR FORECASTING ARCHITECTURE                                 |
|                                                                                                |
|   Input: Multiple Related Time Series (e.g., 10,000 Retail Store SKUs)                         |
|                                                                                                |
|   +-------------------+   +---------------------------------------+   +--------------------+   |
|   | Target Sequence   |   | Dynamic Features (e.g., Price, Promo) |   | Static Cat Features|   |
|   | [12, 15, 14, ...] |   | [0, 0, 1, 1, 0, ...]                  |   | [Store_ID, Cat_ID] |   |
|   +-------------------+   +---------------------------------------+   +--------------------+   |
|             \                             |                             /                      |
|              \                            |                            /                       |
|               v                           v                           v                        |
|   +----------------------------------------------------------------------------------------+   |
|   |              AUTOREGRESSIVE RECURRENT NEURAL NETWORK (DEEPAR GLOBAL MODEL)             |   |
|   +----------------------------------------------------------------------------------------+   |
|                                           |                                                    |
|                                           v                                                    |
|   Output: Probabilistic Forecast Distribution (Quantiles: p10, p50 median, p90)                |
+------------------------------------------------------------------------------------------------+

DeepAR Key Architectural Characteristics

1. Global Model across Multiple Related Time Series:
   • Unlike classical statistical models (ARIMA, Exponential Smoothing) that fit a separate model to each individual time series independently, DeepAR fits a single global RNN model across thousands of interrelated time series.
   • Cross-Series Learning: Learns complex seasonal patterns and promotional dynamics from high-volume series and applies that knowledge to cold-start or low-volume series.
2. Probabilistic Forecasting:
   • Generates probability distributions for future time steps rather than deterministic point estimates.
   • Outputs quantile forecasts (e.g., 0.10, 0.50, 0.90) to support risk-aware operational planning (e.g., ordering inventory at the 90th percentile to prevent stockouts).
3. Covariate Features Support:
   • Static Categorical Features (cat): Attributes that remain constant over time for a given series (e.g., store_id, product_category, brand_tier). DeepAR learns categorical embeddings for these IDs.
   • Dynamic Time-Dependent Features (dynamic_feat): Attributes that change over time and are known into the future (e.g., scheduled marketing promotions, price discounts, holiday calendar flags).
4. Data Format Requirements:
   • JSONLines format (application/jsonlines) or RecordIO-protobuf.
   • Each line is a standalone JSON record containing the required fields:

     {
       "start": "2026-01-01 00:00:00",
       "target": [25.4, 30.1, 28.0, 35.6, 42.1],
       "cat": [3, 12],
       "dynamic_feat": [[0, 0, 1, 1, 0], [10.5, 10.5, 9.99, 9.99, 10.5]]
     }
     

---

5. SageMaker Built-in Algorithm Selection Matrix

---

6. Configuring SageMaker Built-in Estimators via Python SDK

import boto3
import sagemaker
from sagemaker import image_uris
from sagemaker.inputs import TrainingInput

session = sagemaker.Session()
role = sagemaker.get_execution_role()
region = session.boto_region_name

# 1. Retrieve the official built-in container image URI
xgb_container = image_uris.retrieve(
    framework="xgboost",
    region=region,
    version="1.7-1"
)

# 2. Instantiate SageMaker Estimator
xgb_estimator = sagemaker.estimator.Estimator(
    image_uri=xgb_container,
    role=role,
    instance_count=1,
    instance_type="ml.m5.2xlarge",
    output_path="s3://ml-bucket-prod/xgboost-churn-output/",
    sagemaker_session=session
)

# 3. Set Hyperparameters
xgb_estimator.set_hyperparameters(
    max_depth=5,
    eta=0.2,
    gamma=4,
    min_child_weight=6,
    subsample=0.8,
    objective="binary:logistic",
    num_round=150,
    scale_pos_weight=3.5  # Balances 1:3.5 positive:negative class imbalance
)

# 4. Configure S3 Training and Validation Input Channels with FastFile Mode
train_input = TrainingInput(
    s3_data="s3://ml-bucket-prod/train/",
    content_type="text/csv",
    input_mode="FastFile"
)
val_input = TrainingInput(
    s3_data="s3://ml-bucket-prod/validation/",
    content_type="text/csv",
    input_mode="FastFile"
)

# 5. Execute Training Job
xgb_estimator.fit({"train": train_input, "validation": val_input})