13.2 Network Isolation, VPC Endpoints & Data Encryption

Network Isolation, VPC Endpoints & Data Encryption

Securing machine learning workloads against data exfiltration, network eavesdropping, and unauthorized access requires a defense-in-depth architecture. In financial services, healthcare, defense, and enterprise environments, training datasets containing sensitive Personally Identifiable Information (PII) or Protected Health Information (PHI) must never traverse the public internet or be exposed to unmonitored external network connections.

On the AWS Certified Machine Learning Engineer — Associate (MLA-C01) exam, you must demonstrate comprehensive expertise in configuring Amazon Virtual Private Cloud (VPC) network topology for SageMaker, implementing container network isolation, provisioning AWS PrivateLink VPC endpoints, establishing VPC endpoint policies, and enforcing end-to-end encryption across both data at rest and data in transit.

---

1. SageMaker Network Isolation Architecture

Amazon SageMaker supports three distinct networking modes for running training jobs, processing jobs, and hosting real-time inference endpoints:

+--------------------------------------------------------------------------------------------------+
|                             SAGEMAKER NETWORK ARCHITECTURE MODES                                 |
|                                                                                                  |
|   1. DEFAULT MODE (No VPC Attached)                                                              |
|      - Compute runs in AWS-managed service VPC.                                                  |
|      - Internet access enabled via AWS default routes.                                           |
|      - Exfiltration risk: Container can make outbound HTTP/HTTPS requests to external IPs.       |
|                                                                                                  |
|   2. VPC MODE (VpcConfig Attached to Customer Subnets)                                           |
|      - SageMaker provisions Elastic Network Interfaces (ENIs) in customer private subnets.       |
|      - Outbound traffic controlled strictly by customer Security Groups and Route Tables.        |
|      - Communication with S3, ECR, and CloudWatch flows through customer VPC Endpoints or NAT.   |
|                                                                                                  |
|   3. NETWORK ISOLATION MODE (EnableNetworkIsolation = True)                                      |
|      - Network stack inside the container is completely disabled (no eth0/network interfaces).   |
|      - Container cannot make ANY inbound or outbound network calls (no internet, no VPC).        |
|      - Data transfer occurs strictly via filesystem mounts (/opt/ml/input and /opt/ml/output).    |
+--------------------------------------------------------------------------------------------------+

Comparing Network Modes

Mechanics of EnableNetworkIsolation=True

When EnableNetworkIsolation is set to True during CreateTrainingJob or CreateModel:

1. SageMaker launches the Docker container with all networking capabilities turned off (docker run --network none).
2. The training code running inside the container cannot initiate any socket connections, DNS resolutions, or HTTP calls.
3. Before container startup, the SageMaker control plane downloads the S3 training data onto the host EBS volume and mounts it into the container at /opt/ml/input/data/<channel_name>.
4. When training completes, the SageMaker control plane archives everything written to /opt/ml/model and /opt/ml/output and uploads it to Amazon S3.
5. This provides a mathematical guarantee that custom algorithm code or third-party open-source libraries cannot transmit proprietary data to unauthorized external servers.

---

2. AWS PrivateLink & VPC Endpoints for ML

When running SageMaker inside a private VPC with no Internet Gateway and no NAT Gateway (air-gapped private subnet), all communication with AWS services must flow through VPC Endpoints.

+--------------------------------------------------------------------------------------------------+
|                             PRIVATE VPC ENDPOINT TOPOLOGY FOR SAGEMAKER                          |
|                                                                                                  |
|   +------------------------------------------------------------------------------------------+   |
|   | CUSTOMER VIRTUAL PRIVATE CLOUD (VPC: 10.0.0.0/16)                                        |   |
|   |                                                                                          |   |
|   |   +----------------------------------------------------------------------------------+   |   |
|   |   | PRIVATE SUBNET (10.0.1.0/24 - No IGW / No NAT Gateway)                           |   |   |
|   |   |                                                                                  |   |   |
|   |   |   +--------------------------+          +------------------------------------+   |   |   |
|   |   |   | SageMaker Training Node  |          | Interface VPC Endpoints (ENIs)     |   |   |   |
|   |   |   | Elastic Network IF (ENI) | -------- | - sagemaker.api (Management API)   |   |   |   |
|   |   |   +--------------------------+          | - sagemaker.runtime (Invocations)  |   |   |   |
|   |   |                |                        | - ecr.api & ecr.dkr (Container DL) |   |   |   |
|   |   |                |                        | - logs (CloudWatch Log Streams)    |   |   |   |
|   |   |                |                        | - kms (Cryptographic Key Ops)      |   |   |   |
|   |   |                |                        +------------------------------------+   |   |   |
|   |   |                |                                                                 |   |   |
|   |   |                v                                                                 |   |   |
|   |   |   +--------------------------------------------------------------------------+   |   |   |
|   |   |   | VPC ROUTE TABLE: Routes 0.0.0.0/0 = Local | pl-12345678 (S3 Prefix) -> vpce  |   |   |   |
|   |   |   +--------------------------------------------------------------------------+   |   |   |
|   |   +----------------------------------------------------------------------------------+   |
|   +------------------------------------------------------------------------------------------+   |
|                                |                                                                 |
|                                v Gateway VPC Endpoint (Free / Route Table Target)                |
|   +------------------------------------------------------------------------------------------+   |
|   | AMAZON S3 (Training Datasets, Checkpoints, Model Artifacts)                              |   |
|   +------------------------------------------------------------------------------------------+   |
+--------------------------------------------------------------------------------------------------+

Required VPC Endpoints by Service

1. Amazon S3 (Gateway Endpoint):
   • Type: Gateway VPC Endpoint (Configured in VPC route tables; free of charge).
   • Purpose: High-throughput streaming and downloading of training datasets, feature baselines, and model artifacts.
2. SageMaker API & Runtime (Interface Endpoints - PrivateLink):
   • com.amazonaws.<region>.sagemaker.api: Management API calls (CreateTrainingJob, DescribeModel, CreateEndpoint).
   • com.amazonaws.<region>.sagemaker.runtime: Real-time endpoint invocations (InvokeEndpoint).
   • com.amazonaws.<region>.sagemaker.featurestore-runtime: Low-latency online feature retrieval (GetRecord, PutRecord).
3. Amazon ECR (Interface Endpoints - PrivateLink):
   • com.amazonaws.<region>.ecr.api: ECR API operations (authentication and image metadata).
   • com.amazonaws.<region>.ecr.dkr: Docker image registry data transfer (pulling image layers).
4. Amazon CloudWatch Logs (Interface Endpoint - PrivateLink):
   • com.amazonaws.<region>.logs: Streaming training metrics, standard error, and container stdout.
5. AWS KMS (Interface Endpoint - PrivateLink):
   • com.amazonaws.<region>.kms: Decrypting S3 datasets and generating data keys for EBS volume encryption.

VPC Endpoint Policies & S3 Perimeter Security

To ensure that traffic from your private subnets can only access authorized corporate resources (and prevent malicious actors from exfiltrating data to external S3 buckets using your VPC endpoints), attach a VPC Endpoint Policy to the S3 Gateway Endpoint:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "RestrictToCorporateBucketsOnly",
      "Effect": "Allow",
      "Principal": "*",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::corp-training-datasets",
        "arn:aws:s3:::corp-training-datasets/*",
        "arn:aws:s3:::corp-model-artifacts",
        "arn:aws:s3:::corp-model-artifacts/*"
      ]
    }
  ]
}

Complement this with an S3 Bucket Policy enforcing that data can only be accessed from your specific VPC Endpoint ID (aws:sourceVpce):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyAccessOutsideDesignatedVPCEndpoint",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::corp-training-datasets",
        "arn:aws:s3:::corp-training-datasets/*"
      ],
      "Condition": {
        "StringNotEquals": {
          "aws:sourceVpce": "vpce-0123456789abcdef0"
        }
      }
    }
  ]
}

---

3. Comprehensive Data Encryption Framework

An enterprise ML security architecture requires robust encryption across both data at rest and data in transit.

+--------------------------------------------------------------------------------------------------+
|                              DATA ENCRYPTION ARCHITECTURAL SPECTRUM                              |
|                                                                                                  |
|   DATA AT REST                                                                                   |
|   - Amazon S3 Datasets & Models: SSE-S3 (AES-256) or SSE-KMS with Customer Managed Keys (CMKs)   |
|   - SageMaker EBS Storage Volumes: Encrypted with KMS (VolumeKmsKeyId)                           |
|   - SageMaker Studio Home Storage: Amazon EFS user directories encrypted at rest with KMS       |
|   - SageMaker Feature Store: Encrypted at rest (Online Store via KMS, Offline Store via S3 KMS)  |
|                                                                                                  |
|   DATA IN TRANSIT                                                                                |
|   - External API & Invocation Requests: Enforced TLS 1.2+ HTTPS                                  |
|   - Inter-Container Distributed Training: EnableInterContainerTrafficEncryption = True           |
|     (Secures node-to-node gradient synchronization across cluster instances via TLS)            |
+--------------------------------------------------------------------------------------------------+

3.1 Data at Rest Encryption

1. Amazon S3 Storage:
   • SSE-S3: Managed by Amazon S3 using AES-256 encryption. Zero additional cost, but lacks granular key access auditing.
   • SSE-KMS (Customer Managed Key): Grants complete control over key rotation, IAM delegation, and AWS CloudTrail audit logs.
   • S3 Bucket Keys: Enabling S3 Bucket Keys reduces KMS API request traffic from SageMaker by up to 99%, dramatically cutting KMS request costs during distributed training.
2. EBS Volume Encryption for Compute Nodes:
   • Attached EBS volumes store temporary dataset downloads, local checkpoints, and container swap files.
   • By default, EBS volumes attached to SageMaker training and endpoint instances are encrypted with an AWS-managed key. Specify VolumeKmsKeyId in CreateTrainingJob or CreateEndpointConfig to enforce encryption with a dedicated CMK.

3.2 Data in Transit & Inter-Container Encryption

In large distributed multi-node training clusters (such as training foundation models or large vision models across 8x ml.p4de.24xlarge instances):

• Training instances continuously exchange multi-gigabyte gradient tensors and parameter updates over the cluster interconnect fabric (such as Elastic Fabric Adapter - EFA).
• Under standard configuration, this inter-node communication occurs over unencrypted internal network connections.
• Setting EnableInterContainerTrafficEncryption = True (or enable_inter_container_traffic_encryption=True in the SageMaker Python SDK Estimator) instructs SageMaker to generate an ephemeral self-signed TLS certificate for the cluster and encrypt all inter-node traffic across the network.
• Performance Impact: Inter-container encryption introduces a minor compute overhead (typically 5–10% on training throughput), but is strictly required for compliance in PCI-DSS, HIPAA, and military environments.

---

4. Python SDK Implementation: Secure VPC, KMS & Isolated Training

import sagemaker
from sagemaker.estimator import Estimator

session = sagemaker.Session()
role = "arn:aws:iam::123456789012:role/SageMaker-Secure-ExecutionRole"

# Secure Estimator Configuration
secure_estimator = Estimator(
    image_uri="123456789012.dkr.ecr.us-east-1.amazonaws.com/custom-xgboost:latest",
    role=role,
    instance_count=4,
    instance_type="ml.m5.2xlarge",
    output_path="s3://corp-model-artifacts/credit-risk/output/",
    sagemaker_session=session,
    
    # 1. Volume Encryption with Customer Managed Key
    volume_kms_key="arn:aws:kms:us-east-1:123456789012:key/cmk-ebs-ml-key",
    
    # 2. Output Artifact KMS Encryption
    output_kms_key="arn:aws:kms:us-east-1:123456789012:key/cmk-s3-ml-key",
    
    # 3. Customer VPC Subnet and Security Group Attachment
    subnets=["subnet-0a1b2c3d4e5f6g7h8", "subnet-0b2c3d4e5f6g7h8i9"],
    security_group_ids=["sg-0123456789abcdef0"],
    
    # 4. Total Container Network Isolation (No external socket calls)
    enable_network_isolation=True,
    
    # 5. Distributed Multi-Node Inter-Container Traffic Encryption
    encrypt_inter_container_traffic=True
)

[!TIP] Exam Rapid Decision Rules:

• If the question requires preventing training/inference containers from making any outbound network calls or exfiltrating data $\rightarrow$ Set EnableNetworkIsolation=True.
• If the question asks how to run SageMaker in a private VPC with no internet access $\rightarrow$ Provision a Gateway VPC Endpoint for S3 and Interface VPC Endpoints for ECR (ecr.api, ecr.dkr), SageMaker (sagemaker.api), and CloudWatch (logs).
• If the question mandates encrypting inter-node gradient exchanges during distributed training $\rightarrow$ Enable EnableInterContainerTrafficEncryption=True.
• If the question asks how to restrict S3 bucket access strictly to queries from a private VPC $\rightarrow$ Use an S3 Bucket Policy with Condition: { "StringNotEquals": { "aws:sourceVpce": "<vpce-id>" } }.