Large-Scale Data Transfer and Landing-Zone Prerequisites

Key Takeaways

  • AWS DataSync is the online copy service for NFS, SMB, HDFS, and object stores into Amazon S3, Amazon EFS, and Amazon FSx; it is not an SFTP server for partners.
  • AWS Transfer Family provides managed SFTP, FTPS, FTP, and AS2 endpoints in front of S3 or EFS so factory partners keep their existing clients.
  • As of 2026, do not teach Snowcone or Snowmobile as orderable: Snowcone ended 12 November 2024 (support through 12 November 2025), Snowmobile shut down 14 March 2024, and Snowball Edge is existing-customer-only after 7 November 2025, with current models Snowball Edge Storage Optimized 210 TB and Compute Optimized 104 vCPU.
  • Amazon S3 Transfer Acceleration uses CloudFront edge locations and s3-accelerate endpoints for long-distance object uploads; bucket names cannot contain periods.
  • Stand up AWS Organizations, AWS Control Tower, IAM Identity Center, and Directory Service (or an external IdP) before a multi-account migration; then pick internet, Site-to-Site VPN, or Direct Connect based on duration, bandwidth, and encryption needs.
Last updated: September 2026

Match the protocol to the service

Task 4.2’s data-migration list is AWS DataSync, AWS Transfer Family, AWS Snow Family, and Amazon S3 Transfer Acceleration, plus network (Direct Connect, Site-to-Site VPN, Route 53), identity (IAM Identity Center, Directory Service), and governance (Control Tower, Organizations). Professional questions fail candidates who grab whichever service they studied last.

AWS DataSync is an online, agent-accelerated copy. Deploy a DataSync agent as a VM next to NFS or SMB shares, as an HDFS client, or against self-managed/cloud object APIs. DataSync uses a transfer protocol designed by AWS (incremental copy, compression, sparse-file detection, in-line validation and encryption). A single task can saturate about 10 Gbps. Destinations include Amazon S3, Amazon EFS, and Amazon FSx family file systems. Metadata and permissions are preserved on supported pairs. Tasks resume incrementally if the WAN drops. You can use the public service endpoints or VPC endpoints so traffic never uses public IPs. Direct Connect is a common pairing when the project is multi-week and the link is already (or can be) procured from a Direct Connect partner for the migration window. DataSync is not an SFTP listener; partners cannot “point FileZilla at DataSync.”

AWS Transfer Family is the opposite shape: a managed server endpoint so existing clients keep working. Enable SFTP, FTPS, FTP, and/or AS2. Back the endpoint with S3 or EFS. Identity can be service-managed users, Microsoft Active Directory, or a custom identity provider (Okta, Microsoft Entra ID, or a Lambda-backed directory). FTP is VPC-only; if partners on the internet need FTP plus SFTP, that is a separate server endpoint. AS2 is the B2B EDI-style path many factories still run. Transfer Family is how you retire a self-managed sshd farm without rewriting partner jobs.

Amazon S3 Transfer Acceleration is a bucket feature, not a file-transfer protocol. Clients PUT/GET via bucket-name.s3-accelerate.amazonaws.com (or the dual-stack s3-accelerate.dualstack.amazonaws.com for IPv6). Data enters a CloudFront edge and rides the AWS backbone to the bucket. Requirements: DNS-compliant bucket name with no periods, acceleration enabled, virtual-hosted–style requests. Benefit can take up to 20 minutes to show; the accelerate endpoint is usable immediately. It does not support ListBuckets, CreateBucket, DeleteBucket, or cross-Region CopyObject. Use it when clients worldwide upload into one general-purpose bucket, or when you cannot fill your internet pipe toward the bucket’s Region. Use the Speed Comparison tool when the client is already next door to the Region — acceleration can be slower. Transfer Acceleration is not an SFTP server and does not replace Direct Connect for a 50 TB database CDC stream.

Snow Family: current names only

Exam stems still say “Snow Family.” Teach what is actually current as of this lastUpdated date, and do not resurrect retired SKUs.

AWS Snowmobile is in AWS’s full-shutdown list effective 14 March 2024. Do not design with a 100 PB truck.

AWS Snowcone (SSD and HDD) was discontinued 12 November 2024; AWS supported existing Snowcone jobs only through 12 November 2025. Do not order Snowcone in a 2026 design.

Previous-generation Snowball Edge Storage Optimized 80 TB, Compute Optimized 52 vCPU, and Compute Optimized with GPU stopped taking orders 12 November 2024, with support through 12 November 2025.

Devices AWS documented as the remaining Snowball Edge generation are:

Device (current generation)Role
Snowball Edge Storage Optimized 210 TB (NVMe)Bulk offline import/export; AWS cited transfer up to about 1.5 GB/s onto the device
Snowball Edge Compute Optimized (104 vCPU, up to 416 GB customer-usable RAM, 28 TB NVMe)Edge compute (EC2-compatible instances, S3-compatible storage, AWS IoT Greengrass, Lambda) plus data movement

Effective 7 November 2025, Snowball Edge is available only to existing Snow customers. New customers are directed to DataSync (online), AWS Data Transfer Terminal (bring your own storage devices to a reserved, private AWS facility and upload over a fast on-site path; AWS documents Enterprise Support as the current access gate), AWS Marketplace offline partners (examples AWS names include Seagate and Tsecond), and AWS Outposts for edge compute (racks or 2U servers, including DDIL operation for a limited number of days).

A disconnected mine or factory with tens of terabytes therefore branches:

  1. If the AWS account can still order Snowball Edge, pick Storage Optimized 210 TB for bulk files (80 TB fits on one current storage device with headroom) or Compute Optimized if you must process at the edge before shipping.
  2. If the account cannot order Snow, schedule Data Transfer Terminal (when the support tier allows) or a Marketplace sneakernet, or wait until a WAN exists for DataSync.
  3. Do not propose MGN replication to a site with no 443/1500 path.
  4. Do not propose Snowcone or Snowmobile.

Internet versus VPN versus Direct Connect

Use duration and change rate, not habit.

PathStrengthMigration implication
InternetFast to start; TLS on service APIs; S3 Transfer Acceleration helps long-haul object PUTFine for modest data, agent control planes, and bursts; 50 TB at 100 Mbps is ~45–50 days
Site-to-Site VPNIPsec into a VPC in days; encrypted by defaultGood bootstrap and ongoing management; throughput is typically far below a 10 Gbps DataSync task
Direct ConnectDedicated 1 / 10 / 100 Gbps classes, consistent latency, better data-transfer economics at scaleBest for multi-week MGN + DMS + DataSync factories; encrypt with MACsec or a VPN overlay if the stem requires in-transit encryption on the private interconnect; lead time is the hidden cost

Route 53 is the cutover lever after bytes have moved: weighted or failover records from on-premises to Elastic Load Balancing or Aurora endpoints. Hybrid DNS (Resolver inbound/outbound endpoints) belongs with the networking chapter; here, remember that a data copy without a DNS plan leaves users on the source.

Identity and Directory Service during migration

Do not create standing IAM users in every migrated account as the enterprise model. AWS IAM Identity Center (in the Organizations management account) is the landing-zone access plane: permission sets onto member accounts, optional SCIM from an external IdP, or Active Directory as the identity source. Control Tower’s default landing zone can create an Identity Center directory and groups; if Identity Center already exists, Control Tower’s home Region must match that Identity Center Region (with the documented us-east-1 exception).

AWS Directory Service covers directory-aware workloads:

  • AWS Managed Microsoft AD runs Windows Server–based domain controllers as a managed pair across Availability Zones. Use it when migrated EC2, RDS SQL Server Windows authentication, or other AD-aware software must domain-join in AWS. Resource-forest trusts keep identities in the on-premises domain.
  • AD Connector is a gateway with no cloud copy of identities. Use it when AD stays on-premises and AWS services only need to bind against it.

MGN test instances that join AD must not share a live site with production computer accounts unless you intend that collision. Directory design is part of the test-subnet story, not an afterthought.

Control Tower and Organizations as prerequisites

A 400-server plus 50 TB database program that “starts replicating into the management account this week” is already a failed architecture. AWS Organizations (all features) and an AWS Control Tower landing zone give you the management account, a security OU with log-archive and audit accounts, workload OUs, guardrails, and Account Factory for the accounts MGN and DMS will target. You get one landing zone per organization. Enroll accounts before you point replication at them so CloudTrail, Config, and SCPs exist on day one of test launches.

Sequence the factory SFTP story the same way: Transfer Family in a member account whose S3 buckets are already under the landing-zone logging design, AD or Identity Center already chosen, and Direct Connect or internet allow-lists already aimed at the Transfer Family endpoint. Partners should not authenticate to a sandbox that will be deleted at cutover.

Exam traps: DataSync as SFTP; Transfer Family as NFS copy; Snowmobile or Snowcone in a 2026 bill of materials; Snowball Edge for a brand-new AWS customer after the 7 November 2025 change without naming Data Transfer Terminal or partners; Transfer Acceleration with a dotted bucket name; VPN as the only path for a 10 Gbps DataSync task; skipping Control Tower until after cutover; IAM users instead of Identity Center permission sets.

Approximate days to move 50 TB on a dedicated link (no protocol overhead)
Test Your Knowledge

A manufacturer must keep hundreds of factory partners on their existing SFTP clients, depositing production files into Amazon S3, without rewriting partner software. Which service should the architect choose?

A
B
C
D
Test Your Knowledge

A mine site has no usable WAN and must ingest about 80 TB of files into AWS, then process them in the Region. The company already uses AWS. Which physical-transfer statement is accurate for 2026?

A
B
C
D
Test Your Knowledge

Leadership wants to start replicating 400 servers and a 50 TB Oracle database into a single shared account over the public internet this week, then “add governance later.” What should the architect recommend instead?

A
B
C
D