5.2 Route 53 Health Checks & DNS Failover

5.2 Route 53 Health Checks & DNS Failover

CloudOps Blueprint Focus: AWS Certified CloudOps Engineer – Associate (SOA-C03) candidates must design, configure, and maintain high-availability DNS routing using Amazon Route 53. This encompasses mastering endpoint, calculated, and CloudWatch-driven health checks, configuring automated Active-Passive and Active-Active DNS failover policies, and leveraging Route 53 Alias records for zone apex routing and health check inheritance.

Route 53 Health Check Architecture & Mechanics

Amazon Route 53 operates a globally distributed network of health checkers stationed outside the customer VPC across approximately 15 AWS edge locations worldwide. These checkers monitor the availability and performance of application endpoints, updating Route 53 DNS routing tables to automatically bypass failing infrastructure.

+-----------------------------------------------------------------------------------------+
| ROUTE 53 GLOBAL HEALTH CHECKERS (Edge Locations Worldwide)                               |
|   - Location 1 (N. Virginia)    - Location 2 (Frankfurt)    - Location 3 (Tokyo)        |
|   - Location 4 (Oregon)         - Location 5 (Dublin)       - Location 6 (Sydney)       |
+-----------------------------------------------------------------------------------------+
                                             |
                        [ Standard Probe (30s) / Fast Probe (10s) ]
                                             |
                                             v
                         +---------------------------------------+
                         | Target Endpoint: https://app.corp.com |
                         +---------------------------------------+

Route 53 provides three primary categories of health checks:

1. Endpoint Health Checks: Directly probes a specified public IP address or fully qualified domain name (FQDN) over HTTP, HTTPS, or TCP.
   • Request Interval: Defaults to Standard (30 seconds). For faster failure detection and failover, operations teams configure Fast (10 seconds) intervals, which sends probes three times more frequently at an additional monthly fee.
   • Failure Threshold: The number of consecutive failed checks required before Route 53 considers the endpoint UNHEALTHY (range: 1–10; default: 3). With a standard 30s interval and a threshold of 3, failover initiates within approximately 90 seconds. With a fast 10s interval, failover initiates within 30 seconds.
   • Health Checker Consensus: An endpoint is deemed healthy when more than 18% of global health checkers report a healthy status (a configurable threshold, defaulting to a majority consensus).
   • Inverted Health Checks: Inverts the health check evaluation logic (Inverted: true). The check reports healthy when the target fails and unhealthy when the target succeeds. This is used for maintenance window flags or dead-man-switch monitoring.
2. Calculated Health Checks: Monitors the status of up to 256 child health checks and combines them using boolean logic (AND, OR, NOT) or a minimum healthy threshold ($N$ of $M$).
   • Operational Scenario: A regional web platform consists of an authentication service, a product catalog, and a payment processing backend. A calculated health check monitors all three child endpoint checks with an AND rule, or requires at least 2 of 3 to be healthy. This prevents minor, isolated microservice flaps from triggering an abrupt and costly regional DNS failover.
3. CloudWatch Alarm Health Checks: Monitors the operational state of a designated Amazon CloudWatch metric alarm (OK, ALARM, INSUFFICIENT_DATA).
   • The Private VPC Boundary: Because Route 53 endpoint health checkers reside on the public internet, they cannot directly probe private IP addresses or internal AWS resources located inside a VPC (such as internal Application Load Balancers, private EC2 fleets, or Amazon RDS databases).
   • To perform DNS failover for private internal systems, CloudOps engineers create a CloudWatch Alarm monitoring internal metrics (e.g., ALB HTTPCode_Target_5XX_Count or HealthyHostCount). Route 53 binds a CloudWatch Alarm health check to that alarm, transitioning the DNS record status whenever the CloudWatch alarm fires.

---

Deep Payload Inspection: String Matching & Latency Checks

String Matching Health Checks

Standard HTTP/HTTPS endpoint checks only verify that the web server returns an HTTP status code between 200 and 399. However, a crashed application server or failing database connection pool may still return an HTTP 200 OK while rendering a user-facing error message (such as "Database Connection Error").

To detect application-level faults, Route 53 supports String Matching:

• The health checker requests the endpoint and parses the response body for a specific, case-sensitive string (e.g., "STATUS_OPERATIONAL").
• The 5,120-Byte (5 KB) Hard Limit: Route 53 inspects only the first 5,120 bytes of the HTTP/HTTPS response body. If the search string appears at byte 5,121 or later, the health check fails and marks the endpoint UNHEALTHY, even if the HTTP status code is 200.
• Operational Best Practice: Always create a dedicated, lightweight diagnostic endpoint (e.g., /healthz) that returns a minimal JSON payload (e.g., {"status":"OK"}) of less than 500 bytes. Never point string matching checks to full HTML homepages where dynamic marketing banners, tracking scripts, or CSS additions can push the health token past byte 5,120.

Latency Measurement Checks

Route 53 health checks can capture connection latency metrics from global edge locations to the target endpoint. These metrics (ConnectionTime, TimeToFirstByte, SSLHandshakeTime) are published directly to Amazon CloudWatch under the AWS/Route53 namespace, allowing operations teams to establish alarms on regional network degradation before complete outages occur.

---

DNS Failover Topologies: Active-Passive vs. Active-Active

Route 53 automates traffic redirection during outages using failover routing policies:

                     Active-Passive Failover Architecture
                                [ DNS Query ]
                                      |
                                      v
                         [ Route 53 Hosted Zone ]
                                      |
             +------------------------+------------------------+
             | (Primary Healthy)                               | (Primary Unhealthy)
             v                                                 v
   [ Primary Record ]                                 [ Secondary Record ]
   (Role: PRIMARY)                                    (Role: SECONDARY)
   (Health Check Attached)                            (Static S3 / DR Site)
             |
             v
[ Production ALB Fleet ]

Active-Passive Failover

In an Active-Passive architecture, primary infrastructure handles 100% of production traffic under normal conditions, while secondary infrastructure remains in standby (e.g., a warm standby in a secondary Region or a static error page hosted in an Amazon S3 bucket):

• Primary Record: Configured with Routing Policy: Failover, Record Role: PRIMARY, and associated with a Route 53 health check targeting production.
• Secondary Record: Configured with Routing Policy: Failover, Record Role: SECONDARY, pointing to the disaster recovery endpoint.
• Operational Execution: While the primary health check is HEALTHY, Route 53 returns the primary endpoint in all DNS query responses. If the primary health check transitions to UNHEALTHY, Route 53 automatically suppresses the primary record and serves the secondary record IP/hostname to all subsequent DNS queries.

Active-Active Failover

In an Active-Active architecture, multiple geographically dispersed endpoints (or multiple load balancers) actively process production traffic simultaneously:

• Active-Active failover is built using Multivalue Answer, Weighted, Latency-Based, or Geolocation routing policies.
• Every DNS record in the pool is associated with its own individual Route 53 health check.
• Route 53 evaluates all endpoints continuously. When a query arrives, Route 53 responds using only the currently HEALTHY endpoints. If an endpoint becomes unhealthy, Route 53 dynamically extracts it from DNS query answers without manual intervention, distributing traffic across surviving nodes.

---

Alias Records vs. CNAME Records: Architectural Differentiation

Understanding when to use Route 53 Alias records versus standard DNS CNAME records is a core requirement for AWS CloudOps engineers.

The Zone Apex (Naked Domain) Restriction

Internet Engineering Task Force (IETF) standards (RFC 1034 and RFC 1035) dictate that a CNAME record cannot coexist with other record types for the same name. Because a DNS zone apex (e.g., example.com) must contain Start of Authority (SOA) and Name Server (NS) records, standard CNAME records cannot be created at the zone apex. Route 53 Alias records overcome this limitation through proprietary DNS flattening: Route 53 recognizes AWS resource pointers at the apex and dynamically answers with A (IPv4) or AAAA (IPv6) records.

Automatic Health Evaluation (EvaluateTargetHealth)

When creating an Alias record pointing to an Application Load Balancer, Network Load Balancer, or another hosted zone record, administrators can enable EvaluateTargetHealth: true:

{
  "Name": "api.example.com",
  "Type": "A",
  "AliasTarget": {
    "HostedZoneId": "Z35SXDOTRQ7X7K",
    "DNSName": "my-alb-123456789.us-east-1.elb.amazonaws.com.",
    "EvaluateTargetHealth": true
  }
}

When EvaluateTargetHealth is enabled, Route 53 inspects the underlying health of the target resource directly. For an ALB, Route 53 queries the target group's health status: if all targets in the ALB target group fail their health checks, the ALB itself is evaluated as unhealthy by Route 53, triggering instant DNS failover without requiring an independently billed Route 53 endpoint health check.