10.5 Windows Server DNS Policies & DNSSEC

Windows Server DNS Policies & DNSSEC

Modern enterprise security and traffic management require DNS servers to perform intelligent, context-aware query resolution while guaranteeing the cryptographic authenticity and integrity of DNS responses. Windows Server provides two advanced technologies to meet these requirements: DNS Policies (introduced in Windows Server 2016 and refined in 2019/2022/2025) and DNSSEC (Domain Name System Security Extensions). DNS Policies enable geo-location routing, split-brain DNS, and DDoS mitigation, while DNSSEC protects against DNS spoofing and cache poisoning attacks.

---

1. Windows Server DNS Policies Architecture & Processing Pipeline

Traditional DNS servers return identical answers to all querying clients for a given FQDN. DNS Policies allow administrators to define rule-based criteria to alter how the DNS server responds to incoming queries, recursion requests, and zone transfer requests.

+-----------------------------------------------------------------------------+
|                   DNS SERVER POLICY PROCESSING PIPELINE                     |
|                                                                             |
|   [INCOMING DNS QUERY RECEIVED]                                             |
|                 |                                                           |
|                 v                                                           |
|   +----------------------------+                                            |
|   | 1. Recursion Policies      | ---> Allow / Deny recursive resolution     |
|   +----------------------------+      based on Client Subnet                |
|                 |                                                           |
|                 v                                                           |
|   +----------------------------+                                            |
|   | 2. Zone Transfer Policies  | ---> Allow / Deny AXFR / IXFR transfers    |
|   +----------------------------+      to specific secondary servers         |
|                 |                                                           |
|                 v                                                           |
|   +----------------------------+                                            |
|   | 3. Query Resolution Policy | ---> Match Client Subnet, Time of Day, IP  |
|   +----------------------------+      Protocol -> Route to Zone Scope       |
|                 |                                                           |
|                 v                                                           |
|   [RETURN CONTEXTUAL DNS RESPONSE (Zone Scope IP or Default Scope IP)]      |
+-----------------------------------------------------------------------------+

Policy Criteria and Matching Rules:

DNS policies evaluate incoming requests against specific configurable criteria:

• Client Subnet: Matched using pre-defined client subnet objects (Add-DnsServerClientSubnet).
• Transport Protocol: Matched against UDP or TCP.
• Internet Protocol: Matched against IPv4 or IPv6.
• Server Interface IP: The local IP address of the DNS server network interface that received the query.
• FQDN: Fully Qualified Domain Name requested in the query (with wildcard support).
• Time of Day: System time on the DNS server (for business hours vs. off-hours routing).

---

2. Policy Types & Use Cases

+-----------------------------------------------------------------------------+
|                         DNS POLICY TYPES & APPLICATIONS                     |
|                                                                             |
|   POLICY TYPE               PRIMARY OBJECTIVE        COMMON APPLICATION     |
|   ------------------------+------------------------+------------------------|
|   Query Resolution Policy | Context-aware answers  | Geo-Location, Split-   |
|                           | based on client origin | Horizon, Failover      |
|   Zone Transfer Policy    | Restrict AXFR/IXFR     | Block unauthorized     |
|                           | zone transfers         | secondary zone copies  |
|   Recursion Policy        | Control recursive name | Block Open Resolver    |
|                           | resolution paths       | amplification abuse    |
|   Response Rate Limiting  | Throttle repetitive    | Mitigate DNS DDoS      |
|   (RRL)                   | query responses        | amplification attacks  |
+-----------------------------------------------------------------------------+

Core DNS Policy Scenarios:

1. Split-Brain (Split-Horizon) DNS:
   • Internal network clients (e.g., subnet 10.0.0.0/8) querying portal.contoso.com receive the private IP address (10.0.5.50).
   • External or DMZ clients querying the same FQDN receive the public NAT IP address (203.0.113.50).
2. Geo-Location Load Balancing:
   • European clients (192.168.100.0/24) resolving app.contoso.com are directed to the European datacenter (10.20.1.10), while North American clients are routed to the US datacenter (10.10.1.10).
3. DNS Response Rate Limiting (RRL):
   • Prevents the DNS server from being weaponized in DNS amplification and reflection DDoS attacks. RRL detects when high volumes of identical queries are received from the same client subnet and rate-limits or drops the responses.

# PowerShell: Configuring DNS Response Rate Limiting (RRL)
Set-DnsServerResponseRateLimiting `
    -ResponsesPerSec 5 `
    -ErrorsPerSec 3 `
    -WindowInSec 5 `
    -LeakRate 3 `
    -TruncateRate 2 `
    -Mode Enable

---

3. Zone Scopes, Record Scopes & Policy Rules in PowerShell

A Zone Scope is a unique instance of a DNS zone containing its own independent set of DNS resource records. A zone always possesses a default zone scope (the standard zone database) and can host multiple custom zone scopes.

+-----------------------------------------------------------------------------+
|                        ZONE SCOPE ARCHITECTURE                              |
|                                                                             |
|   Zone: "contoso.com"                                                       |
|   +---------------------------------------------------------------------+   |
|   | Default Zone Scope:                                                 |   |
|   | - www.contoso.com  -->  203.0.113.10 (External Public IP)           |   |
|   +---------------------------------------------------------------------+   |
|   | Custom Scope: "InternalScope"                                       |   |
|   | - www.contoso.com  -->  10.10.1.50   (Internal Private IP)          |   |
|   +---------------------------------------------------------------------+   |
|   | Custom Scope: "EuropeScope"                                         |   |
|   | - www.contoso.com  -->  192.168.20.10 (Europe Datacenter IP)        |   |
|   +---------------------------------------------------------------------+   |
+-----------------------------------------------------------------------------+

# PowerShell: Step-by-Step Deployment of Split-Brain DNS Policy

# 1. Define the Client Subnet object for internal LAN clients
Add-DnsServerClientSubnet -Name 'InternalSubnet' -IPv4Subnet '10.0.0.0/8'

# 2. Create the custom Zone Scope within the primary zone
Add-DnsServerZoneScope -ZoneName 'contoso.com' -Name 'InternalScope'

# 3. Add the internal resource record to the custom Zone Scope
Add-DnsServerResourceRecord -ZoneName 'contoso.com' -ZoneScope 'InternalScope' `
    -A -Name 'portal' -IPv4Address '10.0.5.50'

# 4. Add the default (external) resource record to the default zone scope
Add-DnsServerResourceRecordA -ZoneName 'contoso.com' `
    -Name 'portal' -IPv4Address '203.0.113.50'

# 5. Create the Query Resolution Policy linking the subnet to the zone scope
Add-DnsServerQueryResolutionPolicy -Name 'SplitBrainPolicy' `
    -Action ALLOW `
    -ClientSubnet 'eq,InternalSubnet' `
    -ZoneScope 'InternalScope,1' `
    -ZoneName 'contoso.com' `
    -ProcessingOrder 1

---

4. Domain Name System Security Extensions (DNSSEC) Architecture

Standard DNS is an unauthenticated, cleartext protocol vulnerable to DNS Cache Poisoning (e.g., the Kaminsky attack) and DNS Spoofing. An attacker who intercepts or predicts transaction IDs can inject forged IP addresses into a DNS resolver's cache, redirecting users to malicious phishing or malware servers.

DNSSEC (RFC 4033, 4034, 4035) introduces cryptographic authentication and data integrity to DNS through digital signatures. DNSSEC does not encrypt DNS queries (which is handled by DNS-over-HTTPS or DNS-over-TLS); rather, it cryptographically proves that DNS records originated from their authentic authoritative source and were not altered in transit.

+-----------------------------------------------------------------------------+
|                        DNSSEC KEY HIERARCHY & ROLES                         |
|                                                                             |
|   [KEY SIGNING KEY (KSK)]                                                   |
|   - Key Length: 2048 to 4096-bit RSA (or ECDSA P-384)                       |
|   - Contains: Secure Entry Point (SEP) flag (Flag 257)                      |
|   - Function: Signs ONLY the DNSKEY RRset                                   |
|   - Rollover Frequency: Infrequent (e.g., every 1 to 2 years)               |
|   - Hash published to Parent Zone as: DELEGATION SIGNER (DS) RECORD         |
|          |                                                                  |
|          v (Cryptographically validates)                                    |
|   [ZONE SIGNING KEY (ZSK)]                                                  |
|   - Key Length: 1024 to 2048-bit RSA (or ECDSA P-256)                       |
|   - Contains: Zone Key flag (Flag 256)                                      |
|   - Function: Signs ALL standard resource record sets (A, CNAME, MX, etc.)  |
|   - Produces: RRSIG (Resource Record Signature) records                     |
|   - Rollover Frequency: Frequent (e.g., every 30 to 90 days)                |
+-----------------------------------------------------------------------------+

Primary DNSSEC Resource Record Types:

1. RRSIG (Resource Record Signature): Contains the cryptographic signature for a set of identical DNS records (an RRset). Validating resolvers verify the signature against the public ZSK.
2. DNSKEY: Stores the public cryptographic keys (both the KSK and ZSK) for the zone.
3. DS (Delegation Signer): Stored in the parent zone (e.g., .com holds the DS record for contoso.com). Contains a SHA-256 hash of the child zone's public KSK, establishing the cryptographic Chain of Trust.
4. Trust Anchors & Trust Points: A public key or DS digest configured on a recursive DNS resolver that serves as the root of validation for a signed namespace. In Windows Server AD DS, Trust Anchors replicate forest-wide across all DCs in the AD-Integrated Trust Anchor Store.

---

5. Authenticated Denial of Existence: NSEC vs. NSEC3

When a DNS resolver queries a non-existent record (e.g., nonexistent.contoso.com), a signed zone must provide cryptographic proof that the name does not exist (NXDOMAIN) without generating dynamic signatures in real time on the fly.

+-----------------------------------------------------------------------------+
|                          NSEC VS NSEC3 COMPARISON                           |
|                                                                             |
|   FEATURE               NSEC (RFC 4034)             NSEC3 (RFC 5155)        |
|   --------------------+---------------------------+-------------------------|
|   Format              | Plaintext alphabetical    | Salted SHA-1 / SHA-256  |
|                       | linked list               | cryptographic hashes    |
|   Vulnerability       | **Zone Walking** (Full    | Resistant to Zone       |
|                       | zone enumeration)         | Walking attacks         |
|   Opt-Out Feature     | Not Supported             | Supported for insecure  |
|                       |                           | delegations             |
|   Standard Status     | Legacy / Deprecated       | Modern Enterprise Rec.  |
+-----------------------------------------------------------------------------+

• NSEC (Next Secure): Links existing records in alphabetical order (e.g., "apple.contoso.com exists, and the next record is banana.contoso.com"). If a client asks for artichoke.contoso.com, the server returns the signed NSEC record spanning apple to banana. Major Security Flaw: Attackers query sequentially to download the entire contents of the zone (known as Zone Walking).
• NSEC3 (Next Secure v3): Hashes record names with a random cryptographic salt and iterative hashing. The denial of existence proves that no record exists whose hash falls between two signed hashes, preventing zone walking.

---

6. DANE & TLSA Records

DNS-based Authentication of Named Entities (DANE, RFC 6698) leverages a DNSSEC-secured DNS infrastructure to bind Transport Layer Security (TLS) certificates and public keys directly to domain names via TLSA resource records. This eliminates reliance on compromised third-party Certificate Authorities (CAs) by validating that a server certificate presented during HTTPS/TLS handshakes matches the public key signed by the domain's DNSSEC trust chain.

---

7. DNSSEC Signing & Management via PowerShell

# PowerShell: Signing a Primary DNS Zone with DNSSEC

# 1. Sign an AD-Integrated Primary Zone with one default KSK and one default ZSK.
#    Invoke-DnsServerZoneSign is the cmdlet that signs a zone. There is NO
#    Enable-DnsServerDnsSecZoneSetting cmdlet - the DnsServer module ships only the
#    Get-, Set- and Test- forms of *-DnsServerDnsSecZoneSetting.
Invoke-DnsServerZoneSign `
    -ZoneName 'corp.contoso.com' `
    -SignWithDefault `
    -PassThru `
    -Force

# 1b. Tune the DNSSEC parameters of the now-signed zone.
#     -DenialOfExistence accepts only NSec or NSec3.
#     -DistributeTrustAnchor accepts only None or DnsKey ('AD' is not a valid value);
#     DnsKey is what publishes the trust anchor into the AD forest-wide TrustAnchors zone.
Set-DnsServerDnsSecZoneSetting `
    -ZoneName 'corp.contoso.com' `
    -DenialOfExistence NSec3 `
    -DistributeTrustAnchor DnsKey `
    -PassThru

# 2. Add an explicit Key Signing Key (KSK) with RSA-SHA256 (2048-bit)
Add-DnsServerSigningKey `
    -ZoneName 'corp.contoso.com' `
    -Type KeySigningKey `
    -CryptoAlgorithm RsaSha256 `
    -KeyLength 2048 `
    -KeyStorageProvider 'Microsoft Software Key Storage Provider'

# 3. Export the public DS record data to submit to the parent zone registrar
Get-DnsServerDnsSecZoneSetting -ZoneName 'corp.contoso.com' | 
    Select-Object -ExpandProperty KeySigningKeys

# 4. Verify the cryptographic validation state of the signed zone
Test-DnsServerDnsSecZoneSetting -ZoneName 'corp.contoso.com'