5.3 VLAN Configuration & Security Policies

5.3 VLAN Configuration & Security Policies

Virtual Local Area Networks (VLANs) segment physical and virtual networks into distinct Layer-2 broadcast domains. In VMware vSphere, configuring VLANs correctly requires coordinating virtual switch tagging modes with upstream physical switch port configurations. In parallel, Layer-2 security policies prevent rogue virtual machines from sniffing traffic or impersonating network devices.

---

VLAN Tagging Modes in vSphere

vSphere supports three distinct IEEE 802.1Q VLAN tagging modes. The tagging mode determines whether VLAN encapsulation and de-encapsulation occur at the physical switch, the ESXi virtual switch, or within the virtual machine guest operating system.

+--------------------------------------------------------------------------------------------------+
|                                   vSphere VLAN Tagging Modes                                     |
+-----------------------+---------------+-------------------------+--------------------------------+
| Tagging Mode          | VLAN ID Value | Tagging Entity          | Physical Switch Port Mode      |
+-----------------------+---------------+-------------------------+--------------------------------+
| VST (Virtual Switch)  | 1 - 4094      | ESXi Virtual Switch     | Trunk (802.1Q Trunk)           |
| EST (External Switch) | 0 or None     | Physical Access Switch  | Access Mode (Untagged)         |
| VGT (Virtual Guest)   | 4095          | VM Guest Operating Sys  | Trunk (802.1Q Trunk)           |
+-----------------------+---------------+-------------------------+--------------------------------+

1. Virtual Switch Tagging (VST) — Enterprise Standard

Virtual Switch Tagging is the most widely deployed tagging model in enterprise environments.

• VLAN ID Setting: An explicit VLAN ID between 1 and 4094 is configured on the port group.
• Encapsulation Mechanics:
  • Outbound Traffic: When a virtual machine transmits a standard, untagged Ethernet frame, the virtual switch intercepts the frame, adds an IEEE 802.1Q tag containing the port group's VLAN ID, and forwards the tagged frame out of the physical uplink.
  • Inbound Traffic: When a tagged 802.1Q frame arrives on a physical uplink, the virtual switch inspects the tag. If the tag matches the port group's VLAN ID, the virtual switch strips the 802.1Q tag and delivers a standard untagged Ethernet frame to the VM's virtual NIC. If the tag does not match any port group on the switch, the frame is dropped.
• Guest OS Perspective: The virtual machine has no awareness of VLANs. It uses standard network drivers without 802.1Q tagging configuration.
• Physical Switch Requirement: The upstream physical switch port connected to the ESXi uplink must be configured as an 802.1Q Trunk port allowing the required VLAN IDs.

2. External Switch Tagging (EST)

• VLAN ID Setting: The port group VLAN ID is set to 0 (or left blank/None).
• Encapsulation Mechanics: The ESXi virtual switch performs no VLAN tagging or stripping. All packets pass through the virtual switch completely untagged.
• Physical Switch Requirement: The physical switch port connected to the ESXi uplink is configured as an Access Port assigned to a specific physical VLAN. The physical switch inserts tags when frames enter the physical switching fabric and strips tags when sending frames to the ESXi host.
• Limitations: Every VLAN requires dedicated physical network adapters on the host. If a host needs access to 10 VLANs under EST, it requires 10 distinct physical network cables and uplinks, which is inefficient and does not scale.

3. Virtual Guest Tagging (VGT)

• VLAN ID Setting: On a standard switch port group, set the VLAN ID to 4095. On a distributed port group, choose the VLAN trunking type and list the VLAN ranges to pass (for example 100-110), which is more restrictive than passing every VLAN.
• Encapsulation Mechanics: Setting VLAN ID 4095 configures the virtual switch port group as a virtual 802.1Q trunk. The virtual switch preserves all 802.1Q VLAN tags passing through in both directions without modifying or stripping headers.
• Guest OS Requirement: The virtual machine's guest operating system must be equipped with an 802.1Q-aware network driver capable of tagging and inspecting multiple VLAN tags.
• Primary Use Cases:
  • Virtualized routers (e.g., pfSense, VyOS) and firewalls requiring multi-VLAN routing over a single vNIC.
  • Network monitoring and Intrusion Detection System (IDS) sensors inspecting multi-VLAN spans.
  • Nested ESXi deployments where the parent virtual machine runs ESXi and manages its own internal virtual switches.
• Physical Switch Requirement: Upstream physical switch ports must be configured in 802.1Q Trunk mode.

---

Private VLANs (PVLANs) Architecture

Private VLANs (PVLANs) partition a single standard Layer-2 broadcast domain (subnet) into granular sub-domains to enforce micro-segmentation and prevent lateral movement between workloads without requiring separate IP subnets or default gateways. PVLANs are exclusive to vSphere Distributed Switches.

PVLAN Structure & Port Types

A Private VLAN configuration consists of one Primary VLAN and one or more associated Secondary VLANs:

+-------------------------------------------------------------------------+
|                        Primary VLAN (VLAN 100)                          |
|  Promiscuous Port (Default Gateway / Core Router / Firewall Interface)  |
+-------------------------------------------------------------------------+
          |                                                     |
          v Can communicate with both                           v
+-----------------------------------+   +-----------------------------------+
|   Community Secondary (VLAN 101)  |   |    Isolated Secondary (VLAN 102)  |
| - VM-A and VM-B can talk to each  |   | - Web-1 and Web-2 CANNOT talk to  |
|   other inside Community 101      |   |   each other                      |
| - Can talk to Promiscuous Port    |   | - CANNOT talk to Community 101    |
| - CANNOT talk to Isolated 102     |   | - Can ONLY talk to Promiscuous    |
+-----------------------------------+   +-----------------------------------+

1. Primary VLAN (Promiscuous):
   • Identifies the overarching PVLAN domain.
   • Connects to the Promiscuous Port, which is typically the default gateway router, firewall interface, or backup appliance.
   • Packets sent from a promiscuous port can reach all ports in the Primary VLAN and all secondary Community and Isolated ports.
2. Secondary VLAN — Community:
   • Ports assigned to the same Community VLAN can communicate with each other at Layer 2.
   • Community ports can communicate with Promiscuous ports.
   • Community ports cannot communicate with ports in any other Community VLAN or with any Isolated ports.
   • Use Case: Grouping a specific cluster of peer application servers (e.g., database nodes in a cluster) that must exchange heartbeats while isolated from the rest of the subnet.
3. Secondary VLAN — Isolated:
   • Ports assigned to an Isolated VLAN are completely blocked from communicating with any other port in that same Isolated VLAN at Layer 2.
   • Isolated ports cannot communicate with Community ports.
   • Isolated ports can only communicate with the Promiscuous port (the default gateway).
   • Use Case: DMZ web hosting or multi-tenant hosting environments. Even if an attacker compromises Web Server 1, they cannot scan, spoof, or attack Web Server 2 on the same subnet because the virtual switch drops all inter-isolated traffic.

---

Virtual Switch Security Policies

Every standard and distributed port group enforces three fundamental Layer-2 security policies. These policies guard against malicious or misconfigured virtual machine network behaviors.

+--------------------------------------------------------------------------------------------------+
|                               Virtual Switch Security Policies                                   |
+-----------------------+------------------+-----------------------+-------------------------------+
| Security Policy       | Default Setting  | Evaluated Traffic     | Primary Purpose               |
+-----------------------+------------------+-----------------------+-------------------------------+
| Promiscuous Mode      | Reject           | Inbound to vNIC       | Prevents packet sniffing      |
| MAC Address Changes   | Reject           | Inbound to vNIC       | Prevents MAC impersonation    |
| Forged Transmits      | Reject           | Outbound from vNIC    | Prevents source MAC spoofing  |
+-----------------------+------------------+-----------------------+-------------------------------+

1. Promiscuous Mode

• Default Setting: Reject
• Mechanism: In normal operation (Reject), an ESXi virtual switch delivers frames to a virtual machine only if the frame's destination MAC matches that VM's registered vNIC MAC, is an Ethernet broadcast (FF:FF:FF:FF:FF:FF), or matches a registered multicast group. All other frames are withheld.
• Setting to Accept: When changed to Accept, the virtual switch delivers all network frames transiting that port group to the virtual machine, turning the virtual port into a virtual SPAN port.
• Security Risk: Leaving Promiscuous Mode on Accept allows any compromised VM with a packet capture tool (such as Wireshark or tcpdump) to capture all unencrypted traffic traversing that virtual switch port group.
• Valid Use Cases: Network monitoring appliances, intrusion detection virtual machines, or nested ESXi hosts.

2. MAC Address Changes

• Default Setting: Reject (since vSphere 7.0 on standard switches; earlier releases defaulted to Accept. Distributed switches have always defaulted to Reject.)
• Traffic Direction: Governs Inbound traffic destined for the virtual machine.
• Mechanism: When a VM is created, vCenter assigns each vNIC an initial hardware MAC address stored in its configuration file (.vmx). If the guest operating system alters its effective MAC address in the OS network settings:
  • Under Reject, the virtual switch detects the mismatch between the .vmx MAC address and the effective operating system MAC address. The virtual switch immediately drops all inbound frames destined for the new, modified MAC address. The VM will stop receiving incoming traffic.
  • Under Accept, the virtual switch updates its internal port table to reflect the new MAC address and successfully delivers inbound frames addressed to it.
• Valid Use Cases: Guest OS network bonding/teaming, Microsoft Network Load Balancing (NLB) in unicast mode, and virtual routers.

3. Forged Transmits

• Default Setting: Reject (changed from Accept to Reject for standard switches in vSphere 7.0)
• Traffic Direction: Governs Outbound traffic transmitted by the virtual machine.
• Mechanism: When a VM transmits an Ethernet frame, the virtual switch inspects the Source MAC address field in the Layer-2 frame header:
  • Under Reject, if the source MAC address does not match the adapter's authorized MAC address in the .vmx file, the virtual switch drops the outbound frame immediately.
  • Under Accept, the virtual switch transmits the frame out onto the network without verifying whether the source MAC matches the VM's configured adapter.
• Security Risk: Setting Forged Transmits to Accept allows a compromised VM to execute ARP poisoning and MAC spoofing attacks against other network nodes.
• Valid Use Cases: Nested ESXi virtualization (where nested VMs transmit packets with their own unique MAC addresses through the parent VM's vNIC), Microsoft NLB in unicast mode, and high-availability router appliances using VRRP or CARP.

Traffic Shaping and Other Advanced Port Group Options (Objective 4.3)

Objective 4.3 asks you to configure vSphere Standard Switch advanced options. Beyond VLANs and the three security settings, the main per-switch or per-port-group options are traffic shaping, NIC teaming and failover (Section 5.2), and the MTU.

Traffic Shaping

Traffic shaping is disabled by default. When you enable it, three values define the policy:

A burst can run at peak bandwidth only until the burst size is used up. If a port has been sending below its average, it earns burst credit.

MTU and Overrides

• The MTU is set on the switch (for example 9000 for jumbo frames) and on each VMkernel adapter. Every hop must match end to end: VMkernel adapter, switch, physical switch ports, and the target.
• Port groups inherit the switch's security, shaping, and teaming policies. You can override any of them per port group, and on a vDS also per port when port-level overrides are allowed.

Microsoft NLB in Unicast Mode

NLB unicast replaces each node's MAC address with a shared cluster MAC. Besides allowing MAC Address Changes, VMware's guidance for unicast NLB is to set Notify Switches to No on that port group. Otherwise the RARP frames that ESXi sends would teach the physical switches a single port for the shared cluster MAC.

Exam Trap: On a standard switch, traffic shaping controls only outbound traffic from the VMs. If a question asks you to limit traffic arriving at a VM, you need a distributed switch, which shapes both ingress and egress.