6.3 Nutanix Guest Tools (NGT) Architecture, Features, and Deployment

Nutanix Guest Tools (NGT) Architecture, Features, and Deployment

Quick Summary: Nutanix Guest Tools (NGT) is a specialized software bundle installed directly within guest operating systems to establish secure, bidirectional communication with the Nutanix Controller VM (CVM). NGT delivers four core capabilities: optimized VirtIO drivers for high-throughput networking and storage, Nutanix VM Mobility for cross-hypervisor conversions, the Nutanix VSS Hardware Provider for zero-impact application-consistent snapshots, and Self-Service Restore (SSR) for in-guest file-level recovery. Security is maintained via mutual TLS (mTLS) on TCP port 2074, authenticated through cluster-generated cryptographic certificates.

---

1. NGT Architectural Overview and Core Components

While basic virtual machines can run on AHV using standard open-source drivers, enterprise workloads require tight coordination between the guest operating system and the underlying Distributed Storage Fabric (DSF). Nutanix Guest Tools bridges this boundary through an integrated suite of user-space daemons and kernel-space drivers.

+-------------------------------------------------------------------------+
|                        Guest Virtual Machine                            |
|  +-------------------------------------------------------------------+  |
|  | Nutanix Guest Agent (NGA Service / Daemon)                        |  |
|  |   - Self-Service Restore (SSR) Web Service (Port 2105)             |  |
|  |   - Mutual TLS Cryptographic Client Engine                        |  |
|  +-------------------------------------------------------------------+  |
|  +-------------------------------------------------------------------+  |
|  | In-Guest Drivers & Providers                                      |  |
|  |   - Nutanix VSS Hardware Provider (Windows VSS coordination)      |  |
|  |   - Application Freeze / Thaw Scripts (Linux pre/post freeze)     |  |
|  |   - VirtIO Multi-Queue Storage & Network Drivers                  |  |
|  |   - Nutanix VM Mobility Drivers (Cross-Hypervisor Migration)      |  |
|  +-------------------------------------------------------------------+  |
+------------------------------------+------------------------------------+
                                     | (mTLS Encrypted / TCP Port 2074)
                                     v
+------------------------------------+------------------------------------+
|                   Controller VM (Local CVM / Cluster)                  |
|   - Guest Tools Service (Listening on Port 2074)                        |
|   - Internal Certificate Authority (CA) & Handshake Validator           |
|   - Distributed Storage Fabric Snapshot Engine                         |
+-------------------------------------------------------------------------+

The Four Core Functional Pillars of NGT

1. Nutanix Guest Agent (NGA): The primary background service (running as a Windows Service or Linux systemd daemon) that maintains continuous telemetry heartbeats and command execution channels with the cluster CVMs.
2. Nutanix VSS Hardware Provider: A certified VSS hardware provider plugin that hooks into the Microsoft Volume Shadow Copy Service framework, coordinating zero-overhead application quiescing directly with DSF.
3. Self-Service Restore (SSR) Agent: An embedded web server listening locally on TCP port 2105 that empowers guest administrators to browse and mount point-in-time snapshots for file-level recovery.
4. Nutanix VM Mobility Drivers: A comprehensive driver package pre-installing AHV-compatible storage (VirtIO-SCSI) and network (VirtIO-Net) drivers, enabling non-disruptive migration from VMware ESXi to Nutanix AHV.

---

2. Security Architecture: Mutual TLS Handshake over Port 2074

Unlike legacy hypervisor guest tools that rely on unauthenticated memory-bus backdoors or cleartext hypervisor communication channels, Nutanix enforces strict cryptographic isolation for all NGT operations.

+-------------------------------------------------------------------------+
|                   NGT Mutual TLS Security Handshake                     |
+-------------------------------------------------------------------------+
| 1. Admin enables NGT in Prism Element / Prism Central                   |
| 2. Acropolis generates VM-unique SSL/TLS certificate pair               |
| 3. Cluster mounts dynamic NGT ISO containing certs to VM CD-ROM         |
| 4. In-Guest Installer extracts client certs to secure key store         |
| 5. NGA initiates mTLS connection to CVM on TCP Port 2074                |
| 6. CVM validates client certificate; NGA validates cluster CA          |
| 7. Secure, encrypted bi-directional control tunnel established          |
+-------------------------------------------------------------------------+

The mTLS Authentication Workflow

• Dynamic ISO Generation: When an administrator enables NGT for a VM in Prism, the cluster does not attach a generic installer disc. Instead, Acropolis dynamically creates a customized ISO image unique to that specific VM. This ISO contains the NGT installation binaries alongside a unique private key and client certificate signed by the Nutanix cluster's internal Certificate Authority (CA).
• Encrypted Session Establishment: Once installed, the Nutanix Guest Agent initiates an outbound TCP connection to the local CVM on TCP port 2074.
• Mutual Authentication: The CVM verifies the validity of the VM's client certificate against its internal database, and the guest agent validates the CVM's SSL certificate against the cluster CA root. If a certificate is revoked, expired, or cloned from another VM, communication is rejected immediately.

[!IMPORTANT] If a virtual machine with NGT is cloned without sysprepping, or if the Nutanix cluster certificates are regenerated, NGT communication breaks due to certificate mismatch. To remediate, the administrator must toggle Manage Guest Tools in Prism to mount a fresh NGT ISO and refresh the certificates.

---

3. Application-Consistent Snapshots: VSS and Linux Scripts

Data protection strategies depend on the consistency level of recovery snapshots. Nutanix classifies snapshots into two critical tiers:

+-------------------------------------------------------------------------+
|                   Snapshot Consistency Comparison                       |
+-----------------------+-------------------------------------------------+
| Crash-Consistent      | - Instantaneous block-level metadata capture    |
| (Standard DSF)        | - Equivalent to pulling physical power cord     |
|                       | - In-flight memory and unwritten buffers lost   |
|                       | - Requires filesystem journal / DB replay       |
+-----------------------+-------------------------------------------------+
| Application-Consistent| - In-guest caches flushed to disk before snap   |
| (NGT + VSS / Scripts) | - VSS Writers pause transactional activity      |
|                       | - Zero database log replay required on recovery |
|                       | - Maximum recovery integrity for SQL/Exchange   |
+-----------------------+-------------------------------------------------+

Windows: Nutanix VSS Hardware Provider

In a standard crash-consistent snapshot, any data residing in volatile server RAM or uncommitted database buffers is lost. While modern transactional databases (such as Microsoft SQL Server or Oracle) can replay transaction logs upon reboot, recovery takes valuable time and carries the risk of partial corruption.

Nutanix solves this on Windows guests through the Nutanix VSS Hardware Provider:

1. The Nutanix Protection Domain or Prism Central Protection Policy triggers a scheduled snapshot.
2. The CVM sends an RPC command over TCP port 2074 to the in-guest NGA.
3. NGA invokes the native Windows Volume Shadow Copy Service (VSS) framework.
4. VSS notifies registered VSS Writers (Microsoft SQL Server, Exchange Server, Active Directory NTDS, Hyper-V, Oracle VSS Writer).
5. The VSS writers temporarily quiesce application writes, flush all dirty memory buffers to disk, and freeze the database engine.
6. The Nutanix VSS Hardware Provider alerts the CVM, which takes an instantaneous Redirect-on-Write (RoW) snapshot at the Distributed Storage Fabric tier in milliseconds.
7. The CVM signals NGA, which thaws the VSS writers. Normal database transaction processing resumes without application timeout or connection dropping.

Linux: Pre-Freeze and Post-Freeze Scripts

Because Linux operating systems lack a universal native equivalent to Microsoft VSS, NGT implements application consistency via customizable hook scripts:

• Directory Location: The Nutanix Guest Agent monitors /usr/local/sbin/ for two specific executable scripts:
  • /usr/local/sbin/pre_freeze
  • /usr/local/sbin/post_freeze
• Execution Mechanics: When an application-consistent snapshot is triggered, NGA executes pre_freeze. This script contains commands to flush buffers and lock tables (for example, executing FLUSH TABLES WITH READ LOCK; in MySQL/MariaDB or issuing ALTER DATABASE BEGIN BACKUP; in Oracle).
• Snapshot & Thaw: Once pre_freeze exits with return code 0, the CVM captures the DSF snapshot. NGA immediately executes post_freeze (e.g., UNLOCK TABLES; or ALTER DATABASE END BACKUP;) to restore normal read-write database operations.

[!TIP] The pre_freeze and post_freeze scripts must be owned by root, have executable permissions (chmod 700), and execute within the timeout window (default 60 seconds). If pre_freeze fails or times out, the snapshot defaults to crash-consistent.

---

4. Nutanix Self-Service Restore (SSR)

In traditional IT environments, restoring an accidentally deleted file or reverting a corrupted configuration document requires submitting a helpdesk ticket to storage administrators, who must locate the backup catalog, mount a historic volume, and copy the file manually.

Nutanix Self-Service Restore (SSR) eliminates this administrative overhead by providing a secure, web-based file recovery interface directly within the guest operating system.

+-------------------------------------------------------------------------+
|                 Self-Service Restore (SSR) Architecture                 |
+-------------------------------------------------------------------------+
| 1. Guest User browses to: https://localhost:2105                        |
| 2. User logs in with local Windows / Linux OS Admin Credentials         |
| 3. SSR queries CVM (Port 2074) for available historical snapshots       |
| 4. User selects snapshot (e.g., Yesterday 5:00 PM) and clicks [Mount]   |
| 5. CVM dynamically attaches snapshot vDisk as secondary drive (e.g., E:)| 
| 6. User copies desired files directly back to production folders        |
| 7. User clicks [Unmount] (or automatic timeout unmounts after 24 hours) |
+-------------------------------------------------------------------------+

SSR Operational Characteristics

• Local Web Console: SSR hosts a lightweight web management service listening on TCP port 2105 (https://localhost:2105).
• Authentication & RBAC: Users log in using their native guest operating system credentials (local administrator or Active Directory user with administrative rights on that VM). Hypervisor-level credentials are never exposed or required.
• Snapshot Mounting: Upon selecting an available snapshot date/time, the CVM attaches the snapshot vDisk to the VM as a secondary drive letter (in Windows, e.g., E:) or mount point (in Linux, e.g., /mnt/nutanix/ssr/).
• Safety & Auto-Unmount: Snapshot disks are mounted strictly in read-only mode to preserve historical data integrity. To prevent disk sprawl and stranded iSCSI/SCSI attachments, SSR automatically unmounts mounted snapshot disks after a configurable inactivity threshold (default 24 hours).
• Command-Line Interface: SSR includes a CLI utility (nutanix-guest-tools/bin/ssr) allowing power users and backup administrators to script snapshot mount and restore operations.

---

5. Nutanix VM Mobility and Cross-Hypervisor Migration

One of the most complex challenges in enterprise cloud transformation is migrating workloads between disparate hypervisors (such as moving from VMware ESXi to Nutanix AHV, or between on-premises AHV clusters and public cloud Nutanix Cloud Clusters [NC2] on AWS or Azure).

When a virtual machine running on VMware ESXi is converted to AHV, the VM often crashes during its initial boot with a 0x0000007B INACCESSIBLE_BOOT_DEVICE stop error because the guest OS lacks the required KVM/AHV VirtIO storage controller drivers.

Nutanix VM Mobility solves this driver dilemma:

• Pre-Installation: When NGT is deployed to a VM running on VMware ESXi, the VM Mobility service silently pre-injects and registers all requisite Nutanix VirtIO storage and network drivers into the guest operating system's driver store.
• Registry Configuration: VM Mobility configures Windows CriticalDeviceDatabase and hardware registry keys to ensure the VirtIO-SCSI driver initializes early during boot phase.
• Seamless Cutover: When the VM is migrated to AHV (via Nutanix Move or native asynchronous replication), the guest OS detects the virtual VirtIO controller immediately upon first boot and initializes cleanly without manual driver injection or rescue disk recovery.

---

6. NGT Deployment and Lifecycle Management

Nutanix provides flexible options for deploying, managing, and upgrading NGT across single workloads or enterprise-scale estates.

Silent Installation Command Syntax

For enterprise software deployment tools (Microsoft Endpoint Configuration Manager [MECM/SCCM], Group Policy, Ansible, or Puppet), NGT supports fully unattended installations:

• Windows Silent Install (MSI):

  NutanixGuestTools.exe /quiet /norestart ACC=1 SSR=1
  

  Where ACC=1 enables the Application-Consistent snapshot provider, and SSR=1 installs the Self-Service Restore agent.

• Linux Silent Install (RPM / DEB):

  sudo python /mnt/cdrom/installer/linux/install_ngt.py