2.3 Core Distributed Services: Cassandra, Stargate, Curator, Zookeeper, and Genesis
Key Takeaways
- Genesis is the foundational cluster bootstrap daemon that runs on every CVM to initialize services, manage dependencies, and restart failed processes.
- Zookeeper maintains cluster consensus, manages distributed locks, and elects service leaders across a dedicated 3- or 5-node quorum ensemble.
- Cassandra acts as the modified distributed NoSQL metadata store that records all cluster block maps, extent locations, and vDisk pointers in a shared-nothing ring.
- Stargate handles the data plane on every CVM, intercepting guest VM storage I/O, managing caching tiers, and executing synchronous network replications.
- Curator runs periodic MapReduce operations to orchestrate background data optimizations, including ILM tiering, disk balancing, data scrubbing, and space reclamation.
Last updated: September 2026
Core Distributed Services: Cassandra, Stargate, Curator, Zookeeper, and Genesis
Core Concept: Nutanix Acropolis Operating System (AOS) operates as a completely decentralized, shared-nothing distributed operating system. There is no master controller hardware or single point of failure. Instead, a coordinated suite of specialized software services runs inside every Controller VM, dividing responsibilities between real-time data I/O, distributed consensus, metadata indexing, and background optimization.
A central focus of the Nutanix Certified Associate (NCA) examination is identifying the exact responsibilities, operational planes, and interdependencies of these core distributed daemons.
The Shared-Nothing Distributed Services Architecture
In a Nutanix cluster, every node runs an identical software stack. The services communicate with one another using internal RPCs over the cluster's high-speed Ethernet network. These services are cleanly separated into two functional domains:
- Data Plane: Directly handles real-time guest VM read/write storage operations, multi-tiered caching, and synchronous network replication (primarily Stargate).
- Control Plane & Metadata Plane: Governs cluster state, lock leasing, leader elections, configuration persistence, and background analytics (Genesis, Zookeeper, Cassandra, Curator, Cerebro, Pithos).
+-------------------------------------------------------------------------+
| CONTROLLER VM (CVM) SOFTWARE STACK |
| |
| +-----------------------------------------------------------------+ |
| | Genesis (Service Manager & Bootstrapper) | |
| +-----------------------------------------------------------------+ |
| | | | |
| v v v |
| +--------------+ +--------------+ +--------------+ |
| | Zookeeper | | Cassandra | | Stargate | |
| | (Consensus & | | & Medusa | | (Data Plane | |
| | Locks) | | (Metadata) | | I/O Engine) | |
| +--------------+ +--------------+ +--------------+ |
| | ^ | |
| | Elects Master | Extent Queries | Caches & |
| v | v Writes |
| +--------------+ | [ DirectPath I/O ] |
| | Curator |---------------+ [ NVMe / SSD / HDD] |
| | (MapReduce & | (ILM, Balancing, Scrubbing) |
| | Housekeeping)| |
| +--------------+ |
| |
| +-----------------------------------------------------------------+ |
| | Ancillary Services: Cerebro (DR/Snapshots) | Pithos (vDisks) | |
| +-----------------------------------------------------------------+ |
+-------------------------------------------------------------------------+
Genesis: Cluster Initialization and Daemon Management
Genesis is the first AOS daemon to start when a Controller VM boots up, and it serves as the parent supervisor for all other Nutanix services.
- Service Initialization: Genesis runs on every CVM in the cluster. It reads local cluster configuration files (
cluster.json) to understand the node's identity, IP addressing, and cluster membership. - Daemon Supervision: Genesis acts as a watchdog process. It starts, stops, monitors, and restarts all other AOS services in their required dependency order.
- Cluster Formation: When a cluster is first initialized via the CLI (
cluster create) or Foundation tool, Genesis orchestrates service bootstrapping across all participating nodes. - Troubleshooting & Management: Administrators interact with Genesis when issuing cluster-level lifecycle commands from the CVM command line:
cluster status— Queries Genesis across all nodes to display the run state of every service.cluster start/cluster stop— Instructs Genesis to gracefully bring up or halt cluster services.- Log Location:
/home/nutanix/data/logs/genesis.out
Zookeeper: Consensus, Leadership, and Lock Management
Zookeeper is the cluster's distributed consensus manager and configuration authority, built upon the proven Apache ZooKeeper framework.
- Ensemble Architecture: Unlike services that run actively on every CVM, Zookeeper runs as an odd-numbered quorum group known as an ensemble:
- In a 3-node or 4-node cluster, Zookeeper runs on 3 nodes.
- In clusters with 5 or more nodes, Zookeeper runs on 5 nodes.
- An odd number of participants ensures clean mathematical majorities without ties.
- Role and Responsibilities:
- Leader Elections: Elects active cluster component leaders, including the Curator Master and the Prism Leader.
- Distributed Lock Management: Coordinates atomic lock acquisition to ensure two CVMs cannot simultaneously modify the same cluster configuration resource or duplicate tasks.
- Configuration Store: Stores static cluster configuration parameters, physical drive inventories, IP configuration data, and cluster membership tables.
- Log Location:
/home/nutanix/data/logs/zookeeper.out
Cassandra and Medusa: The Distributed Metadata Store
Cassandra is the modified distributed NoSQL database engine responsible for storing and indexing all metadata across the Nutanix cluster.
The Cassandra Ring
Every CVM runs an instance of Cassandra. Together, they form a peer-to-peer, shared-nothing Cassandra Ring:
- Metadata Stored: Cassandra records every vDisk block map, physical extent location, disk tier placement, snapshot pointer, and deduplication checksum.
- Consensus via Paxos: To guarantee strict data consistency without relying on a centralized database master, Cassandra implements the Paxos consensus algorithm. Metadata updates require acknowledgment from a majority of Cassandra replicas before being committed.
- Fault Resiliency: Metadata is replicated according to the cluster's Redundancy Factor (RF2 stores 2 metadata copies; RF3 stores 3 metadata copies).
The Role of Medusa
Medusa is a Nutanix software abstraction layer that interfaces directly between Cassandra and consumers like Stargate and Curator:
- Translates AOS high-level storage queries into Cassandra key-value operations.
- Maintains an in-memory metadata cache inside the CVM to accelerate lookup times for hot block maps.
- Manages metadata pruning and compaction routines.
- Log Location:
/home/nutanix/data/logs/cassandra/system.log
Stargate: The Storage Data Plane Engine
Stargate is arguably the most critical operational daemon in AOS—it represents the entire data plane of the Distributed Storage Fabric.
- Execution: Runs on every CVM in the cluster.
- Protocol Presentation: Presents storage to hypervisors through industry-standard protocols, including iSCSI, NFS v3/v4, and SMB 3.0 (for Hyper-V).
- I/O Interception: All read and write requests initiated by guest VMs pass directly into the local node's Stargate process.
- Caching Hierarchy Management:
- DRAM Cache: Serves ultra-frequently accessed read extents directly from CVM memory.
- OpLog (Write Buffer): Absorbs incoming synchronous random writes on local NVMe/SSD media.
- Extent Store: Manages persistent storage across SSD flash and HDD spinning disks.
- Replication Enforcement: Stargate coordinates synchronous network replication with peer Stargates on remote nodes for RF2 and RF3 write compliance.
- Data Locality Enforcement: Stargate monitors the physical location of requested extents. When a VM runs on a host where its data is remote, Stargate initiates background data shadowing and migration to pull the extents onto the local SSD tier.
- Log Location:
/home/nutanix/data/logs/stargate.out
Curator: Distributed Analytics and Background Housekeeping
While Stargate handles real-time I/O, Curator handles cluster-wide background maintenance, optimization, and lifecycle tasks.
Master/Slave MapReduce Architecture
- Curator Master: Zookeeper elects exactly one CVM in the cluster to act as the Curator Master.
- Curator Slaves: Curator instances running on all other CVMs function as Curator Slaves.
- MapReduce Execution: The Curator Master acts as a job scheduler. It breaks massive cluster-wide analysis jobs into thousands of discrete work units, distributing them across the Curator Slaves using a custom MapReduce framework.
Core Tasks Governed by Curator
- Information Lifecycle Management (ILM):
- Analyzes access timestamps and temperature of all data extents across the cluster.
- Automatically migrates cold data down from flash SSDs to high-capacity HDDs.
- Promotes hot, frequently read extents back into the SSD tier.
- Cluster Disk Balancing:
- Continuously monitors storage utilization across every drive and node in the cluster.
- If a specific node or disk exceeds predefined capacity utilization watermarks, Curator automatically schedules the movement of data extents to underutilized nodes across the network to maintain an even storage distribution.
- Data Scrubbing and Consistency Verification:
- Periodically computes checksums on stored data extents and compares them against Cassandra metadata to detect and heal silent data corruption (bit rot).
- Garbage Collection:
- Scans the storage fabric for orphaned extents resulting from deleted virtual machines, truncated snapshots, or overwritten vDisk blocks, reclaiming the physical space.
- Post-Process Compression and Erasure Coding (EC-X):
- Identifies eligible cold data blocks and coordinates distributed background compression and parity striping.
- Log Location:
/home/nutanix/data/logs/curator.out
Ancillary Distributed Services
In addition to the primary daemons above, several specialized services complete the AOS distributed architecture:
Cerebro: Snapshots and Disaster Recovery
- Role: The disaster recovery and replication synchronization engine.
- Responsibilities: Manages Protection Domains, schedules and executes crash-consistent and application-consistent (VSS) snapshots, and orchestrates asynchronous, NearSync, and Metro Availability replication across sites or to Nutanix Cloud Clusters (NC2).
- Election: Zookeeper elects one Cerebro Master to coordinate replication jobs across Cerebro daemons running on all CVMs.
- Log Location:
/home/nutanix/data/logs/cerebro.out
Pithos: vDisk Configuration and Policies
- Role: Virtual disk configuration manager.
- Responsibilities: Built directly on top of Cassandra, Pithos stores and manages vDisk geometry, disk sizing, container bindings, and storage QoS policies.
- Log Location:
/home/nutanix/data/logs/pithos.out
Chronos: Distributed Task Scheduling
- Role: Distributed task scheduler that ensures periodic maintenance operations (including Curator scan triggers) are executed predictably across the cluster.
Core Distributed Services Matrix
The following table summarizes the key attributes of all AOS core services for rapid exam review:
| Service Name | Operational Plane | Nodes Running | Master Elected? | Primary Core Responsibilities |
|---|---|---|---|---|
| Genesis | Management | Every CVM | No | Service initialization, process watchdog, cluster startup/shutdown |
| Zookeeper | Consensus / Control | 3 or 5 CVMs | Yes | Cluster consensus, leader elections, distributed lock management |
| Cassandra | Metadata Plane | Every CVM | No (Ring/Paxos) | Distributed metadata store for all block maps, vDisks, and extent pointers |
| Medusa | Metadata Abstraction | Every CVM | No | Interfaces Cassandra with Stargate/Curator; metadata caching |
| Stargate | Data Plane | Every CVM | No | Intercepts guest VM I/O, manages read/write caches, enforces RF2/RF3 replication |
| Curator | Maintenance / Analytics | Every CVM | Yes (1 Master) | MapReduce engine for ILM tiering, disk balancing, data scrubbing, garbage collection |
| Cerebro | Data Protection | Every CVM | Yes (1 Master) | Manages protection domains, snapshot schedules, and remote DR replication |
| Pithos | Control Plane | Every CVM | No | Stores vDisk configurations, geometries, and storage QoS policies |
Loading diagram...
Test Your Knowledge
Which distributed AOS service is primarily responsible for electing cluster leaders, storing static configuration parameters, and maintaining distributed lock consensus?
A
B
C
D
Test Your Knowledge
In the Nutanix distributed architecture, what is the specific operational responsibility of the Curator service?
A
B
C
D
Test Your Knowledge
If a storage administrator investigates high guest VM write latency and examines the data plane daemon responsible for intercepting I/O and managing read/write caches on each CVM, which service are they analyzing?
A
B
C
D