6.5 Data Center Facility Requirements & Site Readiness

6.5 Data Center Facility Requirements & Site Readiness

Blueprint anchor: Domain 2 objective 2.6 — "Identify facility requirements." NVIDIA lists this separately from objective 2.3 (power and cooling specifications), which is a strong hint: knowing that a rack draws 40 kW is not the same as knowing whether your building can host it. This section is the site-readiness half of the problem.

1. Why Legacy Facilities Fail an AI Deployment

A conventional enterprise data hall was designed around 5–10 kW racks, raised-floor air distribution, and roughly uniform load. An accelerated cluster violates every one of those assumptions at once: the racks are heavier, hotter, denser, thirstier for power, and far less tolerant of hot spots. A site assessment therefore has to clear six independent gates, and failing any one of them blocks the deployment regardless of the others.

2. Structural and Physical Requirements

Weight is the requirement candidates forget. Accelerated systems are dense metal, heatsinks, and copper:

• A DGX B200 occupies 10U and weighs up to roughly 314 lb (142.4 kg).
• A DGX H100 occupies 8U at roughly 287 lb (130 kg); a DGX A100 occupies 6U at roughly 271 lb (123 kg).
• A fully populated GB200 NVL72 rack — 18 compute trays, 9 switch trays, busbar, manifolds, and coolant — weighs well over a metric ton and is normally installed on slab rather than on a raised floor.

Consequences to plan for:

• Point loading and floor rating. Many raised floors are rated for far less than a loaded AI rack. Slab installation, load-spreading plates, or structural reinforcement may be required.
• Delivery path. Dock height, door and corridor widths, turning radii, freight-elevator capacity, and ramp gradients all have to be verified before the order ships.
• Rack geometry. Deep chassis (a DGX B200 is about 35 in / 897 mm deep) plus cable management need adequate rack depth and rear clearance. Hot-aisle/cold-aisle containment must remain intact.
• Service clearance. Blind-mate liquid couplings, tray removal, and PSU swaps all require defined front and rear working space.

3. Electrical Service Requirements

Section 6.3 covered the physics of 3-phase distribution; the facility question is whether the building can actually deliver it:

• Utility and backup capacity. Total demand includes IT load plus cooling overhead. A 400 kW IT deployment in a facility with a PUE of 1.4 draws roughly 560 kW at the meter.
• Distribution to the position. Enough rack PDUs, correct receptacle types, correct phase balance, and breaker sizing with appropriate derating for continuous load.
• Redundancy topology. Dual feeds (A/B) with UPS and generator, sized so that a single feed failure does not brown out a training run mid-checkpoint.
• Ride-through behaviour. Accelerated racks present sharp, synchronised load steps when thousands of GPUs enter and leave collective phases together; UPS and generator systems must tolerate that transient profile.

4. Thermal and Water Requirements

Heat rejection tracks IT load almost one-for-one: a rack drawing 40 kW puts roughly 40 kW of heat into the room, and that heat must leave from that position.

• Air-cooled sites. Require sufficient CFM and static pressure at the rack, working containment, and a supply temperature within the equipment's operating window (the DGX B200 operating range is 5–30 °C). Beyond roughly 35–40 kW per rack, air becomes impractical.
• Liquid-cooled sites. Add a facility water loop specification: supply temperature and flow rate, allowable pressure, water quality and chemistry, filtration, CDU placement and redundancy, manifold routing, and leak detection with automated shutoff. Warm-water operation (supply in the low-to-mid tens of degrees Celsius) is what lets a site reject heat to dry coolers instead of running mechanical chillers year-round.
• Redundancy and failure envelope. Liquid cooling concentrates risk: know the time-to-thermal-trip if pumps stop, and size N+1 CDU pumps accordingly.

5. Connectivity, Fire, Acoustics, and Commissioning

• Cable reach. At 400 Gb/s, passive direct-attach copper is limited to short intra-rack runs; longer links require active copper or optical transceivers. Because a rail-optimised design needs eight compute links per node plus storage and management, containment and pathway capacity must be planned, not improvised.
• Separate management pathways. The out-of-band management network is a distinct physical network from the compute fabric and must be cabled and powered accordingly (covered in Section 9.4).
• Fire detection and suppression. Suppression agents and detection design must be compatible with high-airflow or liquid-cooled halls.
• Acoustics and safety. Air-cooled AI racks can exceed 85 dBA, which brings hearing-protection and occupational-exposure obligations; liquid cooling is markedly quieter.
• Commissioning and burn-in. Before production, run integrated load tests, verify per-rack power and thermal margins under synthetic full load, and execute a GPU burn-in and diagnostic pass (see dcgmi diag in Section 9.2) so infant-mortality hardware fails during acceptance rather than mid-training.

Scenario tactic: when a stem says "the customer has spare kilowatts in the building," look for the other gate. Aggregate capacity is rarely the binding constraint — floor loading, per-rack cooling delivery, or cable reach usually is.