GB200 NVL72 Infrastructure Design Contribution to OCP

NVIDIA announced at the 2024 OCP Global Summit that it contributed the GB200 NVL72 rack design and the liquid-cooling designs for its compute and switch trays to the Open Compute Project (OCP). For teams planning dense AI infrastructure, the contribution provides a design basis for evaluating rack structure, power distribution, GPU interconnects, and direct liquid cooling together rather than as separate facility projects.

Why GB200 NVL72 requires an infrastructure-level approach

Large-model training and inference typically distribute work across many GPUs. Tensor, pipeline, and expert parallelism can each be part of a deployment strategy, but they also increase the importance of GPU-to-GPU communication. In operations such as AllReduce, GPUs exchange results at each neural-network layer; communication delays can leave accelerators waiting and affect overall system efficiency.

The source describes GB200 NVL72 as extending an NVLink domain to as many as 72 NVIDIA Blackwell GPUs, with 1.8 TB/s GPU communication speed. This changes the design question from selecting individual servers to engineering an integrated rack that can accommodate compute trays, switch trays, NVLink cabling, power delivery, and heat removal. Actual workload performance, topology behavior, and facility impact must be validated against dated NVIDIA documentation and project-specific tests.

Architecture path described in the OCP contribution

The contributed design is based on NVIDIA MGX design principles and is intended as a modular, reusable foundation for high-density AI infrastructure. The source identifies several physical design elements that work together:

• Rack structure: The rack supports 18 compute trays, nine switch trays, and four NVLink cartridges. NVIDIA describes added steel reinforcement and a rear-rack extension for cable support and flow-distribution equipment.
• NVLink cartridges: Four vertically mounted cartridges at the rear of the rack carry more than 5,000 active copper cables. The source states 130 TB/s bandwidth and 260 TB/s AllReduce bandwidth for this arrangement.
• Power distribution: A higher-capacity busbar retains the width of ORV3 while using a deeper profile. The source specifies support for up to 1,400 A.
• Liquid cooling: The design uses direct liquid cooling and describes a 120 kW cooling requirement for the rack, with enhanced blind-mate liquid connections and floating tray connections.
• Serviceability: Blind-mate rails and latching features are intended to support NVLink installation, liquid-cooling integration, and maintenance through blind-mate connectors.

Where this solution may fit

This architecture is relevant when an organization is assessing a high-density AI rack that needs tightly integrated GPU communication and direct liquid cooling. It is especially applicable to teams building facilities or clusters where rack mechanics, coolant distribution, power capacity, cable routing, and service access must be planned together.

It is not a substitute for a complete deployment design. A proposed implementation should establish the exact GB200 NVL72 configuration, tray count, I/O requirements, site electrical design, cooling-loop interfaces, floor loading, and service procedures. The source notes that compute trays have a modular rack design to accommodate different I/O requirements, but it does not define every supported configuration or site prerequisite.

Implementation checkpoints for evaluation

1. Confirm the applicable NVIDIA and OCP design documents by publication date, including the exact rack, tray, busbar, and liquid-cooling specifications.
2. Develop a complete SKU and bill of materials covering compute, switching, NVLink components, power equipment, coolant-distribution components, and required facility interfaces.
3. Validate electrical capacity and protection design against the stated 1,400 A busbar capability and the selected configuration’s documented requirements.
4. Review direct-liquid-cooling compatibility, coolant specifications, connection tolerances, monitoring, leak-response procedures, and maintenance access with the facility and equipment teams.
5. Test representative AI workloads to measure communication behavior, throughput, resilience, and operational procedures before scaling the design.

FAQ

Does the OCP contribution guarantee that a GB200 NVL72 deployment will be faster or less expensive?

No. The contribution provides design information for the rack and liquid-cooled compute and switch trays. Workload outcomes depend on the model, parallelism approach, software stack, cluster configuration, network design, and facility implementation. Claims about deployment time, cost, or efficiency require verification in official documentation and a project evaluation.

What should facilities teams verify before adopting this design approach?

Facilities teams should verify structural loading, rack dimensions, power feeds, busbar interfaces, liquid-cooling capacity, coolant distribution, rear-cable clearance, maintenance access, and emergency operating procedures. The source describes design intent and selected capacities, but site readiness must be assessed against the final equipment BOM and local engineering requirements.

Conclusion

NVIDIA’s 2024 contribution of GB200 NVL72 rack and tray liquid-cooling designs to OCP offers a reference point for integrated, high-density AI infrastructure planning. Its value is in connecting compute, NVLink, power, structural reinforcement, cabling, and cooling decisions. Deployment teams should use it as an evaluation foundation, then validate the final design through dated product documents, a complete BOM, facility engineering review, and workload testing.

After reviewing GB200 NVL72 Infrastructure Design Contribution to OCP, continue with related solutions for related evaluation paths.