NVIDIA GB300 NVL72: GPU Architecture & Performance

NVIDIA’s Blackwell Ultra (GB300 and B300) introduces significant architectural and performance improvements over the Blackwell generation (GB200 and B200), particularly for FP4 formats.

In this post, we analyze per-GPU specifications, key differences with performance implications, GB300 NVL72 rack-level performance, Grace CPU unified memory architecture, and multi-node NVLink and NCCL all-reduce test results.

Blackwell GPU architecture overview (TL;DR)

Each Blackwell GPU superchip configuration consists of 1 Grace CPU and 2 Blackwell GPUs. All Blackwell GPUs feature fifth-generation NVLink with 1.8 TB/s total bandwidth (900 GB/s unidirectional).

Per-GPU specifications

Performance numbers without sparsity (divide by 2):

Key differences between GB300 and B200

• Compute: GB300 dense FP4 performance is 66.67% faster compared to B200 (15 vs 9 PFLOPS), FP8 +11.1% (5 vs 4.5 PFLOPS), and FP16/BF16 +11.1% (2.5 vs 2.25 PFLOPS). TF32 is also slightly faster with a speedup of 13.64% (1.25 vs 1.1 PFLOPS).
• Memory capacity and bandwidth: GB300 memory is 55% bigger compared to B200 (279 vs 180 GB) and HBM bandwidth (8 vs 7.7 TB/s, with a boost of +3.9%).
• Power headroom and clocks: GB300 has 40% higher max GPU TDP (1.4 kW vs 1 kW), which increases headroom to sustain higher SM/Tensor-Core clocks during long GEMM-heavy phases.
• The B200 SFU Problem: In B200, the number of SMs on a single die was reduced to 80. Thus, the performance of the SFU (Special Function Unit) paired with the CUDA cores was not enhanced. This tradeoff sacrifices GEMM performance against creating a compute bottleneck for Softmax in Attention. Blackwell Ultra increases SFU from 5 to 10.7 TeraExponentials/s, doubling B200 performance.
• B200/B300/GB300 GEMM performance): Part of the practical GEMM throughput gain from B200 → B300 (1.0 → 1.1 kW) and further to GB300 (1.4 kW) comes from power headroom that lets boost clocks stay higher for longer. GPUs are unable to sustain their peak clock speed due to power throttling. Thus, the power increase also translates into higher achievable performance.

The equation for the theoretical peak compute is as follows:

GB300 NVL72 rack-aggregate performance

Performance numbers without sparsity (divide by 2):

Grace CPU unified memory & fabric architecture

GB300 NVL72 includes 36 Grace CPUs and 72 GPUs (2 GPUs per Grace CPU), establishing a unified, coherent memory architecture via NVLink-C2C interconnects. This design extends the GPU’s addressable memory beyond local HBM3e, creating a capacity-optimized tier accessible across the fabric.

• Grace LPDDR5X bandwidth depends on capacity SKU, up to 512 GB/s for 120/240 GB configs, and up to 384 GB/s for 480 GB configs.
• GB300 NVL72 averages at ~472 GB LPDDR5X per Grace CPU (17 TB / 36), which aligns with the 480 GB-class config.

The NVIDIA GB200 Grace CPU integrates ARM’s high-performance Neoverse V2 cores. However, the implementation does not use the V2 configuration with a 2 MB L2 cache per core, instead employing a reduced 1 MB L2 cache.

Experimental evaluations have reported a materially higher rate of L1 cache misses on GB200 relative to comparable x86-based platforms (see Megatron DeepSeek V3 Training on GB200 NVL72). In addition, the GB200 CX7 does not include an integrated PCIe switch.

As a result, scale-out RDMA traffic must traverse the full Grace network-on-chip (NoC) and then cross the NVLink-C2C interconnect to reach the Blackwell GPU. Most of these problems are solved in the GB300, which uses CX8 and an external PCIe switch.

Multi-node NVLink & NCCL results

Full rackscale NCCL run (18x nodes) from nccl-tests:

OMPI_ALLOW_RUN_AS_ROOT=1 OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 mpirun -H $(echo pod4-gb300-1-tray{01..18}-f3:4 | tr ' ' , ) --mca btl ^openib --mca btl_openib_warn_no_device_params_found 0 ./build/all_reduce_perf -e 16G -n 1000 -b 1G -f 2 -g 1

• Out-of-bounds values: 0 (OK)
• Average bus bandwidth: 836.792 GB/s

References

1. (Zartbot) Inside Nvidia GPU: Discussing Blackwell's Limitations and Predicting Rubin's Microarchitecture #3
2. White Paper, NVIDIA Blackwell Architecture Technical Brief, table 3. System Specifications, Per GPU specs
3. NVIDIA Blackwell Ultra Datasheet
4. NVIDIA GB300 NVL72 Platform
5. NVIDIA Grace Performance Tuning Guide
6. NVIDIA NVLINK Multi-Node Tuning Guide, NCCL
7. Introducing NVFP4 for Efficient and Accurate Low-Precision Inference
8. NVIDIA DGX B300 Datasheet
9. NVIDIA DGX B200 Datasheet
10. Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data! (Horace He)