DeeperML

NVIDIA's Blackwell platform has achieved top performance in the latest MLPerf Inference v5.0 benchmarks. This marks the first submission for NVIDIA using the GB200 NVL72 system, designed as a rack-scale solution specifically for AI reasoning. The Blackwell platform showcased substantial performance gains, particularly excelling in demanding AI inference scenarios, and highlighted the increasing focus on generative AI within the industry.

The benchmarks included new, challenging workloads such as Llama 3.1 405B and Llama 2 70B Interactive, the latter featuring stricter latency requirements to mirror real-world chatbot operations. The GB200 NVL72, connecting 72 Blackwell GPUs to function as one massive GPU, achieved up to 30x higher throughput on the Llama 3.1 405B benchmark compared to NVIDIA's H200 NVL8 submission. These results underscore the platform's ability to handle complex AI models and deliver high inference throughput with low latency.

ImgSrc: blogs.nvidia.co

References :

• NVIDIA Newsroom: Speed Demon: NVIDIA Blackwell Takes Pole Position in Latest MLPerf Inference Results
• NVIDIA Technical Blog: The compute demands for large language model (LLM) inference are growing rapidly, fueled by the combination of growing model sizes, real-time latency...
• IEEE Spectrum: Nvidia Blackwell Ahead in AI Inference, AMD Second
• insideAI News: MLCommons Releases MLPerf Inference v5.0 Benchmark Results
• insidehpc.com: MLCommons Releases New MLPerf Inference v5.0 Benchmark Results
• www.networkworld.com: Nvidia’s Blackwell raises the bar with new MLPerf Inference V5.0 results

Classification:

• HashTags: #MLPerf #Blackwell #AIInference
• Company: NVIDIA
• Target: AI workloads
• Product: Blackwell
• Feature: AI Inference
• Type: AI
• Severity: Informative