DeeperML

NVIDIA has launched Dynamo, an open-source inference software, designed to accelerate and scale reasoning models within AI factories. Dynamo succeeds the NVIDIA Triton Inference Server, representing a new generation of AI inference software specifically engineered to maximize token revenue generation for AI factories deploying reasoning AI models. The software orchestrates and accelerates inference communication across thousands of GPUs, utilizing disaggregated serving.

Dynamo optimizes AI factories by dynamically managing GPU resources in real-time to adapt to request volumes. Dynamo’s intelligent inference optimizations have shown to boost the number of tokens generated by over 30 times per GPU and has demonstrated the ability to double the performance and revenue of AI factories serving Llama models on NVIDIA’s current Hopper platform.

ImgSrc: www.artificiali

References :

• AI News: NVIDIA Dynamo: Scaling AI inference with open-source efficiency
• BigDATAwire: At its GTC event in San Jose today, Nvidia unveiled updates to its AI infrastructure portfolio, including its next-generation datacenter GPU, the NVIDIA Blackwell Ultra.
• AIwire: Nvidia’s DGX AI Systems Are Faster and Smarter Than Ever
• NVIDIA Newsroom: NVIDIA Blackwell Powers Real-Time AI for Entertainment Workflows
• MarkTechPost: Details the Open Sourcing of Dynamo

Classification:

• HashTags: #NVIDIA #AIInference #OpenSourceAI
• Company: Nvidia
• Target: AI Factories
• Product: Dynamo
• Feature: AI Inference
• Malware: Dynamo
• Type: ProductUpdate
• Severity: Informative