News from the AI & ML world
DeeperML
@cloud.google.com
//
Google Cloud is advancing its AI Hypercomputer with the introduction of Ironwood TPUs, the seventh generation of Tensor Processing Units, designed specifically for AI inference workloads. This integrated supercomputing system combines optimized hardware, open software, and flexible consumption models to deliver high intelligence per dollar for AI workloads. Google Cloud CEO Thomas Kurian highlights that AI has driven adoption of different parts of the platform, enabling companies to perform super-scaled training or inference of their own models. The AI Hypercomputer underpins nearly every AI workload running on Google Cloud, from Vertex AI to direct access for fine-grained control.
Advances in performance-optimized hardware are central to this innovation. Ironwood boasts 5x more peak compute capacity and 6x the high-bandwidth memory (HBM) capacity compared to the prior-generation, Trillium. It comes in two configurations: 256 chips or 9,216 chips, with the larger pod delivering 42.5 exaFLOPS of compute. Moreover, Ironwood is twice as power efficient compared to Trillium, offering significantly more value per watt. Alongside Ironwood, Google Cloud offers A4 and A4X VMs, featuring NVIDIA B200 and GB200 NVL72 GPUs, respectively. These advancements are supported by enhanced networking, including 400G Cloud Interconnect and Cross-Cloud Interconnect, providing up to 4x more bandwidth than the previous 100G offering.
The new Ironwood TPUs are purpose-built for the age of inference, reflecting the increasing focus on deploying AI models. Ironwood incorporates an enhanced SparseCore, which accelerates sparse operations common in ranking and retrieval-based workloads, improving both latency and power consumption. As AI workloads shift from training to inference, Ironwood's design meets the demands of low-latency and high-throughput performance. This new TPU is integrated into Google's AI Hypercomputer, offering developers access through optimized stacks across PyTorch and JAX.
ImgSrc: storage.googlea
References :
Advances in performance-optimized hardware are central to this innovation. Ironwood boasts 5x more peak compute capacity and 6x the high-bandwidth memory (HBM) capacity compared to the prior-generation, Trillium. It comes in two configurations: 256 chips or 9,216 chips, with the larger pod delivering 42.5 exaFLOPS of compute. Moreover, Ironwood is twice as power efficient compared to Trillium, offering significantly more value per watt. Alongside Ironwood, Google Cloud offers A4 and A4X VMs, featuring NVIDIA B200 and GB200 NVL72 GPUs, respectively. These advancements are supported by enhanced networking, including 400G Cloud Interconnect and Cross-Cloud Interconnect, providing up to 4x more bandwidth than the previous 100G offering.
The new Ironwood TPUs are purpose-built for the age of inference, reflecting the increasing focus on deploying AI models. Ironwood incorporates an enhanced SparseCore, which accelerates sparse operations common in ranking and retrieval-based workloads, improving both latency and power consumption. As AI workloads shift from training to inference, Ironwood's design meets the demands of low-latency and high-throughput performance. This new TPU is integrated into Google's AI Hypercomputer, offering developers access through optimized stacks across PyTorch and JAX.
ImgSrc: storage.googlea
Share:
References :
- Compute: Introducing Ironwood TPUs and new innovations in AI Hypercomputer
- www.marktechpost.com: Google AI Introduces Ironwood: A Google TPU Purpose-Built for the Age of Inference
- BigDATAwire: Google Cloud Preps for Agentic AI Era with ‘Ironwood’ TPU, New Models and Software
- cloud.google.com: Today's innovation isn't born in a lab or at a drafting board; it's built on the bedrock of AI infrastructure.
Classification:
- HashTags: #AIHypercomputer #GoogleCloud #AIinfrastructure
- Company: Google
- Target: AI Developers
- Product: AI Hypercomputer
- Feature: AI Hypercomputer
- Type: AI
- Severity: Informative