 |
| Intel's Gaudi 2 |
Intel Fuses Gaudi 3 and NVIDIA Blackwell Technology into Hybrid AI Server
Intel is said to have a shift in strategy for its Gaudi AI platform that involves bolstering it with NVIDIA technologies. SemiAnalysis reports that the company is running a Gaudi 3 hybrid rack-scale system embedding NVIDIA Blackwell B200 GPUs to strengthen its AI inference performance.
The "If You Can't Beat Them, Join Them" Strategy
The move, announced at the recently held OCP Global Summit, presents a peculiar approach for Intel to push Gaudi chip adoption into an AI market heavily dominated by NVIDIA. Instead of head-to-head competition in every respect, Intel's hybrid construct plans to provide a solution that is one part cost-efficient and two parts leveraging the benefits of both platforms.
Working of the Hybrid System
The design of the system is organized to drive AI inference workloads between the two separate chip architectures, enhancing productivity:
- Intel Gaudi 3 Chips: Decode portion of workload aided by memory bandwidth and Ethernet-based scale-out.
- NVIDIA Blackwell B200 GPUs: Compute-intensive 'prefill' stages of the workload in which large matrix multiply is extremely useful.
System Configuration and Networking
Rack-and-scale solutions have been built with enhanced connectivity. One compute tray comprises two Xeon CPUs, four Gaudi 3 AI chips, four NICs, and one NVIDIA BlueField-3 DPU, reported SemiAnalysis, with 16 such trays forming a single rack.
Networking is done by the ConnectX-7 400 GbE NICs from NVIDIA on the compute trays and Broadcom's Tomahawk 5 switches to provide all-to-all connectivity at the rack level.
Performance Claims and Possible Headwinds
Intel has claimed that its hybrid system can achieve 1.7 times faster prefill performance on small dense models than a Blackwell B200 GPU-only server. This has not been independently verified.
Despite innovative hardware design, the system may yield different headwinds: The Gaudi AI platform's software stack is perceived to be less mature; it may never really get far in broader adoption. Also, a question remains as there are expectations for Gaudi architecture to be phased out, will this hybrid solution ever see the mainstream market?
Source: Information via a SemiAnalysis report.
