Arm and Red Hat Partnership Accelerates Agentic AI Scaling Through Integrated AGI CPU Hardware and OpenShift Hybrid Cloud Software Solutions for Improved Power Efficiency
A radical shift is occurring in the datacenter market the primary workload is shifting away from periodic AI training toward always on agentic AI inferencing. In order to overcome the pressure this places on hardware, Arm has partnered with Red Hat to provide a fully integrated, production ready AI stack. This hardware and software integration will "provide a consistent platform for AI agents across cloud and on premise datacenters," according to a press release from Arm.
At the hardware level is the Arm AGI CPU, which is Arm's first purpose built CPU designed for datacenters and their unique performance, and orchestration demands. Built with 136 Arm Neoverse V3 cores, the chip is designed for handling the data preprocessing and orchestrations of agentic AI systems and is capable of moving data at speed with 96 lanes of PCIe Gen6. Memory is pushed with 12 channels of DDR5 running up to 8800 MT/s for the required bandwidth of constant distributed workloads.
At the software level, Arm has partnered with Red Hat for optimized software solutions, allowing enterprises to use a stable operating system compatible with their existing frameworks. This will allow the two to provide a seamless hybrid cloud experience by allowing AI agents and data pipelines to run across both Arm based cloud instances like AWS Graviton and Google Axion, as well as Arm based hardware servers on premise. Furthermore, Red Hat OpenShift can support Arm based instances and allow the deployment of Kubernetes clusters on the new hardware.
The Red Hat OpenShift Virtualization on the AGI CPU offers the ability to have containers and traditional virtual machines on the same physical infrastructure. This will be crucial for organizations transitioning enterprise workloads to more efficient Arm based hardware without having to rework active workflows. Steven Huels, VP of AI Engineering at Red Hat expressed: "the expansion of the Arm ecosystem is delivering a broader set of solutions for AI ready infrastructure built on trusted open foundations."
With agentic AI workloads, power efficiency and density are crucial factors due to the constant computation being done. While x86 alternatives require 500W or higher, the AGI CPU is built with a TDP of 300W. With the required 36kW air cooled rack configuration the CPU will allow for a compute density of 8160 cores per rack compared to 4352 in older architectures. This nearly doubles density and allows for greater scaling of the inference workload without additional space constraints.
For liquid cooled systems, this number grows even larger, with an upper limit of 336 CPUs per rack totaling 45696 cores, a 5X increase in density. While GPUs still remain the primary hardware for initial AI training, the CPU offers a powerful architecture designed for inference and orchestration at scale. This new stack will be available by the end of the year, and has support from Supermicro, Lenovo, and ASRock Rack.
