AMD EPYC Turin and Venice CPUs Redefine Enterprise Agentic Efficiency through Superior Core Density and Per Core Performance within Datacenter Thermal Boundaries
The shift from ad hoc AI experiments to large scale enterprise agentic environments is putting a lot of stress on our current data center CPU architectures. Although GPU hardware handles the actual model execution for inference and training it is still the central processing unit that holds up the majority of the operational environment and serves as the critical layer that handles database transactions, API traffic, web services and all of the logic surrounding an agentic application's existence. Based on the latest projections from AMD those applications are currently CPU bound and the performance scales with active agents; therefore, the amount of CPUs that you can fit within a single rack will dictate the cost efficiency of the business solution.
The ultimate bottleneck in datacenter operation will always be a physical one: the amount of power we can deliver, the space available on the datacenter floor and the amount of heat we can dissipate. In order to achieve a truly comparable benchmark analysts have standardized on a 100KW rack boundary and are not comparing single socket performance on its own. Under this modeled situation two socket servers are packed into a rack until the 100KW boundary is reached and measured. As AMD's internal benchmarks suggest, AMD's server offerings dramatically outperform NVIDIA and Intel server hardware in this comparison.
This shows the throughput of each of the 4 test configurations within a single 100Kw rack boundary, performing 6 separate enterprise workloads which consist of, SPECrate 2017 Integer, server side Java, NGINX web serving, Redis key value storage, Memcached storage and TPROC C relational databases. For clarity in comparing across platforms, the performance of the 88 core NVIDIA Vera processor, tested with 88 cores running a single instance of each of the test workloads, was benchmarked to be 1.00 for ease of scaling purposes.
The 128 core Intel Xeon 6980P had a geometric mean throughput of 1.46 times greater than the NVIDIA Vera processor, at the same 100kw thermal envelope, the 192 core AMD EPYC 9965 (Codename Turin), which is the current latest and greatest of AMD's server platform, showed 2.37x throughput, whereas the next generation 256 core AMD EPYC Venice is predicted to be 3.30x. The above demonstrates that even within a 100Kw thermal envelope simply adding more CPU cores yields more capacity and higher levels of concurrency for each of our applications.
There is no point having a high core density if it cannot be utilized efficiently by the existing infrastructure and software stacks available on the market. In standard liquid cooled datacenters the physical density of CPUs is far more efficient when stacked, AMD's EPYC Turin processor can currently support up to over 27,000 CPU cores per standard 2U rack using Dell PowerEdge IR7000 server chassis and beyond 36,000 cores using the 256 core Venice design. Comparing the physical space available the NVIDIA Vera processor can support about 22,500 cores or sandboxes; most enterprises can already take full advantage of the x86 software ecosystem to add agents and there is no reason to move off of it to use closed proprietary architectures and thus be at the mercy of the vendors that control them.
Although it is important to maximize the number of concurrently available threads to handle the greatest number of agents simultaneously there is still a requirement for increased per core performance when executing certain transactions like the database operations or running analytics programs. It has been measured that the 64 core version of Venice can achieve over 27% greater performance per core than the 88 core NVIDIA Vera, and the higher 96 core version of Venice has an estimated performance increase per core of 11% greater than NVIDIA, while staying within the same power boundaries.
The best solution for increasing the efficiency of enterprise agentic workloads is in building higher densities in a single rack by increasing CPU core count within the established system parameters of power and temperature for existing server infrastructure.
