It was at CES 2022 earlier this year when NVIDIA revealed some of its news for what it presented today, its new servers based on the Grace platform, the latest in HPC and AI. And it is that these servers are going to redefine all the sectors of the market where they enter, since NVIDIA has shown the models HGX Grace Data Center What do you plan to include and… With more than 12,000 cores and 1 TB of RAM, they will be the reference to beat.
Several companies will make available to their customers any of the four types of servers that NVIDIA has designed: ASUS, Foxconn, GIGABYTE, OCT, Supermicro, and Wiwynn, where their customization will elevate the four types of Huang models to over a dozen different business servers. They will arrive in one year, 2023, but we already have the heart and maximum configuration of all of them on the table: HGX Grace CPU Superchips and Grace Hopper Superchip.
NVIDIA HGX Grace: the monstrous server with 12,096 Cores
As always, the details that are not said are the most important, since they reveal data that the company does not want to show even though they are in front of us. HGX Grace type servers represent the biggest giant leap in this world that is remembered.
They will carry the NVIDIA Grace Superchip CPU, that is, two processors coherently connected via NVLink C2C based on ARM V9 Neoverse and designed for AI and HPC infrastructure. It therefore consists of what NVIDIA calls the CPU-CPU module and counts for each PCB that integrates no less than 144 cores with memory LPDDR5X with up to 1TB per rack and a bandwidth of no less than 1TB/s.
all consuming 500 watts and being able to be cooled by air or by water. The most impressive thing here is that NVIDIA allows servers with up to 84 nodes per rack, which is a whopping 12,096 cores total. Considering that the HGX Grace is 1.5 times faster than the DGX A100 to begin with, we can already imagine the beast that the green team has created.
HGX Grace Hopper: CPU and GPU on one PCB
This type of server bets on the second option of NVIDIA. And it is that here we have not two CPUs, but CPU and GPU on the same substrate, which communicate again by NVLink C2C in order to have a coherent high-performance memory model that will be interconnected to 900GB/s being 7 times faster than the PCIe 5.0 bus.
What NVIDIA has created is the ultimate multitasking server, capable of working with any company software stack, be it for HPC, AI or Omniverse, so it is multifaceted. The scheme to follow is simple here with these servers HGX Grace Hoppereach one will integrate a Hopper GPU to 4nm with a Grace CPU, where each one will have its own memory, the first having no less than 80GB of HBM3 and the second 512GB of LPDDR5X available.
This added to the total bandwidth would give us 3.5TB/s with a total consumption of 1000 watts per rack and having the possibility of being cooled by air or water. NVIDIA ensures that it can be installed 42 nodes per rack at HGX Grace Hopper.
Server Designs and Your Portfolio
There will be four specific designs depending on the workload that manufacturers may need, where within these four types each one can configure and customize them according to their needs, which leaves a wide range of benefits and prices:
- The systems NVIDIA HGX Grace Hopper for AI training, inference, and HPC are available with Grace Hopper Superchip and NVIDIA BlueField-3.
- Systems NVIDIA HGX Grace for HPC and supercomputing: They feature a CPU-only design with Grace Superchip CPUs, BlueField-3, and NVIDIA GPUs.
- The systems NVIDIA OVX for digital twins and collaborative workloads are powered by Grace CPU Superchip, BlueField-3, and NVIDIA.
- The systems NVIDIA CGX for graphics and cloud gaming they feature Grace Superchip CPUs, BlueField-3 and NVIDIA A16 GPUs.
As expected, no prices have been revealed yet, since we are a year away from its official launch, so it is to be expected that over this time NVIDIA will reveal the costs, which will surely not be cheap.
