As its name indicates, it is the third generation of the so-called High Bandwidth Memory, a type of memory standard defined by the JEDEC which differs from the classic DDR and GDDR memories due to the way in which their communication interfaces work with the processors to which they are connected. Its particularity is that its wiring instead of being connected horizontally with the CPU or GPU that it accompanies they are to an interposer that they have below with which they have vertical contact, to which their accompanying processor is also connected.
Thanks to this, the interconnections can be made in a matrix instead of in series, which allows increasing the number of them per area. And what effects does this have? One advantage of this is that it allows you to stack multiple memory chips on top of each other. to take up less space. For this they are used vias through silicon or TSV to intercommunicate the upper memory chips in the stack with the lower ones. In this case they work in a very similar way to 3D NAND.
The second advantage of the HBM3 and its predecessors is a considerable reduction in power consumption, since this increases exponentially with the clock speed and not only when a chip processes data, but also when data is sent. The fact of using a greater number of pins, transfer bits per clock cycle, in HBM3 memories and their predecessors is that they do not require reaching such high clock speeds to reach
The advantage? The fact of having an energy consumption, measured in pJ per bit much lower than the rest of memories and be able to reach the same bandwidths as a GDDR6 consuming half the energy with the HBM2E. In addition, the use of the interposer gives it a much lower latency than the classic memories. However, all this has a price and never better said, since despite its advantages they are very expensive to manufacture, which means that these memories are used in high-performance computing environments and far from the reach of the home user and even the professional. that uses workstations.
Changes in HBM3 memory
With the publication by the JEDEC of the HBM3 standard, we can finally talk about it objectively and not about the proposals made by the different manufacturers in all these years in the face of the new standard. And it can be said that there have been significant changes compared to the previous generation of this type of memory.
More speed
One of the changes that has surprised us the most is the fact that instead of doubling the number of pins and using a 2048-bit bus per memory channel. What they have done in HBM3 is to increase from 3.2 Gbps in the most advanced HBM2E to 6.4 Gbps thereby doubling the clock speed. This is an important change in the philosophy of HBM memories that will result in higher energy consumption of the graphics cards that use them.
This change has surely been motivated by the PCI Express interface changes that allow for graphics cards up to 600W. This gives the HBM3 room to achieve twice the bandwidth without making the memory interface twice as wide. However, as occurs with each new iteration of memory, the capacity of the new manufacturing nodes has been used to lower the voltage at periods ranging from 0.4 V and the 1.1 V and thereby mitigate the rise in clock speed.
More memory channels
In HBM3 memory, and its predecessors, the memory channels correspond to the number of chips that can be placed on the stack at most. Although it is possible to assign two channels per memory chip. For example, in the first two generations where the bus was 1024 bits, up to 8 memory channels could be used. So with a bandwidth of 128 bits for each of the 8 stacked chips, but in general for cost reasons 4 chips with two memory channels for each were used.
In HBM3 the norm has changed, now we have 16 memory channels and since the maximum bus is 1024 bit This means that we can arrive at a 16 chip stack with a bus 64 bit each. However, the standard completes 12 chip configurations and therefore of 768 bit bandwidth. Another element that returns from the second generation of this memory is that of the pseudo channels, this time there are 2 per channel, and can reach 32 of them.
All this means that the bandwidth will go in the first generation of the HBM3 of the 614.4GB/s from 12-chip-per-stack configurations to 819.2GB/s for those with 16 chips per stack. Let’s not forget that you can already see HPC graphics cards with 2, 4, 6 and even 8 chips of this type of memory and this in HBM3 means several Terabytes per second per bandwidth.
Larger storage capacity
To finish we have the amount of memory that the HBM3 memory can hold and this wrath of the 8Gb, 1Gb, 32Gb, 4Gb, for each chip in the stack. If we do quick calculations we can see that we are talking about configurations of 12 or 16GB for the first generation of this memory and progressively we could reach the 48 or 64GB. Remember that these capacities are per memory chip. At the moment we have companies like SK Hynix already offering stacks of 12 chips with a total capacity of 24GB.
In other words, it won’t be long before we see graphics cards with hundreds of gigabytes of RAM, at least in the world of high-performance computing. Who will be the first lucky? The timing tells us that it will be the NVIDIA Hopper that should appear at some point in time in 2022 or in the middle of 2023.
