Nvidia has introduced a new High Bandwidth Memory architecture, termed NVHBM, aimed at enhancing bandwidth and reducing power consumption in AI applications. This development marks a significant move as Nvidia decides to create its own memory design rather than relying on existing industry solutions. However, the actual memory chips will still be manufactured by one of the leading memory producers such as Micron, SK Hynix, or Samsung.
The announcement coincided with Nvidia’s latest earnings call and is framed as an extension of the NVLink Fusion platform, which targets hyperscalers and AI firms developing custom accelerators, known as XPUs. Amazon’s Annapurna Labs has been noted as the first partner to work with Nvidia on this technology.
NVHBM distinguishes itself from traditional HBM by relocating the memory controller to the base die of the HBM stack, thus minimizing latency and enhancing data transfer efficiency between the memory and the accelerator. This innovative design results in several potential performance advantages:
- Up to 30% increased memory bandwidth compared to standard HBM4E.
- Up to 15% reduction in power consumption for HBM.
- Up to 25% extra area available on the XPU compute die.
- Up to 67% reduction in physical interface area.
- Up to 80% more usable silicon across the overall layout.
These improvements not only enhance memory performance but also allow designers to include additional computational resources without having to increase the device’s size.
Nvidia is positioning NVHBM as a key part of its ecosystem, making it easier for AI companies and hyperscalers to integrate this technology into their custom silicon, ultimately speeding up the development of specialized AI chips.
According to Nvidia, combining the architectural innovations of NVHBM can yield approximately a 30% boost in the overall performance of end-to-end XPUs, factoring in the improvements in memory bandwidth, additional compute die space, and reduced power consumption.
