NVIDIA has added its next-generation high-bandwidth memory technology, NVIDIA NVHBM, to NVIDIA NVLink Fusion to increase memory performance and efficiency in AI infrastructure. The technology, which moves the memory controller from the XPU to the base of the 3D HBM stack, provides up to 30% higher memory bandwidth, 15% lower HBM power consumption, and up to 25% more space on the XPU compute chip compared with standard HBM4E.
NVHBM will be offered by multiple memory providers. Amazon’s Annapurna Labs unit will work on the technology as part of its collaboration with NVIDIA and will support the NVLink scaling architecture. AWS also plans to adapt next-generation Trainium chips to NVLink Fusion, starting with Trainium4; this will enable Amazon chips and NVIDIA GPU to work together in a shared rack-scale architecture.
Why it matters
This development shows that, in AI infrastructure, memory’s data-transfer capacity and energy consumption are just as important as processing power in determining system design. Moving memory control to a different layer introduces an approach that could directly affect space utilization and power planning in XPU design. The involvement of multiple memory providers in the technology shows that NVLink Fusion is positioned not as a single hardware component but as a structure that brings together solutions from different manufacturers. AWS’s plan to adapt Trainium chips to this architecture raises the issue of technical compatibility for running custom processors and NVIDIA GPU in the same rack-scale environment. However, it has not yet been disclosed which other memory providers will be involved or in which products these configurations will be used.