Recently, Nvidia launched its H200 with 141GB of HBM3E and stole a lot of thunder from AMD. Nvidia’s latest comes as AMD released its Instinct MI300A with 128GB of HBM3 memory and the Instinct MI300X AI accelerator with 192GB of HBM3 memory. However, AMD is developing new versions of Instinct MI300 series products with more memory.
AMD Chief Technology Officer Mark Peppermaster hinted at the possibility during his presentation at the Arte Investor Webinar conference last week.
“We’re not standing still,” the papermaster said (via SeekingAlpha). “We made adjustments around the MI300 family with both memory configurations, a derivative of the MI300, to accelerate our roadmap with the next generation. […] So, we have 8-Hi stacks. We’ve built for a 12-high stack. We are shipping with MI300 HBM3. We have architected for HBM3E.”
It should be noted that AMD’s Instinct MI300A uses an 8-Hi known good stack die (KGSD) with 128GB of HBM3 memory, while the Instinct MI300X comes with 192GB of memory and uses 12-Hi KGDSs. For now, no AMD products use HBM3E, but it looks like the company may use it for some future versions of AMD’s Instinct MI300-series products.
By using HBM3E instead of HBM3, AMD can significantly increase the available memory bandwidth for its AI and HPC GPU or APU. Changing the memory devices attached to its processors is a fairly simple task, so expect AMD to use HBM3E sooner rather than later.
According to AMD’s performance numbers, the Instinct MI300X outperforms Nvidia’s H100 80GB GPU, which is used by major hyperscalers such as Google, Meta (Facebook) and Microsoft. The Instinct MI300X is also likely to be a strong contender against Nvidia’s upcoming H200 141GB GPU. Meanwhile, it remains to be seen what to expect from AMD’s hypothetical Instinct MI300 series with HBM3E memory.
“We’re working very closely with our customers, and what we can tell you, I’ll tell you right now: That race is on, and it’s going to be a competitive race,” Papermaster said. said “You’re going to see it back and forth like you’ve always seen it when you have competition. It’s going to be great for the market. It’s certainly going to encourage our best innovation at AMD. […] Stay tuned as we will be sharing more details on this roadmap as it is truly a multi-year roadmap that we have laid out.”