Checking out the NVIDIA H100 in Our First Look at Hopper

Patrick With The NVIDIA H100 At NVIDIA HQ April 2022
Patrick With The NVIDIA H100 At NVIDIA HQ April 2022

At GTC 2022, NVIDIA had some nice renderings of the NVIDIA H100. A few weeks ago I was able to hold one, and I just got the call that I can now share the photos. For those that prefer live shots instead of renderings, here is our first look at the NVIDIA H100 Hopper GPU that will be a big deal in the market starting later this year.

Checking out the NVIDIA H100 at NVIDIA HQ

Contrary to some rumors, I was not in Santa Clara, California in early April checking out NVIDIA’s newest BBQ gear.

Instead, as you may have seen in our NVIDIA Cedar Fever 1.6Tbps Modules Used in the DGX H100 piece, I was checking out some new hardware. One of those was this covered by the Random Floating Black Box at NVIDIA:

Patrick With A Random Floating Black Box At NVIDIA HQ April 2022
Patrick With A Random Floating Black Box At NVIDIA HQ April 2022

As of today, the RFBB can be lifted, and this was actually me posing with a NVIDIA H100.

Patrick With The NVIDIA H100 At NVIDIA HQ April 2022
Patrick With The NVIDIA H100 At NVIDIA HQ April 2022

Here is the front side of the NVIDIA H100. You can see the SXM packaging is getting fairly packed at this point. There is a lot more here than we saw on the V100 generation.

Patrick With The NVIDIA H100 At NVIDIA HQ April 2022 Front Side
Patrick With The NVIDIA H100 At NVIDIA HQ April 2022 Front Side

The GPU itself is the center die with a CoWoS design and six packages around it. Five of those packages we were told during GTC 2022 are active as HBM and one is there for yield/ structural support. Just for some comparison, here is a V100 from the SC show floor:

Supermicro 8x NVlink NVIDIA Tesla V100 Via Microway
Supermicro 8x NVlink NVIDIA Tesla V100 Via Microway

While I was there, I was allowed to hold the second H100 package that failed and that was not mounted on the SXM PCB. I was just not allowed to take photos of that, even with a RFBB. It was marked as #2. The module itself looks fairly close to the rendering we saw at GTC 2022.

NVIDIA GTC 2022 H100 SXM Module
NVIDIA GTC 2022 H100 SXM Module

Here’s the back of the module:

Patrick With The NVIDIA H100 At NVIDIA HQ April 2022 Backside
Patrick With The NVIDIA H100 At NVIDIA HQ April 2022 Backside

Something that is very different in this generation is the connectors. Since I was not really supposed to take pictures of these (I used my phone, not the Canon R5/ R5 CI had with me for these), I also did not get to take the Molex covers off. In previous generations, like the V100, we would have two long connectors on each side of the GPU. So this is certainly a new connector layout.

Final Words

Since I just got the call that we could publish the photos that I was probably not supposed to take in the first place, I just wanted to get these out there. The H100 is real. What NVIDIA needs is for the next-generation of AMD EPYC and Intel Xeon CPUs to come out for PCIe Gen5 support so that it has a platform for its next-gen PCIe Gen5 GPUs, BlueField-3 DPUs, ConnectX-7, and more. We confirmed in the Cedar Fever article that NVIDIA has motherboard designs for the DGX H100 for both Sapphire Rapids and Genoa, but one will be picked closer to launch (if it has not been in the past month.)

These photos are actually one of the big drivers behind NVIDIA Grace. NVIDIA has the H100 today, but it does not have x86 PCIe Gen5 platforms to sell to customers, and it will not be using Ampere or another Arm solution in this generation. NVIDIA having its own CPU will allow the company to put out new accelerator generations without having to wait for delayed x86 CPUs in the future.

Leave a Comment