Recapping NVIDIA GTC 2024 Keynote

The entire keynote can be summarize in one sentence. NVIDIA aims to bridge the digital and physical world by delivering the hardware, software, and services to change the way we develop solutions to problems, and power the robotics that can interact with the physical world seamlessly.

They highlighted 5 key things:

• NVIDIA Blackwell GPU Platform
• Generative AI & Digitizing the Physical World
• NVIDIA NIMs
• NVIDIA Omniverse Simulation & Digital Twins
• NVIDIA Powered Robotics

These 5 innovations at NVIDIA make up NVIDIA’s mission to connect digital computing to benefit the physical world.

NVIDIA Blackwell GPU - B100, GB200, and DGX GB200 NVL72

NVIDIA Blackwell is the newly announced data center GPU that will supersede the NVIDIA Hopper GPUs. One NVIDIA Blackwell die is composed of 104 billion transistors, but the actual NVIDIA Blackwell GPU is two dies butted up against each other to construct one huge GPU with 208 billion transistors, zero latency, and full cache coherency. With 192GB of HBM3e memory and 20 petaFLOPS of AI performance, NVIDIA Blackwell GPU, or B100, is 5x the AI performance than the Hopper H100. Here’s a slide that details Blackwell’s features.

CEO Jensen Huang, in his keynote, expressed that Blackwell isn’t the GPU itself, but the platform for powering next-gen AI workloads. NVIDIA pairs two Blackwell with one Grace CP, constructing the GB200 Superchip. These Superchips are the building blocks for the liquid cooled 1U server featuring 2 GB200s connected via NVLink and networked with 4 ConnectX-800G InfiniBand and 4 Bluefield 3 SuperNICs DPUs.

These Blackwell Compute Nodes are then deployed in a liquid cooled rack solution called the DGX GB200 NVL72 comprised of:

• 18 Blackwell Compute Nodes
  • 72 Blackwell GPUs and 36 Grace CPUs
  • 72 ConnectX-800G InfiniBand SuperNICs
  • 18 Bluefield-3 DPU with 80GB/s memory bandwidth
• 9 New 5th Gen NVLink Switch Systems
  • Each with 2 NVLink Switch Chip
• 2 Quantum InfiniBand Networking Switch
  • Capable of 800Gb/s throughput per port
  • 230.4Tb/s of aggregated bidirectional bandwidth
• 1 NVLink Spine for connecting everything altogether in a single rack

This is the new DGX. But of course, NVIDIA will also have B100s in the SXM form factor for slotting into existing H100s and catering to the x86 market. They will ship HGX B100 system boards that can be hot swapped to replace DGX and HGX H100 deployments.

Now that the computing is handles, we can go into what NVIDIA has planned for what to compute.

Generative AI - Digitizing Everything

Generative AI has changed the way people interact with information. Now with tools like LLM Chatbots, Generative AI Text to Image Generation, and more, average people like you and I are interacting with the AI off the next generation. What can be digitized can be analyzed and fed into a generative AI model.

We have been processing text, analyzing images, predicting proteins, and parsing genome sequences. We can generate or output text, create new and unique images, fold millions of proteins, and identify new genomes that can possibly cure pathogens.

The compute that powers the trillion parameter generative AI models are NVIDIA. But since these models are huge, this comes a disadvantage at not being extremely skilled. We have AIs everywhere floating around, open source, and difficult to install, run, or even implement with other AIs. Coupled with high complexity, it is difficult to integrate and deploy reliably within a specific workflow. This is where NVIDIA Inference Microservices come to play.

NVIDIA Inference Microservices or NIMs - AIs Assemble

NIMs are specialized pre-trained AI models that are packaged container and optimized to your compute while also being connected to open-source AI APIs. NVIDIA is building out NIMs to bring a new way to program software through gathering a team of NIMs or AI chatbots to tackle a prompt, a problem, a request/event, and more. They will handle the library of AIs and consolidate them in a single location.

You can even customize you NIMs to curate an AI Model for domain complexity, task classification, finetuning, security, and more. Huang expresses his vision to be the AI foundry for the world by building AIs for everyone and anyone to take and run them.

NVIDIA Omniverse and Digital Twins - Digital Before Physical

NVIDA has been talking about Omniverse for years now and highlighting its importance to every single industry in the world. As the Omniverse expands to new use cases, the possibilities of simulation are endless. NVIDIA Omniverse is already being used to simulate warehouses, factories, roads, and even training grounds for robotics. By digitizing the plans, we can analyze and optimize the environment to a specific standard without having to go through physical prototyping. It's like the next level past 3D mapping and planning since you can also train NVIDIA powered robotics within the environment to function optimally prior to deployment for higher degree of efficiency and optimization.

NVIDIA Robotics - Translating Physics based Computing to Physical Intelligence

“Everything that moves will become robotics.” We are already at the forefront of autonomous machinery with cars, manipulating arms, and eventually humanoid robots. NVIDIA’s edge devices inside cars drive in NVIDIA Omniverse for testing, optimization, and finetuning against normal and edge circumstances.

Many robotics deployments like manipulator arms and AGVs are preprogrammed systems that repeat a task or follow digital rails. However, if a small thing goes awry, human intervention is required to clear any stoppages, adjust locations, and are constrained to the vision model and the pre-programming.

With Issac Perceptor, NVIDIA is able to give these systems perception and reasoning. Give it a task or a goal, NVIDIA will run a series of NIMs that have already been finetuned within NVIDIA Omniverse to tackle novel problems in our real world.

NVIDIA Providing the Whole Computing Stack for Physical AI

NVIDIA is providing the whole computing product stack. We already know and love DGX, we were introduced to OVX recently for powering Omniverse, and robotics will be powered by what they call AGX.

NVIDIA DGX will train the foundational AI models for these robotics. LLMs will allow users communicate with their robots with language. NVIDIA OVX will power the NVIDIA Omniverse simulation for training the AI within an environment. NVIDIA AGX is family of systems for self-driving cars and autonomous vehicles, intelligent machines, and medical imaging for inferencing in the physical factory, within humanoid robots, and performing manufacturing plant task on the edge.

Wrapping up the Keynote

NVIDIA is not just an AI foundry but the full computing solution for enabling our digital innovations to interact with the real physical world. DGX will train foundational models, OVX will simulate and train robotics movement, and AGX will power the robotics in the real world.

Every year NVIDIA's GTC conference is a highlight for the AI community showcasing inspiration innovation and discovery for the way we compute things at large. With AI emerging as an essential way to process and interact with information in every industry, NVIDIA is pushing forward to enabling and powering the AI of the next generation.

If you're looking to evaluate and configure a workstation for whatever workload, explore SabrePC's deep learning platforms for training AI. We help with the budgeting and hardware optimization for you so you can get the best bang for your buck. Contact us today to learn more!