ThinkSystem NVIDIA HGX H200 141GB 700W 8-GPU Board

The NVIDIA H200 Tensor Core GPU supercharges generative AI and high-performance computing (HPC) workloads with game-changing performance and memory capabilities. H200 is the newest addition to NVIDIA’s leading AI and high-performance data center GPU portfolio, bringing massive compute to data centers.

The NVIDIA H200 141GB 700W GPU is offered in the ThinkSystem SR680a V3 server, with eight SXM5 form factor GPU modules and NVIDIA® NVLink® Fabric to create an 8-FC (fully connected) NVLink topology per baseboard. Leveraging the power of H200 multi-precision Tensor Cores, an eight-way HGX H200 provides over 32 petaFLOPS of FP8 deep learning compute and over 1.1TB of aggregate HBM memory for the highest performance in generative AI and HPC applications.

Figure 1. ThinkSystem NVIDIA HGX H200 141GB 700W 8-GPU Board in the ThinkSystem SR680a V3 server

To maximize compute performance, H200 is the world’s first GPU with HBM3e memory with 4.8TB/s of memory bandwidth, a 1.4X increase over H100. H200 also expands GPU memory capacity nearly doubled to 141GB. The combination of faster and larger HBM memory accelerates performance of computationally intensive generative AI and HPC applications, while meeting the evolving demands of growing model sizes.

The following table shows the part numbers for the 8-GPU board. Feature code C1HMcontains 8x H200 GPUs in the SXM form factor plus the NVLink high-speed interconnections.

The NVIDIA H200 Tensor Core GPU supercharges generative AI and HPC with game-changing performance and memory capabilities. As the first GPU with HBM3e, H200’s faster, larger memory fuels the acceleration of generative AI and LLMs while advancing scientific computing for HPC workloads.

NVIDIA HGX™ H200, the world’s leading AI computing platform, features the H200 GPU for the fastest performance. An eight-way HGX H200 provides over 32 petaflops of FP8 deep learning compute and 1.1TB of aggregate high-bandwidth memory for the highest performance in generative AI and HPC applications.

Key AI and HPC workload features:

• Unlock Insights With High-Performance LLM Inference

  In the ever-evolving landscape of AI, businesses rely on large language models to address a diverse range of inference needs. An AI inference accelerator must deliver the highest throughput at the lowest TCO when deployed at scale for a massive user base. H200 doubles inference performance compared to H100 when handling LLMs such as Llama2 70B.

• Optimize Generative AI Fine-Tuning Performance

  Large language models can be customized to specific business case needs with fine-tuning, low-rank adaptation (LoRA), or retrieval-augmented generation (RAG) methods. These methods bridge the gap between general pretrained results and task-specific solutions, making them essential tools for industry and research applications.

  NVIDIA H200’s Transformer Engine and fourth-generation Tensor Cores speed up fine-tuning by 5.5X over A100 GPUs. This performance increase allows enterprises and AI practitioners to quickly optimize and deploy generative AI to benefit their business. Compared to fully training foundation models from scratch, fine-tuning offers better energy efficiency and the fastest access to customized solutions needed to grow business.

• Industry-Leading Generative AI Training

  The era of generative AI has arrived, and it requires billion-parameter models to take on the paradigm shift in business operations and customer experiences.

  NVIDIA H200 GPUs feature the Transformer Engine with FP8 precision, which provides up to 5X faster training over A100 GPUs for large language models such as GPT-3 175B. The combination of fourth-generation NVLink, which offers 900GB/s of GPU-to-GPU interconnect, PCIe Gen5, and NVIDIA Magnum IO™ software, delivers efficient scalability from small enterprise to massive unified computing clusters of GPUs. These infrastructure advances, working in tandem with the NVIDIA AI Enterprise software suite, make the NVIDIA H200 the most powerful end-to-end generative AI and HPC data center platform.

• Supercharged High-Performance Computing

  Memory bandwidth is crucial for high-performance computing applications, as it enables faster data transfer and reduces complex processing bottlenecks. For memory-intensive HPC applications like simulations, scientific research, and artificial intelligence, H200’s higher memory bandwidth ensures that data can be accessed and manipulated efficiently, leading to up to a 110X faster time to results.

  The NVIDIA data center platform consistently delivers performance gains beyond Moore’s Law. And H200’s breakthrough AI capabilities further amplify the power of HPC+AI to accelerate time to discovery for scientists and researchers working on solving the world’s most important challenges.

• Reduced Energy and TCO

  In a world where energy conservation and sustainability are top of mind, the concerns of business leaders and enterprises have evolved. Enter accelerated computing, a leader in energy efficiency and TCO, particularly for workloads that thrive on acceleration, such as HPC and generative AI.

  With the introduction of H200, energy efficiency and TCO reach new levels. This cutting-edge technology offers unparalleled performance, all within the same power profile as H100. AI factories and at-scale supercomputing systems that are not only faster but also more eco-friendly deliver an economic edge that propels the AI and scientific community forward. For at-scale deployments, H200 systems provide 5X more energy savings and 4X better cost of ownership savings over the NVIDIA Ampere architecture generation.

The following table lists the NVIDIA H200 GPU specifications.

The following tables list the ThinkSystem servers that are compatible.

Operating system support is based on that of the supported servers. See the SR680a V3 server product guide for details: https://lenovopress.lenovo.com/lp1909-thinksystem-sr680a-v3-server

This section lists the NVIDIA software that is available from Lenovo.

• NVIDIA vGPU Software (vApps, vPC, RTX vWS, and vCS)
• NVIDIA Omniverse Software (OVE)
• NVIDIA AI Enterprise Software
• NVIDIA HPC Compiler Software

• Virtual PC (vPC)

  This product is ideal for users who want a virtual desktop but need great user experience leveraging PC Windows® applications, browsers and high-definition video. NVIDIA Virtual PC delivers a native experience to users in a virtual environment, allowing them to run all their PC applications at full performance.

• NVIDIA RTX Virtual Workstation (RTX vWS)

  NVIDIA RTX vWS is the only virtual workstation that supports NVIDIA RTX technology, bringing advanced features like ray tracing, AI-denoising, and Deep Learning Super Sampling (DLSS) to a virtual environment. Supporting the latest generation of NVIDIA GPUs unlocks the best performance possible, so designers and engineers can create their best work faster. IT can virtualize any application from the data center with an experience that is indistinguishable from a physical workstation — enabling workstation performance from any device.

• Virtual Compute Server (vCS)

  NVIDIA Virtual Compute Server (vCS) enables data centers running on Red Hat Enterprise Linux, Red Hat Virtualization, and other supported KVM-based hypervisors to accelerate server virtualization with the latest NVIDIA data center GPUs, so that the most compute-intensive workloads, such as artificial intelligence, deep learning, and data science, can be run in a virtual machine (VM) powered by NVIDIA vGPU technology.

The following license types are offered:

• Perpetual license

  A non-expiring, permanent software license that can be used on a perpetual basis without the need to renew. Each Lenovo part number includes a fixed number of years of Support, Upgrade and Maintenance (SUMS).

• Annual subscription

  A software license that is active for a fixed period as defined by the terms of the subscription license, typically yearly. The subscription includes Support, Upgrade and Maintenance (SUMS) for the duration of the license term.

• Concurrent User (CCU)

  A method of counting licenses based on active user VMs. If the VM is active and the NVIDIA vGPU software is running, then this counts as one CCU. A vGPU CCU is independent of the connection to the VM.

The following table lists the ordering part numbers and feature codes.

NVIDIA AI Enterprise Software

Lenovo offers the NVIDIA AI Enterprise (NVAIE) cloud-native enterprise software. NVIDIA AI Enterprise is an end-to-end, cloud-native suite of  AI and data analytics software, optimized, certified, and supported by NVIDIA to run on VMware vSphere and bare-metal  with NVIDIA-Certified  Systems™.  It includes key enabling technologies from NVIDIA for rapid deployment, management, and scaling of AI workloads in the modern hybrid cloud.

NVIDIA AI Enterprise is licensed on a per-GPU basis. NVIDIA AI Enterprise products can be purchased as either a perpetual license with support services, or as an annual or multi-year subscription.

The features of NVIDIA AI Enterprise Software are listed in the following table.

Note: Maximum 10 concurrent VMs per product license

The following table lists the ordering part numbers and feature codes.

Find more information in the NVIDIA AI Enterprise Sizing Guide.

NVIDIA HPC Compiler Software

The NVIDIA H200 GPU has the following regulatory approvals:

• RCM
• BSMI
• CE
• FCC
• ICES
• KCC
• cUL, UL
• VCCI

The NVIDIA H200 GPU assumes the server’s base warranty and any warranty upgrades.

The following sales training courses are offered for employees and partners (login required). Courses are listed in date order.

1. Generative AI Overview Foundational
   2024-02-16 | 17 minutes | Employees Only
   Details
   Generative AI Overview Foundational
   
   

   It seems the whole world is excited about Generative AI, and while some of it is just hype, it has become clear that Generative AI has the potential to revolutionize many aspects of our personal and professional lives. In this brief NVIDIA course, we'll explore one aspect of the Generative AI excitement, the value you get from Generative AI technology. We will discuss what Generative AI is, how it works, and how enterprises are planning to use this technology.
   
   By the end of this course, you will be able to discuss the Generative AI market trends and the challenges in this space with your customers. And you will be able to explain what Generative AI is and how the technology works to help enterprises unlock new opportunities for business.

   Published: 2024-02-16
   Length: 17 minutes
   Employee link: Grow@Lenovo
   Course code: DAINVD106
2. Industry Use Cases in Modern Computing Foundational
   2024-02-16 | 9 minutes | Employees Only
   Details
   Industry Use Cases in Modern Computing Foundational
   
   

   As GPU powered computing continues to improve exponentially, applications that were once science fiction are becoming best practice. This is an introductory NVIDIA course that explores some exciting industry focused use cases that are providing companies with faster time to insight, productivity at scale and a great ROI.
   
   By the end of this course, you will be able to explain how companies in a few key industry verticals are benefiting from a variety of accelerated compute use cases.

   Published: 2024-02-16
   Length: 9 minutes
   Employee link: Grow@Lenovo
   Course code: DAINVD105
3. Introduction to Artificial Intelligence Foundational
   2024-02-16 | 10 minutes | Employees Only
   Details
   Introduction to Artificial Intelligence Foundational
   
   

   This NVIDIA course aims to answer questions such as, what is AI and why are enterprises so interested in it? and how does AI happen, why are GPUs so important for it, and what does a good AI solution look like?
   
   By the end of this training, you should be able to describe AI and relate it to some common enterprise use cases. You'll know the difference between training and inference and be able to visualize a typical AI workflow. More importantly, you'll understand the difficulties of traditional CPU-based AI and appreciate why businesses would benefit greatly by adopting GPU-accelerated workflows. Finally, you'll also understand what features contribute to an awesome AI solution and why customers respect and enjoy NVIDIA's solutions.

   Published: 2024-02-16
   Length: 10 minutes
   Employee link: Grow@Lenovo
   Course code: DAINVD104
4. GPU Fundamentals Foundational
   2024-02-16 | 10 minutes | Employees Only
   Details
   GPU Fundamentals Foundational
   
   

   This NVIDIA course introduces you to two devices that a computer typically uses to process information, the CPU and the GPU. We'll discuss their differences and look at how the GPU overcomes the limitations of the CPU. Once you understand the power and advantages of GPU processing, we will talk about the value GPUs bring to modern-day enterprise computing.
   
   By the end of this course, you should know the difference between serial and parallel processing. You will be able to explain what a GPU is in very simple terms and explain the value that GPUs bring to enterprises. Additionally, you'll become familiar with the typical GPU-accelerated enterprise workloads and list one or two use cases under them. By the time you exit this course, you should be able to target various GPU-accelerated computing opportunities with the right NVIDIA GPU.

   Published: 2024-02-16
   Length: 10 minutes
   Employee link: Grow@Lenovo
   Course code: DAINVD103
5. Partner Technical Webinar – NVidia
   2023-12-11 | 60 minutes | Employees and Partners
   Details
   Partner Technical Webinar – NVidia
   
   

   In this 60-minute replay, Brad Davidson of Nvidia will help us recognize AI Trends, and Discuss Industry Verticals Marketing.

   Published: 2023-12-11
   Length: 60 minutes
   Employee link: Grow@Lenovo
   Partner link: Lenovo Partner Learning
   Course code: 120823

For more information, refer to these documents:

• ThinkSystem and ThinkAgile GPU Summary:
  https://lenovopress.lenovo.com/lp0768-thinksystem-thinkagile-gpu-summary
• ServerProven compatibility:
  https://serverproven.lenovo.com/
• NVIDIA H200 product page:
  https://www.nvidia.com/en-us/data-center/h200/
• NVIDIA Hopper Architecture page
  https://www.nvidia.com/en-us/data-center/technologies/hopper-architecture/
• ThinkSystem SR680a V3 product guide
  https://lenovopress.lenovo.com/lp1909-thinksystem-sr680a-v3-server