Elastic GPU Service:GPU-accelerated compute-optimized instance families (gn, ebm, and scc series)

gn8v and gn8v-tee, GPU-accelerated compute-optimized instance families

The gn8v and gn8v-tee instance families are available only in specific regions, including regions outside China. To use the instance families, contact Alibaba Cloud sales personnel.

• Introduction:

  • gn8v: This instance family is an 8th-generation GPU-accelerated compute-optimized instance family provided by Alibaba Cloud for AI model training and the inference tasks of ultra large language models (LLMs). This instance family consists of multiple instance types that provide one, two, four, or eight GPUs per instance.

  • gn8v-tee: To meet security requirements for training and inferring large language models, Alibaba Cloud provides an eighth-generation instance family based on gn8v that includes the confidential computing feature. This instance type encrypts data during GPU computing to ensure user data security.

• Use cases:

  • Multi-GPU parallel inference computing for LLMs that have more than 70 billion parameters

  • Traditional AI model training and autonomous driving training, for which each GPU delivers computing power of up to 39.5 TFLOPS in the single-precision floating-point format (FP32)

  • Small and medium-sized model training scenarios that leverage the NVLink connections among the eight GPUs

• Benefits and positioning:

  • High-speed and large-capacity GPU memory: Each GPU is equipped with 96 GB of HBM3 memory and delivers up to 4 TB/s of memory bandwidth, which greatly accelerates model training and inference.

  • High bandwidth between GPUs: Multiple GPUs are interconnected by using 900 GB/s NVLink connections. The efficiency of multi-GPU training and inference is much higher than that of previous generations of GPU-accelerated instances.

  • Quantization of LLMs: This instance family supports computing power in the 8-bit floating point format (FP8) and optimizes computing power for large-scale parameter training and inference. This significantly improves the computing speed of training and inference and reduces memory usage.

  • (Only for the gn8v-tee instance family) High security: The gn8v-tee instance family supports confidential computing capabilities that cover the full link of model inference tasks. The capabilities include CPU-based Intel Trust Domain Extensions (TDX) confidential computing and GPU-based NVIDIA Confidential Computing (CC). The confidential computing capabilities ensure the security of user inference data and enterprise models in model inference and training.

• Compute:

  • Uses the latest Cloud Infrastructure Processing Unit (CIPU) 1.0 processors.

    • Decouples computing capabilities from storage capabilities, allowing you to flexibly select storage resources based on your business requirements.

    • Provides bare metal capabilities to support peer-to-peer (P2P) communication between GPU-accelerated instances.

  • Uses the 4th-generation Intel Xeon Scalable processors that deliver a base frequency of up to 2.8 GHz and an all-core turbo frequency of up to 3.1 GHz.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • These instances support the NVMe protocol. For more information, see Overview of the NVMe protocol.

  • Supports elastic ephemeral disks, ESSDs, ESSD AutoPL disks, and Regional ESSDs. For information about disks, see Overview of Block Storage.

• Network:

  • Supports IPv4 and IPv6. For more information about IPv6 communication, see IPv6 communication.

  • These instances support jumbo frames. For more information, see Jumbo frames.

  • Provides ultra-high network performance with a packet forwarding rate of up to 30,000,000 pps (for instances equipped with eight GPUs).

  • Supports elastic RDMA interfaces (ERIs).

  • Note

    For information about how to use ERIs, see Enable eRDMA on an enterprise-level instance.

• Security: Supports the Trusted Platform Module (vTPM) feature. This feature is supported by gn8v but not by gn8v-tee. For more information, see Overview of trusted computing capabilities.

gn8v includes the instance types and metric data listed in the following table.

gn8v-tee includes the instance types and metric data listed in the following table.

gn8is, GPU-accelerated compute-optimized instance family

This instance family is available only in specific regions, including regions outside China. To use the instance family, contact Alibaba Cloud sales personnel.

• Introduction: gn8is is the eighth-generation GPU-accelerated compute-optimized instance family from Alibaba Cloud, developed in response to the growth of AI-generated content (AIGC) services. It uses the latest NVIDIA L20 GPUs and provides 1-GPU, 2-GPU, 4-GPU, and 8-GPU instance types, along with instance types with different CPU-to-GPU ratios, to meet various application requirements.

• Benefits and positioning:

  • Graphics processing: This instance family uses 4th-generation Intel Xeon Scalable high-frequency processors to provide sufficient CPU computing power for 3D modeling scenarios, which makes graphics rendering and design smoother.

  • Inference tasks: It uses the new NVIDIA L20 GPU and provides 48 GB of video memory per GPU to accelerate inference tasks. It supports the FP8 floating-point number format and can be used with ACK containers to flexibly support the inference of various AIGC models. It is especially suitable for inference tasks on LLM models with fewer than 70 billion parameters.

• Use cases:

  • Animation, special effects for film and television, and rendering

  • Generation of AIGC images and inference of LLMs

  • Other general-purpose AI recognition, image recognition, and speech recognition scenarios

• Compute:

  • Uses the new NVIDIA L20 enterprise-grade GPUs.

    • Support for acceleration features, such as TensorRT, and the FP8 floating-point format to improve LLM inference performance.

    • Up to 48 GB of memory per GPU and support for the inference of 70B or larger LLMs on a single instance with multiple GPUs.

    • Improved graphic processing capabilities. For example, after you install a GRID driver on a gn8is instance by using Cloud Assistant or an Alibaba Cloud Marketplace image, the instance can provide graphic processing performance twice that of a 7th-generation instance.

  • Key parameters of NVIDIA L20:

    GPU architecture	GPU memory	Compute performance	Video encoding/decoding capabilities	Inter-card connection
    NVIDIA Ada Lovelace	Capacity: 48 GB Bandwidth: 864 GB/s	FP64: N/A FP32: 59.3 TFLOPS FP16/BF16: 119 TFLOPS FP8/INT8: 237 TFLOPS	3 × Video Encoder (+AV1) 3 × Video Decoder 4 × JPEG Decoder	PCIe interface: PCIe Gen4 x16 Bandwidth: 64 GB/s

  • Uses the latest high-frequency Intel^® Xeon^® processors that deliver an all-core turbo frequency of 3.9 GHz to meet complex 3D modeling requirements.

• Storage:

  • Is an instance family in which all instances are I/O optimized.

  • These instances support the NVMe protocol. For more information, see Overview of the NVMe protocol.

  • Supports elastic ephemeral disks, Enterprise SSDs (ESSDs), ESSD AutoPL disks, and Regional ESSDs. For information about disks, see Overview of Block Storage.

• Network:

  • Supports IPv4 and IPv6. For more information about IPv6 communication, see IPv6 communication.

  • Supports ERIs.

    Note

    For information about how to use ERIs, see Enable eRDMA on an enterprise-level instance.

• Security: These instances support the vTPM feature. For more information, see Overview of trusted computing.

gn7e, GPU-accelerated compute-optimized instance family

gn7i, GPU-accelerated compute-optimized instance family

gn7s, GPU-accelerated compute-optimized instance family

To use the gn7s instance family, submit a ticket to apply.

gn7, GPU-accelerated compute-optimized instance family

gn6i, GPU-accelerated compute-optimized instance family

gn6e, GPU-accelerated compute-optimized instance family

gn6v, GPU-accelerated compute-optimized instance family

ebmgn9g, GPU-accelerated compute-optimized ECS Bare Metal Instance family

• Introduction: The ebmgn9g is the ninth-generation, full-featured, and cost-effective GPU-accelerated ECS Bare Metal instance family from Alibaba Cloud. It uses the latest CIPU 2.0 to deliver cloud services. Equipped with a high-clock-speed CPU, large-capacity memory, and new Blackwell-architecture professional GPUs, this instance family provides cost-effective GPU cloud computing for autonomous driving and embodied AI training, large language model inference, film and animation rendering, metaverse and cloud gaming services, and other GPU-accelerated workloads.

• Scenarios and key features:

  • Autonomous driving and embodied AI:
    It offers 256 vCPUs. All CPU cores run at up to 4.2 GHz or higher. Paired with 2.3 TB of memory, it supports the data processing needs for autonomous driving and embodied AI training.

  • Search and recommendation:
    The Blackwell GPU delivers 137 TFLOPS of high-performance TF32 computing power. Each GPU is paired with 32 vCPUs and 153 GB/s of memory bandwidth, delivering an optimal configuration for search and advertising workloads.

  • Large model inference:
    The new-generation GPU delivers higher computing power than the eighth-generation instance family. The GPU memory bandwidth is increased to 1344 GB/s. The new support for FP4 computing significantly improves inference performance and cost efficiency. Eight GPUs are connected via PCIe Gen5, which delivers 128 GB/s of inter-GPU bandwidth and greatly improves multi-GPU parallel inference efficiency.

  • Cloud gaming, rendering, and metaverse:
    The CPU reaches a clock speed of up to 5 GHz, which makes it a top choice for 3D modeling. The GPU natively supports graphics capabilities and includes workstation-class graphics drivers that are certified for professional design. It fully accelerates OpenGL, making it the best choice for high-end film and animation development and computer-aided design (CAD).

• Powered by the latest CIPU 2.0 cloud processor:

  The second-generation CIPU delivers higher cloud processing power and enhanced eRDMA, VPC, and EBS component performance. ECS Bare Metal instances provide direct access to physical resources. They also support workloads that require hardware-bound licenses. Containers, such as Docker, Clear Container, and Pouch, are supported.

• Compute:

  • It features new Blackwell-architecture professional GPUs:

    • It supports OpenGL for professional-grade graphics processing.

    • It supports common acceleration features such as RTX and TensorRT. It is newly upgraded to support FP4 and PCIe Gen5 interconnect.

    • It uses a PCIe Switch interconnect. Compared with a direct CPU connection, NCCL performance is improved by 36%. For multi-GPU sharded large model inference, peak performance is improved by up to 9%.

  • GPU specifications:

    GPU architecture	GPU memory	Computing performance	Video encode/decode capability	Inter-GPU interconnect	Accelerated APIs
    Blackwell	Capacity: 48 GB Bandwidth: 1344 GB/s	TF32: 126 TFLOPS FP32: 52 TFLOPS FP16/BF16: 266 TFLOPS FP8/INT8: 533 TFLOPS FP4: 970 TFLOPS RT core: 196 TFLOPS	3 × Video Encoder 3 × Video Decoder	PCIe Gen5 x16: 128 GB/s P2P supported	Supports DX12, OpenGL 4.6, Vulkan 1.3, CUDA 12.8, OpenCL 3.0, DirectCompute

  • Processor: It uses an AMD Turin-C (SPR) processor with a clock speed of 3.3 GHz to 5 GHz. The all-core frequency is up to 4.2 GHz.

• Storage:

  • This is an I/O optimized instance.

  • It supports the NVMe protocol. For details, see NVMe protocol.

  • Supported disk types include Elastic ephemeral disks, ESSDs, ESSD AutoPL disks, and Regional ESSDs. For more information about disks, see Overview of Block Storage.

• Network:

  • It supports IPv4 and IPv6. For more information about IPv6 communication, see IPv6 communication.

  • It provides ultra-high network performance of up to 30 million PPS for packet forwarding.

  • It supports the Elastic RDMA Interface (ERI). In a VPC network, ERI enables RDMA passthrough acceleration and boosts bandwidth to 360 Gbit/s. You can use it for autonomous driving, embodied AI, computer vision (CV), and traditional model training.

    Note

    For instructions on how to use ERI, see Enable eRDMA on an enterprise-level instance .

ebmgn9g instance types

ebmgn9ge, GPU-accelerated compute-optimized ECS Bare Metal Instance family

• Introduction: The ebmgn9ge family is the ninth-generation, full-featured, and cost-effective GPU-accelerated ECS Bare Metal Instance family from Alibaba Cloud. It uses the latest CIPU 2.0 and features high clock speed CPUs, large memory capacity, and professional graphics cards based on the new Blackwell architecture. This combination delivers cost-effective GPU acceleration for scenarios such as autonomous driving and embodied intelligence training, large language model (LLM) inference, film and animation rendering, and metaverse or cloud gaming services.

• Scenarios and features:

  • Autonomous driving and embodied intelligence:
    These instances provide 256 vCPUs with all cores running at over 4.2 GHz and 2.3 TB of memory, meeting the data processing requirements for autonomous driving and embodied intelligence training.

  • Search and recommendation:
    The Blackwell GPUs deliver 137 TFLOPS of high-performance TF32 computing power. Each GPU is paired with 32 vCPUs and 153 GB/s of memory bandwidth, offering an optimal configuration for search and advertising services.

  • Large model inference:

    The ebmgn9ge instances are designed for large language models. They provide 72 GB of GPU memory per card and a memory bandwidth of 1344 GB/s, enabling high-performance inference for LLM scenarios. With the new FP4 computing architecture and 128 GB/s of fifth-generation PCIe bandwidth, these instances support 8-card parallel inference for models with over 671 billion parameters.

  • Cloud gaming, rendering, and metaverse:
    The CPU achieves a high clock speed of up to 5 GHz, making it ideal for 3D modeling. The GPU natively supports graphics and includes workstation-grade drivers certified for professional design. It also supports full-featured OpenGL acceleration, making it an optimal choice for high-end film and animation development and computer-aided design (CAD).

• Uses the latest CIPU 2.0 cloud processor:

  The second-generation CIPU delivers higher cloud processing power and enhanced computing capabilities for elastic Remote Direct Memory Access (eRDMA), VPC, and EBS components. ECS Bare Metal instances allow direct access to physical resources and are ideal for workloads that require hardware-bound licenses. These instances support containers such as Docker, Clear Container, and Pouch.

• Compute:

  • Uses professional graphics cards based on the new Blackwell architecture:

    • Supports professional-grade OpenGL graphics processing.

    • Supports common acceleration features such as RTX and TensorRT and is newly upgraded to support FP4 and PCIe Gen5 interconnect.

    • Uses a PCIe Switch for interconnection. Compared to a direct CPU connection, this improves NCCL performance by 36% and boosts performance by up to 9% for multi-card sharded large model inference.

  • Key GPU parameters:

    GPU architecture	GPU memory	Compute performance	Video encoding/decoding capability	Inter-card connection	Acceleration APIs
    Blackwell	Capacity: 72 GB Bandwidth: 1344 GB/s	TF32: 126 TFLOPS FP32: 52 TFLOPS FP16/BF16: 266 TFLOPS FP8/INT8: 533 TFLOPS FP4: 970 TFLOPS RT core: 196 TFLOPS	3 × Video Encoder 3 × Video Decoder	PCIe Gen5 x16: 128 GB/s Supports P2P	Supports DX12, OpenGL 4.6, Vulkan 1.3, CUDA 12.8, OpenCL 3.0, and DirectCompute

  • Processor: AMD Turin-C (SPR) processor with a clock speed ranging from 3.3 GHz to 5 GHz. All cores can reach up to 4.2 GHz.

• Storage:

  • I/O optimized instance.

  • Supports the NVMe protocol. For more information, see NVMe protocol.

  • Supported disk types: Elastic ephemeral disks, ESSDs, ESSD AutoPL disks, and Regional ESSDs. For more information about disks, see Overview of Block Storage.

• Network:

  • Supports IPv4 and IPv6. For information about IPv6 communication, see IPv6 communication.

  • Ultra-high network performance with a packet forwarding capability of 30 million PPS.

  • Supports Elastic RDMA Interface (ERI). ERI enables RDMA passthrough for accelerated interconnection within a VPC, increasing bandwidth to 360 Gbit/s. It can be used for training tasks in autonomous driving, embodied intelligence, computer vision (CV), and traditional models.

    Note

    For instructions on how to use ERI, see Enable eRDMA on an enterprise-level instance.

The following table lists the instance types and specifications for the ebmgn9ge family.

ebmgn9gc, GPU-accelerated compute-optimized ECS Bare Metal Instance family

• Introduction: The ebmgn9gc is a ninth-generation, cost-effective, and full-featured GPU-accelerated ECS Bare Metal instance from Alibaba Cloud. It is equipped with the latest CIPU 2.0, a high-clock-speed CPU, large-capacity memory, and professional Blackwell-architecture GPUs. This instance family provides cost-effective GPU cloud computing for workloads such as autonomous driving and embodied AI training, LLM inference, film and animation rendering, and metaverse and cloud gaming services.

• Scenarios and features:

  • Autonomous driving and embodied AI:
    This instance provides 256 vCPUs with all cores running at a clock speed of up to 4.2 GHz, and 2.3 TB of memory. This configuration meets the demanding data processing requirements for autonomous driving and embodied AI training.

  • Search and recommendation:
    The Blackwell GPU delivers 137 TFLOPS of high-performance TF32 computing power. Each GPU is paired with 32 vCPUs and 153 GB/s of memory bandwidth, which delivers an optimal configuration for search and advertising workloads.

  • Model inference:

    The ebmgn9gc instance is specifically designed for large language models, offering 72 GB of GPU memory per GPU and GPU memory bandwidth of up to 1344 GB/s. It provides high-performance inference computing power for LLM scenarios, supports the new FP4 compute architecture, and features 5th-generation PCIe bandwidth of 128 GB/s. This enables parallel inference of large models with more than 671 billion parameters across eight GPUs.

  • Cloud gaming, rendering, and metaverse:
    The CPU has a clock speed of up to 5 GHz, which is ideal for 3D modeling. The GPU natively supports graphics capabilities and includes workstation-class graphics drivers that are certified for professional design applications. The instance fully accelerates OpenGL, which makes it an excellent choice for high-end film and animation development and CAD design.

• Powered by the latest CIPU 2.0 cloud processor:

  The second-generation CIPU delivers enhanced cloud processing capability, including improved performance for eRDMA, VPC, and EBS components. ECS Bare Metal instances provide direct access to physical hardware or support workloads that require hardware-bound licenses. The second-generation CIPU supports containers such as Docker, Clear Containers, and Pouch.

• Compute:

  • This instance uses professional GPUs based on the Blackwell architecture:

    • It supports OpenGL for professional-grade graphics processing.

    • It supports common acceleration features, such as RTX and TensorRT, along with FP4 and PCIe Gen5 interconnects.

    • The instance uses PCIe Switch interconnects. Compared to CPU-direct connections, this improves NCCL performance by 36%. For multi-GPU sharded LLM inference, peak performance is improved by up to 9%.

  • GPU specifications:

    GPU architecture	GPU memory	Compute performance	Video encode/decode capability	GPU-to-GPU interconnect	Accelerated APIs
    Blackwell	Capacity: 72 GB Bandwidth: 1344 GB/s	TF32: 126 TFLOPS FP32: 52 TFLOPS FP16/BF16: 266 TFLOPS FP8/INT8: 533 TFLOPS FP4: 970 TFLOPS RT core: 196 TFLOPS	3 × Video Encoder 3 × Video Decoder	PCIe Gen5 x16: 128 GB/s Peer-to-peer (P2P) supported	Supports DX12, OpenGL 4.6, Vulkan 1.3, CUDA 12.8, OpenCL 3.0, and DirectCompute

  • Processor: This instance uses an AMD Turin-C (SPR) processor with a clock speed of 3.3 GHz to 5 GHz. The all-core clock speed is up to 4.2 GHz.

• Storage:

  • I/O optimized instance.

  • This instance supports the NVMe protocol. For more information, see NVMe protocol.

  • The supported disk types are Elastic ephemeral disks, ESSDs, ESSD AutoPL disks, and Regional ESSDs. For more information about disks, see Overview of Block Storage.

• Network:

  • This instance supports IPv4 and IPv6. For more information about IPv6 communication, see IPv6 communication.

  • The instance provides ultra-high network performance with up to 30 million PPS for packet forwarding.

  • This instance supports Elastic RDMA Interface (ERI). In a VPC, ERI enables RDMA passthrough acceleration and increases bandwidth to 360 Gbit/s. This feature is ideal for autonomous driving, embodied AI, computer vision (CV), and traditional model training.

    Note

    For instructions on how to use ERI, see Enable eRDMA on an enterprise-level instance.

The following table lists the instance types and specifications for the ebmgn9gc family.

ebmgn9gc, GPU-accelerated compute-optimized ECS Bare Metal Instance family

• Introduction: The ebmgn9gc is a ninth-generation, cost-effective, and full-featured GPU-accelerated ECS Bare Metal instance from Alibaba Cloud. It is equipped with the latest CIPU 2.0, a high-clock-speed CPU, large-capacity memory, and professional Blackwell-architecture GPUs. This instance family provides cost-effective GPU cloud computing for workloads such as autonomous driving and embodied AI training, LLM inference, film and animation rendering, and metaverse and cloud gaming services.

• Scenarios and features:

  • Autonomous driving and embodied AI:
    This instance provides 256 vCPUs with all cores running at a clock speed of up to 4.2 GHz, and 2.3 TB of memory. This configuration meets the demanding data processing requirements for autonomous driving and embodied AI training.

  • Search and recommendation:
    The Blackwell GPU delivers 137 TFLOPS of high-performance TF32 computing power. Each GPU is paired with 32 vCPUs and 153 GB/s of memory bandwidth, which delivers an optimal configuration for search and advertising workloads.

  • Model inference:

    The ebmgn9gc instance is specifically designed for large language models, offering 72 GB of GPU memory per GPU and GPU memory bandwidth of up to 1344 GB/s. It provides high-performance inference computing power for LLM scenarios, supports the new FP4 compute architecture, and features 5th-generation PCIe bandwidth of 128 GB/s. This enables parallel inference of large models with more than 671 billion parameters across eight GPUs.

  • Cloud gaming, rendering, and metaverse:
    The CPU has a clock speed of up to 5 GHz, which is ideal for 3D modeling. The GPU natively supports graphics capabilities and includes workstation-class graphics drivers that are certified for professional design applications. The instance fully accelerates OpenGL, which makes it an excellent choice for high-end film and animation development and CAD design.

• Powered by the latest CIPU 2.0 cloud processor:

  The second-generation CIPU delivers enhanced cloud processing capability, including improved performance for eRDMA, VPC, and EBS components. ECS Bare Metal instances provide direct access to physical hardware or support workloads that require hardware-bound licenses. The second-generation CIPU supports containers such as Docker, Clear Containers, and Pouch.

• Compute:

  • This instance uses professional GPUs based on the Blackwell architecture:

    • It supports OpenGL for professional-grade graphics processing.

    • It supports common acceleration features, such as RTX and TensorRT, along with FP4 and PCIe Gen5 interconnects.

    • The instance uses PCIe Switch interconnects. Compared to CPU-direct connections, this improves NCCL performance by 36%. For multi-GPU sharded LLM inference, peak performance is improved by up to 9%.

  • GPU specifications:

    GPU architecture	GPU memory	Compute performance	Video encode/decode capability	GPU-to-GPU interconnect	Accelerated APIs
    Blackwell	Capacity: 72 GB Bandwidth: 1344 GB/s	TF32: 126 TFLOPS FP32: 52 TFLOPS FP16/BF16: 266 TFLOPS FP8/INT8: 533 TFLOPS FP4: 970 TFLOPS RT core: 196 TFLOPS	3 × Video Encoder 3 × Video Decoder	PCIe Gen5 x16: 128 GB/s Peer-to-peer (P2P) supported	Supports DX12, OpenGL 4.6, Vulkan 1.3, CUDA 12.8, OpenCL 3.0, and DirectCompute

  • Processor: This instance uses an AMD Turin-C (SPR) processor with a clock speed of 3.3 GHz to 5 GHz. The all-core clock speed is up to 4.2 GHz.

• Storage:

  • I/O optimized instance.

  • This instance supports the NVMe protocol. For more information, see NVMe protocol.

  • The supported disk types are Elastic ephemeral disks, ESSDs, ESSD AutoPL disks, and Regional ESSDs. For more information about disks, see Overview of Block Storage.

• Network:

  • This instance supports IPv4 and IPv6. For more information about IPv6 communication, see IPv6 communication.

  • The instance provides ultra-high network performance with up to 30 million PPS for packet forwarding.

  • This instance supports Elastic RDMA Interface (ERI). In a VPC, ERI enables RDMA passthrough acceleration and increases bandwidth to 360 Gbit/s. This feature is ideal for autonomous driving, embodied AI, computer vision (CV), and traditional model training.

    Note

    For instructions on how to use ERI, see Enable eRDMA on an enterprise-level instance.

The following table lists the instance types and specifications for the ebmgn9gc family.

ebmgn8v, GPU-accelerated compute-optimized ECS Bare Metal Instance family

This instance family is available only in specific regions, including regions outside China. To use the instance family, contact Alibaba Cloud sales personnel.

• Introduction: This instance family is an 8th-generation GPU-accelerated compute-optimized ECS Bare Metal Instance family provided by Alibaba Cloud for AI model training and ultra-large models. Each instance of this instance family is equipped with eight GPUs.

• Use cases:

  • Multi-GPU parallel inference computing for LLMs that have more than 70 billion parameters

  • Traditional AI model training and autonomous driving training, for which each GPU delivers computing power of up to 39.5 TFLOPS in the single-precision floating-point format (FP32)

  • Small and medium-sized model training scenarios that leverage the NVLink connections among the eight GPUs

• Benefits and positioning:

  • High-speed and large-capacity GPU memory: Each GPU is equipped with 96 GB of HBM3 memory and delivers up to 4 TB/s of memory bandwidth, which greatly accelerates model training and inference.

  • High bandwidth between GPUs: Multiple GPUs are interconnected by using 900 GB/s NVLink connections. The efficiency of multi-GPU training and inference is much higher than that of previous generations of GPU-accelerated instances.

  • Quantization of large models: This instance family supports computing power in the 8-bit floating point format (FP8) and optimizes computing power for large-scale parameter training and inference. This significantly improves the computing speed of training and inference and reduces memory usage.

• Compute:

  • Uses the latest CIPU 1.0 processors.

    • Decouples computing capabilities from storage capabilities, allowing you to flexibly select storage resources based on your business requirements, and increases inter-instance bandwidth to 160 Gbit/s for faster data transmission and processing compared with 7th-generation instance families.

    • Uses the bare metal capabilities provided by CIPU processors to support peer-to-peer (P2P) communication between GPU-accelerated instances.

  • Uses the 4th-generation Intel Xeon Scalable processors that deliver an all-core turbo frequency of up to 3.1 GHz and provides 192 vCPUs.

• Storage:

  • These are I/O optimized instances.

  • These instances support the NVMe protocol. For more information, see Overview of the NVMe protocol.

  • Supports elastic ephemeral disks, ESSDs, ESSD AutoPL disks, and Regional ESSDs. For information about disks, see Overview of Block Storage.

• Network:

  • These instances support IPv4 and IPv6. For information about IPv6 communication, see IPv6 communication.

  • Provides ultra-high network performance with a packet forwarding rate of 30,000,000 pps.

  • Supports ERIs to allow inter-instance RDMA-based communication in VPCs and provides up to 160 Gbit/s of bandwidth per instance, which is suitable for training tasks based on CV models and traditional models.

    Note

    For information about how to use ERIs, see Enable eRDMA on an enterprise-level instance.

ebmgn8v instance types

ebmgn8ia, GPU-accelerated compute-optimized ECS Bare Metal Instance family

This instance family is available only in specific regions, including regions outside China. To use the instance family, contact Alibaba Cloud sales personnel.

• Introduction: The instance family is the 8th generation of accelerated computing ECS Bare Metal instances from Alibaba Cloud, designed for applications such as search, recommendation, and simulation that require a high number of vCPU resources per GPU. This family is powered by the latest NVIDIA L20 GPUs, and each instance is a single bare metal host equipped with two high-frequency CPUs and four GPUs.

• Benefits and Use cases:

  • High clock speed: This product is powered by two AMD EPYC™ Genoa 9T34 processors. Each processor has 64 physical cores, providing a total of 256 vCPUs with clock speeds ranging from 3.4 GHz to 3.75 GHz. This configuration significantly improves single-core CPU performance, making it ideal for CAD modeling and accelerating pre-processing for CAE simulations.

  • Sparse resource allocation: Each GPU is paired with an average of 64 vCPUs and 384 GiB of memory, providing an average memory bandwidth of 230 GB/s. This configuration is ideal for GPU computing scenarios that require high I/O throughput, such as advertising, search, recommendations, and traditional CAE simulations. The configuration also supports certain film and television production tasks that rely on CPU rendering.

• Uses the latest CIPU 1.0 processors that provide the following benefits:

  • Decouples computing capabilities from storage capabilities, allowing you to flexibly select storage resources based on your business requirements, and Increases inter-instance bandwidth to 160 Gbit/s for faster data transmission and processing compared with previous-generation instance families.

  • Uses the bare metal capabilities provided by CIPU processors to support Peripheral Component Interconnect Express (PCIe) P2P communication between GPU-accelerated instances.

• Compute:

  • With the new NVIDIA L20 Enterprise GPU:

    • Support for acceleration features such as vGPU, RTX technology, and TensorRT inference engine.

    • Support for 8-bit floating point precision improves computational efficiency.

  • NVIDIA L20 main parameters:

    GPU architecture	GPU memory	Computing performance	Video codec capability	Interconnection between cards
    NVIDIA Ada Lovelace	Capacity: 48 GB Bandwidth: 864 GB/s	FP64: N/A FP32: 59.3 TFLOPS FP16/BF16: 119 TFLOPS FP8/INT8: 237 TFLOPS	3 * Video Encoder (+AV1) 3 * Video Decoder 4 * JPEG Decoder	PCIe interface: PCIe Gen4 x16 Bandwidth: 64 GB/s

  • Processor: AMD EPYC™ Genoa 9T34, 3.4 GHz to 3.75 GHz.

• Storage:

  • These are I/O optimized instances.

  • These instances support the NVMe protocol. For more information, see Overview of the NVMe protocol.

  • Supports elastic ephemeral disks, ESSDs, ESSD AutoPL disks, and Regional ESSDs. For information about disks, see Overview of Block Storage.

• Newtork:

  • These instances support IPv4 and IPv6. For information about IPv6 communication, see IPv6 communication.

  • Provides ultra-high network performance with a packet forwarding rate of 30,000,000 pps.

  • Supports ERIs to allow inter-instance RDMA-based communication in VPCs and provides up to 160 Gbit/s of bandwidth per instance, which is suitable for training tasks based on CV models and traditional models.

    Note

    For information about how to use ERIs, see Enable eRDMA on an enterprise-level instance.

ebmgn8is, GPU-accelerated compute-optimized ECS Bare Metal Instance family

This instance family is available only in specific regions, including regions outside China. To use the instance family, contact Alibaba Cloud sales personnel.

• Introduction: ebmgn8is is the 8th generation of accelerated computing specification family (ECS Bare Metal instance family) launched by Alibaba Cloud to address the recent development of AI generation business. It uses the latest NVIDIA L20 GPU. Each instance is a bare metal host that uses eight GPU compute cards.

• Benefits and positioning:

  • Graphic processing: This instance family uses high-frequency 4th-generation Intel Xeon Scalable processors to deliver sufficient CPU computing power in 3D modeling scenarios and achieve smooth graphics rendering and design.

  • Inference tasks: uses the new NVIDIA L20 GPU memory of 48 GB on a single GPU card to accelerate inference tasks. It supports the FP8 floating-point format and supports inference based on various AIGC models, especially for inference tasks that use LLM models of less than 70 MB.

  • Training tasks: This instance family provides cost-effective computing capabilities and delivers the FP32 computing performance double that of the 7th-generation inference instances. Instances of this instance family are suitable for training FP32-based CV models and other small and medium-sized models.

• Use cases:

  • Production and rendering of special effects for animation, film, and television based on workstation-level graphics processing capabilities in scenarios in which Alibaba Cloud Marketplace GRID images are used, the GRID driver is installed, and OpenGL and Direct3D graphics capabilities are enabled

  • Scenarios in which the management services provided by Container Service for Kubernetes (ACK) for containerized applications are used to support AI-generated graphic content and LLM inference tasks with up to 130 billion parameters

  • Other general-purpose AI recognition, image recognition, and speech recognition scenarios

• Uses the latest CIPU 1.0 processors that provide the following benefits:

  • Decouples computing capabilities from storage capabilities, allowing you to flexibly select storage resources based on your business requirements, and Increases inter-instance bandwidth to 160 Gbit/s for faster data transmission and processing compared with previous-generation instance families.

  • Uses the bare metal capabilities provided by CIPU processors to support PCIe P2P communication between GPU-accelerated instances.

• Compute:

  • With the new NVIDIA L20 Enterprise GPU:

    • Support for acceleration features such as vGPU, RTX technology, and TensorRT inference engine

    • Support for PCIe Switch interconnect, which achieves a 36% increase in NVIDIA Collective Communications Library (NCCL) performance compared with the CPU direct connection scheme and helps improve inference performance by up to 9% when you run LLM inference tasks on multiple GPUs in parallel

  • NVIDIA L20 main parameters:

    GPU architecture	GPU memory	Computing performance	Video codec capability	Interconnection between cards
    NVIDIA Ada Lovelace	Capacity: 48 GB Bandwidth: 864 GB/s	FP64: N/A FP32: 59.3 TFLOPS FP16/BF16: 119 TFLOPS FP8/INT8: 237 TFLOPS	3 * Video Encoder(+AV1) 3 * Video Decoder 4 * JPEG Decoder	PCIe interface: PCIe Gen4 x16 Bandwidth: 64 GB/s

  • Uses 3.4 GHz Intel^® Xeon^® Scalable processors (SPR) that deliver an all-core turbo frequency of up to 3.9 GHz.

• Storage:

  • These are I/O optimized instances.

  • These instances support the NVMe protocol. For more information, see Overview of the NVMe protocol.

  • Supports elastic ephemeral disks, ESSDs, ESSD AutoPL disks, and Regional ESSDs. For information about disks, see Overview of Block Storage.

• Network:

  • These instances support IPv4 and IPv6. For information about IPv6 communication, see IPv6 communication.

  • Provides ultra-high network performance with a packet forwarding rate of 30,000,000 pps.

  • Supports ERIs to allow inter-instance RDMA-based communication in VPCs and provides up to 160 Gbit/s of bandwidth per instance, which is suitable for training tasks based on CV models and traditional models.

    Note

    For information about how to use ERIs, see Enable eRDMA on an enterprise-level instance.

ebmgn7e, GPU-accelerated compute-optimized ECS Bare Metal Instance family

You must check the status of the multi-instance GPU (MIG) feature and enable or disable the MIG feature after you start an ebmgn7e instance. For information about the MIG feature, see NVIDIA Multi-Instance GPU User Guide.

The following table describes whether the MIG feature is supported by the instance types in the ebmgn7e instance family.

ebmgn7i, GPU-accelerated compute-optimized ECS Bare Metal Instance family

GPU-accelerated compute-optimized ECS Bare Metal Instance family ebmgn7

Check the status of the MIG (Multi-Instance GPU) feature and enable or disable it after you start an ebmgn7 instance. The system does not guarantee whether the MIG (Multi-Instance GPU) feature is enabled or disabled. For more information about MIG (Multi-Instance GPU), see NVIDIA Multi-Instance GPU User Guide.

The following table describes whether the MIG feature is supported by ebmgn7 instances:

ebmgn6e, GPU-accelerated compute-optimized ECS Bare Metal Instance family

ebmgn6v, GPU-accelerated compute-optimized ECS Bare Metal Instance family

ebmgn6i, GPU-accelerated compute-optimized ECS Bare Metal Instance family

gn5i, GPU-accelerated compute-optimized instance family

gn5, GPU-accelerated compute-optimized instance family