cGPU is a kernel-based GPU sharing technology from Alibaba Cloud that lets multiple isolated containers share a single physical GPU. By virtualizing GPU resources into vGPUs at the kernel driver level, cGPU enforces per-container isolation of GPU memory and computing power—without compromising performance. This improves hardware utilization and reduces GPU infrastructure costs.
How it works
Alibaba Cloud's kernel driver creates vGPU devices for each container running on the shared GPU. The driver enforces isolation at the GPU memory and computing layer, so containers cannot interfere with each other's workloads. This makes it possible to run multiple training or inference jobs on a single GPU simultaneously.
Capabilities
Isolation and resource control
GPU memory isolation: Allocate GPU memory at MB-level granularity, enforced per container.
Computing power isolation: Set GPU utilization per container and configure a minimum computing power ratio as low as 2%.
Preemption: High-priority tasks can preempt lower-priority ones to guarantee service-level objectives.
Compatibility
Works with Docker, containerd, and Kubernetes without modifying your AI applications.
No need to recompile code or replace Compute Unified Device Architecture (CUDA) libraries.
Supported instance types
cGPU supports GPU-accelerated Elastic Compute Service (ECS) Bare Metal Instances, virtualized instances, and vGPU-accelerated instances.virtualized
Operations
Hot upgrade: Upgrade the kernel driver without interrupting running workloads.
Multi-GPU allocation: Assign multiple vGPUs to a single container.
Monitoring and management: High O&M capabilities for tracking and managing GPU resource usage.
Use cases
Online/offline workload colocation: Run latency-sensitive online services and batch offline jobs on the same GPU to maximize utilization.
AI training and inference: Isolate multiple CUDA-based training or inference jobs on a single GPU with guaranteed resource boundaries.
Rendering: Share GPU resources across rendering tasks in multi-tenant environments.