KubeRay enables you to deploy, scale, and manage Ray clusters on ACK for distributed AI workloads.
The Ray computing framework has three layers: Ray AI Libraries, Ray Core, and Ray clusters.
KubeRay overview
KubeRay is an open-source Kubernetes operator for deploying and managing Ray applications. It provides three custom resources -- RayCluster, RayJob, and RayService -- through a declarative Kubernetes API.
Container Service for Kubernetes (ACK) provides enterprise-grade container management with certified Kubernetes conformance. Use KubeRay to deploy Ray clusters in ACK.
Ray on ACK integrates with Alibaba Cloud services:
| Alibaba Cloud service | Capability |
|---|---|
| Simple Log Service | Log management |
| Managed Service for Prometheus | Observability |
| Tair (Redis OSS-Compatible) | Improved availability |
| Ray autoscaler + ACK autoscaler | On-demand resource scaling |
Kuberay-Operator
Install KubeRay from the Add-ons page of the ACK console. ACK provides Kuberay-Operator, based on the open-source KubeRay operator, which enables the following ACK capabilities:
-
Scheduling
-
Elastic quotas
-
Priority-based resource scheduling
-
Integration with Simple Log Service, Managed Service for Prometheus, and Object Storage Service (OSS)
After installation, ACK manages Kuberay-Operator and creates the RayCluster, RayJob, and RayService custom resources on the data plane.
Custom resources
RayCluster
A RayCluster creates a Ray cluster consisting of a head pod and worker pods on an ACK cluster. See RayCluster Configuration.
RayJob
A RayJob (in K8sJobMode mode) manages a RayCluster and a Kubernetes batch job. The RayCluster provides compute resources, while the batch job runs ray job submit to submit a Ray job. See RayJob Configuration.
RayService
A RayService manages a RayCluster and Ray Serve applications. The RayCluster provides compute resources, and Ray Serve handles model deployment and inference.
Shared responsibilities for Ray on ACK
When you run Ray workloads in ACK clusters with KubeRay, you and Alibaba Cloud share responsibility for environment security and management.
Alibaba Cloud responsibilities
ACK manages and secures Kuberay-Operator:
-
Ensures Kuberay-Operator images comply with security hardening standards.
-
Ensures Kuberay-Operator stability and availability.
-
Maintains Kuberay-Operator versions.
-
Manages RayCluster, RayJob, and RayService custom resources.
Customer responsibilities
You are responsible for the security and configuration of Ray applications deployed with RayCluster, RayJob, and RayService.
-
Follow the best practices for Ray cluster protection.
-
Update and maintain container images for Ray head and worker pods.
-
Update and maintain Ray versions for head and worker pods.
-
Configure resource requirements for Ray clusters, including CPU, GPU, and memory.
-
Monitor Ray application status and availability.