Container Service for Kubernetes:Accelerate Stable Diffusion XL Turbo text-to-image inference with CPU

The g8i instance family

The g8i general-purpose ECS instance family is powered by Cloud Infrastructure Processing Units (CIPUs) and Apsara Stack. It uses 5th Gen Intel® Xeon® Scalable processors (code-named Emerald Rapids) with AMX for AI acceleration. All g8i instances support Intel® TDX, enabling Trusted Execution Environment (TEE) workloads with minimal performance overhead and without code changes.

See g8i, general-purpose instance family.

Intel® TDX

Intel® TDX is a hardware-based TEE technology that isolates and encrypts ECS instances, protecting CPU registers, memory, and interrupt injections at runtime. It helps prevent unauthorized access to processes and sensitive data without code changes.

See Intel® Trust Domain Extensions (Intel® TDX).

IPEX

Intel® Extension for PyTorch (IPEX) is an open source PyTorch extension that accelerates AI on Intel processors, continuously optimized for the latest Intel hardware and software.

See IPEX.

Full values.yaml reference

The Helm chart supports additional options beyond resources and model source. Default values:

# Number of pod replicas.
replicaCount: 1

# Container image configuration.
image:
  repository: registry-vpc.cn-beijing.aliyuncs.com/eric-dev/stable-diffusion-ipex
  pullPolicy: IfNotPresent
  tag: "v0.1.5"              # Bundles the official stabilityai/sdxl-turbo model
  tagOnlyApi: "v0.1.5-lite"  # API-only image; requires mounting the model manually (see useCustomModels)

# Credentials for pulling a private container image.
imagePullSecrets: []

# Output path for generated images inside the container.
outputDirPath: /tmp/sd

# Set to true to use a custom model mounted via the volumes.models PVC.
# When false, the image.tag image (which includes the model) is used.
useCustomModels: false

volumes:
  # Volume for the image output path.
  output:
    name: output-volume
    emptyDir: {}
  # Volume for the custom model. Only active when useCustomModels: true.
  # Place the model in the stabilityai/sdxl-turbo subdirectory of the mount path.
  models:
    name: data-volume
    persistentVolumeClaim:
      claimName: models-oss-pvc
  # Alternatively, use a host path:
  # models:
  #   hostPath:
  #     path: /data/models
  #     type: DirectoryOrCreate

# Service configuration.
service:
  type: ClusterIP
  port: 5000

# Container resource limits and requests.
resources:
  limits:
    cpu: "16"
    memory: 32Gi
  requests:
    cpu: "14"
    memory: 24Gi

# Workload update strategy.
strategy:
  type: RollingUpdate

# Scheduling configuration.
nodeSelector: {}
tolerations: []
affinity: {}

# Container security settings.
securityContext:
  capabilities:
    drop:
    - ALL
  runAsNonRoot: true
  runAsUser: 1000

# Horizontal Pod Autoscaler (HPA) configuration.
# https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/
autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 3
  targetCPUUtilizationPercentage: 80
  targetMemoryUtilizationPercentage: 90

Performance benchmarks

Average generation times on g8i instance types (batch: 1, step: 4). Results are for reference only.

Recommendation: ecs.g8i.8xlarge offers the best cost-to-throughput balance. At step=4, batch=16, it generates 1.2 images/s without compromising image quality.

Prerequisites

A TDX confidential VM node pool exists in the ACK cluster with the following configuration:

• Instance type: At least 16 vCPUs. Recommended: ecs.g8i.4xlarge.

• Disk space: At least 200 GiB per node.

• Node label: nodepool-label=tdx-vm-pool.

See Create a node pool that supports TDX confidential VMs.

Migrate the service

1. Create tdx_values.yaml with the following node selector. Replace tdx-vm-pool if you used a different label value.

   nodeSelector:
     nodepool-label: tdx-vm-pool

2. Upgrade the Helm release to reschedule the pod onto the TDX node pool.

   helm upgrade stable-diffusion-ipex \
     https://aliacs-app-catalog.oss-cn-hangzhou.aliyuncs.com/pre/charts-incubator/stable-diffusion-ipex-0.1.7.tgz \
     -f tdx_values.yaml

   Expected output:

   Release "stable-diffusion-ipex" has been upgraded. Happy Helming!
   NAME: stable-diffusion-ipex
   LAST DEPLOYED: Wed Jan 24 16:38:04 2024
   NAMESPACE: default
   STATUS: deployed
   REVISION: 2
   TEST SUITE: None

3. Wait about 10 minutes, then verify the pod is running on the TDX node pool.

   kubectl get pod | grep stable-diffusion-ipex

   Expected output:

   stable-diffusion-ipex-7f8c4f88f5-r478t   1/1     Running   0   1m44s

4. Repeat Step 3: Test the service to verify the model works in the TDX node pool.

Request syntax

POST /api/text2image

CPU acceleration benchmarks

Performance results (images/s):

GPU acceleration benchmarks

Cost comparison

These estimates compare g8i CPU instances against ecs.gn7i-c8g1.2xlarge (GPU). For current prices, see the Pricing tab on the Elastic Compute Service page.

Use ecs.g8i.8xlarge when you need both cost savings and throughput above 1 image/s. Use ecs.g8i.4xlarge when cost reduction, TEE-based data confidentiality, or large-scale CPU resource supply is the priority and 0.5 images/s meets your requirements.ecs.g8i.4xlarge when cost reduction is the primary goal and 0.5 images/s meets your requirements.