Container Service for Kubernetes:Accelerate Stable Diffusion XL Turbo text-to-image inference using CPU in a TDX node pool

The g8i instance family

The g8i general-purpose ECS instance family is powered by Cloud Infrastructure Processing Units (CIPUs) and Apsara Stack. It uses 5th Gen Intel® Xeon® Scalable processors (code-named Emerald Rapids) with AMX for enhanced AI performance. All g8i instances support Intel® TDX, which lets you run workloads in a Trusted Execution Environment (TEE) without modifying your application code.

For specifications, see g8i, general-purpose instance family.

Intel® TDX

Intel® TDX is a CPU hardware-based technology that provides hardware-assisted isolation and encryption for ECS instances, protecting CPU registers, memory data, and interrupt injections at runtime. It helps prevent unauthorized access to running processes and sensitive data without requiring application code changes.

For more information, see Intel® Trust Domain Extensions (Intel® TDX).

IPEX

Intel® Extension for PyTorch (IPEX) is an open source PyTorch extension that improves AI application performance on Intel processors. It is optimized continuously with the latest Intel hardware and software technologies.

For more information, see IPEX.

Full values.yaml reference

The Helm chart supports additional configuration options beyond resources and model source. The full default values are:

# Number of pod replicas.
replicaCount: 1

# Container image configuration.
image:
  repository: registry-vpc.cn-beijing.aliyuncs.com/eric-dev/stable-diffusion-ipex
  pullPolicy: IfNotPresent
  tag: "v0.1.5"              # Bundles the official stabilityai/sdxl-turbo model
  tagOnlyApi: "v0.1.5-lite"  # API-only image; requires mounting the model manually (see useCustomModels)

# Credentials for pulling a private container image.
imagePullSecrets: []

# Output path for generated images inside the container.
outputDirPath: /tmp/sd

# Set to true to use a custom model mounted via the volumes.models PVC.
# When false, the image.tag image (which includes the model) is used.
useCustomModels: false

volumes:
  # Volume for the image output path.
  output:
    name: output-volume
    emptyDir: {}
  # Volume for the custom model. Only active when useCustomModels: true.
  # Place the model in the stabilityai/sdxl-turbo subdirectory of the mount path.
  models:
    name: data-volume
    persistentVolumeClaim:
      claimName: models-oss-pvc
  # Alternatively, use a host path:
  # models:
  #   hostPath:
  #     path: /data/models
  #     type: DirectoryOrCreate

# Service configuration.
service:
  type: ClusterIP
  port: 5000

# Container resource limits and requests.
resources:
  limits:
    cpu: "16"
    memory: 32Gi
  requests:
    cpu: "14"
    memory: 24Gi

# Workload update strategy.
strategy:
  type: RollingUpdate

# Scheduling configuration.
nodeSelector: {}
tolerations: []
affinity: {}

# Container security settings.
securityContext:
  capabilities:
    drop:
    - ALL
  runAsNonRoot: true
  runAsUser: 1000

# Horizontal Pod Autoscaler (HPA) configuration.
# https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/
autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 3
  targetCPUUtilizationPercentage: 80
  targetMemoryUtilizationPercentage: 90

Performance benchmarks

The following table shows average generation times on different g8i instance types (batch: 1, step: 4). Results are for reference only.

Recommendation: ecs.g8i.8xlarge offers the best balance of cost and throughput. At step=4, batch=16, it generates 1.2 images/s — above one image per second without compromising image quality.

Prerequisites

A TDX confidential VM node pool exists in the ACK cluster with the following configuration:

• Instance type: At least 16 vCPUs. Recommended: ecs.g8i.4xlarge.

• Disk space: At least 200 GiB per node.

• Node label: nodepool-label=tdx-vm-pool.

For setup instructions, see Create a node pool that supports TDX confidential VMs.

Migrate the service

1. Create a file named tdx_values.yaml with the following node selector. Replace tdx-vm-pool if you used a different label value for the node pool.

   nodeSelector:
     nodepool-label: tdx-vm-pool

2. Upgrade the Helm release to reschedule the pod onto the TDX node pool.

   helm upgrade stable-diffusion-ipex \
     https://aliacs-app-catalog.oss-cn-hangzhou.aliyuncs.com/pre/charts-incubator/stable-diffusion-ipex-0.1.7.tgz \
     -f tdx_values.yaml

   Expected output:

   Release "stable-diffusion-ipex" has been upgraded. Happy Helming!
   NAME: stable-diffusion-ipex
   LAST DEPLOYED: Wed Jan 24 16:38:04 2024
   NAMESPACE: default
   STATUS: deployed
   REVISION: 2
   TEST SUITE: None

3. Wait about 10 minutes, then verify the pod is running on the TDX node pool.

   kubectl get pod | grep stable-diffusion-ipex

   Expected output:

   stable-diffusion-ipex-7f8c4f88f5-r478t   1/1     Running   0   1m44s

4. Repeat Step 3: Test the service to verify the model works correctly in the TDX node pool.

Request syntax

POST /api/text2image

CPU acceleration benchmarks

Performance results (images/s):

GPU acceleration benchmarks

Cost comparison

The following estimates compare g8i CPU instances against ecs.gn7i-c8g1.2xlarge (GPU). For current prices, see the Pricing tab on the Elastic Compute Service page.

Use ecs.g8i.8xlarge when you need both cost savings and throughput above 1 image/s. Use ecs.g8i.4xlarge when cost reduction is the primary goal and 0.5 images/s meets your requirements.