Platform For AI:Use EAS spot instances for cost-effective inference

Use cases

• Non-critical workloads: Applications that tolerate occasional interruptions.

• Fault-tolerant applications: Applications that handle unavailability through retries or other mechanisms.

• Cost-sensitive projects: Projects focused on reducing operational costs.

Configure a spot service

1. Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).

2. On the Inference Service page, click Deploy Service. In the Custom Model Deployment section, click Custom Deployment.

3. In the Resource Deployment section, select Public Resources as the resource type and select an instance type (L or H series recommended). Turn on bidding mode and enter a bid price. View the historical price curve for the market price trend over the past 48 hours.

   

   Important

   • Your bid price is the maximum you are willing to pay, not the actual charge. You pay the current market price as long as it stays below your bid, which prevents reclamation due to price fluctuations.

   • Set your bid price at 20% of the list price. For example, if the pay-as-you-go list price for an 8-core, single-card L20 instance is 14.4 CNY/hour, the spot market price rarely exceeds 20% of the list price (2.88 CNY/hour). This balances cost savings with stable resource availability. Select spot instances.

   • Configure on-demand instances as a fallback in case spot instance allocation fails.

4. After deployment, monitor spot price fluctuations on the EAS service details page.

Spot instance reclamation and fault tolerance

EAS reclamation

EAS receives a warning approximately five minutes before reclamation. It initiates a graceful shutdown to migrate traffic and simultaneously launches a replacement instance based on your configured instance type order.

Reverting to spot instances

If a spot instance is reclaimed and replaced by an on-demand instance, use Rebuild Instance to switch back to a spot instance after inventory recovers.

Rebuild Instance releases current resources and recreates an instance with the same configuration:

Recommended configuration strategies

When a spot instance is reclaimed, EAS performs a graceful shutdown and launches a replacement. The following strategies accelerate startup and improve availability.

Configure multiple instance types

Select multiple instance types and include at least one on-demand instance (with bidding mode turned off) as a fallback. EAS deploys instances in the order you specify. Select multiple instance types.

Enable memory caching

Use spare node memory to cache model files, so new instances load from cache instead of the mounted data source. This reduces startup time after spot instance reclamation.

Enable Memory Caching during service creation to improve scale-out efficiency. Model caching acceleration.

With Stable Diffusion models, cachefs lets new instances load models from other instances' memory (cachefs remote hit) during scale-out, instead of reading from a mounted OSS directory.

Model	Model size	Model loading time (s)
		OSS mount	cachefs remote hit
anything-v4.5.safetensors	7.2 GB	89.88	15.18
Anything-v5.0-PRT-RE.safetensors	2.0 GB	16.73	5.46
cetusMix_Coda2.safetensors	3.6 GB	24.76	7.13
chilloutmix_NiPrunedFp32Fix.safetensors	4.0 GB	48.79	8.47
CounterfeitV30_v30.safetensors	4.0 GB	64.99	7.94
deliberate_v2.safetensors	2.0 GB	16.33	5.55
DreamShaper_6_NoVae.safetensors	5.6 GB	71.78	10.17
pastelmix-fp32.ckpt	4.0 GB	43.88	9.23
revAnimated_v122.safetensors	4.0 GB	69.38	3.20

Use ACR Enterprise Edition

To speed up image pulling after reclamation, use ACR Enterprise Edition with image acceleration. Append the _accelerated suffix to the standard image URL and select the VPC of your ACR Enterprise Edition instance when deploying.

In your ACR Enterprise Edition instance, select Standard for Instance Type. Enable image acceleration when creating an image repository to improve scale-out efficiency. For more information, see Image acceleration.

Image configuration for a custom deployment in EAS:

Purchase an ACR Standard Edition instance:

Enable image acceleration when creating an image repository:

Combine with auto scaling

For workloads with significant traffic fluctuations, combine horizontal auto scaling with a hybrid strategy: dedicated resource group for baseline, spot instances for elastic scaling, and on-demand instances for fallback.

1. Purchase a subscription or pay-as-you-go dedicated resource group as the baseline. Configure GPU, CPU, and memory requirements.

2. Enable the Elastic Resource Pool and configure multiple instance types. Prioritize spot instances, with on-demand instances as fallback. During traffic peaks, the service scales out with spot instances first, failing over to on-demand if unavailable.

   

3. After deployment, enable horizontal Auto Scaling on the service details page. Supported metrics: QPS, CPU utilization, GPU utilization, queue length, and custom metrics.

   

   Enable Scheduled Scaling to handle predictable traffic patterns.

   

Instances are created from the dedicated resource group first. If more resources are required, the system uses the Elastic Resource Pool, cycling through instance types in your specified order.

• Example: if the dedicated resource group is fully utilized and you need multiple instances from the Elastic Resource Pool, the system first checks inventory for ecs.gn8v-8x.48xlarge. If insufficient, it checks ecs.gn8is-8x.32large. If both are insufficient, it uses ml.gu7i.c8m30.1-gu30 for the remaining instances.

• Example: assume your service runs on 2 instances in a dedicated resource group and needs 8 additional instances. The allocation might be: 3 ecs.gn8v-8x.48xlarge spot instances, 2 ecs.gn8is-8x.32large spot instances, and 3 ml.gu7i.c8m30.1-gu30 on-demand instances, based on availability and your configuration.

Related auto scaling topics:

• Horizontal Auto Scaling

• Scheduled Scaling

• Elastic Resource Pool