Public resources are ideal for testing or for services with fluctuating traffic that benefit from an elastic resource pool. They help minimize costs, but their availability is not guaranteed. When you deploy services using public resources, you can use spot instances to further reduce costs. You can also configure multiple instance types to lower the deployment risk caused by a single instance type being out of stock. This topic describes how to deploy model services using public resources.
Billing information
Public resources are billed based on actual usage. For more information, see EAS billing information.
Billing starts
Public resources support deploying model services using machine resources or instance types. Billing starts once the service is deployed and enters the Running state.
PAI provides 30 GiB of free system disk capacity for each machine node in the public resource group. You can expand the system disk on a pay-as-you-go basis. Billing for the system disk starts after it is successfully created.
Billing stops
On the Elastic Algorithm Service (EAS) page, go to the Inference Service tab. In the Actions column for the target service, click Stop to stop the model service and its billing.
Stop model services that are no longer needed to avoid unnecessary charges.
Ensure that the service you are stopping is no longer required to prevent business loss.
When using public resources, if an instance fails to be created due to insufficient resources, the system automatically retries creation once resources become available. Stop or delete such model services if they are not needed.
To determine whether a failure is caused by insufficient resources, click the service name to go to the service details page and check the instance status.
Core concepts
Spot instances
A spot instance is an instance type that you can deploy by setting a maximum price in a preemptible model. It provides cost-effective compute resources.
Advantages
Cost savings: Spot instances offer low prices. The price changes in real time based on supply and demand and is usually lower than that of regular pay-as-you-go instances in the public resource group.
Price tiers: Spot instances are available with or without a protection period. Prices, from lowest to highest, are: no protection period < with protection period < regular instance.
Resource preemption conditions
The spot instance inventory is sufficient, and your bid is not lower than the current market price.
Resource release conditions: This depends on the spot instance retention period setting.
Set instance usage to 1 hour: This provides a one-hour protection period. The instance will not be released during this period. After the protection period ends, it may be automatically released.
No specific protection period: Continuous usage is not guaranteed. The instance may be automatically released due to changes in inventory or market price.
Billing model
Spot instances use a pay-as-you-go billing model. Charges are calculated based on the real-time market price.
Multi-specification instances
If you specify only a single instance type when you deploy a service, resources of this type may be insufficient and the service cannot be launched. To resolve this issue, the EAS deployment phase supports selecting multiple instance types. The system traverses the list of instance types provided in the configuration file to launch resources, thereby greatly reducing the deployment risk caused by insufficient resources of a single instance type.
Instance usage order
When you create or update a service, you can specify multiple instance types, such as spot instances and regular instances. During deployment, the system attempts to use these instances in the order they are configured. If a bid for an instance type fails or the instance type is out of stock, the system automatically switches to the next available type in the list.
Resource release and reallocation
If a configured spot instance is released due to changes in inventory or market price, EAS reallocates the highest-priority available resource based on the configuration file to ensure service continuity.
System disk
PAI provides 30 GiB of free system disk capacity for each machine node in the public resource group. If you need more capacity, you are charged based on actual usage. For billing details, see Elastic Algorithm Service (EAS) billing information.
The maximum system disk size is 2000 GiB. If you exceed this limit, the model service deployment will fail.
Procedure
Configure using the console
This section uses custom deployment as an example.
Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Elastic Algorithm Service (EAS).
Create a service: On the Inference Service tab, click Deploy Service. Then, select Custom Model Deployment > Custom Deployment.
Update a service: On the Inference Service tab, find the target service in the service list. In the Actions column, click Update.
In the Resource Information section, set Resource Type to Public Resource Group. Click the instance type field and select the required instance type from the list.
(Optional) Enable spot instances. Turn on the Bidding switch, set a bid price, and select a retention period.
NoteThe Bidding switch can only be enabled for instance types that support spot instances.
When using spot instances, also configure regular instances to prevent deployment failures if the bid fails.
(Optional) Configure multiple instance types. Click the add button to configure multiple instance types.
Configure the system disk size.
Configure using JSON
After you configure the parameters in the console, you can obtain the JSON configuration in the Service Configuration section. You can also directly edit the JSON parameters to configure the service.
The following example shows the JSON parameters related to resource deployment:
{
"metadata": {
"name": "test",
"instance": 1,
"workspace_id": "your-workspace-id",
"disk": "40Gi"
},
"cloud": {
"computing": {
"instances": [
{
"type": "ecs.c8i.2xlarge",
"spot_price_limit": 1
},
{
"type": "ecs.c8i.xlarge"
}
],
"disable_spot_protection_period": true
}
},
"containers": [
{
"image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/python-inference:py39-ubuntu2004",
"script": "python app.py",
"port": 8000
}
]
}Parameter | Description | ||
metadata | instance | The number of replicas to launch for the service. In the preceding JSON file, this is set to 1. Note EAS supports single-node and multi-node distributed inference.
| |
disk | The system disk size. The public resource group provides 30 GiB for free. If you need more capacity, you are charged based on actual usage. The maximum value is 2000 GiB. | ||
cloud | computing | instances | Specifies the allowed instance types. You can configure multiple types. If a bid for an instance type fails or the instance type is out of stock, the system tries to create the service using the next instance type in the configured order.
|
disable_spot_protection_period | The following values are supported:
| ||
FAQ
What do I do if public resources are out of stock?
When you deploy popular models with many parameters, public resources may be out of stock. Consider the following solutions:
Switch regions. Resource availability varies by region. You can switch to a different region to find available public resources.
ImportantConsider switching to the Ulanqab region to use Lingjun preemptible resources (no whitelist required). Preemptible resources may be reclaimed, so monitor your bid price.
Use an EAS resource group. Some instance types are not available as public resources. Go to EAS Dedicated Resource Subscription to purchase dedicated EAS resources.
ImportantPay-as-you-go dedicated resources start billing immediately after purchase, regardless of whether they are used to deploy services. Delete unused pay-as-you-go machines promptly to avoid unnecessary charges.
References
Public resources do not guarantee resource availability. Use dedicated resources to deploy services. For more information, see Use an EAS resource group.
You can configure Simple Log Service for public resources to enable real-time monitoring and troubleshooting. For more information, see Configure Simple Log Service for a resource group.