use public resources to create PAI eas service - Platform For AI

Public resources are ideal for testing scenarios or businesses experiencing significant fluctuations in traffic, as they combine dedicated resources with elastic resource pools to minimize costs. However, public resources do not guarantee resource availability. When deploying services with public resources, you can also opt for spot instances (preemptible instances) to further reduce expenses and select multiple instance types to reduce deployment risks due to limited availability of a single type. This topic explains how to deploy model services using public resources.

Billing description

Public resources are billed based on actual usage. For more information, see EAS billing description.

Start Billing

Public resources support deploying model services using machine resources or machine models. Once the service is deployed and in Running status, billing commences.
PAI provides 30 GB of system disk capacity for each instance node of public resources at no cost. Additional system disks can be added and are billed on a pay-as-you-go basis, with billing starting after the system disk is successfully created.

Stop Billing

On the PAI EAS Model Online Service page, in the Service List tab, click Stop in the Operation column of the target service to halt the model service and billing.

Important

We recommend stopping any unused model services to avoid unnecessary costs.
Ensure that the stopped model service is no longer needed to prevent business disruption.

Spot instances

Spot instances are a cost-effective solution for deploying instances in preemptible mode with a set price limit.

Advantages
- Cost Savings: Spot instances are priced lower than standard pay-as-you-go instances, with real-time price fluctuations based on market supply and demand.
- Price Levels: Spot instances come in two types: with or without a protection period. The price hierarchy is as follows: no protection period price < protection period price < standard instance price.
Resource Preemption Conditions
- Spot instances are available when inventory is sufficient and the user's bid meets or exceeds the current market price.
Resource Release Conditions
- Instances with Protection Period: A minimum 1-hour protection period is provided by default. After this period, if inventory is low or the bid falls below the market price, the instance may be reclaimed.
- Instances without Protection Period: Instances may be reclaimed if inventory is low or the bid falls below the market price.
Billing Mode
- Spot instances are billed on a pay-as-you-go basis, with fees based on real-time market prices.

Multiple instance types

Specifying only one instance type during service deployment may lead to a shortage of resources, preventing the service from being created promptly. To mitigate this, the EAS deployment process accommodates multiple instance types. It cycles through the list of specifications in the configuration file to allocate resources, greatly minimizing the risk of deployment delays due to a lack of available resources for a single type.

Instance Usage Order

You can specify a sequence of instance types, such as spot and regular instances, for service creation or updating. The system will use these instances in the specified order, automatically switching to the next available type if the current one is unavailable or fails to meet the bid.
Resource Release and Reallocation

If spot instances are reclaimed due to inventory or price changes, EAS will reallocate the highest priority available resources according to the configuration to maintain service continuity.

Usage methods

Configure through the console

The following steps illustrate custom deployment.

Log in to the PAI console, select the destination region at the top of the page, and choose the target workspace on the right. Then click Enter EAS.
- Create a New Service: In the Inference Service tab, click Deploy Service, select Custom Model Deployment >Custom Deployment.
- Update Service: In the service list of the Inference Service tab, find the service to operate, and click Update in the Operation column.
In the Resource Deployment area, select Resource Type as Public Resources, and click the resource specification to choose the required resource specification from the pop-up list.
(Optional) Enable bidding. Activate the Bidding switch, set the bid, and select the bidding retention duration.
Note
- The Bidding switch is only available for resource specifications that support bidding.
- We recommend including regular instances when using spot instances to prevent deployment issues due to unsuccessful spot instance bidding.

4. (Optional) Configure multiple instance types. Click the add button to configure multiple instances.

Configure through the EASCMD client

For detailed steps on deploying model services through the EASCMD client, see Service Deployment: EASCMD.

When using the EASCMD client for the first time, you can retrieve the JSON configuration directly in the Service Configuration area after setting parameters in the console.

Examples of JSON parameters related to resource deployment are as follows:

{
    "metadata": {
        "instance": 1
    },
    "cloud": {
        "computing": {
            "instances": [
                {
                    "type": "ecs.c8i.2xlarge",
                    "spot_price_limit": 1
                    
                },
                {
                    "type": "ecs.c8i.xlarge"
                }
            ],
            "disable_spot_protection_period": false
        }
    }
}

Parameter	Description
metadata.instance	The number of service instances. In this example, one instance is specified in the JSON configuration file.
cloud.computing.instances	One or more instance types that you can specify. If resources are insufficient for the first instance type, the system prepares resources for the next instance type. type: Indicates the configured instance type. spot_price_limit is an optional parameter: If you configure this parameter, preemptible instances whose prices are less than the value specified by this parameter are used. The unit is USD, supporting pay-as-you-go. If this field is not specified, the instance type indicates a standard pay-as-you-go instance.
cloud.computing.disable_spot_protection_period	Values in the Event Source drop-down list: false (default value): Indicates that after a spot instance is successfully created, there is a default 1-hour protection period. During the protection period, even if the market price is higher than the bidding price, the instance cannot be released. true: Indicates that the protection period is disabled. Instances without a protection period are always about 10% cheaper than those with a protection period.

References

Public resources do not ensure resource availability. You can use dedicated resources to deploy services. For more information, see Use EAS Resource Group.
If you need to reduce network latency through VPC direct connection or if EAS services must access other cloud products within the same VPC, configure network connectivity. For more information, see Configure Network Connectivity.
Configure Simple Log Service for public resources to facilitate real-time monitoring of EAS services. Logs generated by EAS services using public resources are stored in Simple Log Service. For more information, see Configure Resource Group Simple Log Service.