Specify multiple instance types - Platform For AI - Alibaba Cloud Documentation Center

If you specify only a single instance type when you deploy a service to Elastic Algorithm Service (EAS), resources of this type may be insufficient and the service cannot be created. To address this issue, EAS allows you to specify multiple instance types when you create a service. Then, the system prepares resources based on the instance types you specified in the configuration file. This method reduces the chances of insufficient resources in scenarios where you specify only a single instance type. This topic describes how to specify multiple instance types.

Prerequisites

If you want to specify multiple instance types by using a client, make sure that the EASCMD client is installed, and you have completed user authentication. For more information, see Download the EASCMD client and complete identity authentication.

Limits

You can specify multiple instance types only for EAS services deployed in the public resource group.

Specify multiple instance types when you deploy a service

Configure in the console

Go to the Deploy Service page. For more information, see Model service deployment by using the PAI console and Machine Learning Designer.

On the Deploy Service page, configure the following parameters. For more information about other parameters, see Model service deployment by using the PAI console and Machine Learning Designer. image..png

Parameter	Description
Resource Group Type	Select Public Resource Group.
Resource Configuration Mode	Select Advanced.
Preemptible Instance Protection Period	This configuration takes effect only when you set Resource Type to Spot. Valid values: 1-Hour Protection Period: You can set a protection period of 1 hour for a preemptible instance. This means that during a protection period of 1 hour, the system ensures your access to the instance. No Protection Period: The preemptible instance does not have a protection period. For more information about preemptible instances, see EAS preemptible instance Spot Instances.
Deployment	Resource Type: Spot and Common are supported. Node Type: Select an appropriate node type. Bid Price: You need to set a bidding price for only preemptible instances. The system automatically bids for preemptible instances based on the bid price. If the resources required to create the instance are sufficient and your bidding price is greater than or equal to the market price of the instance, the instances can be created. Specify multiple instance types: You can click icon to specify multiple instance types. You can add up to five instance types. Resources are started based on the sequence in which the instance types are configured.

Click Deploy, When Service Status changes to Running, the service is deployed.

Configure in the client

In this topic, a Windows 64 server is used as an example.

Create a service configuration file named service.json in the directory where the client is located. Sample content:

{
  "name": "service_example",
  "model_path": "http://examplebucket.aliyuncs.com/models%2Fmnist_saved_model.tar.gz",
  "processor": "tensorflow_cpu_1.12",
  "metadata": {
    "instance": 1,
  }
  "cloud": {
    "computing": {
      "instances": [
        {"type": "ecs.g7.2xlarge", "spot_price_limit": 2.00},
        {"type": "ecs.g7.4xlarge", "spot_price_limit": 4.00},
        {"type": "ecs.g7.2xlarge"},
        {"type": "ecs.g7.4xlarge"}
      ],
      "disable_spot_protection_period": true
    }
  }
}

The following table describes the key parameters that are required in the file. For more information about other parameters, see Run commands to use the EASCMD client.

Parameter	Description
instance	The number of service instances. In this example, one instance is specified in the JSON configuration file.
instances	Instance types. You can specify one or more instance types. If resources are insufficient for the first instance type, the system prepares resources for the next instance type. type: the instance type. spot_price_limit: your highest bidding price for preemptible instances. This field is optional. If this field is specified, preemptible instances whose prices are less than the value specified by this field are used. Unit: USD. The pay-as-you-go billing method is supported. If this field is not specified, standard pay-as-you-go instances are used.
disable_spot_protection_period	Specifies whether to disable the instance protection period. Default value: false. Valid values: false: The preemptible instance has a 1-hour protection period after the instance is created. During the protection period, the instance provides services even if the market price is higher than the bidding price. true: The preemptible instance does not have a protection period. Instances without a protection period are 10% cheaper than instances that have a protection period.

Run the following command in the directory where the JSON file is located to create the service:
```
eascmdwin64.exe create <service.json>
```
Replace <service.json> with the name of the JSON file that you created.

Specify multiple instance types when you modify the settings of a deployed service

Modify in the console

Go to the EAS-Online Model Services page.
1. Log on to the PAI console.
2. In the left-side navigation pane, click Workspaces. On the Workspaces page, click the name of the workspace that you want to manage.
3. In the left-side navigation pane, choose Model Deployment > Elastic Algorithm Service (EAS). The EAS-Online Model Services page appears.
Find the service that is deployed in the public resource group and click the service name to go to the Service Details page.
On the Service Details tab, click Resource Configuration in the Resource Information section.
On the Advanced tab of the Resource Configuration dialog box, select multiple instance types in the Deployment section. For more information, see Configure in the console.
Click OK.

Modify in the client

In this topic, a Windows 64 server is used as an example.

Create a file named instances.json in the directory where the client is located. Sample content:

{
  "cloud": {
    "computing": {
      "instances": [
        {"type": "ecs.g7.2xlarge", "spot_price_limit": 2.00},
        {"type": "ecs.g7.4xlarge", "spot_price_limit": 4.00},
        {"type": "ecs.g7.2xlarge"},
        {"type": "ecs.g7.4xlarge"}
      ],
      "disable_spot_protection_period": true
    }
  }
}

For information about the key parameters, see the "Specify preemptible instances when you create a service" section in this topic.

Run the following command to modify the service configuration file to use multiple instance types:
```
eascmd modify <service_name> -s instances.json
```
<service_name> indicates the name of the EAS service that you want to manage.