Machine Learning Platform for AI:Use preemptible instances

Background information

• Preemptible instances

  You can specify preemptible instances that cost less than standard instances when you deploy a service to EAS. To specify preemptible instances, you must offer your highest bidding price to compete for the instances. For more information about preemptible instances, see Overview.

  Preemptible instances are more cost-effective than pay-as-you-go instances of the public resource group. Prices of preemptible instances fluctuate in real time based on the changes in supply and demand. You can configure a protection period for preemptible instances. Different categories of instances can be ranked in ascending order of their prices:

  preemptible instances without a protection period < preemptible instances with a protection period < standard instances

  .

  Note

  Preemptible instances can be released as the market price or stock changes. If a preemptible instance of a service is released, EAS scans the instance types configured for the service in the configuration file and selects the type with the highest specifications to create another instance.

• How can preemptible instances be created?

  When you deploy a service, you can specify whether to use preemptible instances for the service and set the highest bidding price for preemptible instances in the service configuration file. Then, preemptible instances can be created based on the service configuration file.

  Preemptible instances can be created if the following requirements are met:

  • The resources required to create the preemptible instances are sufficient.

  • Your highest bidding price is greater than or equal to the market prices of the preemptible instances.

• Usage of preemptible instances

  • By default, if a preemptible instance is created, you can use it for at least 1 hour. This means that the instance has a protection period of 1 hour, during which the system ensures your access to the instance.

  • After the 1-hour protection period expires, you can continue to use the instance if the resources required to create the instance are sufficient and your previous bidding price is greater than or equal to the market price of the instance.

    If the preceding requirements are not met, your instance may be released after the protection period expires.

• Release of preemptible instances

  • A preemptible instance is released if the 1-hour protection period expires and the resources required to create the instance are insufficient.

  • A preemptible instance is released if the 1-hour protection period expires and your previous bidding price is less than the market price of the instance.

• How are preemptible instances billed?

  Preemptible instances are billed on a pay-as-you-go basis. You are charged based on spot prices.

• In which order are preemptible instances used?

  When you create or update a service by using a service configuration file, you can specify multiple instance types in the file. For example, you can specify a preemptible instance type and a standard instance type. Then, the system chooses the first instance type to create or update the service. If your bidding price is less than the market price of the instance type or if the resources required to create instances of the instance type are insufficient, the second instance type is used.

Prerequisites

The EASCMD client is installed, and you have completed user authentication. For more information, see Download the EASCMD client and complete user authentication.

Limits

You can specify preemptible instances only when you create or modify a service by using the EASCMD client.

Procedure

Specify preemptible instances when you create a service

1. Prepare a service configuration file named service.json.

   {
     "name": "service_example",
     "model_path": "http://examplebucket.aliyuncs.com/models%2Fmnist_saved_model.tar.gz",
     "processor": "tensorflow_cpu_1.12",
     "metadata": {
       "instance": 1,
     }
     "cloud": {
       "computing": {
         "instances": [
           {"type": "ecs.g7.2xlarge", "spot_price_limit": 2.00},
           {"type": "ecs.g7.4xlarge", "spot_price_limit": 4.00},
           {"type": "ecs.g7.2xlarge"},
           {"type": "ecs.g7.4xlarge"}
         ],
         "disable_spot_protection_period": true
       }
     }
   }

   The following table describes the key parameters that you must specify in the file. For more information about other parameters, see Run commands to use the EASCMD client.

   Parameter	Description
   instance	The number of service instances. In this example, one instance is specified in the JSON configuration file.
   instances	One or more instance types that you can specify. If resources are insufficient for the first instance type, the system prepares resources for the next instance type. type: the instance type. spot_price_limit: your highest bidding price for preemptible instances. This field is optional. If this field is specified, the instance type indicates a preemptible instance whose price is less than the value specified by this field. Unit: USD. The pay-as-you-go billing method is supported. If this field is not specified, the instance type indicates a standard pay-as-you-go instance.
   disable_spot_protection_period	Specifies whether to disable the instance protection period. Default value: false. Valid values: false: The preemptible instance has a 1-hour protection period after the instance is created. During the protection period, even if the market price is higher than the bidding price, the instance cannot be released. true: The preemptible instance does not have a protection period. Instances without a protection period are 10% cheaper than instances that have a protection period.

2. Run the following command to create the service:

   eascmd create service.json

Specify preemptible instances when you modify the settings of a service

1. Prepare a service configuration file named instances.json.

   {
     "cloud": {
       "computing": {
         "instances": [
           {"type": "ecs.g7.2xlarge", "spot_price_limit": 2.00},
           {"type": "ecs.g7.4xlarge", "spot_price_limit": 4.00},
           {"type": "ecs.g7.2xlarge"},
           {"type": "ecs.g7.4xlarge"}
         ],
         "disable_spot_protection_period": true
       }
     }
   }

   For information about the key parameters, see Specify preemptible instances when you create a service.

2. Run the following command to modify the service configuration file to use multiple preemptible instance types:

   eascmd modify <service_name> -s instances.json

<service_name> indicates the name of the EAS service that you want to manage.