During business development, environmental dependencies are often complex. If you use a processor to deploy a model service, you must package shared libraries to the processor. You cannot install the dependency to a path of the system by running the yum install command. This method is less flexible. Therefore, Elastic Algorithm Service (EAS) provides a new feature that allows you to use a custom image to deploy a model service. This topic describes how to use a custom image to deploy a model service.

Prerequisites

Background information

Compared with the deployment by using processors, the deployment by using custom images has the following advantages in usability:
  • The deployment by using custom images is simple. To deploy a model service, you can provide an image rather than preparing parameters and configuration files.
  • The offline environment is consistent with the online environment. This prevents the issue that the online service cannot work as expected after the offline model passes the test because a dependency cannot be found.
Compared with the deployment by using processors, the deployment by using custom images has the following disadvantages:
  • To ensure security, EAS needs to provide URLs that contain authentication information for services and cannot provide services by using the HTTP URLs of custom images.
  • Specific features such as authentication, monitoring, and traffic depend on the EAS engine. For example, traffic needs to be forwarded to your image by using the EAS engine. In scenarios where the concurrency is high and the amount of traffic is large, traffic forwarding leads to performance overhead.

When you deploy a model service by using a custom image, the EAS engine acts as a sidecar container and runs with your container.

Usage notes

If you want to use a custom image to deploy a model service, take note of the following items:
  • EAS does not support Internet access. Therefore, you need to upload your image to an image repository in Container Registry. When you deploy a model service, you can access the image by using the virtual private cloud (VPC) endpoint of the image repository. Sample VPC endpoint: registry-vpc.cn-shanghai.aliyuncs.com.
  • If you use the SDK for Python provided by EAS as a service image and deploy a model service by using the image, the services in the image cannot use port 8080. The port configurations are in the MyProcessor constructor. For more information, see Develop custom processors by using Python.

Deploy a model service

  1. Create a service description file named image.json. The following code provides an example of the file content:
    {
        "containers":[
            {
                "image":"registry-vpc.cn-shanghai.aliyuncs.com/easapp/eas-container-deploy-test:20201009****",
                "env":
                [
                    {
                        "name":"VAR_NAME",
                        "value":"var_value"
                    }
                ],
                "command":"/data/eas/ENV/bin/python /data/eas/app1.py",
                "port":8000
            }
        ],
        "dockerAuth":"YWJjZDphYmNkZTEy****",
        "metadata":{
            "cpu":1,
            "instance":1,
            "memory":1000
        },
        "name":"registry_test"
    }
    The following table describes the parameters in the JSON file.
    Parameter Required Description
    containers image Yes The URL of the image that you want to use to deploy a model service.

    EAS does not support Internet access. Therefore, you can access the image by using the VPC endpoint of the image repository to which the image is uploaded. Sample VPC endpoint: registry-vpc.cn-shanghai.aliyuncs.com.

    env name Yes The name of the environment variable used when the image is executed.
    value Yes The value of the environment variable used when the image is executed.
    command Yes The commands in the image that are run to start containers.
    port Yes The container port.
    Notice You cannot specify port 8080 because the EAS engine listens on port 8080.
    dockerAuth No The authentication information used to log on to the Docker registry. The authentication information is required when the image is in a private image repository. The authentication information must be in the format of username:password and encoded in Base64.

    For example, if the value of username:password is abcd:abcde12345, you can run the echo -n "abcd:abcde12345" | base64 command to encode the value. Then, you can specify the output echo -n "abcd:abcde12345" | base64 as the value of the dockerAuth parameter.

    metadata No The metadata of the service. For more information, see Table 1.
    name Yes The name of the service, which must be unique in a region.
  2. Deploy your model service by running the following create command in the EASCMD client:
    eascmd create image.json
    image.json indicates the service description file that is created in the preceding step.
    The system displays information including the service endpoint and the token for authentication. The information is similar to the following output:
    [RequestId]: BFFFE5F5-1F07-437E-B59A-AF1F2B66****
    +-------------------+-----------------------------------------------------------------------------------+
    | Internet Endpoint | http://1828488879222746.cn-shanghai.pai-eas.aliyuncs.com/api/predict/registry_test     |
    | Intranet Endpoint | http://1828488879222746.vpc.cn-shanghai.pai-eas.aliyuncs.com/api/predict/registry_test |
    |             Token | NjA4MzQxOWQ0MTY2M2Y4OGY0NjgwODkwZTZmYWJmZWU1ZmY0Njhk****                          |
    +-------------------+-----------------------------------------------------------------------------------+
    [OK] Service is now creating
    [OK] Waiting [Total: 2, Pending: 2, Running: 0]
    [OK] Running [Total: 2, Pending: 0, Running: 2]
    [OK] Service is running
  3. If you need to modify service configurations, you can run the modify command in the EASCMD client:
    To modify the configurations of the service that you deploy in this topic, run the following command. For more information about the modify command, see Modify a service.
    eascmd modify registry_test -s image.json
    In the preceding command, registry_test indicates the name of the service, and image.json indicates the name of the service description file.

What to do next

After a model service is deployed, you can access the service for online prediction by using an endpoint in the output of the create command.

To access the service, append the application directory in the image to the endpoint to form the complete endpoint of the service For example, the endpoint returned by the create command is http://1828488879222746.cn-shanghai.pai-eas.aliyuncs.com/api/predict/registry_test, and the application directory in the image is /hello/model. The complete endpoint of the service is http://1828488879222746.cn-shanghai.pai-eas.aliyuncs.com/api/predict/registry_test/hello/model. For more information about how to access a service, see Call a service over a public endpoint.