Platform For AI:Develop custom processors by using Python

Background information

Note

• We recommend that you separate the model file from the prediction logic of the processor. This helps you reuse the processor for future deployment when the model requires modification. To obtain the path of the model file within the processor, call the get_model_path() method. This path is then used to load the model.

• If a custom processor has excessive dependencies or the processor package is large in size, we recommend that you use an image to deploy the model. For more information about the two deployment methods, see the "Deployment method" section in the Overview topic.

Procedure for developing a custom processor by using Python:

1. Step 1: Create a Python environment

   Elastic Algorithm Service (EAS) SDK for Python supports multiple machine learning frameworks and can integrate with data analysis and processing frameworks such as Pandas. This topic describes how to create and upload an on-premises environment for Python development.

2. Step 2: Add prediction logic

   EAS SDK for Python adopts a high-performance remote procedure call (RPC) framework and contains necessary internal interfaces to facilitate interaction among EAS clusters. You need to only implement several functions in the prediction logic before you deploy the model in EAS.

3. Step 3: Run on-premises tests

   After you add the prediction logic, you must run on-premises tests to ensure that the service can work as expected after you deploy the service.

4. Step 4: Package the Python code and environment

   Package the Python code and environment in the required format.

5. Step 5: Upload the package

   Upload the package file and the model file to OSS.

6. Step 6: Deploy and test the service

   Use the custom processor to deploy the model service.

Prerequisites

The model file is prepared.

Step 1: Create a Python environment

Platform for AI (PAI) provides three methods to create a Python environment. You can use the EASCMD client or an official image for faster creation, or use package management tools such as Conda and Pyenv to manually create a Python environment that meets your business requirements. If you have customization requirements, you can manually build a development environment.

# Install and initialize EASCMD. In this example, the EASCMD client for Linux is installed. 
$ wget https://eas-data.oss-cn-shanghai.aliyuncs.com/tools/eascmd/v2/eascmd64
# After you install EASCMD, modify the access permissions by configuring your AccessKey pair. 
$ chmod +x eascmd64
$ ./eascmd64 config -i <access_id> -k <access_key>

# Initialize the environment. 
$ ./eascmd64 pysdk init ./pysdk_demo

[PYSDK] conda install complete, execute 'source ~/.bashrc' , and try again!

Step 2: Add prediction logic

# -*- coding: utf-8 -*-
import allspark


class MyProcessor(allspark.BaseProcessor):
    """ MyProcessor is a example
        you can send mesage like this to predict
        curl -v http://127.0.0.1:8080/api/predict/service_name -d '2 105'
    """
    def initialize(self):
        """ load module, executed once at the start of the service
             do service intialization and load models in this function.
        """
        self.module = {'w0': 100, 'w1': 2}
        # model_dir = self.get_model_path().decode()
        # Define the load_model function. If you need to load the model.pt model file, you can implement the function as torch.load(model_dir + "/model.pt"). 
        # self.model = load_model(model_dir)

    def pre_process(self, data):
        """ data format pre process
        """
        x, y = data.split(b' ')
        return int(x), int(y)

    def post_process(self, data):
        """ process after process
        """
        return bytes(data, encoding='utf8')

    def process(self, data):
        """ process the request data
        """
        x, y = self.pre_process(data)
        w0 = self.module['w0']
        w1 = self.module['w1']
        y1 = w1 * x + w0
        if y1 >= y:
            return self.post_process("True"), 200
        else:
            return self.post_process("False"), 400


if __name__ == '__main__':
    # parameter worker_threads indicates concurrency of processing
    runner = MyProcessor(worker_threads=10)
    runner.run()

model_dir = self.get_model_path().decode()
self.model = load_model(model_dir)

Step 3: Run on-premises tests

1. In the directory that contains the app.py file, run the following command to launch the Python project:

   ./ENV/bin/python app.py

   Sample success output:

   [INFO] waiting for service initialization to complete...
   [INFO] service initialization complete
   [INFO] create service
   [INFO] rpc binds to predefined port 8080
   [INFO] install builtin handler call to /api/builtin/call
   [INFO] install builtin handler eastool to /api/builtin/eastool
   [INFO] install builtin handler monitor to /api/builtin/monitor
   [INFO] install builtin handler ping to /api/builtin/ping
   [INFO] install builtin handler prop to /api/builtin/prop
   [INFO] install builtin handler realtime_metrics to /api/builtin/realtime_metrics
   [INFO] install builtin handler tell to /api/builtin/tell
   [INFO] install builtin handler term to /api/builtin/term
   [INFO] Service start successfully

2. Open another terminal and run the following command to send two requests.

   Verify the responses based on the sample code in the "Step 2: Add prediction logic" section of this topic.

   curl http://127.0.0.1:8080/test  -d '10 20'

Step 4: Package the Python code and environment

You can use the following methods to package the Python code and environment:

Step 5: Upload the package

After you package the Python code and environment, upload the package and the model file to OSS. This way, you can use the files when you deploy the service. The package is in the .zip or .tar.gz format. For information about how to upload files to OSS, see Get started with ossutil.

Step 6: Deploy and test the service

You can deploy the model service in the PAI console or on the EASCMD client.

1. Deploy the service.

   Use the PAI console

   1. Go to the Deploy Service page. For more information, see Model service deployment by using the PAI console.

   2. On the Deploy Service page, configure the parameters. The following table describes key parameters. For more information, see Model service deployment by using the PAI console.

      Parameter	Description
      Deployment Method	Select Deploy Service by Using Model and Processor.
      Model File	Configure this parameter based on your business requirements.
      Processor Type	Select Custom Processor.
      Processor Language	Select python.
      Processor Package	Select Import OSS File and select the OSS path where the package file is stored.
      Processor Main File	Set the value to ./app.py.

   3. Optional. Add the data_image parameter in the Configuration Editor section. Set the value to the image path that you configure when you package the files.

      Note

      Configure the data_image parameter only if you use an image to upload the development environment.Step 4: Package the Python code and environment

   4. Click Deploy.

   Use EASCMD

   The following section uses the Linux operating system as an example.

   1. Download the EASCMD client and perform identity authentication. For more information, see Download the EASCMD client and complete user authentication.

   2. Create a JSON file named app.json in the directory where the client is downloaded. The file contains the following content:

      • Sample file if the processor is packaged manually or by using the EASCMD client:

        {
          "name": "pysdk_demo",
          "processor_entry": "./app.py",
          "processor_type": "python",
          "processor_path": "oss://examplebucket/exampledirectory/pysdk_demo.tar.gz",
          "model_path": "oss://examplebucket/exampledirectory/model",
          "cloud": {
                "computing": {
                    "instance_type": "ecs.c7.large"
                }
          },
          "metadata": {
            "instance": 1,
            }
        }

      • Sample file if the development environment is uploaded by using an image:

        {
          "name": "pysdk_demo",
          "processor_entry": "./app.py",
          "processor_type": "python",
          "processor_path": "http://eas-data.oss-cn-shanghai.aliyuncs.com/demo/app.py",
          "data_image": "registry.cn-shanghai.aliyuncs.com/eas-service/develop:latest",
          "model_path": "oss://examplebucket/exampledirectory/model",
          "cloud": {
                "computing": {
                    "instance_type": "ecs.c7.large"
                }
          },
          "metadata": {
            "instance": 1,
            }
        }

   3. Open a terminal window. In the directory where the JSON file resides, run the following command to deploy the service:

      $ ./eascmd64 create app.json

      Sample success output:

      [RequestId]: 1202D427-8187-4BCB-8D32-D7096E95B5CA
      +-------------------+-------------------------------------------------------------------+
      | Intranet Endpoint | http://182848887922****.vpc.cn-beijing.pai-eas.aliyuncs.com/api/predict/pysdk_demo |
      |             Token | ZTBhZTY3ZjgwMmMyMTQ5OTgyMTQ5YmM0NjdiMmNiNmJkY2M5ODI0****          |
      +-------------------+-------------------------------------------------------------------+
      [OK] Waiting task server to be ready
      [OK] Fetching processor from [oss://eas-model-beijing/195557026392****/pysdk_demo.tar.gz]
      [OK] Building image [registry-vpc.cn-beijing.aliyuncs.com/eas/pysdk_demo_cn-beijing:v0.0.1-20190806082810]
      [OK] Pushing image [registry-vpc.cn-beijing.aliyuncs.com/eas/pysdk_demo_cn-beijing:v0.0.1-20190806082810]
      [OK] Waiting [Total: 1, Pending: 1, Running: 0]
      [OK] Service is running

2. Test the service.

   1. Go to the EAS-Online Model Services page. For more information, see Model service deployment by using the PAI console..

   2. Find the service that you want to test and click Invocation Method in the Service Type column to obtain the public endpoint and token.

   3. Run the following command in the terminal window to call the service.

      $ curl <service_url> -H 'Authorization: <token>' -d '10 20'

      Modify the following parameters:

      • Replace <service_url> with the public endpoint that you obtained in step b. Example: http://182848887922****.vpc.cn-beijing.pai-eas.aliyuncs.com/api/predict/pysdk_demo.

      • Replace <token> with the token that you obtained in step b. Example: ZTBhZTY3ZjgwMmMyMTQ5OTgyMTQ5YmM0NjdiMmNiNmJkY2M5ODI0****.

      • Use the -d option to specify the input parameters of the service.

References

• For information about how to resolve issues that you may encounter when you develop custom processors, see FAQ about custom processors.

• For information about how to develop custom processors by using Java, see Develop custom processors by using Java.