Develop custom processors by using Python

• EASCMD (for Linux only)
  EASCMD is a client provided by EAS. This tool encapsulates the logic for initializing the SDK for Python. After you download EASCMD, you can initialize the SDK for Python environment by running only one command. The corresponding file template is generated.

  # Install and initialize EASCMD. In this example, the EASCMD client for Linux is installed.
  $ wget http://eas-data.oss-cn-shanghai.aliyuncs.com/tools/eascmd64
  # After you download EASCMD, modify access permissions and configure your AccessKey pair information.
  $ chmod +x eascmd64
  $ ./eascmd64 config -i <access_id> -k <access_key>
  
  # Initialize the environment.
  $ ./eascmd64 pysdk init ./pysdk_demo

  Then, enter the Python version. The default version is 3.6. The ENV directory, the prediction service template app.py, and the service deployment template app.json are automatically created for the Python environment.
• Manual mode
  If EASCMD cannot meet your requirements or you encounter problems during the initialization process, you can try to manually initialize the development environment. We recommend that you use Conda to deploy the environment.

  mkdir demo
  cd demo
  # Use Conda to create a Python environment. The directory must be ENV.
  conda create -p ENV python=2.6
  # Install the SDK for Python of EAS.
  ENV/bin/pip install http://eas-data.oss-cn-shanghai.aliyuncs.com/sdk/allspark-0.9-py2.py3-none-any.whl
  # Install other dependencies, such as TensorFlow 1.14.
  ENV/bin/pip install tensorflow==1.14

  If you have not locally installed Conda, run the following commands to install it:

  $ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
  $ sh Miniconda3-latest-Linux-x86_64.sh

• Pre-built development image (recommended)
  This section uses a pre-built development image provided by EAS, for which Conda has been pre-installed and a Python ENV environment is generated. EAS provides the following three pre-built images:

  # The base image installed with only Conda.
  registry.cn-shanghai.aliyuncs.com/eas/eas-python-base-image:latest
  # The SDK for Python installed with Conda, Python 2.7, and EAS Allspark 0.8.
  registry.cn-shanghai.aliyuncs.com/eas/eas-python-base-image:py2.7-allspark-0.8
  # The SDK for Python installed with Conda, Python 3.6, and EAS Allspark 0.8.
  registry.cn-shanghai.aliyuncs.com/eas/eas-python-base-image:py3.6-allspark-0.8

  Run the following run command on the image to obtain the Python development environment:

  $sudo docker run -ti registry.cn-shanghai.aliyuncs.com/eas/eas-python-base-image:py3.6-allspark-0.8
  (/data/eas/ENV) [root@487a04df4b21 eas]#
  (/data/eas/ENV) [root@487a04df4b21 eas]# ENV/bin/python app.py
  [INFO] initialize new lua plugin
  [INFO] loading builtin config script
  [INFO] current meritc id:0
  [INFO] loading builtin lua scripts
  [INFO] Success load all lua scripts.
  [INFO] create service
  [INFO] rpc binds to predefined port 8080
  [INFO] updating rpc port to 8080
  [INFO] install builtin handler call to /api/builtin/call
  [INFO] install builtin handler eastool to /api/builtin/eastool
  [INFO] install builtin handler monitor to /api/builtin/monitor
  [INFO] install builtin handler ping to /api/builtin/ping
  [INFO] install builtin handler prop to /api/builtin/prop
  [INFO] install builtin handler realtime_metrics to /api/builtin/realtime_metrics
  [INFO] install builtin handler tell to /api/builtin/tell
  [INFO] install builtin handler term to /api/builtin/term
  [INFO] Service start successfully
  [INFO] shutting down context ... press Ctrl+C again to force quit

  You can install your dependency libraries such as TensorFlow 1.12 based on the ENV environment of the base image, and then submit the modified container as a data image.

  ENV/bin/pip install tensorflow==1.12

  You can also build an ENV environment outside Docker and then copy the deployment package to the /data/eas/ directory of a Docker image. If you build a development environment by using an image, you do not need to upload the ENV environment package upon each deployment, which speeds up deployment.

# -*- coding: utf-8 -*-
import allspark
class MyProcessor(allspark.BaseProcessor):
    """ MyProcessor is a example
        you can send mesage like this to predict
        curl -v http://127.0.0.1:8080/api/predict/service_name -d '2 105'
    """
    def initialize(self):
        """ load module, executed once at the start of the service
             do service intialization and load models in this function.
        """
        self.module = {'w0': 100, 'w1': 2}
    def pre_process(self, data):
        """ data format pre process
        """
        x, y = data.split(b' ')
        return int(x), int(y)
    def post_process(self, data):
        """ process after process
        """
        return bytes(data, encoding='utf8')
    def process(self, data):
        """ process the request data
        """
        x, y = self.pre_process(data)
        w0 = self.module['w0']
        w1 = self.module['w1']
        y1 = w1 * x + w0
        if y1 >= y:
            return self.post_process("True"), 200
        else:
            return self.post_process("False"), 400
if __name__ == '__main__':
    # paramter worker_threads indicates concurrency of processing
    runner = MyProcessor(worker_threads=10)
    runner.run()