Deployment of deep learning models based on Serverless Devs

Deployment of Deep Learning Model Based on Serverless Devs

to configure

Basic configuration of function calculation

After downloading the Serverless Devs developer tool and completing the Alibaba Cloud key configuration.

Use the command's init 'to complete the basic configuration of the project. Taking Alibaba Cloud as an example, select the Python 3 environment with an HTTP trigger and enter the key obtained in the first step.

Modify the configuration file s.yaml of the project, complete the configuration of the project name and description

Configuring NAS and VPC

In the deployment of deep learning models, the most difficult issues to deal with are the installation of large machine learning dependencies and the storage of deep learning model parameters. Generally speaking, the library size of machine learning can approach 1G, while deep models typically start at 100M. For function computing, in order to reduce platform cold starts, a code package size limit is set, and Alibaba Cloud's function computing code package must be within 100MB. Therefore, we need a cloud storage to solve the problem of dependent invocation and model parameter loading. In this example, we use NAS (Network Attached Storage) to store our dependencies and models. NAS, in simple terms, is a storage system that is connected to a network and has data storage capabilities. We can access NAS through mounting.

Activate Alibaba Cloud NAS services and create a file system. We can choose different configurations based on business needs.

Write the obtained NAS address into the configuration and set VPC to automatic.

Configuration considerations

Due to the long initialization time of the deep learning model, it is recommended to adjust the initialization time to more than 120s

Due to the high memory usage of the deep learning model, it is recommended to adjust the maximum memorySize to over 521MB

For more configurations, please refer to the S tool configuration document.

Code Architecture

Simply put, we can divide the serverless devs based project into three parts The S folder contains some configuration files, as well as dependencies and code generated by our build; The src folder stores our source code and deep learning model parameters; s. The yaml file stores our configuration for function calculations.

Among them, the index. py file in the src folder usually stores our function entry, and some sample code is provided here. In this project, we need to process the images uploaded by users. For the convenience of obtaining images, we use the Flask framework here. It should be noted that for operations that occur every time they are executed, such as loading model parameters, in order to reduce model inference time, we place them in the initialize function, so that this code will only be executed once.

Local debugging

Due to the fact that the local debugging environment is often different from the remote function calculation environment, for compatibility, it is best to use Docker pull functions to calculate the remote image for debugging.

Download Docker (if there is no Docker on this machine)

Using the command s build -- use docker for dependency installation

Use the command's local invoice/start 'for local debugging.

Build simple tests and make calls to local interfaces.

Upload Dependencies and Model Parameters

Upload dependencies and model parameters to NAS

Upload Dependency

Upload Model

Check if the upload was successful

View installed dependencies

After successful upload, we can simply use the path in the code to obtain the corresponding file:)

Model deployment

When debugging locally, our model parameters and dependent paths are both local paths. Now, we need to convert these paths into NAS paths and remove local model parameters and debugging generated dependencies (otherwise the code package would be too large to deploy). It should be noted that for dependencies, we need to modify the reference path of Python, which is reflected in our configuration file.

Next, run the command's deploy 'to deploy the code remotely.

Use the above test code to call the remote interface for testing.