This topic describes how to construct a request for a Caffe service that is based on a universal processor.
Notes on input data
Elastic Algorithm Service (EAS) provides a built-in Caffe processor for you to deploy Caffe models as services. To guarantee the performance, you must ensure that the input and output data is in the Protocol Buffers format.
Examples
A public test model is deployed as a service in the China (Shanghai) region. The service
name is caffenet_serving_example. The service is accessible to all users in VPCs in this region. No access token is
specified. You can send requests to the http://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/caffenet_serving_example endpoint to call the service. The following section describes how to call the service:
- Obtain the model information.
The Caffe model file stores information of each layer of the model. You can open the file to view relevant information. In this example, the model file contains the following content:
This is a classic CaffeNet model. The layer that has the type parameter set to Input defines the input of the model. Typically, it is the first layer. The last layer defines the output of the model. The input shape of the model is [10, 3, 227, 227]. The first dimension is batch_size. If a request contains only one image, batch_size is set to 1. When you construct the input, you must flatten it into a one-dimensional vector regardless of the value of shape. In this example, the one-dimensional vector of 1 by 3 by 227 by 227 is used. If the input shape specified in the request does not match the input shape of the model, the request fails.name: "CaffeNet" layer { name: "data" type: "Input" top: "data" input_param { shape { dim: 10 dim: 3 dim: 227 dim: 227 } } } .... layer { name: "prob" type: "Softmax" bottom: "fc8" top: "prob" }
- Install Protocol Buffers and call the service. This topic describes how to use a Python
client to call a Caffe service.
EAS provides a Protocol Buffers package for Python clients. You can run the following command to install it:
The following sample code is used to call the service to make a prediction:$ pip install http://eas-data.oss-cn-shanghai.aliyuncs.com/pai_caffe_predict_proto-1.0-py2.py3-none-any.whl
#! /usr/bin/env python # -*- coding: UTF-8 -*- import requests from pai_caffe_predict_proto import caffe_predict_pb2 # build request. request = caffe_predict_pb2.PredictRequest() request.input_name.extend(['data']) array_proto = caffe_predict_pb2.ArrayProto() array_proto.shape.dim.extend([1, 3, 227, 227]) array_proto.data.extend([1.0]*3*227*227) request.input_data.extend([array_proto]) # Serialize data in the Protocol Buffers format to a string and transfer the string. data = request.SerializeToString() # The API for prediction must be called from a VPC in the China (Shanghai) region. url = 'http://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/caffenet_serving_example' s = requests.Session() resp = s.post(url, data=data) if resp.status_code != 200: print resp.content else: response = caffe_predict_pb2.PredictResponse() response.ParseFromString(resp.content) print(response)
Use a client in other languages to call the service
If you use a client in a language other than Python, you must manually generate the
prediction request code file based on the .proto file. The following section shows the sample code:
- Prepare a request code file, such as caffe.proto, which contains the following content:
In the file,syntax = "proto2"; package caffe.eas; option java_package = "com.aliyun.openservices.eas.predict.proto"; option java_outer_classname = "CaffePredictProtos"; message ArrayShape { repeated int64 dim = 1 [packed = true]; } message ArrayProto { optional ArrayShape shape = 1; repeated float data = 2 [packed = true]; } message PredictRequest { repeated string input_name = 1; repeated ArrayProto input_data = 2; repeated string output_filter = 3; } message PredictResponse { repeated string output_name = 1; repeated ArrayProto output_data = 2; }
PredictRequest
defines the input format of the Caffe service, andPredictResponse
defines the output format of the service. For more information about Protocol Buffers, see Protocol Buffers. - Install protoc.
#/bin/bash PROTOC_ZIP=protoc-3.3.0-linux-x86_64.zip curl -OL https://github.com/google/protobuf/releases/download/v3.3.0/$PROTOC_ZIP unzip -o $PROTOC_ZIP -d ./ bin/protoc rm -f $PROTOC_ZIP
- Generate the request code file.
- Java
After the command completes, the request code file com/aliyun/openservices/eas/predict/proto/CaffePredictProtos.java is generated in the current directory. Import the file to the project.$ bin/protoc --java_out=./ caffe.proto
- Python
After the command completes, the request code file caffe_pb2.py is generated in the current directory. Run the$ bin/protoc --python_out=./ caffe.proto
import
command to import the file to the project. - C++
After the command completes, the request code files including caffe.pb.cc and caffe.pb.h are generated in the current directory. Add the$ bin/protoc --cpp_out=./ caffe.proto
include caffe.pb.h
command to the code and add caffe.pb.cc to the compile list.
- Java