We recommend that you use official Elastic Algorithm Service (EAS) SDKs provided by Machine Learning Platform for AI (PAI) to call the services deployed based on models. This reduces the time required for defining the call logic and improves the call stability. This topic describes EAS SDK for Go in detail. This topic also provides demos to describe how to use EAS SDK for Go to call services. In these demos, inputs and outputs are of commonly used types.

Background information

When you use EAS SDK for Go to call services, the package manager of this SDK automatically downloads the code of this SDK to your on-premises machine from GitHub. Therefore, you can use this SDK to call services without the need to install it in advance. To customize specific part of the call logic, you can download the code of EAS SDK for Go and modify the code.

Methods

Class Method Detailed information
PredictClient NewPredictClient(endpoint string, serviceName string) *PredictClient
  • Description: Creates a client object of the PredictClient class.
  • Parameters:
    • endpoint: required. The endpoint of the server. To call a service in regular mode, set this parameter to the endpoint of the default gateway.
    • serviceName: required. The name of the service to be called.
  • Return value: the client object that you create.
SetEndpoint(endpointName string)
  • Description: Specifies the endpoint of the server.
  • Parameter: endpointName: the endpoint of the server. To call a service in regular mode, set this parameter to the endpoint of the default gateway.
SetServiceName(serviceName string)
  • Description: Specifies the name of the service.
  • Parameter: serviceName: the name of the service.
SetEndpointType(endpointType string)
  • Description: Specifies the gateway type of the server.
  • Parameter: endpointType: the gateway type to be used. The following gateway types are supported:
    • "DEFAULT": the default gateway. If you do not specify the gateway type, the default gateway is used.
    • "DIRECT": Virtual Private Cloud (VPC) direct connection channels.
SetToken(token string)
  • Description: Specifies the token for service access.
  • Parameter: token: the token that is used for service authentication.
SetHttpTransport(transport *http.Transport)
  • Description: Sets the transport attribute of the HTTP client.
  • Parameter: transport: the transport object that is used to send HTTP requests.
SetRetryCount(max_retry_count int)
  • Description: Specifies the maximum number of retries allowed after a request failure.
  • Parameter: max_retry_count: the maximum number of retries allowed after a request failure. Default value: 5.
    Note The client must resend requests if process errors occur on the server, server errors occur, or persistent connections to gateways are closed. Therefore, you cannot set this parameter to 0.
SetTimeout(timeout int)
  • Description: Sets the timeout period of a request.
  • Parameter: timeout: the timeout period of a request. Unit: milliseconds. Default value: 5000.
Init() Description: Initializes a client object. After all the preceding methods that are used to set parameters are called, the parameters take effect only after you call the Init() method.
Predict(request Request) Response
  • Description: Sends a prediction request to the online prediction service.
  • Parameter: Request: the request to be sent. It can be a string, a TFRequest object, or a TorchRequest object.
  • Return value: the response to the prediction request. It can be a string, a TFResponse object, or a TorchResponse request.
StringPredict(request string) string
  • Description: Sends a prediction request to the online prediction service.
  • Parameter: request: the request string to be sent.
  • Return value: the response to the prediction request, in the form of a string.
TorchPredict(request TorchRequest) TorchResponse
  • Description: Sends a PyTorch prediction request to the online prediction service.
  • Parameter: request: the request to be sent, which is a TorchRequest object.
  • Return value: the response to the prediction request. It is a TorchResponse object.
TFPredict(request TFRequest) TFResponse
  • Description: Sends a prediction request to the online prediction service.
  • Parameter: request: the request to be sent, which is a TFRequest object.
  • Return value: the response to the prediction request. It is a TFResponse object.
TFRequest TFRequest(signatureName string)
  • Description: Creates an object of the TFRequest class.
  • Parameter: signatureName: the signature name of the model of the service to be called.
AddFeed(?)(inputName string, shape []int64{}, content []?)
  • Description: Specifies the input tensor of the TensorFlow model of the online prediction service to be called.
  • Parameters:
    • inputName: the alias of the input tensor.
    • shape: the shape of the input tensor.
    • content: the data of the input tensor. Specify the value in the form of a one-dimensional array. The data type can be INT32, INT64, FLOAT32, FLOAT64, STRING, or BOOL. The name of this method is determined by the specific data type that is used. Example: AddFeedInt32(). If you want to use other data types, construct them in the protocol buffer (PB) format as needed based on the given code.
AddFetch(outputName string)
  • Description: Specifies the alias of the output tensor to be exported of the TensorFlow model.
  • Parameter: outputName: the alias of the output tensor to be exported.

    If the TensorFlow model is in the SavedModel format, this parameter is optional. If this parameter is not specified, all output tensors are exported.

    If the TensorFlow model is a frozen model, this parameter is required.

TFResponse GetTensorShape(outputName string) []int64
  • Description: Queries the shape of the output tensor identified by the specified alias.
  • Parameter: outputName: the alias of the output tensor whose shape you want to query.
  • Return value: the shape of the output tensor. Each dimension is displayed in the form of an array.
Get(?)Val(outputName string) [](?)
  • Description: Queries the data of the specified output tensor. The return value is in the form of a one-dimensional array. You can call this method together with the GetTensorShape() method to query the shape of the output tensor. The return value is in the form of a multi-dimensional array. The data type can be FLOAT, DOUBLE, INT, INT64, STRING, or BOOL. The name of this method is determined by the specific data type that is used. Example: GetFloatVal().
  • Parameter: outputName: the alias of the output tensor whose data you want to query.
  • Return value: a one-dimensional array converted from the retrieved output tensor data.
TorchRequest TorchRequest() Description: Creates an object of the TFRequest class.
AddFeed(?)(index int, shape []int64{}, content []?)
  • Description: Specifies the input tensor of the PyTorch model of the online prediction service to be called.
  • Parameters:
    • index: the index of the input tensor.
    • shape: the shape of the input tensor.
    • content: the data of the input tensor. Specify the value in the form of a one-dimensional array. The data type can be INT32, INT64, FLOAT32, or FLOAT64. The name of this method is determined by the specific data type that is used. Example: AddFeedInt32(). If you want to use other data types, construct them in the PB format as needed based on the given code.
AddFetch(outputIndex int)
  • Description: Specifies the index of the output tensor to be exported of the PyTorch model. This method is optional. If you do not call this method to set the index of the output tensor, all output tensors are exported.
  • Parameter: outputIndex: the index of the output tensor to be exported.
TorchResponse GetTensorShape(outputIndex int) []int64
  • Description: Queries the shape of the output tensor identified by the specified index.
  • Parameter: outputName: the alias of the output tensor whose shape you want to query.
  • Return value: the shape of the output tensor. Each dimension is displayed in the form of an array.
Get(?)Val(outputIndex int) [](?)
  • Description: Queries the data of the specified output tensor. The return value is in the form of a one-dimensional array. You can call this method together with the GetTensorShape() method to obtain the shape of the output tensor. The return value is in the form of a multi-dimensional array. The data type can be FLOAT, DOUBLE, INT, or INT64. The name of this method is determined by the specific data type that is used. Example: GetFloatVal().
  • Parameter: outputName: the index of the output tensor whose data you want to query.
  • Return value: a one-dimensional array converted from the retrieved output tensor data.

Demos

  • Input and output as strings
    If you use custom processors to deploy models as services, strings are often used to call the services, such as a service deployed based on a Predictive Model Markup Language (PMML) model. The following demo is for your reference:
    package main
    
    import (
            "fmt"
            "github.com/pai-eas/eas-golang-sdk/eas"
    )
    
    func main() {
        client := eas.NewPredictClient("182848887922****.cn-shanghai.pai-eas.aliyuncs.com", "scorecard_pmml_example")
        client.SetToken("YWFlMDYyZDNmNTc3M2I3MzMwYmY0MmYwM2Y2MTYxMTY4NzBkNzdj****")
        client.Init()
        req := "[{\"fea1\": 1, \"fea2\": 2}]"
        for i := 0; i < 100; i++ {
            resp, err := client.StringPredict(req)
            if err != nil {
                fmt.Printf("failed to predict: %v\n", err.Error())
            } else {
                fmt.Printf("%v\n", resp)
            }
        }
    }
  • Call a TensorFlow model
    If you use TensorFlow to deploy models as services, you must use the TFRequest and TFResponse classes to call the services. The following demo is for your reference:
    package main
    
    import (
            "fmt"
            "github.com/pai-eas/eas-golang-sdk/eas"
    )
    
    func main() {
        client := eas.NewPredictClient("182848887922****.cn-shanghai.pai-eas.aliyuncs.com", "mnist_saved_model_example")
        client.SetToken("YTg2ZjE0ZjM4ZmE3OTc0NzYxZDMyNmYzMTJjZTQ1YmU0N2FjMTAy****")
        client.Init()
    
        tfreq := eas.TFRequest{}
        tfreq.SetSignatureName("predict_images")
        tfreq.AddFeedFloat32("images", []int64{1, 784}, make([]float32, 784))
    
        for i := 0; i < 100; i++ {
            resp, err := client.TFPredict(tfreq)
            if err != nil {
                fmt.Printf("failed to predict: %v", err)
            } else {
                fmt.Printf("%v\n", resp)
            }
        }
    }
  • Call a PyTorch model
    If you use PyTorch to deploy models as services, you must use the TorchRequest and TorchResponse classes to call the services. The following demo is for your reference:
    package main
    
    import (
            "fmt"
            "github.com/pai-eas/eas-golang-sdk/eas"
    )
    
    func main() {
        client := eas.NewPredictClient("182848887922****.cn-shanghai.pai-eas.aliyuncs.com", "pytorch_resnet_example")
        client.SetTimeout(500)
        client.SetToken("ZjdjZDg1NWVlMWI2NTU5YzJiMmY5ZmE5OTBmYzZkMjI0YjlmYWVl****")
        client.Init()
        req := eas.TorchRequest{}
        req.AddFeedFloat32(0, []int64{1, 3, 224, 224}, make([]float32, 150528))
        req.AddFetch(0)
        for i := 0; i < 10; i++ {
            resp, err := client.TorchPredict(req)
            if err != nil {
                fmt.Printf("failed to predict: %v", err)
            } else {
                fmt.Println(resp.GetTensorShape(0), resp.GetFloatVal(0))
            }
        }
    }
  • Use a VPC direct connection channel to call a service
    You can use a VPC direct connection channel to access only the services that are deployed in the dedicated resource group for EAS. In addition, to use the channel, the dedicated resource group for EAS and the specified vSwitch must be connected to the VPC. For more information, see Dedicated resource groups and VPC direct connection channel. Compared with the regular mode, this mode contains an additional line of code: client.SetEndpointType(eas.EndpointTypeDirect). You can use this mode in high-concurrency and heavy-traffic scenarios. The following demo is for your reference:
    package main
    
    import (
            "fmt"
            "github.com/pai-eas/eas-golang-sdk/eas"
    )
    
    func main() {
        client := eas.NewPredictClient("pai-eas-vpc.cn-shanghai.aliyuncs.com", "scorecard_pmml_example")
        client.SetToken("YWFlMDYyZDNmNTc3M2I3MzMwYmY0MmYwM2Y2MTYxMTY4NzBkNzdj****")
        client.SetEndpointType(eas.EndpointTypeDirect)
        client.Init()
        req := "[{\"fea1\": 1, \"fea2\": 2}]"
        for i := 0; i < 100; i++ {
            resp, err := client.StringPredict(req)
            if err != nil {
                fmt.Printf("failed to predict: %v\n", err.Error())
            } else {
                fmt.Printf("%v\n", resp)
            }
        }
    }
  • Set the connection parameters of the client
    You can set the connection parameters of the client by using the http.Transport attribute. The following demo is for your reference:
    package main
    
    import (
            "fmt"
            "github.com/pai-eas/eas-golang-sdk/eas"
    )
    
    func main() {
        client := eas.NewPredictClient("pai-eas-vpc.cn-shanghai.aliyuncs.com", "network_test")
        client.SetToken("MDAwZDQ3NjE3OThhOTI4ODFmMjJiYzE0MDk1NWRkOGI1MmVhMGI0****")
        client.SetEndpointType(eas.EndpointTypeDirect)
        client.SetHttpTransport(&http.Transport{
            MaxConnsPerHost:       300,
            TLSHandshakeTimeout:   100 * time.Millisecond,
            ResponseHeaderTimeout: 200 * time.Millisecond,
            ExpectContinueTimeout: 200 * time.Millisecond,
        })
    }