This guide explains the APIs of the official SDK for Go and provides complete code examples for common input and output formats.
For information about the SDK's use cases and invocation principles, see Service invocation SDKs.
Prerequisites
When you call an inference service using the SDK for Go, the Go package manager automatically downloads the SDK source code from GitHub during compilation. You do not need to install the SDK beforehand. If you need to customize the invocation logic, you can download the SDK for Go source code and modify it locally.
To import the SDK, use the following code:
import (
"github.com/pai-eas/eas-golang-sdk/eas"
)Quick start
Select the Request class that corresponds to your model's input data format. The following example shows a minimal, end-to-end invocation using a string request.
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
)
func main() {
client := eas.NewPredictClient("182848887922****.cn-shanghai.pai-eas.aliyuncs.com", "my_service")
client.SetToken("YOUR_SERVICE_TOKEN")
client.Init()
resp, err := client.StringPredict("[{}]")
if err != nil {
fmt.Printf("failed to predict: %v\n", err.Error())
} else {
fmt.Printf("%v\n", resp)
}
}API reference
The SDK for Go provides the following classes, grouped by purpose:
Group | Description |
Main client | PredictClient: Configures service information such as endpoint, service name, and token, sends requests, and receives responses. |
Input and output |
|
Queuing service |
|
PredictClient
The main client class used to configure service information, send requests, and receive prediction results.
Method | Description |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Initializes the PredictClient object. After setting parameters with the preceding methods, you must call |
|
|
|
|
|
|
|
|
TFRequest
Builds input data for TensorFlow models.
Method | Description |
|
|
|
|
|
|
TFResponse
Parses output data from TensorFlow models.
Method | Description |
|
|
|
|
TorchRequest
Builds input data for PyTorch models.
Method | Description |
| Creates a |
|
|
|
|
TorchResponse
Parses output data from PyTorch models.
Method | Description |
|
|
|
|
QueueClient
Interacts with the EAS queuing service to produce, consume, and manage data.
Method | Description |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
types.Watcher
Reads pushed data from the subscription channel of the queuing service.
Method | Description |
|
|
| Closes a watcher object and its backend data connection. Note A client can have only one active watcher object at a time. You must close the current watcher object before creating a new one. |
Examples
Synchronous inference by format
Choose the code sample based on your service's input and output types.
String
If you deploy a service with a custom processor, you typically invoke it using strings, for example, when calling a PMML model service. The following program shows a complete example.
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
)
func main() {
client := eas.NewPredictClient("182848887922****.cn-shanghai.pai-eas.aliyuncs.com", "scorecard_pmml_example")
client.SetToken("YWFlMDYyZDNmNTc3M2I3MzMwYmY0MmYwM2Y2MTYxMTY4NzBkNzdj****")
client.Init()
req := "[{\"fea1\": 1, \"fea2\": 2}]"
for i := 0; i < 100; i++ {
resp, err := client.StringPredict(req)
if err != nil {
fmt.Printf("failed to predict: %v\n", err.Error())
} else {
fmt.Printf("%v\n", resp)
}
}
}TensorFlow
For TensorFlow models, use TFRequest and TFResponse as the input and output data formats, respectively. The following program shows a complete example.
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
)
func main() {
client := eas.NewPredictClient("182848887922****.cn-shanghai.pai-eas.aliyuncs.com", "mnist_saved_model_example")
client.SetToken("YTg2ZjE0ZjM4ZmE3OTc0NzYxZDMyNmYzMTJjZTQ1YmU0N2FjMTAy****")
client.Init()
tfreq := eas.TFRequest{}
tfreq.SetSignatureName("predict_images")
tfreq.AddFeedFloat32("images", []int64{1, 784}, make([]float32, 784))
for i := 0; i < 100; i++ {
resp, err := client.TFPredict(tfreq)
if err != nil {
fmt.Printf("failed to predict: %v", err)
} else {
fmt.Printf("%v\n", resp)
}
}
}PyTorch
For PyTorch models, use TorchRequest and TorchResponse as the input and output data formats, respectively. The following program shows a complete example.
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
)
func main() {
client := eas.NewPredictClient("182848887922****.cn-shanghai.pai-eas.aliyuncs.com", "pytorch_resnet_example")
client.SetTimeout(500)
client.SetToken("ZjdjZDg1NWVlMWI2NTU5YzJiMmY5ZmE5OTBmYzZkMjI0YjlmYWVl****")
client.Init()
req := eas.TorchRequest{}
req.AddFeedFloat32(0, []int64{1, 3, 224, 224}, make([]float32, 150528))
req.AddFetch(0)
for i := 0; i < 10; i++ {
resp, err := client.TorchPredict(req)
if err != nil {
fmt.Printf("failed to predict: %v", err)
} else {
fmt.Println(resp.GetTensorShape(0), resp.GetFloatVal(0))
}
}
}VPC direct connection
A VPC direct connection lets you access services deployed in an Elastic Algorithm Service (EAS) dedicated resource group. You must also connect the resource group to the specified vSwitch before you can use this mode. For information about how to purchase an EAS dedicated resource group and configure network connectivity, see Use EAS resource groups and Configure EAS to access public or internal resources. This method differs from a standard invocation by requiring only one additional line of code: client.SetEndpointType(eas.EndpointTypeDirect). This mode is ideal for high-traffic, high-concurrency services. The following is a code sample:
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
)
func main() {
// Format of a VPC direct connection endpoint: {uid}.vpc.{region-id}.pai-eas.aliyuncs.com. You can find the endpoint on the Invocation Information tab of the service details page in the EAS console.
client := eas.NewPredictClient("182848887922****.vpc.cn-shanghai.pai-eas.aliyuncs.com", "scorecard_pmml_example")
client.SetToken("YWFlMDYyZDNmNTc3M2I3MzMwYmY0MmYwM2Y2MTYxMTY4NzBkNzdj****")
client.SetEndpointType(eas.EndpointTypeDirect)
client.Init()
req := "[{\"fea1\": 1, \"fea2\": 2}]"
for i := 0; i < 100; i++ {
resp, err := client.StringPredict(req)
if err != nil {
fmt.Printf("failed to predict: %v\n", err.Error())
} else {
fmt.Printf("%v\n", resp)
}
}
}Client connection parameters
You can set the connection parameters for the client by using the http.Transport property. The following example demonstrates how to configure these settings:
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
"net/http"
"time"
)
func main() {
// Format of a VPC direct connection endpoint: {uid}.vpc.{region-id}.pai-eas.aliyuncs.com. You can find the endpoint on the Invocation Information tab of the service details page in the EAS console.
client := eas.NewPredictClient("182848887922****.vpc.cn-shanghai.pai-eas.aliyuncs.com", "network_test")
client.SetToken("MDAwZDQ3NjE3OThhOTI4ODFmMjJiYzE0MDk1NWRkOGI1MmVhMGI0****")
client.SetEndpointType(eas.EndpointTypeDirect)
client.SetHttpTransport(&http.Transport{
MaxConnsPerHost: 300,
TLSHandshakeTimeout: 100 * time.Millisecond,
ResponseHeaderTimeout: 200 * time.Millisecond,
ExpectContinueTimeout: 200 * time.Millisecond,
})
}Queuing service
You can use QueueClient to send data to a queuing service, query data, query the status of the queuing service, and subscribe to data pushes from the queuing service. In this example, one goroutine sends data to the queuing service, while another uses a watcher to subscribe to and receive that data.
When you deploy an asynchronous inference service in EAS, an input queue and an output queue are automatically generated. The addresses are typically in the following formats:
Input queue: <domain>/api/predict/<service_name>
Output queue: <domain>/api/predict/<service_name>/sink
Use <service_name> or <service_name>/sink to build the QueueClient based on your requirements.
const (
QueueEndpoint = "182848887922****.cn-shanghai.pai-eas.aliyuncs.com"
// For example, if the EAS service name is test_qservice, the input queue name is test_qservice, and the output queue name is test_qservice/sink.
QueueName = "test_qservice"
QueueToken = "YmE3NDkyMzdiMzNmMGM3ZmE4ZmNjZDk0M2NiMDA3OTZmNzc1MTUx****"
)
queue, err := NewQueueClient(QueueEndpoint, QueueName, QueueToken)
// truncate all messages in the queue
attrs, err := queue.Attributes()
if index, ok := attrs["stream.lastEntry"]; ok {
idx, _ := strconv.ParseUint(index, 10, 64)
queue.Truncate(context.Background(), idx+1)
}
ctx, cancel := context.WithCancel(context.Background())
// create a goroutine to send messages to the queue
go func() {
i := 0
for {
select {
case <-time.NewTicker(time.Microsecond * 1).C:
_, _, err := queue.Put(context.Background(), []byte(strconv.Itoa(i)), types.Tags{})
if err != nil {
fmt.Printf("Error occured, retry to handle it: %v\n", err)
}
i += 1
case <-ctx.Done():
break
}
}
}()
// create a watcher to watch the messages from the queue
watcher, err := queue.Watch(context.Background(), 0, 5, false, false)
if err != nil {
fmt.Printf("Failed to create a watcher to watch the queue: %v\n", err)
return
}
// read messages from the queue and commit manually
for i := 0; i < 100; i++ {
df := <-watcher.FrameChan()
err := queue.Commit(context.Background(), df.Index.Uint64())
if err != nil {
fmt.Printf("Failed to commit index: %v(%v)\n", df.Index, err)
}
}
// everything is done, close the watcher
watcher.Close()
cancel()Troubleshooting
For troubleshooting common issues when calling services with the SDK for Go, including their causes and solutions, see the "Troubleshoot invocation exceptions" section in Service invocation SDKs.
For a complete list of service status codes, error messages, and recommended actions, see Appendix: Service status codes and common errors.