PAI-EAS supports model inference over gRPC. Compared to HTTP/JSON, gRPC provides more efficient serialization and better streaming support, ideal for latency-sensitive or streaming inference workloads.
Prerequisites
Before you begin:
-
You have deployed a PAI-EAS inference service.
-
You have configured a Go development environment (Go 1.18 or later).
-
You have installed the required gRPC dependency packages.
Key points
Endpoint format
To call a service through the gateway, get the endpoint from the PAI-EAS console, extract the domain name, and append port :80 as described in Obtain the access endpoint and token.
The port is fixed at 80 because PAI-EAS exposes gRPC services through a gateway that listens on port 80.
Example endpoints:
|
Access method |
Console endpoint |
gRPC endpoint (append |
|
Public network |
|
|
|
VPC |
|
|
Note: A gRPC endpoint does not include the http:// prefix.
Token authentication
PAI-EAS uses Bearer Token authentication. Pass the token in gRPC request metadata by setting the authorization field as follows:
authorization: Bearer <Token>
Prefix the token with Bearer and a space. Get your token from the service details page in the PAI-EAS console.
Code examples
Before coding, prepare the proto file for your service and compile the client stub.
These examples show the core gRPC call workflow. The corresponding proto file and server code are in the gRPC service demo appendix.
package main
import (
"context"
"log"
"time"
pb "your-project/proto"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/metadata"
)
func main() {
// Replace with your gRPC endpoint. You can find it in the call info panel in the PAI-EAS console.
// The public network endpoint format is: service-name.{uid}.{region}.pai-eas.aliyuncs.com:80 (the port is fixed at 80).
host := "your-service-endpoint:your port"
// Replace with your service token. You can find it on the service details page in the PAI-EAS console.
token := "your-service-token"
log.Printf("Connecting to the PAI-EAS gRPC service (%s)...", host)
// 1. Establish a connection. Connections are thread-safe. We recommend that you create one connection and reuse it.
conn, err := grpc.NewClient(
host,
grpc.WithTransportCredentials(insecure.NewCredentials()),
)
if err != nil {
log.Fatalf("Failed to connect: %v", err)
}
defer conn.Close()
// 2. Create a client.
client := pb.NewGreeterServiceClient(conn)
// 3. Inject the authentication token by using gRPC metadata.
md := metadata.Pairs("authorization", "Bearer " + token)
ctx := metadata.NewOutgoingContext(context.Background(), md)
// 4. Set a timeout.
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
// 5. Construct the request.
req := &pb.HelloRequest{Name: "World"}
// 6. Make the call.
resp, err := client.SayHello(ctx, req)
if err != nil {
log.Fatalf("Call failed: %v", err)
}
log.Printf("Received response: %s", resp.Message)
}import grpc
import helloworld_pb2
import helloworld_pb2_grpc
# Replace with your gRPC endpoint. You can find it in the call info panel in the PAI-EAS console.
# The public network endpoint format is: service-name.{uid}.{region}.pai-eas.aliyuncs.com:80 (the port is fixed at 80).
host = "your-service-endpoint:your port"
# Replace with your service token. You can find it on the service details page in the PAI-EAS console.
token = "your-service-token"
# 1. Create a channel to connect to the server.
channel = grpc.insecure_channel(host)
# 2. Initialize the stub (client handle).
stub = helloworld_pb2_grpc.GreeterServiceStub(channel)
# 3. Construct the request.
request = helloworld_pb2.HelloRequest(name="World")
# 4. Construct the authentication metadata to pass the token to the PAI-EAS service.
metadata = (("authorization", "Bearer " + token),)
print("Initiating gRPC call...")
try:
# 5. Make a synchronous request. Pass the metadata for authentication.
response = stub.SayHello(request, metadata=metadata, timeout=5.0)
print(response.message)
except grpc.RpcError as e:
print(f"Call failed: {e.code()} - {e.details()}")
finally:
channel.close()Best practices
-
Connection reuse: gRPC connections are thread-safe. Create one connection at application startup and reuse it for all requests.
ImportantIdle connections may be dropped by intermediate devices. In production, enable keepalive: use
KeepaliveParamsin Go or theoptionsparameter in Python. -
Timeout: Set a timeout for each call to prevent requests from hanging on network or service failures.
FAQ
Authentication fails (UNAUTHENTICATED)
Verify that your token is valid and not expired. Confirm the metadata field is named authorization exactly, and the value starts with Bearer followed by a space.
Connection times out (UNAVAILABLE)
Verify that the gRPC endpoint is correct and that the network is reachable.
For VPC endpoints, confirm your client and the PAI-EAS service are in the same VPC and region. Check that security group rules allow traffic on the relevant port.
Appendix
gRPC service demo
API definition
-
In your project's root directory, create a file named
helloworld.prototo define aGreeterServiceservice that includes aSayHellomethod.-
Request:
HelloRequest, which contains a string field namedname. -
Response:
HelloResponse, which contains a string field namedmessage.
-
-
Compile the
.protofile to generate thehelloworld_pb2_grpc.pyandhelloworld_pb2.pycode files. The following command uses Python as an example.python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. helloworld.proto
2. Server deployment
Deploy the service to PAI-EAS. The project structure is as follows:
my-grpc-demo
├── helloworld_pb2_grpc.py
├── helloworld_pb2.py
└── server.py