Official Elastic Algorithm Service (EAS) SDKs are provided to call services deployed based on their models. EAS SDKs can help reduce the amount of time required to define the call logic and improve call stability. This topic describes EAS SDK for Java. This topic also provides demos to describe how to use EAS SDK for Java to call services. In the first demo, the input and output are strings. In the second demo, the input and output are tensors.
Add dependencies
If you write code in Java, Maven is required to manage projects. You need to add the eas-sdk dependency for the client to the pom.xml file. The latest version of the eas-sdk dependency is 2.0.11. Add the following code to the pom.xml file:
<dependency>
<groupId>com.aliyun.openservices.eas</groupId>
<artifactId>eas-sdk</artifactId>
<version>2.0.11</version>
</dependency>V2.0.5 and later support the QueueService client and support asynchronous queue services with multiple priorities. To use this feature and prevent version conflicts in the dependency package, you must add the following dependency packages and modify the dependency packages based on the following example.
<dependency>
<groupId>org.java-websocket</groupId>
<artifactId>Java-WebSocket</artifactId>
<version>1.5.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.1</version>
</dependency>Methods
Class | Method | Details |
PredictClient |
|
|
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
HttpConfig |
|
|
|
| |
|
| |
|
| |
|
| |
|
| |
| Returns the error code of the last call. | |
| Returns the error message of the last call. | |
TFRequest |
|
|
|
| |
|
| |
TFResponse |
|
|
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
QueueClient |
|
|
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
| Description: disables the queue service. | |
DataFrame |
|
|
|
| |
|
|
Demos
Input and output as strings
In most cases when you use custom processors to deploy models as services, strings are used to call the services, such as a service deployed based on a Predictive Model Markup Language (PMML) model. The following demo is for your reference:
import com.aliyun.openservices.eas.predict.http.PredictClient;
import com.aliyun.openservices.eas.predict.http.HttpConfig;
public class TestString {
public static void main(String[] args) throws Exception {
// Start and initialize the client. A single client object must be shared by multiple requests. You cannot create a client object each time you send a request.
PredictClient client = new PredictClient(new HttpConfig());
client.setToken("YWFlMDYyZDNmNTc3M2I3MzMwYmY0MmYwM2Y2MTYxMTY4NzBkNzdj****");
// To use a direct connection channel, call the setDirectEndpoint method.
// Example: client.setDirectEndpoint("pai-eas-vpc.cn-shanghai.aliyuncs.com");
// You must enable the direct connection channel feature on the EAS page of the PAI console. This feature can provide a source vSwitch to access EAS. After you enable the direct connection channel feature, you can call services deployed in EAS by using the server load balancing algorithm without the need to use a gateway. This way, improved stability and performance are achieved.
// Note: If you want to call a service by using a gateway, the endpoint that you use must start with your user ID. To obtain the endpoint, find the service that you want to call on the EAS page, and click Invocation Method in the Service Type column. In the Invocation Method dialog box, you can view the endpoint. If you want to call a service by using a direct connection channel, the endpoint that you use must be in the pai-eas-vpc.{region_id}.aliyuncs.com format.
client.setEndpoint("182848887922****.vpc.cn-shanghai.pai-eas.aliyuncs.com");
client.setModelName("scorecard_pmml_example");
// Define the input string.
String request = "[{\"money_credit\": 3000000}, {\"money_credit\": 10000}]";
System.out.println(request);
// Return a string by using EAS.
try {
String response = client.predict(request);
System.out.println(response);
} catch (Exception e) {
e.printStackTrace();
}
// Shut down the client.
client.shutdown();
return;
}
}The preceding demo shows that you must perform the following steps to call a service by using EAS SDK for Java:
Call the
PredictClientmethod to create a client object. If multiple services are involved, create multiple client objects.Set the token, endpoint, and modelname parameters for the client object.
Define the request content of the STRING type as the input, and call the
client.predictmethod to send the HTTP request. Then, the system returns the response.
Input and output as tensors
If you use TensorFlow to deploy models as services, you must use the TFRequest and TFResponse classes to call the services. The following demo is for your reference:
import java.util.List;
import com.aliyun.openservices.eas.predict.http.PredictClient;
import com.aliyun.openservices.eas.predict.http.HttpConfig;
import com.aliyun.openservices.eas.predict.request.TFDataType;
import com.aliyun.openservices.eas.predict.request.TFRequest;
import com.aliyun.openservices.eas.predict.response.TFResponse;
public class TestTF {
public static TFRequest buildPredictRequest() {
TFRequest request = new TFRequest();
request.setSignatureName("predict_images");
float[] content = new float[784];
for (int i = 0; i < content.length; i++) {
content[i] = (float) 0.0;
}
request.addFeed("images", TFDataType.DT_FLOAT, new long[]{1, 784}, content);
request.addFetch("scores");
return request;
}
public static void main(String[] args) throws Exception {
PredictClient client = new PredictClient(new HttpConfig());
// To use a direct connection channel, call the setDirectEndpoint method.
// Example: client.setDirectEndpoint("pai-eas-vpc.cn-shanghai.aliyuncs.com");
// You must enable the direct connection channel feature on the EAS page of the PAI console. This feature can provide a source vSwitch to access EAS. After you enable the direct connection channel feature, you can call services deployed in EAS by using the server load balancing algorithm without the need to use a gateway. This way, improved stability and performance are achieved.
// Note: If you want to call a service by using a gateway, the endpoint that you use must start with your user ID. To obtain the endpoint, find the service that you want to call on the EAS page, and click Invocation Method in the Service Type column. In the Invocation Method dialog box, you can view the endpoint. If you want to call a service by using a direct connection channel, the endpoint that you use must be in the pai-eas-vpc.{region_id}.aliyuncs.com format.
client.setEndpoint("182848887922****.vpc.cn-shanghai.pai-eas.aliyuncs.com");
client.setModelName("mnist_saved_model_example");
client.setToken("YTg2ZjE0ZjM4ZmE3OTc0NzYxZDMyNmYzMTJjZTQ1YmU0N2FjMTAy****");
long startTime = System.currentTimeMillis();
int count = 1000;
for (int i = 0; i < count; i++) {
try {
TFResponse response = client.predict(buildPredictRequest());
List<Float> result = response.getFloatVals("scores");
System.out.print("Predict Result: [");
for (int j = 0; j < result.size(); j++) {
System.out.print(result.get(j).floatValue());
if (j != result.size() - 1) {
System.out.print(", ");
}
}
System.out.print("]\n");
} catch (Exception e) {
e.printStackTrace();
}
}
long endTime = System.currentTimeMillis();
System.out.println("Spend Time: " + (endTime - startTime) + "ms");
client.shutdown();
}
}The preceding demo shows that you must perform the following steps to call a service whose model is a TensorFlow model by using EAS SDK for Java:
Call the
PredictClientmethod to create a client object. If multiple services are involved, create multiple client objects.Set the token, endpoint, and modelname parameters for the client object.
Encapsulate the input by using the TFRequest class and the output by using the TFResponse class.
QueueService client
You can use the Queue Service feature by using the QueueClient operations. The following demo is for your reference:
import com.alibaba.fastjson.JSONObject;
import com.aliyun.openservices.eas.predict.http.HttpConfig;
import com.aliyun.openservices.eas.predict.http.QueueClient;
import com.aliyun.openservices.eas.predict.queue_client.QueueUser;
import com.aliyun.openservices.eas.predict.queue_client.WebSocketWatcher;
public class DemoWatch {
public static void main(String[] args) throws Exception {
/**Create a queue service client.*/
String queueEndpoint = "18*******.cn-hangzhou.pai-eas.aliyuncs.com";
String inputQueueName = "test_qservice";
String sinkQueueName = "test_qservice/sink";
String queueToken = "test-token";
/**Input queue. Add data to the input queue. The inference service automatically reads the request data from the input queue.*/
QueueClient inputqueue =
new QueueClient(queueEndpoint, inputQueueName, queueToken, new HttpConfig(), new QueueUser());
/**Output queue. The inference service processes the input data and writes the results to the output queue.*/
QueueClient sinkqueue =
new QueueClient(queueEndpoint, sinkQueueName, queueToken, new HttpConfig(), new QueueUser());
/**Clear queue data!!! Proceed with caution*/
inputqueue.clear();
sinkqueue.clear();
/**Add data to the input queue.*/
int count = 10;
for (int i = 0; i < count; ++i) {
String data = Integer.toString(i);
input_queue.put(data.getBytes(), null);
/**The queue service supports multi-priority queues. You can use the put function to set the data priority. The default priority is 0.*/
// inputQueue.put(data.getBytes(), 0L, null);
}
/**Subscribe to the data of the output queue by using the watch function. The window size is 5.*/
WebSocketWatcher watcher = sink_queue.watch(0L, 5L, false, true, null);
/**You can use the WatchConfig parameter to specify the number of retries, retry interval (in seconds), and whether to limit the maximum number of retries. If you do not specify the WatchConfig parameter, the default number of retries is 3, and the retry interval is 5. */
// WebSocketWatcher watcher = sink_queue.watch(0L, 5L, false, true, null, new WatchConfig(3, 1));
// WebSocketWatcher watcher = sink_queue.watch(0L, 5L, false, true, null, new WatchConfig(true, 10));
/**Obtain output data. * /
for (int i = 0; i < count; ++i) {
try {
/**The getDataFrame function is used to obtain the DataFrame data class. If no data is available, the function is blocked.*/
byte[] data = watcher.getDataFrame().getData();
System.out.println("[watch] data = " + new String(data));
} catch (RuntimeException ex) {
System.out.println("[watch] error = " + ex.getMessage());
break;
}
}
/**Disable a watcher object. Each client instance can have only one watcher object. If you do not disable a watcher object, an error is reported when you rerun the instance. * /
watcher.close();
Thread.sleep(2000);
JSONObject attrs = sink_queue.attributes();
System.out.println(attrs.toString());
/**Shut down the client.*/
inputqueue.shutdown();
sinkqueue.shutdown();
}
}The preceding demo shows that you must perform the following steps to call a service by using EAS SDK for Java:
Creates a queue service client object through the
QueueClientoperations. If you create an inference service, you must create both input queue and output queue objects.Use the
put()function to send data to the input queue, and use thewatch()function to subscribe to data from the output queue.NoteData transmission and subscription can occur in different threads. However, in this example, they are completed in the same thread, data transmission by the put function and subscription by the watch function.
Compress request data
If the amount of request data is large, EAS allows you to compress the data before sending the data to the server. Zlib and Gzip compression formats are supported. To compress data, you need to specify the rpc.decompressor parameter in the service configuration.
Sample service configuration:
"metadata": {
"rpc": {
"decompressor": "zlib"
}
}The following code provides an example on how to call the SDK:
package com.aliyun.openservices.eas.predict;
import com.aliyun.openservices.eas.predict.http.Compressor;
import com.aliyun.openservices.eas.predict.http.PredictClient;
import com.aliyun.openservices.eas.predict.http.HttpConfig;
public class TestString {
public static void main(String[] args) throws Exception{
// Start and initialize the client.
PredictClient client = new PredictClient(new HttpConfig());
client.setEndpoint("18*******.cn-hangzhou.pai-eas.aliyuncs.com");
client.setModelName("echo_compress");
client.setToken("YzZjZjQwN2E4NGRkMDMxNDk5NzhhZDcwZDBjOTZjOGYwZDYxZGM2****");
// or use Compressor.Gzip.
client.setCompressor(Compressor.Zlib);
// Specify the input string.
String request = "[{\"money_credit\": 3000000}, {\"money_credit\": 10000}]";
System.out.println(request);
// Return a string by using EAS.
String response = client.predict(request);
System.out.println(response);
// Shut down the client.
client.shutdown();
return;
}
}