This topic provides a detailed API reference for the official Python SDK and includes complete code examples for common input and output formats.
To learn about the use cases and working principles of the SDK, see SDK for service calls.
Prerequisites
pip install -U eas-prediction --userQuick start
Choose a Request class that matches your model's input data format. The following code shows a minimal example of an end-to-end service call that uses a string request. For more examples, see Examples.
#!/usr/bin/env python
from eas_prediction import PredictClient
from eas_prediction import StringRequest
if __name__ == '__main__':
client = PredictClient('http://182848887922****.cn-shanghai.pai-eas.aliyuncs.com', 'my_service')
client.set_token('YOUR_SERVICE_TOKEN')
client.init()
request = StringRequest('[{}]')
resp = client.predict(request)
print(resp)API reference
The Python SDK provides the following classes, which are grouped by purpose:
Group | Description |
Main client |
|
Input and output |
|
Queue service |
|
Common parameters
Endpoint: The endpoint address of the server.
For a standard service, set this parameter to the default gateway endpoint. Example:
182848887922***.cn-shanghai.pai-eas.aliyuncs.com.For a VPC direct connection request, set this parameter to the service's VPC direct connection endpoint. The format is
<uid>.vpc.<region-id>.pai-eas.aliyuncs.com. Example:182848887922****.vpc.cn-shanghai.pai-eas.aliyuncs.com.
PredictClient class
Method | Description |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Initialize the PredictClient object. After you set the parameters, you need to call the |
|
|
StringRequest class
Method | Description |
|
|
StringResponse class
Method | Description |
|
|
TFRequest class
Method | Description |
|
|
|
|
|
|
|
|
TFResponse class
Method | Description |
|
|
|
|
TorchRequest class
Method | Description |
| Creates a |
|
|
|
|
|
|
TorchResponse class
Method | Description |
|
|
|
|
QueueClient class
Method | Description |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Watcher class
Method | Description |
|
|
| Description: Closes a Watcher object to terminate the back-end data connection. Note A client can have only one active Watcher at a time. You must close the current Watcher before starting a new one. |
Examples
Synchronous inference example (by input/output format)
Select the sample code based on the input and output types of the service.
String
Users who deploy services with a custom processor typically use strings for service calls, such as for PMML model services. The following code shows a sample program.
#!/usr/bin/env python
from eas_prediction import PredictClient
from eas_prediction import StringRequest
if __name__ == '__main__':
client = PredictClient('http://182848887922****.cn-shanghai.pai-eas.aliyuncs.com', 'scorecard_pmml_example')
client.set_token('YWFlMDYyZDNmNTc3M2I3MzMwYmY0MmYwM2Y2MTYxMTY4NzBkNzdj****')
client.init()
request = StringRequest('[{"fea1": 1, "fea2": 2}]')
for x in range(0, 1000000):
resp = client.predict(request)
print(resp)TensorFlow
If you use TensorFlow, use TFRequest and TFResponse as the input and output data formats. The following code shows a sample.
#!/usr/bin/env python
from eas_prediction import PredictClient
from eas_prediction import StringRequest
from eas_prediction import TFRequest
if __name__ == '__main__':
client = PredictClient('http://182848887922****.cn-shanghai.pai-eas.aliyuncs.com', 'mnist_saved_model_example')
client.set_token('YTg2ZjE0ZjM4ZmE3OTc0NzYxZDMyNmYzMTJjZTQ1YmU0N2FjMTAy****')
client.init()
#request = StringRequest('[{}]')
req = TFRequest('predict_images')
req.add_feed('images', [1, 784], TFRequest.DT_FLOAT, [1] * 784)
for x in range(0, 1000000):
resp = client.predict(req)
print(resp)PyTorch
If you use PyTorch, use TorchRequest and TorchResponse as the input and output data formats. The following code shows a sample.
#!/usr/bin/env python
from eas_prediction import PredictClient
from eas_prediction import TorchRequest
if __name__ == '__main__':
client = PredictClient('http://182848887922****.cn-shanghai.pai-eas.aliyuncs.com', 'pytorch_gpu_wl')
client.init()
req = TorchRequest()
req.add_feed(0, [1, 3, 224, 224], TorchRequest.DT_FLOAT, [1] * 150528)
# req.add_fetch(0)
import time
st = time.time()
timer = 0
for x in range(0, 10):
resp = client.predict(req)
timer += (time.time() - st)
st = time.time()
print(resp.get_tensor_shape(0))
# print(resp)
print("average response time: %s s" % (timer / 10) )BladeProcessor
If you use BladeProcessor, use BladeRequest and BladeResponse as the input and output data formats. The following code shows a sample.
#!/usr/bin/env python
from eas_prediction import PredictClient
from eas_prediction import BladeRequest
if __name__ == '__main__':
client = PredictClient('http://182848887922****.cn-shanghai.pai-eas.aliyuncs.com', 'nlp_model_example')
client.init()
req = BladeRequest()
req.add_feed('input_data', 1, [1, 360, 128], BladeRequest.DT_FLOAT, [0.8] * 85680)
req.add_feed('input_length', 1, [1], BladeRequest.DT_INT32, [187])
req.add_feed('start_token', 1, [1], BladeRequest.DT_INT32, [104])
req.add_fetch('output', BladeRequest.DT_FLOAT)
import time
st = time.time()
timer = 0
for x in range(0, 10):
resp = client.predict(req)
timer += (time.time() - st)
st = time.time()
# print(resp)
# print(resp.get_values('output'))
print(resp.get_tensor_shape('output'))
print("average response time: %s s" % (timer / 10) )BladeProcessor compatible with the default TensorFlow interface
BladeProcessor users can use TFRequest and TFResponse as the input and output data formats. These formats are compatible with the default TensorFlow interface of Elastic Algorithm Service (EAS). The following code shows a sample.
#!/usr/bin/env python
from eas_prediction import PredictClient
from eas_prediction.blade_tf_request import TFRequest # Need Importing blade TFRequest
if __name__ == '__main__':
client = PredictClient('http://182848887922****.cn-shanghai.pai-eas.aliyuncs.com', 'nlp_model_example')
client.init()
req = TFRequest(signature_name='predict_words')
req.add_feed('input_data', [1, 360, 128], TFRequest.DT_FLOAT, [0.8] * 85680)
req.add_feed('input_length', [1], TFRequest.DT_INT32, [187])
req.add_feed('start_token', [1], TFRequest.DT_INT32, [104])
req.add_fetch('output')
import time
st = time.time()
timer = 0
for x in range(0, 10):
resp = client.predict(req)
timer += (time.time() - st)
st = time.time()
# print(resp)
# print(resp.get_values('output'))
print(resp.get_tensor_shape('output'))
print("average response time: %s s" % (timer / 10) )Example of calling a service using a VPC direct connection
To use a direct connection, configure a Virtual Private Cloud (VPC) for the service. For more information, see Use Elastic Algorithm Service (EAS) resource groups and Access public or internal resources from EAS. This calling method requires only one additional line of code compared to the normal method: client.set_endpoint_type(ENDPOINT_TYPE_DIRECT). This method is ideal for services with high traffic and concurrency. The following code provides an example.
#!/usr/bin/env python
from eas_prediction import PredictClient
from eas_prediction import StringRequest
from eas_prediction import TFRequest
from eas_prediction import ENDPOINT_TYPE_DIRECT
if __name__ == '__main__':
# The format of a VPC direct connection endpoint is <uid>.vpc.<region-id>.pai-eas.aliyuncs.com. You can find the endpoint on the Invocation Information tab of the service details page in the EAS console.
client = PredictClient('http://182848887922****.vpc.cn-hangzhou.pai-eas.aliyuncs.com', 'mnist_saved_model_example')
client.set_token('M2FhNjJlZDBmMzBmMzE4NjFiNzZhMmUxY2IxZjkyMDczNzAzYjFi****')
client.set_endpoint_type(ENDPOINT_TYPE_DIRECT)
client.init()
request = TFRequest('predict_images')
request.add_feed('images', [1, 784], TFRequest.DT_FLOAT, [1] * 784)
for x in range(0, 1000000):
resp = client.predict(request)
print(resp)Example: Sending and subscribing to data in a queue service
You can use QueueClient to send data to a queue service, query data, check the service status, and subscribe to data pushes. The following demo shows an example with two threads. One thread sends data to the queue service. The other thread uses a Watcher to subscribe to the pushed data.
#!/usr/bin/env python
from eas_prediction import QueueClient
import threading
if __name__ == '__main__':
endpoint = '182848887922****.cn-shanghai.pai-eas.aliyuncs.com'
queue_name = 'test_group.qservice/sink'
token = 'YmE3NDkyMzdiMzNmMGM3ZmE4ZmNjZDk0M2NiMDA3OTZmNzc1MTUx****'
queue = QueueClient(endpoint, queue_name)
queue.set_token(token)
queue.init()
queue.set_timeout(30000)
# truncate all messages in the queue
attributes = queue.attributes()
if 'stream.lastEntry' in attributes:
queue.truncate(int(attributes['stream.lastEntry']) + 1)
count = 100
# create a thread to send messages to the queue
def send_thread():
for i in range(count):
index, request_id = queue.put('[{}]')
print('send: ', i, index, request_id)
# create a thread to watch messages from the queue
def watch_thread():
watcher = queue.watch(0, 5, auto_commit=True)
i = 0
for x in watcher.run():
print('recv: ', i, x.index, x.tags['requestId'])
i += 1
if i == count:
break
watcher.close()
thread1 = threading.Thread(target=watch_thread)
thread2 = threading.Thread(target=send_thread)
thread1.start()
thread2.start()
thread1.join()
thread2.join()Troubleshooting
For information about the symptoms, causes, and troubleshooting of exceptions from Python SDK calls, see the 'Troubleshooting call exceptions' section in Service Call SDK. These exceptions include common issues with authentication, routing, connection, and the server.
For a complete list of service status codes, error message descriptions, and suggested solutions, see Appendix: Service status codes and common errors.