[DSW Gallery] Introducing how to use the EAS command line tool in DSW

overview
For online reasoning scenarios, the PAI platform provides online prediction service PAI-EAS (Elastic Algorithm Service), which supports model loading based on heterogeneous hardware (CPU and GPU) and real-time response to data requests. With PAI-EAS, you can quickly deploy the model as a RESTful API, and then invoke the service through HTTP requests.
At the same time, you can use the command tool provided by EAS: eascmd to manage PAI-EAS service. This article introduces you how to use the eascmd client to upload files, create services, and view service lists. And, use the prediction SDK of EAS to call the EAS service with code.
prerequisite
Before trying out the content introduced in this document, please make sure that your account meets the following conditions:
• Open PAI-EAS (it will also have a public resource group), for details, see: "Activation and Purchase"
• You have obtained the AccessKey ID and AccessKey Secret of your Alibaba Cloud account. For details, see Obtaining an AccessKey.
• In the current JupyterLab environment, eascmd is installed. For the installation steps, please refer to: Download and authenticate the client (if you use PAI-DSW, the CLI has been preset in the initial image)
• Because some python libraries are used in the sample code, please decide whether to execute the following python package installation commands as needed
Preparations before deployment
Step 1: Install the Python packages required by the example
# --------If you need to switch to the domestic pypi source, please uncomment the following --------
# !pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
# This example requires tensorflow2.6 or higher version, if your DSW version is lower than this requirement, please upgrade the relevant python library
# --------You can uncomment the following, run and install the corresponding python library --------
# !pip install tensorflow tensorflow_datasets
# !pip install opencv-python
!pip install eas-prediction
Step 2: Train and produce a model
We refer to the basic example of tensorflow (https://www.tensorflow.org/datasets/keras_example ) to train a model to demonstrate the use process of EAS.
import tensorflow as tf
import tensorflow_datasets as tfds
(ds_train, ds_test), ds_info = tfds.load(
'mnist',
split=['train', 'test'],
data_dir='./cached_datasets',
shuffle_files=True,
as_supervised=True,
with_info=True,
)
def normalize_img(image, label):
"""Normalizes images: `uint8` -> `float32`."""
return tf.cast(image, tf.float32) / 255., label
ds_train = ds_train. map(
normalize_img, num_parallel_calls=tf.data.AUTOTUNE)
ds_train = ds_train.cache()
ds_train = ds_train.shuffle(ds_info.splits['train'].num_examples)
ds_train = ds_train. batch(128)
ds_train = ds_train.prefetch(tf.data.AUTOTUNE)
ds_test = ds_test.map(normalize_img, num_parallel_calls=tf.data.AUTOTUNE)
ds_test = ds_test. batch(128)
ds_test = ds_test.cache()
ds_test = ds_test.prefetch(tf.data.AUTOTUNE)
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(10)
])
model.compile(
optimizer=tf.keras.optimizers.Adam(0.001),
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=[tf.keras.metrics.SparseCategoricalAccuracy()],
)
model.fit(
ds_train,
epochs=6,
validation_data=ds_test,
)
model.save('./eas_demo_output')
So far, we have trained a simple tensorflow model and output it to the directory: ./eas_demo_output/
In the next step, we will deploy this model to EAS and verify that the service is normal.
Deploy the model to EAS (based on eascmd)
PAI-EAS provides the eascmd client management tool, which can create, modify, switch, delete and other operations on services.
Step 1: Authenticate the client
Before using the eascmd client, you need to download the client and complete user authentication. For details on client download and authentication, please refer to: "Download and Authenticate Client"
If the user uses the PAI-DSW product, eascmd has been preset in the instance image, you can complete the following command and run it to complete the authentication:
# -------- Need to improve the command information, and cancel the code comment before running --------
# !eascmd64 config -i -k [-e ]
• yourAccessKeyID - the AccessKey ID of your Alibaba Cloud account.
• yourAccessKeySecret - the AccessKey Secret of your Alibaba Cloud account.
• yourEndpoint - The default PAI-EAS service region is East China 2 (Shanghai). If you need to deploy the model to other regions, you can use the -e parameter to specify the Endpoint corresponding to the region
Step 2: Package and upload the model
• Command instructions: https://help.aliyun.com/document_detail/111031.html
# !eascmd64 upload eas_demo_output.tar.gz --inner
Step 3: Create EAS service
# !eascmd64 create eas_demo_service_desc.json
Step 4: Check the EAS service status
# !eascmd64 ls
Call EAS service for prediction
Step 1: Create a test sample
# import tensorflow.compat.v2 as tf
import tensorflow_datasets as tfds
import matplotlib.pyplot as plt
import numpy as np
# Construct a tf.data.Dataset
ds = tfds. load('mnist', split='train', shuffle_files=False)
# Build your input pipeline
ds = ds.shuffle(1024).take(1)
target = []
for example in ds.take(3):
image, label = example['image'], example['label']
print(label)
target = np. reshape(image, 784)
plt.imshow(tf.squeeze(image))
plt. show()
tf.Tensor(3, shape=(), dtype=int64)
2022-06-23 19:56:30.621935: W tensorflow/core/kernels/data/cache_dataset_ops.cc:856] The calling iterator did not fully read the dataset being cached. In order to avoid unexpected truncation of the dataset, the partially cached contents of the dataset will be discarded. This can happen if you have an input pipeline similar to `dataset.cache().take(k).repeat()`. You should use `dataset.take(k).cache( ).repeat()` instead.
Step 2: Make Online Predictions
Using eas_prediction, call the deployed service. For more information, please refer to Public Network Address Call
#!/usr/bin/env python
from eas_prediction import PredictClient, TFRequest
# http://1848217********.cn-shanghai.pai-eas.aliyuncs.com/api/predict/kears_model
client = PredictClient('http://1157703270994901.cn-shanghai.pai-eas.aliyuncs.com', 'dsw_deploy_model_sample_1')
# Note that the information filled in the above client = PredictClient() is obtained by splitting the access address obtained in the call information window (below)
client.set_token('MjNiNzJmNWRlYjdkNjgxYjY1YTNkMGY4NDhhNjNmMjI5ZjhjM2E0MQ==')
# Token information is obtained in "EAS Console - Service List - Service - Call Information - Public Network Address Call - Token"
client.init()
req = TFRequest('serving_default') # signature_name parameter: serving_default
req.add_feed('flatten_input', [1, 28, 28], TFRequest.DT_FLOAT, target)
resp = client. predict(req)
print((resp. response. outputs). keys)
array_shape {
dim: 1
dim: 10
}
float_val: -2063.09912109375
float_val: 125.97838592529297
float_val: -99.17234802246094
float_val: 956.5430908203125
float_val: -1272.6309814453125
float_val: -2213.461181640625
float_val: -2570.403076171875
float_val: 1942.651611328125
float_val: -1032.1619873046875
float_val: -217.21417236328125
}>