【DSW Gallery】Use EAS Python SDK to complete model deployment

overview
For online reasoning scenarios, the PAI platform provides online prediction service PAI-EAS (Elastic Algorithm Service), which supports model loading based on heterogeneous hardware (CPU and GPU) and real-time response to data requests. With PAI-EAS, you can quickly deploy the model as a RESTful API, and then invoke the service through HTTP requests.
At the same time, you can use the command tools and Python SDK provided by EAS to manage your model services. This article introduces you how to use Python to create a service and view the service list. And, use the prediction SDK of EAS to call the EAS service with code.
prerequisite
Before trying out the content introduced in this document, please make sure that your account meets the following conditions:
• Open PAI-EAS (it will also have a public resource group), for details, see: "Activation and Purchase"
• You have obtained the AccessKey ID and AccessKey Secret of your Alibaba Cloud account. For details, see Obtaining an AccessKey.
• In the current JupyterLab environment, eascmd is installed. For the installation steps, please refer to: Download and authenticate the client (if you use PAI-DSW, the CLI has been preset in the initial image)
• Because some python libraries are used in the sample code, please decide whether to execute the following python package installation commands as needed
Preparations before deployment
Step 1: Install the Python packages required by the example
# !pip install tensorflow tensorflow_datasets
# !pip install opencv-python
!pip install eas-prediction alibabacloud_eas20210701==1.1.2 --upgrade
!pip install tensorflow_datasets
Step 2: Train and produce a model
We refer to the basic example of tensorflow (https://www.tensorflow.org/datasets/keras_example ) to train a model to demonstrate the use process of EAS.
import tensorflow as tf
import tensorflow_datasets as tfds
(ds_train, ds_test), ds_info = tfds.load(
'mnist',
split=['train', 'test'],
data_dir='./cached_datasets',
shuffle_files=True,
as_supervised=True,
with_info=True,
)
def normalize_img(image, label):
"""Normalizes images: `uint8` -> `float32`."""
return tf.cast(image, tf.float32) / 255., label
ds_train = ds_train. map(
normalize_img)
ds_train = ds_train.cache()
ds_train = ds_train.shuffle(ds_info.splits['train'].num_examples)
ds_train = ds_train. batch(128)
ds_train = ds_train. prefetch(10)
ds_test = ds_test. map(normalize_img)
ds_test = ds_test. batch(128)
ds_test = ds_test.cache()
ds_test = ds_test.prefetch(10)
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(10)
])
model.compile(
optimizer=tf.keras.optimizers.Adam(0.001),
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=[tf.keras.metrics.SparseCategoricalAccuracy()],
)
model.fit(
ds_train,
epochs=6,
validation_data=ds_test,
)
model.save('./eas_demo_output3')
So far, we have trained a simple tensorflow model and output it to the directory: ./eas_demo_output3/
In the next step, we will deploy this model to EAS and verify that the service is normal.
Deploy the model to EAS (based on EAS Python SDK)
PAI-EAS provides the eascmd client management tool, which can create, modify, switch, delete and other operations on services, and also provides Python SDK for programming. Of course, their backends are all EAS Open APIs.
Step 1: Create an EAS client object
from alibabacloud_eas20210701.client import Client as eas20210701Client
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_eas20210701 import models as eas_20210701_models
from alibabacloud_tea_util import models as util_models
from alibabacloud_tea_util.client import Client as UtilClient
from alibabacloud_eas20210701.models import (ListServicesRequest, CreateServiceRequest)
access_key_id = "*Key ID of your Alibaba Cloud account*"
access_key_secret = "*Your Alibaba Cloud installed Key Secret*"
import os.path
config_path="/mnt/data/pai.config
)
# The domain name to visit
region = "cn-beijing"
config.endpoint = f'pai-eas.{region}.aliyuncs.com'
eas_client = eas20210701Client(config)
Step 2: Prepare model files
EAS supports reading model files from your OSS Bucket, provided that the authorization to PAI EAS is completed.
# Copy the model directory to OSS as needed
# !mkdir /mnt/data_oss/my_model_dir
# !cp -r ./eas_demo_output3/* /mnt/data_oss/my_model_dir/
Step 3: Create EAS service
import json
resource_config = { "instance": 1,
"memory": 7000,
"cpu": 4}
model_path = "oss://dlcyolo/dlc_demo/dsw_demo_model/"
service_config = {"name": "service_from_dsw",
"model_path": model_path,
"processor": "tensorflow_cpu_2.4",
"metadata": resource_config}
print(json. dumps(service_config))
service1 = eas_client.create_service(CreateServiceRequest(body=json.dumps(service_config))).body
print(service1)
Step 4: Check the EAS service status
service2 = eas_client.describe_service(cluster_id='cn-beijing', service_name=service1.service_name).body
print(service2. status)
Call EAS service for prediction
Step 1: Create a test sample
# import tensorflow.compat.v2 as tf
import tensorflow_datasets as tfds
import matplotlib.pyplot as plt
import numpy as np
# Construct a tf.data.Dataset
ds = tfds.load('mnist', split='train', data_dir='./cached_datasets', shuffle_files=False)
# Build your input pipeline
ds = ds.shuffle(1024).take(3)
target = []
for example in ds.take(1):
image, label = example['image'], example['label']
print(label)
target = np. reshape(image, 784)
plt.imshow(tf.squeeze(image))
plt. show()
tf.Tensor(6, shape=(), dtype=int64)
Step 2: Make Online Predictions
Using eas_prediction, call the deployed service. For details, please refer to Public Network Address Calling.
View all services:
res = eas_client. list_services(ListServicesRequest())
for s in res.body.services:
print(s. service_name, s. status)
mnist_first Running
mnist_first2 Stopped
service_from_dsw Running
Select a service for prediction:
service3 = eas_client.describe_service(cluster_id='cn-beijing', service_name='service_from_dsw').body
print(service3)
Construct a PredictClient object to call the service:
from eas_prediction import PredictClient, TFRequest
Impo