All Products
Search
Document Center

EASCMD Client Manual

Last Updated: Jul 21, 2019

You can use the EASCMD client to create services, modify services, delete services, and view service status. You can download the EASCMD client from the following links:

Authenticate users

The online model service uses an Alibaba Cloud AccessKey pair to authenticate a user. When you submit a model deployment task, you must provide your Alibaba Cloud AccessKey ID and AccessKey Secret at the same time.

Run the following command to configure the AccessKey information for EASCMD. Modify the command according to the actual name of the client installation package. The following command is for the EASCMD client running in Linux 64.

  1. ./eascmd64 config -i <AccessKeyId> -k <AccessKeySecret>

The default region endpoint is cn-shanghai. If you want to deploy the model in other regions,

or you want to use the model in DSW, set the -e parameter to manually specify the endpoint of the region.

For example, you can run the following command to set the region to China (Beijing):

  1. ./eascmd64 config -i <AccessKeyId> -k <AccessKeySecret> -e pai-eas.cn-beijing.aliyuncs.com

The public cloud offerings of Alibaba Cloud support the following region endpoints:

Region Endpoint
China (Shanghai) pai-eas.cn-shanghai.aliyuncs.com
China (Beijing) pai-eas.cn-beijing.aliyuncs.com
China (Shanghai), only for services deployed by using Data Science Workshop (DSW) pai-eas-share.cn-shanghai.aliyuncs.com
DSW in China (Beijing), only for services deployed by using DSW pai-eas-share.cn-beijing.aliyuncs.com
China (Hangzhou), coming soon pai-eas.cn-hangzhou.aliyuncs.com

Upload files

When you create a service, you must specify the HTTP address or OSS address of the model or processor. EAS provides an OSS bucket for each user to store the data. You can run the upload command on the EASCMD client to upload files, and then obtain the OSS address of the uploaded files.

  1. eascmd upload [filename]

If you run the command in DSW, set the --inner parameter at the same time.

  1. eascmd upload [filename] --inner

The following example shows how to upload the savedmodel package to OSS:

  1. $ eascmd upload savedmodel_example/savedmodel_example.tar.gz
  2. [OK] oss endpoint: [http://oss-cn-shanghai.aliyuncs.com]
  3. [OK] oss target path: [oss://eas-model-shanghai/1828488879222746/savedmodel_example/savedmodel_example.tar.gz]
  4. Succeed: Total num: 1, size: 33,013. OK num: 1(upload 1 files).

You can use the OSS path of the package in the preceding example to deploy the service: oss://eas-model-shanghai/1828488879222746/savedmodel_example/savedmodel_example.tar.gz

Create a servcie

You can run the create command to create a service. When you create a service, you must specify the HTTP address of the resource (model or processor). You can upload the resource to OSS and then specify the OSS path of the resource. Make sure that the resource address is accessible.

  1. eascmd create [service_desc_json]

When you deploy a service, you can use a JSON file to describe the service information, such as the location of the model and the used resources. The service_desc_json file contains the service information. The metadata list contains the resource information and advanced configuration.

  1. {
  2. "name": "mnist_saved_model_example",
  3. "generate_token": "true",
  4. "model_path": "http://eas-data.oss-cn-shanghai.aliyuncs.com/models%2Fmnist_saved_model.tar.gz",
  5. "processor": "tensorflow_cpu",
  6. "metadata": {
  7. "instance": 1,
  8. "cpu": 1,
  9. }
  10. }

The descriptions for the fields in the preceding example are as follows:

Field Description
name The name of the service, which must be unique in a region.
generate_token Indicates whether a token is generated. If it is set to true, then the service requires authentication. HTTP requests sent to this service must contain this token in their header. If it is set to false, then this indicates that the service is a public service, which does not require authentication.
token Optional. This field specifies the token for authentication. If you do not specify a token and generate_token is set to true, the token is automatically generated.
model_path The path of the model package. For more information, see the note under this table.
model_entry Optional. This field specifies the model input files. If no model input file is specified, the file specified by the model_path field is applied. You can specify arbitrary files in this field. The path of the primary file will be passed to the Load() function in the processor.
model_config Optional. This field specifies the configuration of the model. The configuration can be in any text format. This setting is passed to the second parameter of the LoadWithConfig() function in the processor.
processor Optional. Set this field to use the system-defined processor to provide the service. When this field is set, the processor_path, processor_entry, processor_mainclass, and processor_type fields are ignored.
metadata The metadata of the service.

Note: The model_path and processor_path fields specify the paths of the model input and processor input, respectively. You can set the paths to HTTP addresses or OSS paths. When you use the test command to debug the model on your local host, you can set them to local paths. When an HTTP address is used, the required files must be compressed into the tar.gz, tar.bz2, or ZIP format.

Metadata descriptions:

Common parameter
workers Optional. This parameter specifies the number of threads that are used by each instance to handle requests in parallel. Default value: 5.
instance The number of instances that are required to run the service.
CPU The number of CPUs that are required by each instance.
GPU The number of GPUs that are required by each instance.
resource The name of the resource group. This parameter is not required by CPU compute. GPU compute supports the “P4_4CORE” and “P4_8CORE” resource groups.
Advanced parameter (edit with caution)
rpc.batching Optional. This parameter indicates whether batch processing is enabled on the server end for GPU compute acceleration. Default value: false.
rpc.keepalive Optional. This parameter specifies the maximum allowed time for processing a request, in milliseconds. When this threshold is exceeded, the server returns a 408 timeout error and closes the connection. Default value: 5,000 milliseconds.
rpc.io_threads Optional. This parameter specifies the number of threads that are used by each instance to handle the input and output data. Default value: 4.
rpc.max_batch_size Optional. This parameter specifies the maximum size of each batch. The parameter takes effect only when batch processing is enabled. Default value: 16.
rpc.max_batch_timeout Optional. This parameter specifies the maximum timeout for each batch. The parameter takes effect only when batch processing is enabled. Default value: 50 milliseconds.
rpc.max_queue_size Optional. This parameter specifies the length of the request queue. When the queue is full, the server returns a 450 error and closes the connection. The request queue function prevents the server from overload and notifies the client to send requests to other instances when the queue is full. If the server’s response time is too high, set this parameter to a smaller value to prevent requests from timing out. Default value: 64.
rpc.worker_threads Optional. This parameter specifies the number of threads that are used by each instance to handle requests in parallel. This parameter has the same function as the workers parameter. Default value: 5.
Examples {

“metadata”: {
“cpu”: 4,
“rpc.max_queue_size”: 32,

}
}

Examples:

  1. $ eascmd create pmml.json
  2. [RequestId]: 1651567F-8F8D-4A2B-933D-F8D3E2DDEB2D
  3. --------------+------------+-------------------+-----------------+----------------
  4. | Intranet Endpoint | http://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/savedmodel_exanple |
  5. | Token | YjQxZDYzZTBiZTZjMzQ5ZmE0MzczZjIxMGZiNzZmMDBkY2VjMDg4MQ== |
  6. --------------+------------+-------------------+-----------------+----------------
  7. [OK] Creating api gateway
  8. [OK] Building image [registry-vpc.cn-shanghai.aliyuncs.com/eas/savedmodel_exanple_cn-shanghai:v0.0.1-20190224001315]
  9. [OK] Pushing image [registry-vpc.cn-shanghai.aliyuncs.com/eas/savedmodel_exanple_cn-shanghai:v0.0.1-20190224001315]
  10. [OK] Waiting [Total: 1, Pending: 1, Running: 0]
  11. [OK] Waiting [Total: 1, Pending: 1, Running: 0]
  12. [OK] Service is running

Modify configuration

You can run the modify command and set the -D parameter to modify the metadata parameters, such as the process (instance), CPU, and memory parameters.

  1. eascmd modify [service_name] -Dmetadata.[attr_name]=[attr_value]

For example, you can run the following command to set the number of instances to 10:

  1. eascmd modify service_test -Dmetadata.instance=10

You can also set these parameters in one command. For example, you can set the number of instances to 10 and the quota for each instance to 5 (5 cores and 20 GB of memory).

  1. eascmd modify service_test -Dmetadata.instance=10 -Dmetadata.cpu=5

Modify a service

You can run the modify command to modify the deployed service.

  1. eascmd modify [service_name] -s [service_desc_json]

Note: If you only want to modify the resource usage for the service, then you only need to specify the metadata configuration in the service description file.

Delete a service

You can run the delete command to delete a service. You can only delete services in the current region.

  1. eascmd delete [service_name]

Examples:

  1. $ eascmd delete savedmodel_exanple
  2. Are you sure to delete the service [savedmodel_exanple] in [cn-shanghai]? [Y/n]
  3. [RequestId]: 1651567F-8F8D-4A2B-933D-F8D3E2DDEB2D
  4. [OK] Service [savedmodel_exanple] in region [cn-shanghai] is terminating
  5. [OK] Service is terminating
  6. [OK] Service is terminating
  7. [OK] Service was deleted successfully

Switch service version

You can run the desc command to view the current service version and the latest service version, and run the version command to switch the service version.

  1. eascmd version [service_name] [version_id]

View service list

You can run the list(ls) command to obtain a list of services that are deployed by the current account.

  1. $ eascmd ls
  2. [RequestId]: 83945D4E-ED3E-4D35-A989-831E36BBA39F
  3. +---------------------------+-------------+----------+---------------------+---------------------+---------+--------+----------------------------------------+
  4. | SERVICENAME | REGION | INSTANCE | CREATETIME | UPDATETIME | STATUS | WEIGHT | SERVICEPATH |
  5. +---------------------------+-------------+----------+---------------------+---------------------+---------+--------+----------------------------------------+
  6. | mnist_saved_model_example | cn-shanghai | 1 | 2019-02-21 16:35:41 | 2019-02-21 16:35:41 | Running | 0 | /api/predict/mnist_saved_model_example |
  7. +---------------------------+-------------+----------+---------------------+---------------------+---------+--------+----------------------------------------+

View service information

You can run the desc command to view detailed information of a deployed service.

  1. eascmd desc [service_name]

Examples:

  1. $ eascmd desc mnist_saved_model_example
  2. +---------------------+--------------------------------------------------------------------------------------------------------------+
  3. | Status | Running |
  4. | ServiceName | mnist_saved_model_example |
  5. | Region | cn-shanghai |
  6. | CreateTime | 2019-02-21 16:35:41 |
  7. | UpdateTime | 2019-02-21 16:35:41 |
  8. | AccessToken | |
  9. | PrivateToken | ZWNjMTNkNDExMmExNjZkYTM4YWQ5YTY0YmFjNjk3YWYzZTRjM2Y2ZA== |
  10. | TotalInstance | 1 |
  11. | RunningInstance | 1 |
  12. | PendingInstance | 0 |
  13. | CPU | 1 |
  14. | GPU | 0 |
  15. | Memory | 1000M |
  16. | Image | registry-vpc.cn-shanghai.aliyuncs.com/eas/mnist_saved_model_example_cn-shanghai:v0.0.1-20190221163541 |
  17. | Weight | 0 |
  18. | LatestVersion | 1 |
  19. | CurrentVersion | 1 |
  20. | Message | Service start successfully |
  21. | APIGatewayUrl | 1c3b37ea83c047efa0dc6df0cacb70d3-cn-shanghai.alicloudapi.com/EAPI_1828488879222746_mnist_saved_model_example |
  22. | APIGatewayAppKey | 25641710 |
  23. | APIGatewayAppSecret | 12562a7b8858bbba2c2e9c4517ff5da5 |
  24. | IntranetEndpoint | http://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/mnist_saved_model_example |
  25. | ServiceConfig | { |
  26. | | "generate_token": "false", |
  27. | | "metadata": { |
  28. | | "cpu": 1, |
  29. | | "instance": 1, | |
  30. | | "region": "cn-shanghai" |
  31. | | }, |
  32. | | "model_path": |
  33. | | "http://eas-data.oss-cn-shanghai.aliyuncs.com/models%2Fmnist_saved_model.tar.gz", |
  34. | | "name": |
  35. | | "mnist_saved_model_example", |
  36. | | "processor": |
  37. | | "tensorflow_cpu" |
  38. | | } |
  39. +---------------------+--------------------------------------------------------------------------------------------------------------+

View service processes

You can run the showworkers(w) command to view the status of the service processes.

  1. eascmd w [service_name]

Examples:

  1. $ eascmd w mnist_saved_model_example
  2. [RequestId]: 4E905404-E617-4BD8-85D6-EC5C6A0D0211
  3. +--------------+----------------+---------------------+----------+---------+-------+--------+
  4. | INNERIP | HOSTIP | STARTAT | RESTARTS | STATUS | READY | REASON |
  5. +--------------+----------------+---------------------+----------+---------+-------+--------+
  6. | 172.24.5.183 | 192.168.65.121 | 2019-02-21 16:35:58 | 0 | Running | [1/1] | |
  7. +--------------+----------------+---------------------+----------+---------+-------+--------+

Call a service

You can use the HTTP URL generated when you create a service to call the service. The input and output of the service are defined by the processor. For more information, see Create a PMML processor request.