This topic describes how to use the Model Export component to export EasyRec models.
Prerequisites
You have trained a model by using the Model Training component. For more information, see Model Training.
Configure the component in Machine Learning Designer
Input stubs
Stub (from left to right) | Data type and upstream component | Parameter of PAI commands | Required |
input model dir | Object Storage Service (OSS) path / Model Training | model_dir | Yes |
Component parameters
Tab | Parameter | Required | Description | Parameters of PAI commands | Default value |
Parameters Setting | EasyRec Configuration | No | The EasyRec Configuration is the pipeline.config file generated in the Model Training-Model Path section. You need to either configure the input model dir stub, or specify a model path in EasyRec Configuration. If both are configured, the input model dir stub takes precedence. | config | None |
checkpoint_path | No | The OSS path of the checkpoint. If you set this parameter, the model path specified in input model dir stub is ignored. | checkpoint_path | None | |
export_dir | Yes | The directory of the exported model. | export_dir | None | |
extra_params | No | Other parameters that are not specified in the pipeline, such as assert_files. Example: -- assert_files oss://xxx. | extra_params | None | |
Specify the algorithm version | No | Select Advanced Options, you can select an algorithm package to run. 1. Generate a TAR package. For more information, see Release & Upgrade in the EasyRec documentation. 2. Upload the TAR package to OSS. For more information, see Upload objects. 3. Select the uploaded file in this parameter. | script | None | |
Tuning | Worker Count | None | The number of worker nodes. | The parameters on the Tuning tab are passed in as the cluster parameter. | 1 |
Worker CPU | No | The number of CPU cores for each worker node. A value of 1 indicates one CPU core. | 8 | ||
Worker Memory | No | The memory size of each worker node. A value of 100 indicates 100 MB. | 40000 | ||
Worker GPU | No | GPUs are not required in most EasyRec trainings. |
Component parameters
Output stub (from left to right) | Data type | Downstream component |
export model dir | OSS path | Update EAS Service(Beta) |
PAI commands and description
PAI -project algo_public -name easy_rec_ext
-Dcmd="export"
-Dconfig="oss://rec_sln_demo/EasyRec/deploy/rec_sln_demo_rec_sln_demo_sorting_v2/20230425/pipeline.config"
-Dmodel_dir="oss://rec_sln_demo/EasyRec/deploy/rec_sln_demo_rec_sln_demo_sorting_v2/20230425"
-Dexport_dir="oss://lcl-hz/rec_sln_demo/EasyRec/deploy/rec_sln_demo_rec_sln_demo_sorting_v2/export/20230425/final_witn_fg"
-Darn="acs:ram::xxxx:role/aliyunodpspaidefaultrole"
-Dbuckets="oss://rec_sln_demo/"
-Dcluster="{\"worker\": {\"count\": 1, \"cpu\": 800, \"gpu\": 0, \"memory\": 40000}}"
-Dextra_params="--asset_files oss://rec_sln_demo/EasyRec/deploy/rec_sln_demo_rec_sln_demo_sorting_v2/fg.json"
-Dlifecycle="28"
-DossHost="oss-cn-hangzhou-internal.aliyuncs.com";Parameter | Description | Required |
config | The pipeline.config file generated in the training. If you specify the model_dir parameter, the model path in the pipeline.config file is ignored. This parameter is used for periodic scheduling. | No |
model_dir | The OSS path in which the trained model is stored. If you specify the model_dir parameter, the model path in the pipeline.config file is ignored. This parameter is used for periodic scheduling. | No |
cmd | To export the model, set the parameter to export. | Yes |
export_dir | The OSS directory to which the model is exported. | Yes |
arn | The authorization information. You can log on to the PAI console. Choose Activation & Authorization > Dependent Services. Find the cloud service that you want to manage in the Designer section and click View Authorization. | Yes |
ossHost | The endpoint of OSS. For more information about how to obtain the endpoint, see Regions and endpoints. | Yes |
buckets | The bucket where the config file resides and the bucket where the model is stored. If multiple buckets are used, separate multiple buckets with commas (,). Example: | Yes |
extra_params | Other parameters that are not specified in the pipeline. | No |
Example
Create a pipeline as shown in the following figure.
Section | Description |
1, 2, 3 | Components used in model training. For more information, see the "Example" section in the Model Training topic. |
4 | Configure the export_dir parameter. |
After the pipeline is run, you can view the exported model in the OSS path that you specified by the export_dir parameter.