Platform for AI (PAI) provides a PAI-TensorFlow deep learning framework that supports single-node and distributed training. This reference covers the command parameters and I/O parameters for running PAI-TensorFlow tasks.
GPU-accelerated servers will be phased out. Submit TensorFlow tasks on CPU servers only. To train models on GPU instances, use Deep Learning Containers (DLC) instead. See Submit training jobs.
Commands and parameters
Run PAI-TensorFlow tasks from any of the following interfaces:
The MaxCompute client
An SQL node in the DataWorks console
The Machine Learning Designer page in the PAI console
TensorFlow components in Machine Learning Designer
All tasks use the same PAI command syntax:
pai -name tensorflow1120_ext
-project algo_public
-Dscript= 'oss://<bucket_name>.<oss_host>.aliyuncs.com/*.tar.gz'
-DentryFile='entry_file.py'
-Dbuckets='oss://<bucket_name>.<oss_host>.aliyuncs.com/<path>'
-Dtables='odps://prj_name/tables/table_name'
-Doutputs='odps://prj_name/tables/table_name'
-DcheckpointDir='oss://<bucket_name>.<oss_host>.aliyuncs.com/<path>'
-Dcluster="{\"ps\":{\"count\":1},\"worker\":{\"count\":2,\"gpu\":100}}"
-Darn="acs:ram::******:role/aliyunodpspaidefaultrole"
-DossHost="oss-cn-beijing-internal.aliyuncs.com"The name and project parameters have fixed values (tensorflow1120_ext and algo_public) and cannot be changed.
The following table describes all command parameters.
| Parameter | Description | Example | Default | Required |
|---|---|---|---|---|
script | The TensorFlow algorithm script for the task. Accepted formats: file:///path/to/file (absolute local path), project_name/resources/resource_name, oss://..aliyuncs.com/.tar.gz, or oss://..aliyuncs.com/*.py. The script can be a local file, a local TAR package (.tar.gz, gzip-compressed), or a Python file. | oss://demo-yuze.oss-cn-beijing-internal.aliyuncs.com/deepfm/deepfm.tar.gz | None | Yes |
entryFile | The entry script. Required when script points to a TAR package. If script is a single file, this parameter is not needed. | main.py | Not required for single-file scripts | Yes |
buckets | The input OSS buckets. Separate multiple buckets with commas. Each bucket path must end with a forward slash (/). | oss://..aliyuncs.com/ | None | No |
tables | The input MaxCompute tables. Separate multiple tables with commas. | odps:///tables/ | None | No |
outputs | The output MaxCompute tables. Separate multiple tables with commas. | odps:///tables/ | None | No |
gpuRequired | The number of GPUs for standalone training. 100 = 1 GPU, 200 = 2 GPUs. Set to 0 to use CPU only. For distributed training, configure GPUs through the cluster parameter instead. Available for TensorFlow 1120 only. | 100 | None | No |
checkpointDir | The OSS path for storing TensorFlow checkpoints. | oss://..aliyuncs.com/ | None | No |
cluster | The distributed training configuration, as an escaped JSON string. See Cluster parameters for details. | {\"ps\":{\"count\":1},\"worker\":{\"count\":2,\"gpu\":100}} | None | No |
enableDynamicCluster | Specifies whether to enable failover for individual worker nodes. When set to true, a failed worker node restarts automatically so the task can continue. Valid values: true, false. | false | false | No |
jobName | The name of the experiment. Use a descriptive name (not test) to make historical data searchable and performance analysis meaningful. | jk_wdl_online_job | None | Yes |
maxHungTimeBeforeGCInSeconds | The maximum duration (in seconds) a GPU can remain suspended before automatic reclamation. Set to 0 to disable automatic reclamation. | 3600 | 3600 | No |
ossHost | The OSS endpoint. See Regions and endpoints for valid values. | oss-cn-beijing-internal.aliyuncs.com | None | No |
Cluster parameters
Use the cluster parameter to configure distributed training across parameter servers (PSs) and workers. The value must be a JSON object with escaped quotation marks. Example:
{
"ps": {
"count": 2
},
"worker": {
"count": 4
}
}The JSON object supports two keys: ps (parameter server) and worker. Each key accepts the following sub-parameters.
| Parameter | Description | Default | Required |
|---|---|---|---|
count | The number of PSs or workers. | None | Yes |
gpu | The number of GPUs per PS or worker. 100 = 1 GPU. Set to 0 under worker to use CPU clusters with no GPU consumption. | 0 for PS; 100 for worker | No |
cpu | The number of CPU cores per PS or worker. 100 = 1 CPU core. | 600 | No |
memory | The memory per PS or worker, in MB. 100 = 100 MB. | 30000 | No |
I/O parameters
The following table describes the I/O parameters for PAI-TensorFlow tasks.
| Parameter | Description |
|---|---|
tables | The MaxCompute table path to read data from. |
outputs | The MaxCompute table path to write data to. Separate multiple paths with commas. Path formats: <br>- Non-partitioned table: odps://<prj_name>/tables/<table_name> <br>- Partitioned table: odps://<proj_name>/tables/<table_name>/<pt_key1=v1> <br>- Multi-level partitioned table: odps://<prj_name>/tables/<table_name>/<pt_key1=v1>/<pt_key2=v2> |
buckets | The OSS bucket that stores objects for the algorithm to read. Reading from OSS requires the role_arn and host parameters. To get the role_arn value, go to the PAI console, navigate to Dependent Services, find OSS in the Designer section, and click View authorization. See Grant the permissions required to use Machine Learning Designer for details. |
checkpointDir | The OSS bucket path to write output data to. |