You can use the O&M Assistant feature to create, run, and delete commands on an exclusive resource group for scheduling. You can also use the feature to view the running results of the commands. This topic describes how to create and manage a command on an exclusive resource group for scheduling by using the O&M Assistant feature.
Limits
- Feature usage
- After you create a command, you cannot edit the command. If you want to edit the command, delete it and create another command.
- You cannot use the O&M Assistant feature to run a yum command on an exclusive resource group for scheduling.
- Only exclusive resource groups for scheduling support the O&M Assistant feature.
- You can use the O&M Assistant feature to upload only resources whose size is not greater than 50 MB in an exclusive resource group for scheduling. We recommend that you use DataWorks to upload resources in a visualized manner. For more information about how to upload resources in a visualized manner, see Create a MaxCompute resource.
- You can use a PyODPS node to reference a third-party Python package installed by using the O&M Assistant feature only when an exclusive resource group for scheduling is used to run the code of the node on an on-premises machine. For information about how to reference a third-party Python package in a Python user-defined function (UDF) of MaxCompute, see Reference third-party packages in Python UDFs. For information about how to use a PyODPS node to reference a third-party package, see Use a PyODPS node to reference a third-party package.
- Permission management
Only RAM users to which the AliyunDataWorksFullAccess policy is attached or the ModifyResourceGroup permission is granted can use the O&M Assistant feature. For more information about permission management and authorization, see Manage permissions on the DataWorks services and the entities in the DataWorks console by using RAM policies.
Usage notes
- The O&M feature allows you to upload resources by running commands or installing built-in Python packages. For more information, see Create a command.
- The success rate of manually running commands cannot be ensured. For information about the third-party Python packages that are available for quick installation, see Third-party Python packages that are available for quick installation.
Go to the O&M Assistant page
- Log on to the DataWorks console.
- In the left-side navigation pane, click Resource Groups. The Exclusive Resource Groups tab of the Resource Groups page appears.
- On this tab, find the required exclusive resource group for scheduling. Move the pointer over the More icon and select O&M Assistant in the Actions column.
On the O&M Assistant page of the resource group, you can perform the following operations: create commands, run commands, and view running results of commands.
Create a command
- On the O&M Assistant page, click Create Command.
- In the Create Command panel, configure the parameters.
Parameter Description Command Name The name of the command. You can customize a command name. Command Type The method used to create the command. Valid values: Quick Installation and Manual Installation. - Quick Installation: DataWorks provides built-in Python packages. You can select a package based on the Python version for quick installation. Note For information about the third-party Python packages that can be quickly installed on an exclusive resource group for scheduling, see Third-party Python packages that are available for quick installation. The third-party Python packages are updated until November 25, 2022.
- Manual Installation: You can manually enter a Shell command to upload a package or resource file. If the required Python package is not included in the built-in Python packages provided by DataWorks, you can set the Command Type parameter to Manual Installation. After the resource file is uploaded, you must reference the file on a node based on the absolute path of the resource file. Note The success rate of manually running commands cannot be ensured. We recommend that you use DataWorks to upload resources whose size is not greater than 50 MB in a visualized manner. For example, you can upload a MaxCompute resource in a visualized manner. For more information, see Create a MaxCompute resource.
Command Content The content of the command. Installation Directories The parent directory of the directory in which you want to run the command. The specified parent directory is added to the directory whitelist. Separate multiple directories with semicolons (;). Note- You can write data only to the /home/admin/usertools/tools/ directory of an exclusive resource group for scheduling.
- If you do not specify an installation directory in which you want to run the command, the default installation directory /home/admin/usertools/tools/ is used.
Timeout The timeout period for running the command. Unit: seconds. If the timeout period is reached, the system forcefully stops running the command. - Quick Installation: DataWorks provides built-in Python packages. You can select a package based on the Python version for quick installation.
- After you configure the preceding parameters, click Create. Then, you can view the command in the command list of the O&M Assistant page.
Manage a command
- Run the command: You can run the command on an exclusive resource group for scheduling.
- View the running result: You can view the running result and details of the command.
- Delete the command: You can delete the command if you no longer need to use it.
Third-party Python packages that are available for quick installation
The following table lists the third-party Python packages that can be quickly installed on an exclusive resource group for scheduling. The third-party Python packages are updated until November 25, 2022.
python2 | python3 |
---|---|
aliyun-python-sdk-core | aliyun-python-sdk-core |
aliyun-python-sdk-ecs | aliyun-python-sdk-schedulerx2 |
jieba | Cython |
matplotlib | fbprophet |
numpy | gensim |
openpyxl | jieba |
oss2 | lxml |
pandas | numpy |
prophet | openpyxl |
pyhive | oss2 |
requests | pandas |
scikit-learn | prophet |
thrift | pyhive |
thrift_sasl | pymysql |
- | pystan |
- | requests |
- | scikit-learn |
- | sqlalchemy |
- | thrift |
- | thrift_sasl |
- | xgboost |