This topic describes how to use a PyODPS node in DataWorks to reference a third-party package. You can reference a common Python script or a third-party open source package.
Reference a common Python script
Reference a third-party open source package
Before you reference a third-party open source package, you must use pip to install
the package and make sure that the following requirements are met:
- An exclusive resource group for scheduling is available. For more information, see Create an exclusive resource group for scheduling.
- The third-party open source package is installed in O&M Assistant of the exclusive
resource group for scheduling. For more information, see O&M Assistant. PyODPS nodes include PyODPS 2 nodes and PyODPS 3 nodes.
- If you want to use a PyODPS 2 node to reference the third-party open source package,
run the following command to install the package:
pip install <Package that you want to reference> -i https://pypi.tuna.tsinghua.edu.cn/simple
If you are prompted to upgrade pip after you run the preceding command, run the following command:pip install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple
- If you want to use a PyODPS 3 node to reference the third-party open source package,
run the following command to install the package:
/home/tops/bin/pip3 install <Package that you want to reference> -i https://pypi.tuna.tsinghua.edu.cn/simple
After the package is installed, run the
import
command to import the package. For example, use O&M Assistant to run thepip3 -install oss2
command to install the package oss2. Then, run theimport oss2
command in the PyODPS 3 node to import and reference oss2.If you are prompted to upgrade pip after you run the preceding commands, run the following command:/home/tops/bin/pip3 install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple
If the following error occurs when you use the PyODPS 3 node, submit a ticket to apply for permissions."/home/admin/usertools/tools/cmd-0.sh:Line 3: /home/tops/bin/python3: The file or directory does not exist."
- If you want to use a PyODPS 2 node to reference the third-party open source package,
run the following command to install the package: