This topic describes how to prepare the Mars runtime environment.
To run Mars in MaxCompute, you must prepare the Mars runtime environment by using
one of the following methods:
- DataWorks
- Create a DataWorks PyODPS 3 node, which provides features of PyODPS and Mars.
You can run the following commands in the new PyODPS 3 node to check the versions of PyODPS and Mars. Make sure that the versions meet the requirements.
from odps import __version__ as odps_version from mars import __version__ as mars_version print(odps_version) print(mars_version)
odps_version: the version of PyODPS. Make sure that the PyODPS version is V0.9.3.1 or later. mars_version: the version of Mars. Make sure that the Mars version is V0.4.4 or later.
- Initialize a MaxCompute entry.
You can use the MaxCompute entry provided by the DataWorks PyODPS 3 node.
- Create a DataWorks PyODPS 3 node, which provides features of PyODPS and Mars.
- Other environments
- Install pip. After pip is installed, install PyODPS and Mars by running the pip install
command in a command-line interface such as the Command Prompt in Windows. The following
commands show an example of how to use the pip install command:
-
(Optional)Make sure that pip is in the latest version.
pip install -U pip
-
Install the latest version of PyODPS. In the command, https://mirrors.aliyun.com/pypi/simple/ is the URL of the Python Package Index (PyPI) mirror that Alibaba Cloud provides to accelerate package download.
pip install pyodps -i https://mirrors.aliyun.com/pypi/simple/
-
Install the latest version of Mars.
pip install pymars -i https://mirrors.aliyun.com/pypi/simple/
-
Install the latest version of protocol buffers.
pip install protobuf -i https://mirrors.aliyun.com/pypi/simple/
-
(Optional)Install the latest version of PyArrow to accelerate job execution in Mars.
pip install pyarrow -i https://mirrors.aliyun.com/pypi/simple/
Note For more information about how to install pip, see Installation in the pip documentation. -
- Initialize a MaxCompute entry.
You must use your AccessKey ID and AccessKey secret to initialize the MaxCompute entry. For more information about how to initialize a MaxCompute entry, see PyODPS: ODPS Python SDK and data analysis framework.
- Install pip. After pip is installed, install PyODPS and Mars by running the pip install
command in a command-line interface such as the Command Prompt in Windows. The following
commands show an example of how to use the pip install command: