Install the OSS Connector for AI/ML to efficiently access and store datasets in OSS for your PyTorch training jobs.
Prerequisites
-
Operating system: Linux x86-64
-
glibc: >=2.17
-
Python: 3.8–3.13
-
PyTorch: >=2.0
-
To use the OSS Checkpoint feature, your Linux kernel must support userfaultfd.
NoteFor example, on an Ubuntu system, run the
sudo grep CONFIG_USERFAULTFD /boot/config-$(uname -r)command to check if your Linux kernel supports userfaultfd. If the command returnsCONFIG_USERFAULTFD=y, your kernel supports userfaultfd. If it returnsCONFIG_USERFAULTFD=n, it does not, and you cannot use the OSS Checkpoint feature.
Procedure
The following example shows how to install the OSS Connector for AI/ML for Python 3.12.
-
On your Linux operating system or in a container built from a Linux image, run the
pip3.12 install osstorchconnectorcommand to install the OSS Connector for AI/ML.pip3.12 install osstorchconnector -
Run
pip3.12 show osstorchconnectorto verify the installation.pip3.12 show osstorchconnectorA successful installation displays the version information for osstorchconnector. The output is similar to the following:
Name: osstorchconnector Version: 1.0.0rc1 Summary: OSS connector for AI/ML Home-page: Author: Author-email: License: Location: /usr/local/lib/python3.12/dist-packages Requires: torch Required-by:
Next steps
After installation, configure the connector's access credentials and settings. This allows the connector to communicate with OSS and initialize correctly. For instructions, see Configure OSS Connector for AI/ML.