All Products
Search
Document Center

Import job files from an OSS bucket to a cluster

Last Updated: Sep 29, 2021

You can configure job data in advance and upload a job data file and executable file to an Object Storage Service (OSS) bucket. When you create a job, you can directly import job files that are stored in an OSS bucket so that you can improve your work efficiency. This topic describes how to upload job files to an OSS bucket and import job files to a cluster.

Upload job files to an OSS bucket

To upload a job data file and executable file to an OSS bucket, perform the following steps:

  1. Select the same region as your cluster.

    For more information, see Activate OSS.

  2. Create a bucket to store job files.

    For more information, see Create buckets.

  3. Upload job files.

    For more information, see Upload objects.

Import job files from an OSS bucket to a cluster

  1. Log on to the cluster.

    For more information, see Log on to a cluster.

  2. Select one of the following methods to import job files:

    • Import job files by accessing the URL of OSS

      1. Obtain the URL of the OSS service.

        For more information, see Use a public endpoint to access OSS and Use an internal endpoint to access OSS.

      2. Run the following commands to download the job data file and script file:

        In this example, the script file is test.py and the job data file is test.data.

        • GNU Wget is installed on your computer:

          cd /ehpcdata
          wget http://<Bucket>.<Internal endpoint>//test.py
          wget http://<Bucket>.<Internal endpoint>//test.data 
        • GNU Wget is not installed on your computer:

          cd /ehpcdata
          curl -O http://<Bucket>.<Internal endpoint>//test.py
          curl -O http://<Bucket>.<Internal endpoint>//test.data 
    • Import job files by accessing the temporary URL of job files

      1. Obtain the temporarily signed URL of job files.

        For more information, see Authorize third-party users to download objects.

      2. Run the following commands to download the job data file and script file:

        In this example, the script file is test.py and the job data file is test.data.

        • GNU Wget is installed on your computer:

          cd /ehpcdata
          wget http://<bucket>.<region>.aliyuncs.com/<object>?OSSAccessKeyId=<user access_key_id>&Expires=<unix time>&Signature=<signature_string>//test.py
          wget http://<bucket>.<region>.aliyuncs.com/<object>?OSSAccessKeyId=<user access_key_id>&Expires=<unix time>&Signature=<signature_string>//test.data
        • GNU Wget is not installed on your computer:

          curl -O http://<bucket>.<region>.aliyuncs.com/<object>?OSSAccessKeyId=<user access_key_id>&Expires=<unix time>&Signature=<signature_string>//test.py
          curl -O http://<bucket>.<region>.aliyuncs.com