All Products
Search

This Product

    Submit job

    Document Center

    Submit job

    Last Updated: May 11, 2018

    This article describes the commands used to submit various types of jobs.

    1. Submit a simplest job

    If you have configured the default image, instance type, and network type according to Preparations, use the following simple command to submit a job.

    1. bcs sub "echo 123" # Submit and run a single-task job "echo 123".

    If default settings are unavailable for the BatchCompute-cli tool, you must specify more parameters when submitting a job.

    1. bcs sub "echo 123" -t ecs.sn1ne.large -i img-ubuntu-vpc --vpc_cidr_block 192.168.0.0/16

    For the remaining examples in this article, it is assumed that you have configured the default settings.

    2. Submit a Python script job

    1. bcs sub "python test.py" -p ./test.py # "-p" indicates that test.py is packed into worker.tar.gz and then uploaded to OSS before the job is submitted.

    3. Submit a directory (multi-file) job

    Generally, the src directory contains multiple files, for example:

    1. src
    2.   |-- test.py
    3.   |-- dep.py

    Test.py must depend on dep.py. Therefore, the entire directory can be packed.

    1. bcs sub "python test.py" -p ./src/ # Pack all files under the src directory into worker.tar.gz, upload it to the OSS, and then submit the job.

    Specify multiple files at a time and separate these files by commas (,):

    1. cd src  # Switch to the src directory.
    2. bcs sub "python test.py" -p test.py,dep.py # Pack the two files into worker.tar.gz.

    If you do not switch to the src directory, submit the job as follows:

    1. bcs sub "python test.py" -p src/test.py,src/dep.py # Pack the two files into worker.tar.gz.

    Then, use the following command to view the content of worker.tar.gz:

    1. tar -tvf worker.tar.gz

    The content of worker.tar.gz is as follows:

    1. test.py
    2. dep.py

    4. Submit a job by mounting a task program

    Assume that you have uploaded test.py to oss://mybucket/test/test.py.You can mount the OSS directory “oss://mybucket/test/“ to a VM local directory, for example, “/home/admin/test/“. Then use the command “python /home/admin/test/test.py” to run the program.

    Submit a job as follows:

    1. bcs sub "python /home/admin/test/test.py" -r oss://mybucket/test/:/home/admin/test/

    The parameter -r oss://mybucket/test/:/home/admin/test/ indicates read-only mounting and “oss://mybucket/test/“ is mounted onto “/home/admin/test/“.

    In this way, you do not need to pack test.py and dep.py into worker.tar.gz.

    For more information about how to submit a job, run “bcs sub -h” to view the help information.

    Other tasks

    1. Mount the input data

    Assume that your data has been uploaded to the directory “oss://my-bucket/inputs/“.

    1. bcs sub "python test.py" -p ./src/  -r oss://my-bucket/inputs/:/home/admin/inputs/
    • “-r” indicates read-only mounting and “oss://my-bucket/inputs/“ is mounted to “/home/admin/inputs/“. In this way, your program can read files under the directory “/home/admin/inputs/“ same as reading local files.
    • All mounted directory cannot be system directories, such as “/bin” and “/usr”. The directory “/home/“ is recommended.
    • To mount multiple directories, separate them by using commas (,). For example, -r oss://my-bucket/inputs/:/home/admin/inputs/,oss://my-bucket/inputs2/:/home/admin/inputs2/.

    2. Use the mounting method to automatically upload the program running results

    Your program will write the running results to the directory “/home/admin/outputs/“. To upload all data under this directory to OSS, run the following command:

    1. bcs sub "python test.py" -p ./src/ -r oss://my-bucket/inputs/:/home/admin/inputs/ -w oss://my-bucket/outputs/:/home/admin/outputs/
    • “-w” indicates writable mounting and all data written to this directory will be automatically uploaded to the corresponding OSS directory after the program finishes running.
    • All mounted directory cannot be system directories, such as “/bin” and “/usr”. The directory “/home/“ is recommended.
    • To mount multiple directories, separate them by using commas (,). For example, -w oss://my-bucket/outputs/:/home/admin/outputs/,oss://my-bucket/outputs2/:/home/admin/outputs2/.

    3. Use the custom image and instance type

    1. bcs sub "python test.py" -p ./src/ -c img=img-ubuntu-vpc:type=ecs.sn1ne.large

    • “-c” indicates that a cluster is used. Next to “-c”, you can specify the cluster ID or the AutoCluster configuration. The AutoCluster configuration format is as follows: img=${ImageId}:type=${InstanceType}.

    • “img=img-ubuntu-vpc” indicates that an image is used. You can specify a custom image.

    • “type=ecs.sn1ne.large” indicates that an instance type is used. You can run bcs it to view the list of the supported instance types.

    • Specify only -c img=${ImageId} or only -c type=${InstanceType}. If none of them are specified, the Default image and default instance types are used.

    4. Use a cluster

    1. bcs sub "python test.py" -p ./src/ -c cls-xxxxxxx
    • “-c cls-xxxxxxx” indicates that the cluster with the ID set to cls-xxxxxxx is used.

    For more information about the cluster, see Use a cluster.

    5. Use Docker

    1. bcs sub "python test.py" -p ./src/ --docker myubuntu@oss://my-bucket/dockers/

    • “myubuntu” is short for “localhost:5000/myubuntu”. “oss://my-bucket/dockers/“ indicates the path of the OSS Docker registry.

    • For more information, see Use the Docker.

    6. Custom disks

    1. bcs sub "python test.py" --disk system:cloud_efficiency:50,data:cloud_efficiency:200

    • The command is valid only when AutoCluster is used. System disks and one data disk (optional) can be configured. The format is similar to —disk system:cloud_efficiency:40,data:cloud_efficiency:50:/home/disk1. The system disk configuration is separated from the data disk configuration by a comma (,). You can specify only the system disk or only the data disk, for example:—disk system:cloud_efficiency:40.

    • By default, only one system disk is mounted, and the size is 40 GB.

    • The format of the system disk configuration is as follows: system:< cloud_efficiency|cloud_ssd>:<40-500>. For example, “system:cloud_efficiency:40” indicates that the system disk is mounted to a 40 GB ultra cloud disk.

    • The format of the data disk configuration is as follows: data:< cloud_efficiency|cloud_ssd>:<5-2000>:For example, “data:cloud_ssd:50:/home/disk1” indicates that a 50 GB SSD cloud disk is mounted as a data disk. In the Windows operating system, the data disk can be mounted only to the drive. For example, “data:cloud_ssd:50:E” indicates that the data disk is mounted to the E drive.

    Note: For an old and dedicated instance that starts with “bcs”, set the disk type to ephemeral, and set the size of the data disk to a value ranging from 5 GB to 1024 GB.

    View job information

    For more information, see How to view the job information.