Elastic High Performance Computing:Submit a job

Prerequisites

• The cluster and its nodes are in the Running state.

• A cluster user must be created. For more information, see user management.

• If you submit a job using the console, ensure the job script is in the cluster.

• If you submit a job using the CLI, the cluster scheduler must be Slurm, OpenPBS.

Submit a job using the console

1. Go to the Cluster Details page.

   1. Log on to the E-HPC console.

   2. In the left part of the top navigation bar, select a region.

   3. In the left-side navigation pane, click Cluster.

   4. On the Cluster List page, find the cluster that you want to manage and click the Cluster Name.

2. In the left-side navigation pane, click Job Management.

3. Click Create Job and configure the following parameters.

   • Basic settings

     Parameter	Required	Description
     Job Name	Yes	The name of the job.
     Scheduler Queue	Yes	The queue in the cluster where the job runs.
     Start Job Array	No	Specifies whether to enable the Job Array feature of the scheduler. A Job Array allows you to define custom rules for job execution. Minimum Index: The starting index for tasks in the Job Array. Maximum Index: The ending index for tasks in the Job Array. Step: The increment between task indices. If not specified, the default value is 1.
     Job Priority	No	The job execution priority depends on the scheduler. The value is an integer of 0 or greater. A larger value indicates a higher priority. Note If the cluster scheduling policy is set to prioritize jobs, higher-priority jobs are scheduled to run first. If you submit multiple jobs, you can set a higher priority for jobs that need to be executed first.
     Run Command	Yes	The command submitted to the scheduler to run the job. This can be a job script, such as job.pbs in the /home/test directory, or a command string. If the script file is executable, enter its relative path, such as ./job.pbs. If the script file does not have execute permissions, enter an execution command and prefix it with --. For example: --/opt/mpi/bin/mpirun /home/test/job.pbs.
     Nodes	No	The number of compute nodes for the job.
     Requested CPUs	No	The memory and CPU resources that each node requires to run the job's tasks. Note Set these parameters based on the actual requirements of your job. Insufficient memory may cause the job to fail or its performance to degrade.
     Requested Memory	No

   • Advanced settings

     Parameter	Required	Description
     Stdout Path	No	The redirection path for the Linux stderr and stdout streams. The path must include the output filename. stdout: The path to the standard output file. stderr: The path to the standard error file. The cluster user must have write permissions on this path. By default, output files are generated based on the scheduler settings.
     Stderr Path	No
     Environment Variables	No	Custom environment variables for the job. You can access these variables from within your job script.

4. After confirming the settings, click Confirm Create at the bottom of the page.

Submit a job using the CLI

1. Connect to the logon node of the cluster. For more information, see Connect to a cluster.

2. Depending on your cluster's scheduler, write and submit a job script. The following sections provide examples.

   Note

   In auto scaling scenarios, E-HPC does not support memory-based scaling. We recommend that you specify the number of vCPUs that the job requires when you submit it.

   Slurm

   1. Run the following command to create and edit a job script named jobscript.slurm:

      vim jobscript.slurm

      The following example shows the content of the jobscript.slurm file. For more information about Slurm commands, see the official Slurm documentation.

      #!/bin/sh
      #SBATCH --job-name=slurm-quickstart       # Job name
      #SBATCH --output=test_slurm.log           # Standard output file
      #SBATCH --nodes=1                         # Number of nodes
      #SBATCH --ntasks=1                        # Number of tasks
      #SBATCH --cpus-per-task=1                 # vCPUs per task
      #SBATCH --time=00:10:00                   # Estimated runtime
      #SBATCH --mem-per-cpu=1024                # Memory per vCPU
      cd $HOME
      test.py test.data

   2. Run the following command to submit the job:

      sbatch jobscript.slurm

   OpenPBS

   1. Run the following command to create and edit a job script named jobscript.pbs:

      vim jobscript.pbs

      The following example shows the content of the jobscript.pbs file. For more information about OpenPBS commands, see the official OpenPBS documentation.

      #!/bin/sh
      #PBS -l ncpus=4,mem=1gb       # Required compute resources
      #PBS -l walltime=00:10:00     # Maximum wall time
      #PBS -o test_pbs.log          # Standard output file
      #PBS -j oe                    # Merge stderr with stdout
      cd $HOME
      test.py -i test.data

   2. Run the following command to submit the job:

      qsub jobscript.pbs

Submit a job using the E-HPC Portal

For more information, see Submit a job using a submitter.

Related documentation

You can also submit a job by calling the CreateJob API.