Use the CLI to submit a job - Elastic High Performance Computing

E-HPC supports three job schedulers: PBS, SLURM, and SGE. This guide walks you through connecting to a cluster and submitting a job using the CLI for each scheduler.

Prerequisites

Before you begin, ensure that you have:

A cluster user account. See Create a user. This guide uses the username testuser with sudo permissions.
Your execution program and data files in place. This guide uses the following paths:
```
$HOME/test.py       # execution program
$HOME/test.data     # job data file
```

Important

Do not submit jobs as the root user. A non-root user prevents accidental cluster data loss caused by errors in the job script.

Connect to the cluster

Connect using either the E-HPC client or the E-HPC console. After connecting, you are logged in to /home/testuser automatically.

Option 1: E-HPC client (PBS clusters only)

Make sure you have downloaded and installed the E-HPC client and set up the required environment. See Deploy an environment for an E-HPC client.

Start the E-HPC client and log in.
In the left-side navigation pane, click Session Management.
In the upper-right corner of the Session Management page, click terminal.

Option 2: E-HPC console

Log in to the E-HPC console.
In the upper-left corner of the top navigation bar, select a region.
In the left-side navigation pane, click Cluster.
Find your cluster and click Connect.
In the Connect panel, enter your username and password, then click Connect via SSH.

Submit a job

In auto scaling scenarios, E-HPC scales based on vCPU count, not memory. Always specify the number of vCPUs when submitting a job.

The following table summarizes the submit command for each scheduler:

Scheduler	Submit command
PBS	`qsub jobscript.pbs`
SLURM	`sbatch jobscript.slurm`
SGE	`qsub -V jobscript.sh`

Select the section for your scheduler.

PBS

This script requests 4 vCPUs and 1 GB of memory, runs for up to 10 minutes, and writes all output to test_pbs.log.

Create the job script.
```
vim jobscript.pbs
```
Paste the following content into the file.
Line Description
(1) Compute resources: 4 vCPUs and 1 GB of memory
(2) Walltime limit: the job is stopped if it exceeds 10 minutes
(3) stdout output file
(4) Redirects stderr to the same file as stdout
```
#!/bin/sh
# PBS -l ncpus=4,mem=1gb    # (1)
# PBS -l walltime=00:10:00  # (2)
# PBS -o test_pbs.log       # (3)
# PBS -j oe                 # (4)
cd $HOME
test.py -i test.data
```
For the full PBS CLI reference, see the PBS official website.
Submit the job.
```
qsub jobscript.pbs
```

Line	Description
(1)	Compute resources: 4 vCPUs and 1 GB of memory
(2)	Walltime limit: the job is stopped if it exceeds 10 minutes
(3)	stdout output file
(4)	Redirects stderr to the same file as stdout

SLURM

This script runs a single task on one node using 1 vCPU and 1024 MB of memory, with a 10-minute time limit. Output goes to test_slurm.log.

Create the job script.
```
vim jobscript.slurm
```

Paste the following content into the file.

Line	Description
(1)	Job name shown in `squeue` output
(2)	stdout output file
(3)	Number of nodes to allocate
(4)	Number of tasks to run
(5)	vCPUs per task
(6)	Time limit: the job is stopped if it exceeds 10 minutes
(7)	Memory allocated per vCPU (MB)

#!/bin/sh
# SBATCH --job-name=slurm-quickstart  # (1)
# SBATCH --output=test_slurm.log      # (2)
# SBATCH --nodes=1                    # (3)
# SBATCH --ntasks=1                   # (4)
# SBATCH --cpus-per-task=1            # (5)
# SBATCH --time=00:10:00              # (6)
# SBATCH --mem-per-cpu=1024           # (7)
cd $HOME
test.py test.data

For the full SLURM CLI reference, see the SLURM official website.

Submit the job.
```
sbatch jobscript.slurm
```

SGE

This script runs the job in the current working directory using 2 vCPUs and 1 GB of memory, writing logs to /home/testuser.

Important

E-HPC selects ECS instance types based on the vCPU count during auto scaling. Use -pe smp to specify the number of vCPUs required; omitting this directive prevents correct instance selection.

Create the job script.
```
vim jobscript.sh
```

Paste the following content into the file.

Line	Description
(1)	Sets the working directory to the current directory
(2)	Job name
(3)	Queue to submit to
(4)	Number of vCPUs required (used for auto scaling instance selection)
(5)	Memory required: 1 GB
(6)	stdout log path
(7)	stderr log path

#!/bin/bash
#$ -cwd         # (1)
#$ -N test1     # (2)
#$ -q all.q     # (3)
#$ -pe smp 2    # (4)
#$ -l vf=1g     # (5)
#$ -o /home/testuser  # (6)
#$ -e /home/testuser  # (7)
cd $HOME
test.py test.data

For the full SGE CLI reference, see the SGE official website.

Submit the job.
```
qsub -V jobscript.sh
```