All Products
Search

This Product

    Elastic High Performance Computing:Test cluster communication performance using IMB and MPI libraries

    Document Center

    Elastic High Performance Computing:Test cluster communication performance using IMB and MPI libraries

    Last Updated:Apr 01, 2026

    Use Intel MPI Benchmarks (IMB) and a Message Passing Interface (MPI) library to test the communication performance of an E-HPC cluster. This tutorial covers point-to-point latency (pingpong) and collective operations (allreduce, alltoall).

    Prerequisites

    Before you begin, make sure you have:

    • An SCC-based E-HPC cluster with one management node and two compute nodes. For setup instructions, see Create a cluster by using the wizard. Configure the cluster with the following settings:

      SettingValue
      Hardware settingsOne management node and two compute nodes. Use an ECS instance type with eight or more vCPUs per compute node. This example uses two ecs.c7.2xlarge instances.
      Software settingsCentOS 7.6 public image and the PBS scheduler.

    • A cluster user named mpitest in the sudo permission group. For setup instructions, see Create a user.

    • The following software installed on the cluster. For installation instructions, see Install software.

      • Intel MPI 2018

      • Intel MPI Benchmarks (IMB) 2019

    Key concepts

    Intel MPI Benchmarks (IMB) measures point-to-point and global communication performance in an HPC cluster for various granularities of messages.

    Message Passing Interface (MPI) is a standardized and portable message-passing standard for parallel computing. It supports multiple programming languages and provides high performance, concurrency, portability, and scalability.

    Step 1: Connect to the cluster

    Connect to the cluster using one of the following methods. This example uses mpitest as the username. After connecting, the shell opens at /home/mpitest.

    Option 1: E-HPC client (requires PBS scheduler)

    Make sure you have deployed the E-HPC client environment. For setup instructions, see Deploy an environment for an E-HPC client.

    1. Start and log on to your E-HPC client.

    2. In the left-side navigation pane, click Session Management.

    3. In the upper-right corner of the Session Management page, click terminal.

    Option 2: E-HPC console

    1. Log on to the E-HPC console.

    2. In the upper-left corner of the top navigation bar, select a region.

    3. In the left-side navigation pane, click Cluster.

    4. On the Cluster page, find the cluster and click Connect.

    5. In the Connect panel, enter the username and password, then click Connect via SSH.

    Step 2: Submit a job

    All examples in this step use mpirun with the -genv I_MPI_DEBUG 5 flag to enable MPI debug output at level 5.

    1. Create a job script file named IMB.pbs:

      vim IMB.pbs

      Use the following script:

      #!/bin/sh
#PBS -j oe
#PBS -l select=2:ncpus=8:mpiprocs=1
export MODULEPATH=/opt/ehpcmodulefiles/
module load intel-mpi/2018
module load intel-mpi-benchmarks/2019
echo "run at the beginning"
/opt/intel/impi/2018.3.222/bin64/mpirun -genv I_MPI_DEBUG 5 -np 2 -ppn 1 -host compute000,compute001 /opt/intel-mpi-benchmarks/2019/IMB-MPI1 pingpong > IMB-pingpong

      The #PBS -l select=2:ncpus=8:mpiprocs=1 line requests two nodes, eight vCPUs per node, and one MPI process per node. Adjust these values to match your cluster configuration.

      Configure the mpirun parameters based on your requirements:

      ParameterDescription
      -npTotal number of MPI processes across all nodes. For example, two nodes with one process each requires -np 2.
      -ppnNumber of MPI processes per node.
      -hostComma-separated list of nodes to run the job on.
      -npminMinimum number of processes that can run.
      -msglogRange of segment granularities.
      Note: To view all IMB parameters and supported communication modes, run /opt/intel-mpi-benchmarks/2019/IMB-MPI1 -h.

    2. (Optional) Run additional benchmark tests by modifying the mpirun command in the script.

      Allreduce — measures collective reduction performance across N nodes. Use this to test the performance of all-reduce communication between N nodes. On each node, two processes are started to obtain the time consumed to communicate messages of various message granularities.

      /opt/intel/impi/2018.3.222/bin64/mpirun -genv I_MPI_DEBUG 5 -np <N*2> -ppn 2 -host <node0>,...,<nodeN> /opt/intel-mpi-benchmarks/2019/IMB-MPI1 -npmin 2 -msglog 19:21 allreduce

      For example, to test two nodes with two processes each, set -np 4 -ppn 2 -host compute000,compute001.

      Alltoall — measures all-to-all collective performance across N nodes. Use this to test the performance of all-to-all communication among N nodes. On each node, a process is started to obtain the time consumed to communicate messages of various granularities.

      /opt/intel/impi/2018.3.222/bin64/mpirun -genv I_MPI_DEBUG 5 -np <N> -ppn 1 -host <node0>,...,<nodeN> /opt/intel-mpi-benchmarks/2019/IMB-MPI1 -npmin 1 -msglog 15:17 alltoall

      For example, to test two nodes with one process each, set -np 2 -ppn 1 -host compute000,compute001.

    3. Submit the job:

      qsub imb.pbs

      The output returns the job ID:

      0.manager

    Step 3: View the job result

    1. Check the job status:

      qstat -x 0.manager

      The output is similar to the following. In the S column, R means the job is running and F means it has completed.

      Job id            Name             User              Time Use S Queue
----------------  ---------------- ----------------  -------- - -----
0.manager         imb.pbs          mpitest           00:00:04 F workq

    2. View the pingpong test results:

      cat /home/mpitest/IMB-pingpong

      The test results are shown in the following image:

      IMB-Results.png