Use Environment Modules to manage software - Elastic High Performance Computing

This topic describes how to use Environment Modules to configure environment settings of modules for software. This way, the modules can be directly loaded and used in an Elastic High Performance Computing (E-HPC) cluster.

Background information

In some cases, you may need to install different compilers and Message Passing Interface (MPI) libraries for your E-HPC cluster. Common compilers include GNU Compiler Collection (GCC) and Intel Fortran Compiler. Common MPI libraries include Open MPI and MPICH. When you use a piece of software, different compiler settings are required to obtain different versions of executable programs and dynamic-link libraries, or different versions of environment variables. In addition, dependencies exist between multiple programs due to the variety of compilers or MPI libraries. Therefore, environment variables are hard to be modified when you execute a specific version of a program.

The Environment Modules package simplifies shell initialization and allows you to modify the environment during the session by using modulefiles. Each modulefile contains the information needed to configure the shell for a piece of software. Modulefiles can be shared by many users of a cluster. Users can have their own collection to supplement or replace the shared modulefiles.

Environment Modules commands

By default, Environment Modules is installed in each E-HPC cluster to manage different runtime environments.

Available modules are stored in the modulepath=/opt/ehpcmodulefiles directory. Modulefiles are usually named as a software version number, for example, openmpi/3.0.0.

The following Environment Modules commands are commonly used:

module avail: lists available modules.
module load: loads modules.
module unload: unloads modules.
module list: lists loaded modules.

Procedure

The following example shows how to use Environment Modules to set modulefiles by unloading the openmpi/3.0.0 module and then loading the openmpi/3.0.1 module:

Log on to the E-HPC console.
In the upper-left corner of the top navigation bar, select a region.
In the left-side navigation pane, click Cluster.
On the Cluster page, find the cluster to which you want to log on. Click Connect.
In the Connect panel, enter the name of the root user, logon password, and port number. Then, click Connect via SSH.
Run the cd /opt/ehpcmodulefiles command to go to the ehpcmodulefiles directory.
Run the module list command to list the loaded modules.
The openmpi/3.0.0 module is loaded, as shown in the following figure.
Run the module unload openmpi/3.0.0 command to unload the openmpi/3.0.0 module.

Run the module load openmpi/3.0.1 command to load the openmpi/3.0.1 module.

You can modify the variables in the openmpi/3.0.1 module based on your business requirements. You can modify the following variables:

PATH: the path where executable programs are located.
LIBRARY_PATH: the path to the shared library when searching for the dynamic-link library
LD_LIBRARY_PATH: a path other than the system default path when searching for the dynamic link library.

installpath: the path where the software is installed.

 
#%Module1.0#####################################################################
##
## modulefiles/openmpi-3.0.1  Generated by EHS 1.0.0
##
proc ModulesHelp { } {
  puts stderr "\tmodules - loads the modules software & application environment"
  puts stderr "\n\tThis loads environment of openmpi-3.0.1"
}

module-whatis   "loads the openmpi-3.0.1 environment"
prepend-path PATH  ${installpath}/bin  
prepend-path LD_LIBRARY_PATH  ${installpath}/lib
prepend-path LIBRARY_PATH  ${installpath}/lib