Modules Environment

What is the main purpose of the Modules environment?

Environment modules allow you to easily adapt your shell's environment (PATH, MANPATH, INCLUDE, LD_LIBRARY_PATH, etc.) to your needs. Use the Modules environment commands to coherently set and unset all paths and environment variables that are necessary to use available (and sometimes conflicting) software packages on our HPC systems, by simply loading or unloading the corresponding module files. We offer module files for the following categories:

Application-Software
Compilers
Parallel-Environments
Libraries
Debugging-and-Profiling / Development

Most installed third party and open source software packages are contained in a two-level structure of module files application/version. Eg. the Intel 11.1 compiler is in module file intel/11.1. To load or unload a module, simply issue the command

  module [un]load application/version

This will set/unset all the module's environment variables in your current shell.

There is one exception: All software which comes pre-installed with the operating system, such as the default Gnu Compiler Collection (GCC), is not provided by means of the Modules environment, because all binaries, libraries, etc. for these packets are already contained within the default system paths. We may, however, offer newer versions of the same software by means of the Modules environment.

Setting up the Modules environment

The Modules environment will automatically be set during your cluster login. While the general usage is quite self-explanatory, please note the following usage guidelines for UIBK:

Setup for a specific compiler

With the exception of the compiler modules, all module files (in particular, libraries) are set up so they will automatically be compatible with your preferred compiler.
This setup is different from most other sites, where you need to use explicitly compiler-specific module commands.

Setting the preferred compiler is easy: just load the respective compiler module before all libraries. The following two command lines

module load intel/11.1
module load netcdf/4.0.1

will activate the Intel 11.1 compiler and a version of the NetCDF 4.0.1 library which is compatible with the Intel compiler. If no special compiler module is loaded, the GCC that came with the OS is selected as the preferred compiler.

If for some reason you need to set up a specific module for another compiler than the preferred one, you can enforce this behavior by setting the environment variable


export PREFERRED_MC={gnu-$HPC_SYSTEM|intel[-version]|pgi[-version]}

where PREFERRED_MC simply stands for Preferred Module Compiler. So e.g. if you want as preferred compiler the Intel compiler, but for some reasons the netcdf libraries for the system specific Gnu compilers, the following command lines will yield the desired result:

module load intel/11.1
export PREFERRED_MC=gnu-$HPC_SYSTEM
module load netcdf/4.0.1
unset PREFERRED_MC
module load further_modules

Modules environment and batch jobs

There are (among other methods) two proposed ways to ensure that the required modules are available for your batch jobs:

You can put your required module commands (module load module_name) into your $HOME/.bashrc file. This is the preferred method if you constantly use the same module files for interactive use and all your job runs.
The second and more flexible way is to load a certain module environment as part of your job. This method is recommended if you need to use different environments for different job runs.

The following script file script.sh is an example:
#!/bin/bash
module load module_name_1 module load module_name_2 ... program_name <parameters>

Make sure that the script.sh file has execute permission (chmod +x script.sh).

Note: If you submit parallel jobs to the SGE, PBS or SLURM batch system please be aware that normally batch systems do not export your interactive shell session's environment variables to the job. As a result, modules that you have loaded interactively in your login session will have no effect when a job that you have submitted is executed. Use either of above methods to set your job's execution environment.

Working with the Module Environment

The module command has a number of sub-commands. In the following sections, we briefly discuss the most important ones.

module avail

With the avail subcommand, a list of available modules on the system is printed in the formmat:

 $ module avail

----------- /path_to_module_categories/{Application-Software|Compiler|...} -----------

application1/version1   application2/version2   application3/version3   ...

module display

With the display subcommand you can see what a given modulefile will do to your environment. This way, you can also find out about where application binaries or libraries for linking your own programs are located. For your convenience and for portability between our HPC systems we provide special environment variables prefixed with UIBK_ for this purpose.

The following example illustrates the display subcommand:

 $ module display intel/11.1
-------------------------------------------------------------------
/usr/site/hpc/modules/leo2/Compiler/intel/11.1:

conflict         intel pgi 
setenv           UIBK_INTEL_BIN /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/bin/intel64 
setenv           UIBK_INTEL_INC /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/include/intel64 
setenv           UIBK_INTEL_LIB /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/lib/intel64 
prepend-path     PATH /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/bin/intel64 
prepend-path     INCLUDE /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/include 
prepend-path     INCLUDE /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/include/intel64 
prepend-path     LD_LIBRARY_PATH /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/lib/intel64 
prepend-path     MANPATH /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/man/en_US 
prepend-path     INTEL_LICENSE_FILE /usr/site/hpc/x86_64/generic/intel/license/intel-flexlm.lic 
setenv           CC icc 
setenv           CXX icpc 
setenv           FC ifort 
setenv           IDB idb 
-------------------------------------------------------------------

module load

With the load subcommand you can load one or more of the available module files:

 $ module load intel/11.1 netcdf/4.0.1
  Loading intel/11.1
  Loading netcdf/4.0.1 for compiler intel-11.1

module list

With the list subcommand you get the list of all currently loaded module files:

 $ module list

Currently Loaded Modulefiles:
  1) intel/11.1     2) netcdf/4.0.1

module unload/purge

With the unload subcommand you can unload one or more of the loaded module files (see list of loaded modules above):

 $ module unload intel/11.1 netcdf/4.0.1
  Unloading intel/11.1
  Unloading netcdf/4.0.1 for compiler intel-11.1

Similarly, with the command

 $ module purge

all loaded modules are unloaded at once.

module help

The help subcommand on its own gives general information about module usage. When adding a specific module to the help subcommand, some more information about this module is displayed:

 $ module help netcdf/4.0.1
----------- Module Specific Help for 'netcdf/4.0.1' ---------------

NetCDF (network Common Data Form) is a set of software libraries and machine-independent
data formats that support the creation, access, and sharing of array-oriented scientific
data. Please load this module subsequently to any Matlab module!!!

Additional information about the Modules environment

For further information concerning the Modules environment please have a look at the man pages (man module).
The sources and further documentation can be found at the SourceForge website