The Modules Environment at the ITP Cluster
What is the main purpose of Modules Environment?
Environment modules provide a great way to easily customize your shell environment (PATH, MANPATH, INCLUDE, LD_LIBRARY_PATH), especially on the fly. Using Modules environment allows you to set and unset cleanly your paths and environment variables especially with loading or unloading a module file. At /opt/Modules there exist four (4) subdirectories:
- Applications
- Compiler
- Libraries
- Parallel-Environments
In this new structure you can see a wide varity of installed third party software, as e.g. the Intel-compiler.
Which modules are actual available on the cluster?
Here you can see the actual Module Environments from the ITP Compute Cluster Regulus and the Opteron Cluster Teazer. Further details about the installed software you can find here.
Setting up the Modules environment
The Modules environment will automatically be set during your cluster login (using the /etc/profile.d/zzmodule.{sh,csh} scripts). The zzmodule-script sources the module initialization file /opt/Modules/3.2.6/init/$modules_shell (where $modules_shell=bash for bash or $modules_shell=csh for csh as login shell, etc.) and sets the MODULEPATH variable.
If you want to submit jobs using the SGE (Sun Grid Engine) please be aware that the SGE does not export the actual PATH variable to the remote cluster nodes. If you load modules on the master node teazer in an interactive way and then submit jobs with the SGE, the locally loaded modules are not transferred to the compute nodes.
There are two ways to ensure that the required modules are available at the compute nodes:
- You can put your required modules into your
.bashrcfile in your home directory (module load module_name). This is the preferred method if you use the same modulefiles for all job runs. - The second and more flexible way is to load a certain module environment as part of your job (for further details please refer to Short tutorial for using the SGE ). This method is recommended if you have to use different environments for different job runs.
The following script file "script.sh" serves as an example:
Assure that "script.sh" has execution rights (chmod +x script.sh).#!/bin/bash
module load module_name_1
module load module_name_2
...
program_name <parameters>
Working with the Module Environment
avail
With the subcommand avail you get the available list of the module files:
[user@teazer ~]$ module avail
------------------------------ /opt/Modules/Applications -------------------------------
java/1.6.0u12 matlab/R2008a scipy
mathematica/6.0 matlab/R2008b(default)
mathematica/7.0(default) numpy
-------------------------------- /opt/Modules/Compiler ---------------------------------
intel/10.1.018 intel/11.0.081(default)
------------------------------- /opt/Modules/Libraries ---------------------------------
acml/4.2.0_gnu-multiproc atlas/3.8.2 intel-ipp/6.0.1
acml/4.2.0_gnu-singleproc fftw/3.1.2_gnu-4.1.2 intel-mkl/10.0.4
acml/4.2.0_intel-multiproc fftw/3.1.2_intel-10.1 intel-mkl/10.1.1(default)
acml/4.2.0_intel-singleproc fftw/3.2.1_gnu-4.1.2 intel-tbb/2.1
-------------------------- /opt/Modules/Parallel-Environments --------------------------
mpich/1.2.7p1_gnu-4.1.2 openmpi/1.2.6_gnu-4.1.2
The notation for the modulefilename is "name of the application/version/[compiler] and description".
help
With the subcommand help you get more information about module usage:
[user@teazer ~]$ module --help
Modules Release 3.2.6 2007-02-14 (Copyright GNU GPL v2 1991):
Usage: module [ switches ] [ subcommand ] [subcommand-args ]
Switches:
-H|--help this usage info
-V|--version modules version & configuration options
-f|--force force active dependency resolution
-t|--terse terse format avail and list format
-l|--long long format avail and list format
-h|--human readable format avail and list format
-v|--verbose enable verbose messages
-s|--silent disable verbose messages
-c|--create create caches for avail and apropos
-i|--icase case insensitive
-u|--userlvl <lvl> set user level to (nov[ice],exp[ert],adv[anced])
Available SubCommands and Args:
+ add|load modulefile [modulefile ...]
+ rm|unload modulefile [modulefile ...]
+ switch|swap [modulefile1] modulefile2
+ display|show modulefile [modulefile ...]
+ avail [modulefile [modulefile ...]]
+ use [-a|--append] dir [dir ...]
+ unuse dir [dir ...]
+ update
+ refresh
+ purge
+ list
+ clear
+ help [modulefile [modulefile ...]]
+ whatis [modulefile [modulefile ...]]
+ apropos|keyword string
+ initadd modulefile [modulefile ...]
+ initprepend modulefile [modulefile ...]
+ initrm modulefile [modulefile ...]
+ initswitch modulefile1 modulefile2
+ initlist
+ initclear
You can also get more information about particular modules:
[user@teazer ~]$ module help intel/10.1.018
----------- Module Specific Help for 'intel/10.1.018' -------------
Sets up the paths you need to use Intel Compiler Suite (64bit)
Version 10.1 (10.1.018) with intel debugger (idb)
display
With the subcommand display you can exactly see what a given modulefile will do to your environment. In fact all changes to the PATH, MANPATH etc. will be displayed.
[user@teazer ~]$ module display intel/10.1.018
-------------------------------------------------------------------
/opt/Modules/Compiler/intel/10.1.018:
module-whatis Adds Intel Compiler Suite 10.1.018 to your environment variables
conflict intel
prepend-path PATH /opt/intel/cce/10.1.018/bin
prepend-path INCLUDE /opt/intel/cce/10.1.018/include
prepend-path LD_LIBRARY_PATH /opt/intel/cce/10.1.018/lib
prepend-path MANPATH /opt/intel/cce/10.1.018/man
prepend-path INTEL_LICENSE_FILE /opt/intel/licenses
setenv CC icc
setenv CXX icpc
prepend-path PATH /opt/intel/fce/10.1.018/bin
prepend-path INCLUDE /opt/intel/fce/10.1.018/include
prepend-path LD_LIBRARY_PATH /opt/intel/fce/10.1.018/lib
prepend-path MANPATH /opt/intel/fce/10.1.018/man
setenv FC ifort
prepend-path PATH /opt/intel/idbe/10.1.018/bin
prepend-path INCLUDE /opt/intel/idbe/10.1.018/include
prepend-path LD_LIBRARY_PATH /opt/intel/idbe/10.1.018/lib
prepend-path MANPATH /opt/intel/idbe/10.1.018/man
setenv IDB idb
-------------------------------------------------------------------
load
With the subcommand load you can load one or more of the available module files:
syntax: module load modulefile [modulefile ...]
[user@teazer ~]$ module load intel/10.1.018 intel-mkl/10.0.4
list
With the subcommand list you get the current loaded module files:
[user@teazer ~]$ module list
Currently Loaded Modulefiles:
1) intel/10.1.018 2) intel-mkl/10.0.4
unload
With the subcommand unload you can unload one or more of the loaded module files (seen in list before):
syntax: module unload modulefile [modulefile ...]
[user@teazer ~]$ module unload intel/10.1.018 intel-mkl/10.0.4
Information about Modules Environment
For further information concerning the Modules environment please have a look at the man pages (man module, man modulefile). The sources and further documentation can be found at the SourceForge website.
Adapted from http://www.uibk.ac.at/zid/systeme/hpc-systeme by courtesy of the ZID HPC Team.