Accessing Software using Distributed Environment Modules

Introduction

This page covers use of the module command in the OSG computing environment.

Environment modules have historically been used in HPC environments to provide users with an easy way to access different versions of software and to access various libraries, compilers, and software (c.f. the wikipedia reference). OSG has implemented a version based on Lmod to provide the typical module commands on any site in the OSG. You can test workflows on the OSG Connect login node and then submit the same workflow without any changes.

Using modules on OSG Connect

Use module avail to see available software applications and libraries:

$ module avail

ANTS/1.9.4                  gcc/4.6.4                  pbsuite/14.9.9
ANTS/2.1.0           (D)    gcc/4.8.1                  pcre/8.35
Julia/0.6.0                 gcc/4.9.2           (D)    pegasus/4.4.2-image_tools
MUMmer3.23/3.23             gcc/4.9.3                  pegasus/4.5.3
OpenBUGS/3.2.3              gcc/6.2.0                  pegasus/4.6.0dev
OpenBUGS-3.2.3/3.2.3        gd/2.1.1                   pegasus/4.6.0cvs
R/3.1.1              (D)    gdal/2.0.0                 pegasus/4.6.0
R/3.2.0                     geant4/9.4p02              pegasus/4.6.1dev
R/3.2.1                     geant4/10.02               pegasus/4.6.1
R/3.2.2                     geant4/10.3p01      (D)    pegasus/4.7.0
R/3.3.1                     geos/3.4.2                 pegasus/4.7.1
R/3.3.2                     gfal/7.20                  pegasus/4.7.3
RAxML/8.2.9                 git/1.9.0                  pegasus/4.7.4             (D)
SeqGen/1.3.3                glpk/4.54                  phenix/1.10
Shelx/2015                  gmp/6.0.0                  poppler/0.24.1            (D)
SitePackage                 gnome_libs/1.0             poppler/0.32
SparseSuite/4.2.1           gnuplot/4.6.5              povray/3.7
ViennaRNA/2.2               graphviz/2.38.0            proj/4.9.1
abyss/2.0.2                 grass/6.4.4                proot/2014
ant/1.9.4                   gromacs/4.6.5              protobuf/2.5
apr/1.5.1                   gromacs/5.0.0       (D)    psi4/0.3.74
aprutil/1.5.3               gromacs/5.0.5.cuda         python/2.7                (D)
arc-lite/2015               gromacs/5.0.5              python/3.4
atlas/3.10.1                gromacs/5.1.2-cuda         python/3.5.2
atlas/3.10.2         (D)    gsl/1.16                   qhull/2012.1
autodock/4.2.6              gsl/2.3             (D)    root/5.34-32-py34
bedtools/2.21               hdf5/1.8.9                 root/5.34-32
binutils/2.26               hdf5/1.8.12-cxx11          root/6.06-02-py34         (D)
blasr/1.3.1                 hdf5/1.8.12                rosetta/2015
blast                       hdf5/1.8.13-cxx11          rosetta/2016-02
blender                     hdf5/1.8.13         (D)    rosetta/2016-32           (D)
boost/1.50.0                healpix/3.30               ruby/2.1
boost/1.56                  hisat2/2.0.3-beta          rucio/1.6.6
boost/1.57.0                hmmer/3.1                  sac/10.1.6a
boost/1.62.0-cxx11          igraph/0.7.1               saga/2.2.0
boost/1.62.0         (D)    imagemagick/7.0.2          samtools/0.1.17
bowtie/2.2.3                intelMKL/11.3.0.109        samtools/1.3.1            (D)
bowtie/2.2.9         (D)    ipopt/3.12.6               sca/10.1.6a
bwa/0.7.12                  jasper/1.900.1             scons/2.3.4
bwa/2014             (D)    java/7u71                  sdpa/7.3.8
bzip2/1.0.6                 java/8u25           (D)    serf/1.37
canopy/1.4.1                jpeg/6b                    settarg/5.6.2
casino/2.13.211             jpeg/9a             (D)    shelx/2015
cblosc/1.7.1                julia/0.6.0                shrimp/2.2.3
ccp4/2015                   lammps/2.0                 siesta/3.2
cctools/4.4.2               lammps/15May15      (D)    simbody/3.5.3
cctools/5.2.3               lapack/3.5.0               snappy/1.1.3
cctools/5.4.7               lapack/3.6.1        (D)    sqlite/3.8.11.1
cctools/6.0.7        (D)    libXpm/3.5.10              sra/2.5.4
cdo/1.6.4                   libgfortran/4.4.7          sra/2.8.0                 (D)
cfitsio/3.37                libtiff/4.0.4              stashcp/2.2
circos/0.68                 llvm/3.6                   stashcp/2.5
clhep/2.1.0.1               llvm/3.7                   stashcp/2.6
clhep/2.2.0.8               llvm/3.8.0          (D)    stashcp/3.0
clhep/2.3.1.0               lmod/5.6.2                 stashcp/3.1
clhep/2.3.1.1               madgraph/2.1.2             stashcp/3.2
clhep/2.3.4.4        (D)    madgraph/2.2.2      (D)    stashcp/3.3
cmake/3.0.1                 matlab/2013b               stashcp/3.4
cmake/3.4.1                 matlab/2014a               stashcp/4.0
cmake/3.8.0          (D)    matlab/2014b               stashcp/4.0.1
connect-client/0.2.1        matlab/2015a               stashcp/4.1.0
connect-client/0.3.0        matlab/2015b               stashcp/4.1.1
connect-client/0.4.0        matlab/2016b        (D)    stashcp/4.2.0
connect-client/0.5.3 (D)    mercurial/1.9.1            stashcp/4.2.1
cp2k/2.5.1                  mixmodlib/3.1              stashcp/4.3.0
cpan/perl-5.10              mono/4.2.1                 stashcp/4.3.1             (D)
cufflinks/2.2.1             mothur/1.39.0              stringtie/1.1.2
curl/7.37.1                 mpc/1.0.3                  stringtie/1.2.2           (D)
dakota/6.4.0                mpfr/3.1.3                 subversion/1.8.10
dmtcp/2.5.0                 mplayer/1.1                sundials/2.5
ectools                     mrbayes/3.2.2              swift/0.94.1
eemt/0.1                    muscle/3.8.31              swift/0.96.2              (D)
elastix/2015                mysql/5.1.73               tassel/5.0
entropy/2017.03.16          namd/2.9                   tcl/8.6.2
espresso/5.1                namd/2.10.cuda             tcsh/6.20.00
espresso/5.2         (D)    namd/2.10           (D)    tophat/2.0.13
ete2/2.3.8                  nco/4.3.0                  tophat/2.1.1              (D)
expat/2.1.0                 netcdf/4.2.0               transabyss/1.5.5
ffmpeg/0.10.15              ngsTools/2017.03.16        tutorial/1.0
ffmpeg/2.5.2         (D)    octave/3.8.1               uclust/2.22
fftw/3.3.4-gromacs          openbabel/2.3.2            udunits/2.2.17
fftw/3.3.4           (D)    opencv/2.4.10              unixodbc/2.3.2
fiji/2.0                    opensees/6482              valgrind/3.10
fpc/2.6.4                   opensim/3.3                vmd/1.9.1
freesurfer/5.1.0            orca/3.0.3                 wget/1.15
freesurfer/5.3.0            papi/5.3.2                 wxgtk/3.0.2
freesurfer/6.0.0     (D)    pari/2.7.5                 xrootd/4.1.1
freetype/2.5.5              pax/evan-testing           xrootd/4.2.1              (D)
fsl/5.0.8                   pax/4.5.0                  xz/5.2.2
gamess/2013                 pax/4.6.1                  zlib/1.2.8
gate/7.2                    pax/4.9.1
gcc/4.6.2                   pax/4.11.0          (D)

Where:
(D):  Default Module

Note that currently, most of our login and compute nodes are based on RHEL 6. RHEL 7 hosts can be loaded with Singularity and the software catalog availalbe for that operating system is not as extensive, but growing:

bowtie/2.2.9       gsl/2.3           (D)    mpfr/3.1.3             sra/2.5.4
cmake/3.8.0        java/8u25                pegasus/4.7.3          sra/2.8.0       (D)
cufflinks/2.2.1    lapack/3.6.1             pegasus/4.7.4   (D)    stashcp/3.1
gcc/4.9.2          libgfortran/4.4.7        python/2.6.9           stringtie/1.2.2
gmp/6.0.0          mixmodlib/3.1            samtools/1.3.1         tophat/2.1.1
gsl/1.16           mpc/1.0.3                sqlite/3.8.11.1

In order to load a module, you should run module load [modulename].

For example:

$ module load python/2.7

Some modules require other modules to be loaded. If that occurs then you'll get an error message indicating this:

$ module load all-pkgs

Lmod has detected the following error: Cannot load module "all-pkgs" without these modules loaded:
atlas, lapack


While processing the following module(s):

Module fullname  Module Filename
---------------  ---------------
all-pkgs         /cvmfs/oasis.opensciencegrid.org/osg/modules/modulefiles/python/2.7.7/all-pkgs.lua

In addition, some modules such as the all-pkgs for python won't be available until the relevant packages are loaded.

For example:

$ module avail

---- /cvmfs/oasis.opensciencegrid.org/osg/modules/modulefiles/Core -----
atlas      fftw/fftw-3.3.4-gromacs    lapack              lmod/5.6.2 (D)    
blast      gromacs/4.6.5              lmod/SiteHook       namd/2.9          
blender    jpeg                       lmod/SitePackage    python/2.7 (D)
python/3.4 settarg/5.6.2

Where:
(D):  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

$ module load python/2.7
$ module avail

--- /cvmfs/oasis.opensciencegrid.org/osg/modules/modulefiles/python/2.7.7 ---
all-pkgs

---- /cvmfs/oasis.opensciencegrid.org/osg/modules/modulefiles/Core -----
atlas      fftw/fftw-3.3.4-gromacs    lapack              lmod/5.6.2 (D)    
blast      gromacs/4.6.5              lmod/SiteHook       namd/2.9          
blender    jpeg                       lmod/SitePackage    python/2.7 (D)
python/3.4 settarg/5.6.2
Where:
(D):  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

To unload a package and remove it from the environment, use `module unload [modulename]`.

For example:

$ module unload python/2.7

To see currently loaded modules, use module list.

For example:

$ module list
Currently Loaded Modules:
1) python/2.7

Finally, module help will give you more detailed information.

Using environment modules in jobs

Not all resources available through OSG Connect support distributed environment modules. In order to make sure that the jobs you submit run on resources that do support distributed environment modules, you will need to add the following condition to the requirements in your HTCondor job submission file. E.g. :

Requirements = (HAS_MODULES =?= true) && (OSGVO_OS_STRING == "RHEL 6") && (OpSys == "LINUX")

or

Requirements = [Other requirements ] && (HAS_MODULES =?= true) && (OSGVO_OS_STRING == "RHEL 6") && (OpSys == "LINUX")

if you already have other requirements specified and need to append the OASIS requirement.

Jobs submitted with the HTCondor requirement given above will automatically have the module system set up for them when run. So the only change needed to job scripts would be to add module load commands to load the appropriate modules.

Available Software

A list of currently installed software is available here. If your software is not already listed, you can request it.

Using modules outside of OSG Connect

In order to use distributed environment modules outside of OSG Connect, you'll need to load the modules code manually from OASIS. You can do this by running

$ source /cvmfs/oasis.opensciencegrid.org/osg/modules/lmod/current/init/SHELL

where SHELL is the shell that you are using (this is usually bash).

 

This page was updated on Dec 19, 2018 at 07:01 from setting-up-and-using-environment-modules.md.