The modules system¶

Since October 2016 we are using a set of tools called EasyBuild and Lmod to install and manage software packages and their accompanying modules.

The modules are installed hierarchically in such a way that not all modules are visible initially.

The EasyBuild Hierarchial Module Naming System (HMNS for short) ensures that the correct libraries are available for a specific application while keeping the output of module avail to a minimum. See the section on using Lmod further down on this page, for more information.

Verify that your environment works

Before trying anything else please check if your environment is correctly setup to work on our systems. Version number changes as we make updates.

Check the output of this command:

module --version 2>&1 | head -n 2

On Kebnekaise it should show something like:

Modules based on Lua: Version 8.6.2  2021-12-21 19:39 -06:00

if so, then your enviroment is working. If not, send a mail to support@hpc2n.umu.se asking for help. Please make sure to state your username.

Compilers and compiler toolchains¶

The module hierarchy is based on compiler toolchains.

A compiler toolchain is a bundling of software that makes up a complete environment for compiling software (or using a specific prebuilt software), usually including a compiler suite, an MPI version, BLAS, LAPACK, ScaLapack, and FFTW versions.

It may also be just a compiler, like GCC alone.

An example of a compiler toolchain is foss, which is based on the GCC compiler.

$ module add foss/2019b
$ module list

Currently Loaded Modules:
  1) snicenvironment (S)   8) XZ/5.2.4           15) OpenMPI/3.1.4
  2) systemdefault   (S)   9) libxml2/2.9.9      16) OpenBLAS/0.3.7
  3) GCCcore/8.3.0        10) libpciaccess/0.14  17) FFTW/3.3.8
  4) zlib/1.2.11          11) hwloc/1.11.12      18) ScaLAPACK/2.0.2
  5) binutils/2.32        12) libevent/2.1.11    19) foss/2019b
  6) GCC/8.3.0            13) PMIx/3.1.4
  7) numactl/2.0.12       14) UCX/1.6.1

  Where:
   S:  Module is Sticky, requires --force to unload or purge

There are multiple toolchains, each one based on one of the compiler suites, GCC, icc/ifort, or others. You can always find an updated list with module avail

Available toolchains include:

• GCC: GCC only
• foss: GCC, OpenMPI, OpenBLAS/LAPACK, FFTW, ScaLAPACK
• fosscuda: gompic, OpenBLAS/LAPACK, FFTW, ScaLAPACK
• gcccuda: GCC and CUDA
• gomkl: GCC, OpenMPI, IntelMKL
• gompi: GCC, OpenMPI
• gompic: GCC, OpenMPI, CUDA
• iccifort: icc, ifort
• iccifortcuda: icc, ifort, CUDA
• iimpi: icc, ifort, IntelMPI
• imkl: icc, ifort, IntelMKL
• intel: icc, ifort, IntelMPI, IntelMKL
• intelcuda: intel, CUDA
• iompi: icc, ifort, OpenMPI

The main toolchains to use are foss, intel, fosscuda, intelcuda. More toolchains may be available in time.

By doing module spider TOOLCHAIN for a specific TOOLCHAIN, you will get a list of available versions.

How does the Hierarchial Module Naming System work¶

Initially you only see a smaller set of modules when doing module avail, these modules only depend on the libraries installed with the operating system and those modules which can be loaded without prerequisite modules.

When loading one of the compiler toolchains you get access to modules built with that specific toolchain.

What happens is that the toolchain module adds another module path which contains some modules built with that specific toolchain.

How to find a specific software¶

Since not every module is visible initially you can not use the module avail command to find software like GROMACS, NAMD, VASP, etc.

To find software hidden by the hierarchy you should use the module spider command.

$ module spider

----------------------------------------------------------------------------------------
The following is a list of the modules and extensions currently available:
----------------------------------------------------------------------------------------
  ABySS: ABySS/2.2.5
    Assembly By Short Sequences - a de novo, parallel, paired-end sequence assembler

  ACTC: ACTC/1.1
    ACTC converts independent triangles into triangle strips or fans.

  ADGofTest: ADGofTest/0.3 (E)

  AICcmodavg: AICcmodavg/2.3-0 (E), AICcmodavg/2.3-1 (E)

  ALDEx2: ALDEx2/1.20.0 (E), ALDEx2/1.26.0 (E)

  ALL: ALL/1.30.0 (E), ALL/1.36.0 (E)

  AMGX: AMGX/2.2.0-CUDA-11.3.1
    Distributed multigrid linear solver library on GPU

...

Setting up the environment using Lmod¶

To set up your environment for using a particular (set of) software package(s), you can use the modules that are provided centrally.

Interacting with the modules is done via Lmod, using the module command or the handy shortcut command ml.

Module command cheatsheet¶

Examples

$ module spider fosscuda

----------------------------------------------------------------------------
fosscuda:
----------------------------------------------------------------------------
  Description:
    GCC based compiler toolchain __with CUDA support__, and including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK.

  Versions:
     fosscuda/2019a
     fosscuda/2019b
     fosscuda/2020a
     fosscuda/2020b

----------------------------------------------------------------------------
  For detailed information about a specific "fosscuda" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider fosscuda/2020b
----------------------------------------------------------------------------

$ module spider fosscuda/2019b

----------------------------------------------------------------------------
  fosscuda: fosscuda/2019b
----------------------------------------------------------------------------
    Description:
      GCC based compiler toolchain __with CUDA support__, and including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK.


    This module can be loaded directly: module load fosscuda/2019b

    Help:
      Description
      ===========
      GCC based compiler toolchain __with CUDA support__, and including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK.


      More information
      ================
       - Homepage: (none)

$ module load fosscuda/2019b
$

$ module list

Currently Loaded Modules:
  1) snicenvironment (S)   8) numactl/2.0.12     15) UCX/1.6.1-CUDA-10.1.243
  2) systemdefault   (S)   9) XZ/5.2.4           16) OpenMPI/3.1.4
  3) GCCcore/8.3.0        10) libxml2/2.9.9      17) OpenBLAS/0.3.7
  4) zlib/1.2.11          11) libpciaccess/0.14  18) FFTW/3.3.8
  5) binutils/2.32        12) hwloc/1.11.12      19) ScaLAPACK/2.0.2
  6) GCC/8.3.0            13) libevent/2.1.11    20) fosscuda/2019b
  7) CUDA/10.1.243        14) PMIx/3.1.4

  Where:
   S:  Module is Sticky, requires --force to unload or purge

$ module spider Python/3.9.5

----------------------------------------------------------------------------------------------------------------------------------------------------------
  Python: Python/3.9.5
----------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Python is a programming language that lets you work more quickly and integrate your systems more effectively.


    You will need to load all module(s) on any one of the lines below before the "Python/3.9.5" module is available to load.

      GCCcore/10.3.0

    This module provides the following extensions:

       alabaster/0.7.12 (E), appdirs/1.4.4 (E), asn1crypto/1.4.0 (E), atomicwrites/1.4.0 (E), attrs/21.2.0 (E), 
       Babel/2.9.1 (E), bcrypt/3.2.0 (E), bitstring/3.1.7 (E), blist/1.3.6 (E), CacheControl/0.12.6 (E), 
       cachy/0.3.0 (E), certifi/2020.12.5 (E), cffi/1.14.5 (E), chardet/4.0.0 (E), cleo/0.8.1 (E), click/7.1.2 (E), 
       clikit/0.6.2 (E), colorama/0.4.4 (E), crashtest/0.3.1 (E), cryptography/3.4.7 (E), Cython/0.29.23 (E), 
       decorator/5.0.7 (E), distlib/0.3.1 (E), docopt/0.6.2 (E), docutils/0.17.1 (E), ecdsa/0.16.1 (E), 
       filelock/3.0.12 (E), flit-core/3.2.0 (E), flit/3.2.0 (E), fsspec/2021.4.0 (E), future/0.18.2 (E), 
       html5lib/1.1 (E), idna/2.10 (E), imagesize/1.2.0 (E), importlib_metadata/4.0.1 (E), iniconfig/1.1.1 (E), 
       intervaltree/3.1.0 (E), intreehooks/1.0 (E), ipaddress/1.0.23 (E), jeepney/0.6.0 (E), Jinja2/2.11.3 (E), 
       joblib/1.0.1 (E), jsonschema/3.2.0 (E), keyring/21.8.0 (E), keyrings.alt/4.0.2 (E), liac-arff/2.5.0 (E), 
       lockfile/0.12.2 (E), MarkupSafe/1.1.1 (E), mock/4.0.3 (E), more-itertools/8.7.0 (E), msgpack/1.0.2 (E), 
       netaddr/0.8.0 (E), netifaces/0.10.9 (E), nose/1.3.7 (E), packaging/20.9 (E), paramiko/2.7.2 (E), 
       pastel/0.2.1 (E), pathlib2/2.3.5 (E), paycheck/1.0.2 (E), pbr/5.6.0 (E), pexpect/4.8.0 (E), pip/21.1.1 (E), 
       pkginfo/1.7.0 (E), pluggy/0.13.1 (E), poetry-core/1.0.3 (E), poetry/1.1.6 (E), psutil/5.8.0 (E), 
       ptyprocess/0.7.0 (E), py/1.10.0 (E), py_expression_eval/0.3.13 (E), pyasn1/0.4.8 (E), pycparser/2.20 (E), 
       pycrypto/2.6.1 (E), Pygments/2.9.0 (E), pylev/1.3.0 (E), PyNaCl/1.4.0 (E), pyparsing/2.4.7 (E), 
       pyrsistent/0.17.3 (E), pytest/6.2.4 (E), python-dateutil/2.8.1 (E), pytoml/0.1.21 (E), pytz/2021.1 (E), 
       regex/2021.4.4 (E), requests-toolbelt/0.9.1 (E), requests/2.25.1 (E), scandir/1.10.0 (E), SecretStorage/3.3.1 (E), 

...

$ ml GCCcore/10.3.0 Python/3.9.5
$

$ ml spider gromacs

----------------------------------------------------------------------------------------------------------------------------------------------------------
  GROMACS:
----------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      GROMACS is a versatile package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles.

     Versions:
        GROMACS/3.3.3-cphmd-1.3
        GROMACS/3.3.3-cphmd-1.3-1lpernode
        GROMACS/3.3.3-cphmd-1.4
        GROMACS/4.0.7-ionstr
        GROMACS/5.1.4
        GROMACS/2016.x-drude-20180214-g3f7439a
        GROMACS/2016.x-drude-20220117-g78fe3d1e
        GROMACS/2016.x-drude-20220120-ge35ae4e2
        GROMACS/2016.4
        GROMACS/2018.8
        GROMACS/2019
        GROMACS/2019.4-PLUMED-2.5.4
        GROMACS/2021
        GROMACS/2022.4

----------------------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "GROMACS" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider GROMACS/2022.4
----------------------------------------------------------------------------------------------------------------------------------------------------------

Advanced information about modules¶

In the current module naming scheme, each module name consists of two parts:

• the part before the first /, corresponding to the software name; and
• the remainder, corresponding to the software version and a possible version suffix

For example, the module name iccifort/2017.0.098-GCC-5.4.0-2.26 would set up the environment for using icc and ifort version 2017.0.098, with the GCC/5.4.0-2.26 compiler toolchain.

The (D) indicates that this particular version of the module is the default, but we strongly recommend to not rely on this as the default can change at any point. Usually, the default will point to the latest version available.

Conflicting modules¶

It is important to note that only modules that are compatible with each other can be loaded together. In particular, modules must be installed either with the same toolchain as the modules that are already loaded, or with a compatible (sub)toolchain.

For example, once you have loaded one or more modules that were installed with the foss/2019b toolchain, all other modules that you load should have been installed with the same toolchain. Due to the nature of the hierarchial module naming system, only modules built with that same toolchain are actually visible, thus reducing the probablility of making a mistake.

In addition, only one single version of each software package can be loaded at a particular time. For example, once you have loaded the module Python/3.9.5, you can not load a different version of Python in the same session/job script; neither directly, nor indirectly as a dependency of another module you want to load.

Module collections¶

If you have a set of modules that you need to load often, you can save these in a collection.

$ ml GCC/11.2.0  OpenMPI/4.1.1 SciPy-bundle/2021.10 matplotlib/3.4.3

$ ml save my-collection
Saved current collection of modules to: my-collection

$ ml restore my-collection
Restoring modules to user's my-collection

$ ml savelist
Named collection list :
 1) my-collection

Lmod vs Tcl-based environment modules¶

In October 2016, we switched to Lmod as a modules tool, a modern alternative to the outdated & no longer actively maintained Tcl-based environment modules tool.

Consult the Lmod documentation web site for more information.

Benefits

• significantly more responsive module commands, in particular module avail
• a better and easier to use interface (e.g. case-insensitive avail, the ml command, etc.)
• additional useful features, like defining & restoring module collections
• drop-in replacement for Tcl-based environment modules (existing Tcl module files do not need to be modified to work)
• module files can be written in either Tcl or Lua syntax (and both types of modules can be mixed together)

Key differences

The switch to Lmod (or more specifically the use of EasyBuild for building and installing software) means that we have a completely different module naming system.

There are a couple of minor differences between Lmod & the old modules tool you should be aware of, if you are used to Tcl-based modules:

• module conflicts are strictly enforced, meaning conflicting modules cannot be loaded together
• module purge does not unload the snicenvironment and systemdefault modules

Using module purge effectively resets your environment to a pristine working state, i.e. the snicenvironment module stays loaded after the purge.

(This text is in large parts copied from https://www.vscentrum.be/cluster-doc/software/modules-lmod with their consent)