Using modules¶

HPC cluster systems typically have a large number of software packages installed. Often there will be several versions provided for a package where it will be necessary for a user to choose between them. Equally, two different packages may clash with each other: for example, the commands for Intel MPI and Open MPI would overlap if simultaneously installed.

On Apocrita we use the modules package to manage the user environment for the installed packages. This makes it simple to use different packages or switch between versions of the same package without conflicts.

Module structure¶

On Apocrita, we have provide two sets of modules:

Software compiled with Spack
Software manually installed or compiled

Both sets are visible by running module avail, but display in different sections. When viewing the available modules, ensure to check both sections.

Usage commands¶

This section documents common commands to manage your module environment on Apocrita using the module command, which is considered the long format of the command; The ml alias is the short form of the module command. See the ml man page (man ml) for example usage of this command, including shortcuts for loading and unloading modules.

The examples below use the long format of the module command.

Listing available modules¶

To view the list of all available modules, use the following command:

module avail

To view a list of all versions available for a specific package (i.e. MATLAB), use the following command:

module avail matlab

Loading modules¶

Customised Environments

While we encourage users to customise their Apocrita environment to make their workflow easier, please be aware that customisations which change the user's environment for example by setting variables in the ~/.bash_profile file, or by using python's pip to create a ~/.local folder, may cause problems with modules which can be difficult to troubleshoot.

To load a module into your environment to start using an application, use the following command:

module load <package>

For example, to load the default version of MATLAB, run module load matlab.

Listing loaded modules¶

To view the list of currently loaded modules, use the following command:

module list

Unloading modules¶

To unload a specific module, use the following command:

module unload <package>

For example, to unload the default version of MATLAB previously loaded into the environment, run module unload matlab.

Purging modules¶

To unload (purge) all loaded modules, use the following command:

module purge

Suppressing module messages¶

Suppressing module output

Suppressing output from the module commands can make output from your job scripts tidier, but it can potentially hide important diagnostic information about your work. Be sure to display the output from module commands when performing unfamiliar work, when looking at potential problems or seeking help from others.

Loading a module can result in a message being displayed, such as warnings about a module's behaviour or other modules being loaded to satisfy dependencies. Examples of these are given in the following sections.

To suppress these messages, the -s option can be used with the module command:

$ module load use.dev
--------------------------------------------------------------------------
Loading Development Modules

The modules in the development environment are for testing purposes only.
Modules may be removed without prior warning.
Please do not use these modules for production jobs.

--------------------------------------------------------------------------

$ module purge
$ module -s load use.dev
$

Naming conventions¶

All software available via modules on Apocrita follow the NAME/VERSION convention. Spack-generated modules also include the compiler used to build the software, and also if applicable, the MPI flavour and/or the CUDA library version. Any dependent modules will be automatically loaded to avoid runtime compatibility issues. See the dependencies section for more information about module dependencies.

The following examples explain the naming convention for modules generated by Spack:

app/1.1.0-gcc-12.2.0 - software named app version 1.1.0 that was compiled with GCC 12.2
app/1.1.0-openmpi-5.0.3-gcc-12.2.0 - software named app version 1.1.0 that was compiled with GCC 12.2 and OpenMPI 5.0.3
app/1.1.0-openmpi-5.0.3-cuda-12.4.0-gcc-12.2.0 - software named app version 1.1.0 that was compiled with GCC 12.2, OpenMPI 5.0.3 and CUDA 12.4.

Please contact us if you have any queries regarding our module naming standards.

Default versions¶

Environment modules also support the concept of default versions, to identify which module should be loaded if a version number is not provided (i.e. module load exampleApp, rather than module load exampleApp/1.2.3). The module avail output underlines the default version of each module.

The below example demonstrates how to load the default and non-default version of the MATLAB application (matlab module):

To load the default version, simply run module load matlab.

To load a non-default version, first run module avail matlab to find the available versions, and then run module load matlab/VERSION, substituting VERSION with the actual version number, matching the module name (for example: module load matlab/2024a to load MATLAB 2024a).

When adding a new version of an existing module, we may bump up the default version at our discretion (i.e. for backward-compatible applications), or stick with an older version. A primary use case for pinning an older version would be to avoid introducing incompatibilities or inconsistent results on a version not widely tested by several users of the cluster.

Dependencies¶

Some of the software provided by a module may in turn depend on software provided by another module. For example, an application compiled with MPI support will require a suitable MPI runtime when running the application. These cases may be handled by module dependencies.

On Apocrita, the module system is set up to load dependent modules automatically rather than requiring users to satisfy the dependencies first.

To demonstrate, let's load a HDF5 module:

$ module load hdf5
Loading hdf5/1.14.3-openmpi-5.0.3-gcc-12.2.0
  Loading requirement: openmpi/5.0.3-gcc-12.2.0

Here we see that loading the default version of HDF5 has also loaded an OpenMPI module - this is the same version of OpenMPI we used to compile HDF5, so we would expect there to be no runtime incompatibilities when running HDF5.

Conflicts¶

When two packages may cause problems when both made accessible, their modules will conflict with each other. This means that they cannot both be loaded. In particular, different versions of the same module may not be used at the same time:

When two packages are unable to be loaded into the environment concurrently (for example, two versions of the same package), their respective modules will conflict with each other, for example:

$ module load python/3.12.1-gcc-12.2.0
Loading python/3.12.1-gcc-12.2.0
  Loading requirement: openssl/3.3.0-gcc-12.2.0 sqlite/3.43.2-gcc-12.2.0

$ module load python/3.11.7-gcc-12.2.0
Loading python/3.11.7-gcc-12.2.0
  ERROR: Module cannot be loaded due to a conflict.
    HINT: Might try "module unload python/3.12.1-gcc-12.2.0" first.

In this example, to load the older version of Python, simply follow the hint and unload the previously loaded Python module, then load the desired version.

Development versions¶

When requesting a new version or software package you may be asked to test a development version before it is released. These test installations will be made available through development modules.

To access development modules you can load the use.dev module:

$ module load use.dev
$ module avail
---- /share/apps/rocky9/environmentmodules/apocrita-modules/dev ----
test/2.1.2      testing/2.3

$ module load test/2.1.2

On acceptance, the module will be moved to one of the production module sets, and the use.dev module will no longer be required to access that module.

Deprecated modules¶

Once a software package has been made available with a module in one of the production module sets, it will generally remain available for future use. However, in some circumstances, such as those related to security or performance problems, it may be necessary for modules to be deprecated ahead of eventual removal. A deprecated module will lose its default visibility and we recommend no further use of the module.

Deprecated modules, and installations, will usually have an alternative module available and in the first instance we recommend considering using this. Contact us if you experience problems using an alternative module.

To gain access to modules which have been deprecated you can load the use.deprecated module, but we recommend to not use this widely for production jobs, as deprecated modules may be deleted without warning.

Private modules¶

It is possible to have your own module files to enable you to install and manage specific versions of software without affecting existing installations.

This is mostly useful when you have very specific needs that will not be shared with other users, or if you would like to test a new version of an application before requesting it to be centrally installed by us.

Similarly to development modules, these private modules may be accessed by loading the module use.own.

Basic setup¶

Private modules are defined on a per-user level, likely stored in the user's home directory. In the below examples, we use ~/privatemodules as the target location, and we recommend that you also use this location if setting up private modules.

The following example describes how to use a private module for a self-compiled application named app, version 2.0.0:

# Create the location to store private modules
mkdir ~/privatemodules

# Create the application directory
mkdir ~/privatemodules/app

# using an editor (vim in this case), create a module file called "2.0.0"
# to represent the application version
vim ~/privatemodules/app/2.0.0

Add the contents as described in the below section.

Module Files¶

Modulefiles handle the path adjustments required for an application to work, the full syntax is defined in the MODULEFILE man page, which can be read with man modulefile.

The basics are covered by this example modulefile for python 3.4.3:

#%Module
#
# app 2.0.0 module file

proc ModulesHelp { } {
    puts stderr "\tAdds app 2.0.0 to your environment"
}

module-whatis "Loads app 2.0.0"

prepend-path PATH /path/to/the/application/binaries
prepend-path LD_LIBRARY_PATH /path/to/the/application/libraries
prepend-path C_INCLUDE_PATH /path/to/the/application/headers
prepend-path PKG_CONFIG_PATH /path/to/the/application/pkconfig_files
prepend-path MANPATH /path/to/the/application/man_pages

Edit the paths as appropriate above and note that not all variables will be applicable for all applications (i.e. library-only software may not require the PATH variable setting if it does not contain any binaries).

Usage¶

Once a module is configured in a modulefile and the software has been installed you can use private modules with the following commands:

$ module load use.own
$ module load app/2.0.0
$ module list
Currently Loaded Modulefiles:
 1) use.own   2) app/2.0.0