Using Software Modules

CTIT computing lab uses Environment Modules for managing software. The modules system permits us to set up the shell environment to make running and compiling software easier. It also allows us to make available many software packages and libraries that would otherwise conflict with one another.

When you first log into the cluster, you will be entered into a very barebones user environment with minimal software available. The module system is a script based system used to manage the user environment and to “activate” software packages. In order to access software that is installed on the cluster, you must first load the corresponding software module.

Local Software

The directory /software contains sub-directories where the separate groups
can install software.

Software installation policy.

  • If you need an addition standard package, ask a contact person to arrange for the package to be installed.
  • If you need a package that can be installed in user space then install it yourself. If others will be using the package ask for a directory in /software otherwise use your home directory.
    Tensorflow v1.6+ and Keras require the avx and/or avx2 support, use either the --constraint=avx/avx2 or the main_avx or main_avx2 partitions.
  • If the package is not standard and requires root privileges to install:
    • If it has to run on the Hadoop part: contact Jan Flokstra.
    • If it has to run on the HPC part: contact Geert Jan Laanstra.
  • You are not supposed to contact ICTS without talking to the contact persons.

Installed Software

The preinstalled software on the cluster is a growing list containing the following software :
  • opencv, boost, python2.7/3.5 (ubuntu 16.04), python2.7/3.8 (ubuntu 20.04) etc...

python packages in your home folder.

In order to install your own set of python (conda) packages within your home folder run one of the following commands (depending on your python version).

pip install --user <PackageName>
pip3 install --user <PackageName>
conda install --use-local <PackageName>

The default python versions are 2.7.12 and 3.5.2 (for ubuntu 16.04), 2.7.18rc1 and 3.8.2 (for ubuntu 20.04), for different versions you need to load the correct module file.

Optional Software

Additional software that is not directly available, can be added to your environment using the Environment Modules.
To load this software on login, just add the following command to your .bashrc file:

module load <ModuleFolder>/<ModuleName>

To list the available modules run the following command:

module avail

Below is a list of the available optional software.
Organization Name Version <ModuleFolder> <ModuleName> Examples
Anaconda Python/R Data Science Platform (python 2.7/3.7) 2019.03 anaconda2/anaconda3 2019.03 Anaconda or Jupyter
Comsol Comsol Multiphysics 5.3a, v5.5 comsol vx.x Comsol
nVidia Cuda Toolkit 8.0, 9.0, 9.2, 10.0, 10.1 nvidia cuda-x.x module load nvidia/cuda-10.1
nVidia Deep Neural Network library 5.1, 6.0, 7.0, 7.1, 7.3, 7.4, 7.5, 7.6 nvidia cuda-x.x_cudnn-x.x module load nvidia/cuda-10.1_cudnn-7.6
nVidia Collective Communications Library 1.3, 2.0, 2.3, 2.4 nvidia cuda-x.x_nccl_x.x module load nvidia/cuda-9.2_nccl-2.3
Mathworks Matlab r2017a, r2018a, r2018b, r2019a mathworks matlab_rxxxxy Matlab
Mathworks Matlab Compiler Runtime r2017a, r2018a, r2018b, r2019a mathworks mcr_rxxxxy Matlab
Microsoft Visual Studio Code 1.33.1 vscode 1.33.1 module load vscode/1.33.1
Python Python Software Foundation 3.7.3 python 3.7.3 module load python/3.7.3
r-project The R Project for Statistical Computing 3.6.0 R 3.6.0 module load R/3.6.0
Wolfram Mathematica v11.1 wolfram mathematica_11.1 module load wolfram/mathematica_11.1

FMT software.

The FMT group also uses environment modules to manage its software.
To use these modules add the following commands to your .bashrc file:

module use /software/fmt/other/modules
module use /software/fmt/easybuild/modules/all

Other modules.

The other hierarchy contains modules with software that needs to be installed by hand.
Currently, it contains installations of
  • Java (Various SE versions)
  • CADP

Easybuild modules.

EasyBuild is a software build system. The provided
configurations allow build of a long list of HPC software, including various compiler
tool chains.

When a package can be built automatically, adding it as a module here has the great advantage
that you can share the build configuration with other people. This directory contains modules for:
  • mCRL2
  • LTSmin

The configs for these are archived at https://github.com/utwente-fmt/easyconfigs.