HPC/Module naming scheme 2016

Introduction

(*) Environment modules are the means by which software applications are made available to users. Using modules allows users and administrators to pick, by user or even by compute job, a desired or default version of an application from typically several current and historic versions persisting on the system. Older versions are kept to improve reproducibility of results, an important characteristic of scientific computing.

On Carbon, the naming scheme for environment modules* and the set of default modules being loaded for users has changed. This step was necessary because of increasing diversity and interdependence of applications, their modules, and the underlying operating system.

Nomenclature

The module handling now is as follows:

Modules have longer and more complex names.

Modules have several name components, separated by slash, "/", between the module's main name and the previously only other component, a version and build number. The number and specificity of added name components varies depending on the nature of the application.

• First, an abbreviated version number is used as first component after the name, typically either major version number, or in the form major.minor. This enables you to select the features and binary compatibility level that you need without necessarily having to give a complete module name all the way down to a build number.
• A compiler name component is typically present when an application was compiled on Carbon. Is is typically not present, because not needed, for applications installed from binary packages, notably commercial applications and, naturally, compilers themselves.
• An MPI name component is present when an application uses MPI for parallel computations.
• The added name components make it more apparent which set of tools was used to produce the given application, and thus usually which tools and modules are needed to satify the application's runtime requirements.

Example: Names for the FFT3 library module in the two schemes:

Scheme 1	Scheme 2
$ module -t avail fftw3 /opt/soft/modulefiles: fftw3/3.2.1-1 fftw3/3.2.2-1 fftw3/3.3-1 fftw3/3.3.2-1 fftw3/3.3.2-4 /opt/intel/modulefiles: /usr/share/Modules/modulefiles: /etc/modulefiles:	$ module -t avail fftw3 /opt/apps/M/x86_64/EL6: /opt/apps/M/x86_64/EL: fftw3/3.3/impi-5/intel/3.3.4-10 # uses Intel-MPI fftw3/3.3/intel/3.3.2-1 # build without MPI fftw3/3.3/openmpi-1.10/intel/3.3.4-4 # uses OpenMPI fftw3/3.3/openmpi-1.4/intel/3.3.2-4 # … fftw3/3.3/openmpi-1.4/intel/3.3.4-2 fftw3/3.3/openmpi-1.8/intel/3.3.4-3 /usr/share/Modules/modulefiles: /etc/modulefiles:

• The -t option of the module avail command shows the output in "terse" form, one entry per line. Lines ending in ":" are the search directories traversed, as defined by module use dirname statements.

A set of default modules is not loaded by the system

• Users must load all applicable modules in their shell setup files or job files, in suitable order.

This is born of necessity because still useful older applications were compiled with older MPI versions (typically OpenMPI-1.4) which partially interfere with newer versions (OpenMPI-1.8, 1.10, or Intel-MPI-5.x). In particular, commands like mpirun and mpifort are typically provided by each MPI flavor.

Modules do not load dependencies.

• A module does not implicitly load other modules that it might depend on, such as modules for compilers, an MPI flavor, or specialized libraries. While a minor burden at first, this will make operations more explicit and predictable.
• You need to recognize error messages issued by module load when a dependent module is found missing.
• You must load prerequisite modules yourself before modules that depend on them. The typical order is: compiler, MPI, other (dynamic) libraries, and finally your intended module.
• Different MPI flavors can (and may have to) be loaded at the same time. In this situation, MPI commands will have to be called by absolute path, e.g. $OPENMPI_HOME/bin/mpirun …

Again, this is because the system is less homogeneous than in the past: it is impractical or even impracticable to upgrade and maintain applications at a single "one true" MPI implementation.

The set of available modules differs between naming schemes.

• Newer modules will only be made available in the newer scheme 2.

The set of available modules differs between operating system releases.

• Applications are typically compiled and installed on a node that runs a recent OS release. Some compiled applications are backwards-compatible with a previous OS release, and will be made available there. If we find that is not the case, the respective module will be visible only on the suitable OS release.

Configuration files used

You have files: .bashrc and …		Remark	CentOS-5 uses:		CentOS-6 uses:
			files	module names	files	module names
–	–	Starting situation.	.bashrc only	scheme 1	.bashrc only	scheme 2
–	.modules-2	Switch over, recommended.	.modules-2 and .bashrc	scheme 2	.modules-2 and .bashrc
.modules-1	–	Only recommended during transition.	.modules-1 and .bashrc	scheme 1	.bashrc only
.modules-1	.modules-2	For advanced users.	.modules-1 and .bashrc	scheme 1	.modules-2 and .bashrc

New-style module names

You may need to adapt the module names that you placed in your shell startup and job files to the new more hierarchical scheme. For most modules, with exceptions show below, the leading name component (the part before any "/") is the same in the old and new naming schemes. What always differs are the name parts after the first slash.

Name change rules

• To use the latest or automatically selected version of a package, remove version numbers from old-style module names of the formpackagename/version, leaving only packagename. This is the recommended approach, as you will automatically benefit from future updates and maintenance builds.
• To insist on a specific version for a package in new style names:
  • Inspect the available flavors and versions (some older modules were not migrated):

    module avail packagename.

  • Choose the new-style name up to the desired specificity. You may leave out trailing name or directory parts.

    For instance, instead of vasp5/5.3/openmpi-1.4/intel/5.3.3p3-mkl-3 you may write vasp5/5.3/openmpi-1.4 or vasp5/5.3, letting the system choose the versions for MPI and compiler that are chosen as defaults at a given time.

Renamed modules

For the following modules the newer naming convention allowed for and thus uses more consistent names:

OLD		NEW
-------------------------------------
asap3		asap/3

ase2		ase/2
ase3		ase/3

g09		gaussian/09
GaussView	gaussview  (lowercase)

The modules fftw3 and vasp5 did not change name due to more entrenched usage in the package itself, Unix group names, and compilation dependencies.

Determining default module versions

The module avail command under CentOS-6 no longer includes the marker "(default)" when one has been set in a .version file. I am not sure if this is a bug or by design, but the change certainly makes the output more consistent.

To determine which module will be loaded when an abbreviated name is used, I recommend to inspect the first relevant line in the output of one of these commands:

module show name
module help name

Customizing module selection by OS release

Configure in .bashrc

To switch over to new-style module names entirely, on both CentOS releases, you could continue using only ~/.bashrc. To do this:

• Tell CentOS-5 nodes to offer you the new-style module catalog instead of the old one. To do so, simply create an empty customization file:

touch ~/.modules-2

• Then, apply the changes shown in the #Name change rules section above:

vi .bashrc
# or:
nano .bashrc

Use a text editor of your choice, such as nano or vi.

Configure in dedicated files

It is perhaps cleaner to perform the module selection in files dedicated for each CentOS release, so that you can make adjustments independently.

To migrate your existing configuration, manage the following files:

.bashrc
.modules-1
.modules-2

Use a helper application to get you started:

modules-migrate

Here is an example of what you might see:

$ modules-migrate  

This application will help you adapt your environment-modules customizations to
Carbon's naming scheme updated for 2016.

Your module commands will be moved from .bashrc into files .modules-1 and .modules-2,
so that a node's operating system sees only module names in the appropriate
scheme.

The file text manipulation done here is fairly basic. You will have
opportunities to review and edit your changes.

Continue? [Y/n] y
======================================================================
diff of changes from .bashrc:
======================================================================

19,21c19,21
< module load vasp5
< module load g09/D.01.x86_64-2
< module load GaussView
---
> #	module load vasp5
> #	module load g09/D.01.x86_64-2
> #	module load GaussView
Accept changes in .bashrc? [Y/n] y
Putting in place ...
`.bashrc.tentative' -> `.bashrc'

Opening text editor for .modules-1.tentative .modules-2.tentative ...
Continue? [Y/n] y

(editor session omitted.)

~
~
~
======================================================================
Review: "module" lines in new files:                                                                                                                 ======================================================================

==> .modules-1.tentative <==
# Carbon modules initialization for v1-style module names, used on CentOS-5.
# To stop using v1-style modules, remove this file.
#
# Initially extracted from .bashrc by modules-migrate .
#
# This file is in TCL syntax.

module load vasp5
module load g09/D.01.x86_64-2
module load GaussView

# vim:syntax=tcl:

==> .modules-2.tentative <==
# Carbon modules initialization for v2-style module names, used on CentOS-6.
# Also used on CentOS-5 if .modules-1 does not exist.
#
# Initially extracted from .bashrc by modules-migrate .
#
# This file is in TCL syntax.

module load vasp5
module load gaussian/09/D.01.x86_64-2
module load gaussview

# vim:syntax=tcl:
======================================================================
Accept changes (a), Edit (e), or Quit (q)? a
Renaming files ...
`.modules-1.tentative' -> `.modules-1'
`.modules-2.tentative' -> `.modules-2'
Done.

• Test – same as in previous section.