HPC/Module naming scheme, 2008 edition: Difference between revisions

From CNM Wiki
< HPC
Jump to navigation Jump to search
(Created page with "== Module Conventions on Carbon, 2008 edition == * Most application software is installed under <code>'''/opt/soft'''</code> * Package directories are named <code>''' ''name-v...")
 
 
(12 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Module Conventions on Carbon, 2008 edition ==
== Module naming scheme ==
* Most application software is installed under <code>'''/opt/soft'''</code>
* Most application software is installed under <code>'''/opt/soft'''</code>
* Package directories are named <code>''' ''name-version-build'' '''</code>, e.g. <code>/opt/soft/jmol-12.1.37-1</code>.
* Package directories are named <code>''' ''name-version-build'' '''</code>, e.g. <code>/opt/soft/jmol-12.1.37-1</code>.
* Module names are organized by a mostly version-less name, with the version following after a slash: <code>''' ''name/version-build'' '''</code>. Using the <code>''name''</code> component alone is possible and will select a default version for <code>module ''subcommand''</code> to act upon. Some packages do carry a major version number in their name, notably fftw3 and vasp5.
* Module names are organized by a mostly version-less name, with the version following after a slash: <code>''' ''name/version-build'' '''</code>. Using the <code>''name''</code> component alone is possible and will select a default version for <code>module ''subcommand''</code> to act upon. Some packages do carry a major version number in their name, notably fftw3 and vasp5.
* <code>module help</code> briefly describes a package, gives its version number (in the module path) and will usually contain a link to its home page.
 
== Default modules being loaded ==
The following modules are loaded by default for all users:
$ '''module list'''
Currently Loaded Modulefiles:
  1) icc/11/11.1.073          4) openmpi/1.4.5-intel11-2
  2) ifort/11/11.1.073        5) profile/user/1.0
  3) mkl/10.3.10
: '''Note''': These is done by the system ''before'' processing <code>.bashrc</code>.
 
== Module conventions on Carbon ==
=== Package description ===
The command <code>module help</code> brings up a brief description of a package, gives its version number (in the form of the full module name) and will usually contain a link to the package home page and/or documentation, e.g.:
  $ '''module help jmol'''
  $ '''module help jmol'''
   
   
Line 15: Line 27:
  http://jmol.sourceforge.net/
  http://jmol.sourceforge.net/
  http://wiki.jmol.org/
  http://wiki.jmol.org/
* Default modules are loaded by <code>/etc/profile.d/zz-moduleuse.sh</code>. (The strange name form ensures this profile segment is loaded last.) They currently are:
$ '''module list'''
Currently Loaded Modulefiles:
  1) moab/6.0.3-1              4) icc/11/11.1.073
  2) gold/2.1.11.0-4          5) ifort/11/11.1.073
  3) openmpi/1.4.3-intel11-1  6) mkl/10.2.6.038
<!-- OBSOLETE:
For now the only applications are the [[HPC/Carbon Cluster - Development tools| Development tools]].
Admin note:  The master copy of these files resides in <code>mgmt{01,02}:/opt/teamhpc/node-skel/etc/profile.d</code> and is distributed by <code>~root/bin/skeldistrib</code>.
-->
=== Package home directory organization ===
A package directory usually contains Unix-style subdirectories for the various files, which the modulefile usually automatically integrates into your user environment by means of standard environment variables.
; <code>bin/</code>: the main package executable and associated tools and utility scripts; added to <code>'''$PATH'''</code>.
; <code>lib/</code>: static and shared libraries; added to <code>'''$LD_LIBRARY_PATH'''</code> if it contains shared libs.
; <code>man/, share/man/, doc/, share/doc/</code>: man pages (added to <code>'''$MANPATH'''</code>) and human-readable documentation.
; <code>include/</code>: C header files and other script integration files for using library packages; added to <code>'''$INCLUDE'''</code>.
; …: and others.


=== Environment variable for package home directory ===
=== Environment variable for package home directory ===
Most modules will set a convenience variable <code>''' $''NAME''_HOME'''</code> which points to the package's toplevel directory.
Most modules will set a convenience variable <code>''' $''NAME''_HOME'''</code> which points to the package's toplevel directory.
Dashes in the package name are converted to underscores in the environment variable.
Dashes in the package name are converted to underscores in the environment variable.
Line 54: Line 45:
  Doc/  atomic_doc/  examples/
  Doc/  atomic_doc/  examples/


; Specifcy link paths in makefiles:
; Specify paths in makefiles:
  $ '''module load fftw3'''
  $ '''module load fftw3'''
  $ cat Makefile
  $ cat Makefile
  …
  …
  LDFLAGS += -L$(FFTW3_HOME)/lib
  CPPFLAGS += -I'''$(FFTW3_HOME)'''/include
FPPFLAGS += -I'''$(FFTW3_HOME)'''/include
  …
  …
LDFLAGS += -L'''$(FFTW3_HOME)'''/lib
=== Package home directory organization ===
A package directory usually contains Unix-style subdirectories for the various files, which the modulefile usually automatically integrates into your user environment by means of standard environment variables.
; <code>bin/</code>: the main package executable and associated tools and utility scripts; added to <code>'''$PATH'''</code>.
; <code>lib/</code>: static and shared libraries; added to <code>'''$LD_LIBRARY_PATH'''</code> if it contains shared libs.
; <code>man/, share/man/, doc/, share/doc/</code>: man pages (added to <code>'''$MANPATH'''</code>) and human-readable documentation.
; <code>include/</code>: C header files and other script integration files for using library packages; added to <code>'''$INCLUDE'''</code>.
; …: and others.


=== Package-specific sample job ===
=== Package-specific sample job ===

Latest revision as of 15:43, June 21, 2017

Module naming scheme

  • Most application software is installed under /opt/soft
  • Package directories are named name-version-build , e.g. /opt/soft/jmol-12.1.37-1.
  • Module names are organized by a mostly version-less name, with the version following after a slash: name/version-build . Using the name component alone is possible and will select a default version for module subcommand to act upon. Some packages do carry a major version number in their name, notably fftw3 and vasp5.

Default modules being loaded

The following modules are loaded by default for all users:

$ module list
Currently Loaded Modulefiles:
  1) icc/11/11.1.073           4) openmpi/1.4.5-intel11-2
  2) ifort/11/11.1.073         5) profile/user/1.0
  3) mkl/10.3.10
Note: These is done by the system before processing .bashrc.

Module conventions on Carbon

Package description

The command module help brings up a brief description of a package, gives its version number (in the form of the full module name) and will usually contain a link to the package home page and/or documentation, e.g.:

$ module help jmol

----------- Module Specific Help for 'jmol/12.0.34-1' -------------

	Jmol is a molecule viewer platform for researchers in chemistry and
	biochemistry, implemented in Java for multi-platform use.  This is the
	standalone application.  It offers high-performance 3D rendering with
	no hardware requirements and supports many popular file formats.

	http://jmol.sourceforge.net/
	http://wiki.jmol.org/

Environment variable for package home directory

Most modules will set a convenience variable $NAME_HOME which points to the package's toplevel directory. Dashes in the package name are converted to underscores in the environment variable. This is a convention on Carbon and is useful to inspect binaries or documentation and auxiliary files of a package:

$ module load quantum-espresso
Inspect package binaries
$ ls $QUANTUM_ESPRESSO_HOME/bin
average.x     dos.x              …
band_plot.x   dynmat.x           …
bands.x       epsilon.x          …
bands_FS.x    ev.x               …
…
Inspect package documentation
$ ls -F $QUANTUM_ESPRESSO_HOME/doc
Doc/  atomic_doc/  examples/
Specify paths in makefiles
$ module load fftw3
$ cat Makefile
…
CPPFLAGS += -I$(FFTW3_HOME)/include
FPPFLAGS += -I$(FFTW3_HOME)/include
…
LDFLAGS += -L$(FFTW3_HOME)/lib
…

Package home directory organization

A package directory usually contains Unix-style subdirectories for the various files, which the modulefile usually automatically integrates into your user environment by means of standard environment variables.

bin/
the main package executable and associated tools and utility scripts; added to $PATH.
lib/
static and shared libraries; added to $LD_LIBRARY_PATH if it contains shared libs.
man/, share/man/, doc/, share/doc/
man pages (added to $MANPATH) and human-readable documentation.
include/
C header files and other script integration files for using library packages; added to $INCLUDE.
and others.

Package-specific sample job

Packages that require more than the standard Carbon job template contain a sample job in the toplevel directory:

module load quantum-espresso
cat $QUANTUM_ESPRESSO_HOME/*.job

which gives:

#!/bin/bash
# Job template for Quantum ESPRESSO 4.x on Carbon
#
#PBS -l nodes=1:ppn=8
#PBS -l walltime=1:00:00
#PBS -N qe_jobname
#PBS -A cnm12345
#
# Output and error log:
#PBS -o job.out
#PBS -e job.err
#
## send mail at begin, end, abort, or never (b, e, a, n):
#PBS -m ea

cd $PBS_O_WORKDIR

# job-specific tmp directories -- each is node-local, wiped on exit
export ESPRESSO_TMPDIR=$TMPDIR

mpirun -x ESPRESSO_TMPDIR \
	-machinefile  $PBS_NODEFILE \
	-np $(wc -l < $PBS_NODEFILE) \
	pw.x \
	< input.txt > output.txt