HPC/Carbon Cluster - Development tools

Compilers and development tools

GNU

• GCC
• GDB
• Make
• … other manuals

Intel

We have supported versions of the following commercial tools, currently installed under /opt/intel:

• Fortran
  • Manual for 10.1
  • /opt/intel/{fc,fce}/version
• C++
  • Manual for 10.1
  • /opt/intel/{cc,cce}/version
• Intel Math Kernel Library (MKL), includes Cluster Edition
  • /opt/intel/mkl/version
• VTune analyzer
• Intel Debugger
  • /opt/intel/{idb,idbe}/version

To access the documentation pages, use firefox on a login node:

• /opt/intel/fce/version/doc/Doc_Index.htm
• /opt/intel/cce/version/doc/Doc_Index.htm
• /opt/intel/mkl/version/doc/Doc_index.htm

Note on binary architectures

CPU architecture labels used by Intel and others can be confusing and have changed over time. Here's a summary of the labels and directory names commonly used:

32, ia32, x86

legacy 32-bit x86 generations up to Pentium 4 and Intel Core

em64t, Intel-64, x86_64, AMD-64

64-bit generations of the Xeon branded CPUs, and the Core 2 generation, mostly compatible with AMD's Opteron

64, ia64, Itanium

Itanium – limited ia32 support, but none for x86_64; not used here.

The cce, fce, and idbe directories above provide the 64-bit em64t versions appropriate for the Xeon processors, with ia32 versions being available under cc, fc, idb to compile and run legacy applications. By default, the 64-bit versions are activated in the user's environment ($PATH and others). This is done automatically through /etc/profile.d/{iccvars,ifortvars,mklvarsem64t}.{sh,csh}.

The environment-modules package is used for version and architecture selection.

module list
module avail
module load ...
module unload ...
module switch old new

Libraries

Intel Math Kernel Library (MKL)

User Notes

• User's Guide – consult Sec. 5 to resolve linker issues, and 32bit vs. 64bit programming
• Reference Manual
• since v10 includes formerly separate Cluster MKL (providing ScaLapack et al.)
• provides BLAS, Lapack, ScaLapack, FFTs (including an FFTW interface)
• Detailed linking instructions; alternate link
• Link example:

 # Sequential:
   -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread

 # OpenMP multi-threaded:
   -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread

• The FFTW interface of the MKL can be linked by prefixing the above options with one of:

-lfftw2xc_intel
-lfftw2xf_intel
-lfftw3xc_intel
-lfftw3xf_intel

Choose the appropriate version between bindings for FFTW v2 vs. v3 and C vs. Fortran.

• Avoid specifying absolute library paths in your Makefile unless you wish to select a specific version. The user environment is normally correctly set up. To force the library version current at the time of writing, add as first flag to the linker options:

-L$MKLROOT/lib/em64t. The variable $MKLROOTis automatically provided.

• see HPC/Carbon_Cluster_-_Getting_started#Example_job_file for controlling OpenMP execution.

VTK

• Intro
• VTK install notes

PETSc

• TODO: Portable, Extensible Toolkit for Scientific Computation

MPI

OpenMPI

This is the primary and recommended MPI version. It supports Ethernet and Infiniband interconnect.

• See also: FAQ

MPICH/MPIC2

TBD

Intel MPI

• Documentation

Resource Management

• MOAB cluster suite
  • Documentation overview
  • Torque
  • Gold allocation manager