HPC/Carbon Cluster - Development tools

Compilers and development tools

Intel

We have supported versions of the following commercial tools, currently installed under /opt/intel:

Fortran
- Manual for 10.1
- /opt/intel/{fc,fce}/version
C++
- Manual for 10.1
- /opt/intel/{cc,cce}/version
Intel Math Kernel Library (MKL), includes Cluster Edition
- /opt/intel/mkl/version
VTune analyzer
Intel Debugger
- /opt/intel/{idb,idbe}/version

To access the documentation pages, use firefox on a login node:

/opt/intel/fce/version/doc/Doc_Index.htm
/opt/intel/cce/version/doc/Doc_Index.htm
/opt/intel/mkl/version/doc/Doc_index.htm

Note on binary architectures

CPU architecture labels used by Intel and others can be confusing and have changed over time. Here's a summary of the labels and directory names commonly used:

32, ia32, x86: legacy 32-bit x86 generations up to Pentium 4 and Intel Core
em64t, Intel-64, x86_64, AMD-64: 64-bit generations of the Xeon branded CPUs, and the Core 2 generation, mostly compatible with AMD's Opteron
64, ia64, Itanium: Itanium – limited ia32 support, but none for x86_64; not used here.

The cce, fce, and idbe directories above provide the 64-bit em64t versions appropriate for the Xeon processors, with ia32 versions being available under cc, fc, idb to compile and run legacy applications. By default, the 64-bit versions are activated in the user's environment ($PATH and others). This is done automatically through /etc/profile.d/{iccvars,ifortvars,mklvarsem64t}.{sh,csh}.

In the future, we will leverage the environment-modules package for version and architecture selection.

Libraries

Intel Math Kernel Library (MKL)

User Notes

User's Guide – consult Sec. 5 to resolve linker issues, and 32bit vs. 64bit programming
Reference Manual
since v10 includes formerly separate Cluster MKL (providing ScaLapack et al.)
provides BLAS, Lapack, ScaLapack, FFTs (including an FFTW interface)
Detailed linking instructions
Link example (from /opt/cnm/examples):

 # Sequential:
   -lmkl_intel_lp64 -lmkl_sequential -lmkl_lapack -lmkl_em64t -lmkl_core -lpthread

 # alternative:
   -Wl,--start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_lapack -lmkl_em64t -lmkl_core -Wl,--end-group \
   -lpthread

 # OpenMP multi-threaded:
   -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_lapack -lmkl_em64t -lmkl_core -Wl,--end-group \
   -lguide -lpthread

The FFTW interface of the MKL can be linked by prefixing the above options with one of:

-lfftw2xc_intel
-lfftw2xf_intel
-lfftw3xc_intel
-lfftw3xf_intel

Choose the appropriate version between bindings for FFTW v2 vs. v3 and C vs. Fortran.

Avoid specifying absolute library paths in your Makefile unless you wish to select a specific version. The user environment is normally correctly set up. To force the library version current at the time of writing, add as first flag to the linker options:

-L$MKLROOT/lib/em64t (automatically set in $LD_LIBRARY_PATH).