HPC/Carbon Cluster - Development tools: Difference between revisions

From CNM Wiki
< HPC
Jump to navigation Jump to search
m (→‎User Notes: version update)
m (→‎User Notes: alternate MKL linkage)
Line 51: Line 51:
* since v10 includes formerly separate Cluster MKL (providing ScaLapack et al.)
* since v10 includes formerly separate Cluster MKL (providing ScaLapack et al.)
* provides BLAS, Lapack, ScaLapack, FFTs (including an [http://www.intel.com/software/products/mkl/docs/fftw3xmkl_notes.htm FFTW interface])
* provides BLAS, Lapack, ScaLapack, FFTs (including an [http://www.intel.com/software/products/mkl/docs/fftw3xmkl_notes.htm FFTW interface])
* [http://support.intel.com/support/performancetools/libraries/mkl/linux/sb/CS-028423.htm Detailed linking instructions]
* [http://support.intel.com/support/performancetools/libraries/mkl/linux/sb/CS-028423.htm Detailed linking instructions]; [http://software.intel.com/en-us/articles/intel-math-kernel-library-intel-mkl-for-linux-linking-applications-with-intel-mkl-version-100/ alternate link]
* Link example (from <code>/opt/cnm/examples</code>):
* Link example:
   # Sequential:
   # Sequential:
     -lmkl_intel_lp64 -lmkl_sequential -lmkl_lapack -lmkl_em64t -lmkl_core -lpthread
     -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread


   # alternative:
   # OpenMP multi-threaded:
     -Wl,--start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_lapack -lmkl_em64t -lmkl_core -Wl,--end-group \
     -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread
    -lpthread


  # OpenMP multi-threaded:
    -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_lapack -lmkl_em64t -lmkl_core -Wl,--end-group \
    -lguide -lpthread
* The FFTW interface of the MKL can be linked by ''prefixing'' the above options with one of:
* The FFTW interface of the MKL can be linked by ''prefixing'' the above options with one of:
  -lfftw2xc_intel
  -lfftw2xc_intel
Line 70: Line 66:
Choose the appropriate version between bindings for FFTW v2 vs. v3 and C vs. Fortran.
Choose the appropriate version between bindings for FFTW v2 vs. v3 and C vs. Fortran.
*  Avoid specifying absolute library paths in your Makefile unless you wish to select a specific version.  The user environment is normally correctly set up.  To force the library version current at the time of writing, add as first flag to the linker options:
*  Avoid specifying absolute library paths in your Makefile unless you wish to select a specific version.  The user environment is normally correctly set up.  To force the library version current at the time of writing, add as first flag to the linker options:
<code>-L$MKLROOT/lib/em64t</code> (automatically set in <code>$LD_LIBRARY_PATH</code>).
<code>-L$MKLROOT/lib/em64t</code>. The variable <code>$MKLROOT</code>is automatically provided.
* see [[HPC/Carbon_Cluster_-_Getting_started#Example_job_file]] for controlling OpenMP execution.
* see [[HPC/Carbon_Cluster_-_Getting_started#Example_job_file]] for controlling OpenMP execution.



Revision as of 08:11, February 23, 2009

Compilers and development tools

GNU

Intel

We have supported versions of the following commercial tools, currently installed under /opt/intel:

To access the documentation pages, use firefox on a login node:

  • /opt/intel/fce/version/doc/Doc_Index.htm
  • /opt/intel/cce/version/doc/Doc_Index.htm
  • /opt/intel/mkl/version/doc/Doc_index.htm

Note on binary architectures

CPU architecture labels used by Intel and others can be confusing and have changed over time. Here's a summary of the labels and directory names commonly used:

32, ia32, x86
legacy 32-bit x86 generations up to Pentium 4 and Intel Core
em64t, Intel-64, x86_64, AMD-64
64-bit generations of the Xeon branded CPUs, and the Core 2 generation, mostly compatible with AMD's Opteron
64, ia64, Itanium
Itanium – limited ia32 support, but none for x86_64; not used here.

The cce, fce, and idbe directories above provide the 64-bit em64t versions appropriate for the Xeon processors, with ia32 versions being available under cc, fc, idb to compile and run legacy applications. By default, the 64-bit versions are activated in the user's environment ($PATH and others). This is done automatically through /etc/profile.d/{iccvars,ifortvars,mklvarsem64t}.{sh,csh}.

In the future, we will leverage the environment-modules package for version and architecture selection.

Libraries

Intel Math Kernel Library (MKL)

User Notes

 # Sequential:
   -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread
 # OpenMP multi-threaded:
   -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread
  • The FFTW interface of the MKL can be linked by prefixing the above options with one of:
-lfftw2xc_intel
-lfftw2xf_intel
-lfftw3xc_intel
-lfftw3xf_intel

Choose the appropriate version between bindings for FFTW v2 vs. v3 and C vs. Fortran.

  • Avoid specifying absolute library paths in your Makefile unless you wish to select a specific version. The user environment is normally correctly set up. To force the library version current at the time of writing, add as first flag to the linker options:

-L$MKLROOT/lib/em64t. The variable $MKLROOTis automatically provided.

VTK

PETSc

  • TODO: Portable, Extensible Toolkit for Scientific Computation

MPI

OpenMPI

This is the primary and recommended MPI version. It supports Ethernet and Infiniband interconnect.

MPICH/MPIC2

TBD

Intel MPI

Resource Management

(description of overlap/differences)