HPC/Carbon Cluster - Development tools: Difference between revisions

From CNM Wiki
< HPC
Jump to navigation Jump to search
(migrated from internal wiki)
 
 
(20 intermediate revisions by the same user not shown)
Line 1: Line 1:
__TOC__
__TOC__


== Compilers and development tools ==
This is an overview of the compilers and MPI libraries available on Carbon.
Each package name leads to its documentation.


=== [http://www.gnu.org/doc/ GNU] ===
; See also: [[HPC/Software/Modules#Intel development tools | Module catalog]].
 
== Compilers ==
 
=== GNU family ===
* [http://gcc.gnu.org/onlinedocs/ GCC]
* [http://gcc.gnu.org/onlinedocs/ GCC]
* [http://www.gnu.org/software/gdb/documentation/ GDB]
* [http://www.gnu.org/software/gdb/documentation/ GDB]
Line 9: Line 14:
* [http://www.gnu.org/manual/manual.html … other manuals]
* [http://www.gnu.org/manual/manual.html … other manuals]


=== [http://www.intel.com/support/performancetools/ Intel] ===
=== [http://software.intel.com/en-us/articles/intel-sdp-products/ Intel Software Development Products] ===
We have supported versions of the following commercial tools, currently installed under <code>/opt/intel</code>:
As of 2011, we have the following tools under support within the [http://software.intel.com/en-us/articles/intel-composer-xe/ ''Intel® Composer XE 2011 for Linux''] bundle.
* [http://www.intel.com/support/performancetools/fortran/linux/ Fortran]
** [http://www.intel.com/cd/software/products/asmo-na/eng/346152.htm Manual for 10.1]
** <code>/opt/intel/{fc,fce}/''version''</code>
* [http://www.intel.com/support/performancetools/c/index.htm C++]
** [http://www.intel.com/cd/software/products/asmo-na/eng/346158.htm Manual for 10.1]
** <code>/opt/intel/{cc,cce}/''version''</code>
* [http://www.intel.com/support/performancetools/libraries/mkl/index.htm Intel Math Kernel Library (MKL)], includes Cluster Edition
** <code>/opt/intel/mkl/''version''</code>
* [http://www.intel.com/support/performancetools/vtune/index1.htm VTune analyzer]
* Intel Debugger
** <code>/opt/intel/{idb,idbe}/''version''</code>
To access the documentation pages, use '''firefox''' on a login node:
* <code>/opt/intel/fce/''version''/doc/Doc_Index.htm</code>
* <code>/opt/intel/cce/''version''/doc/Doc_Index.htm</code>
* <code>/opt/intel/mkl/''version''/doc/Doc_index.htm</code>
 
==== Note on binary architectures ====


CPU architecture labels used by Intel and others can be confusing and have changed over time.
* [http://software.intel.com/en-us/articles/intel-c-compiler-professional-edition-for-linux-documentation/ Intel C/C++]
Here's a summary of the labels and directory names commonly used:
* [http://software.intel.com/en-us/articles/intel-fortran-compiler-professional-edition-for-linux-documentation/ Intel Fortran]
; '''32, ia32, x86''': legacy 32-bit x86 generations up to ''Pentium 4'' and ''[http://en.wikipedia.org/wiki/Intel_Core Intel Core]''
* [http://software.intel.com/en-us/articles/intel-math-kernel-library-documentation/ Math Kernel Library] -- see also [[HPC/Software/Modules/mkl]]
; '''em64t''', '''Intel-64''', '''x86_64''', '''AMD-64''': 64-bit generations of the ''[http://en.wikipedia.org/wiki/Intel_Xeon Xeon]'' branded CPUs, and the ''[http://en.wikipedia.org/wiki/Intel_Core_2 Core 2]'' generation, mostly compatible with AMD's ''[http://en.wikipedia.org/wiki/Opteron Opteron]''
* [http://software.intel.com/en-us/articles/intel-integrated-performance-primitives-documentation/ Integrated Performance Primitives Documentation (IPP)]
; '''64, ia64''', '''Itanium''': ''[http://en.wikipedia.org/wiki/Itanium Itanium]'' – limited ia32 support, but none for x86_64; ''not used here.''
* [http://software.intel.com/en-us/articles/intel-threading-building-blocks-documentation/ Threading Building Blocks Documentation (TBB)]
* [http://software.intel.com/en-us/articles/intel-c-compiler-professional-edition-for-linux-documentation/ Debugger (idb)]


The cce, fce, and idbe directories above provide the 64-bit em64t versions appropriate for the Xeon processors, with ia32 versions being available under cc, fc, idb to compile and run legacy applications.  By default, the 64-bit versions are activated in the user's environment (<code>$PATH</code> and others).   This is done automatically through <code>/etc/profile.d/{iccvars,ifortvars,mklvarsem64t}.{sh,csh}</code>.
Documentation is installed locally mainly under:
  $ICC_HOME/Documentation
<pre>
$ ls -F $ICC_HOME/Documentation/en_US
Release_NotesC.pdf  compiler_c/         documentation_f.htm    getting_started_f.pdf  lgpltext
Release_NotesF.pdf  compiler_f/          flicense              idb/                   mkl/
clicense            documentation_c.htm  getting_started_c.pdf  ipp/                   tbb/
</pre>


In the future, we will leverage the ''[http://modules.sourceforge.net/ environment-modules]'' package for version and architecture selection.
No longer maintained on Carbon:
 
* [http://www.intel.com/support/performancetools/vtune/index1.htm VTune analyzer]
==Libraries==
 
=== [http://www.intel.com/support/performancetools/libraries/mkl/index.htm Intel Math Kernel Library (MKL)] ===
 
==== Admin Notes ====
 
To install or update, follow these steps:
 
* Dowload to and unpack in <code>mgmt01:/usr/local/src/intel</code>
* cd, then run ./install.sh, accepting defaults
* Important: the default procedure installs to /opt, which is symlinked.  A correction is needed in the profile files:
perl -i -pe 's,/drbd/export,,'  /opt/intel/mkl/VERSION/tools/environment/mkl*sh
* to set as default for users ('''Caution - this may break older binaries that are incorrectly linked'''):
** old-style:
cp -p /opt/intel/mkl/VERSION/tools/environment/mkl*sh  /opt/teamhpc/node-skel/etc/profile.d/
skeldistrib -v
** new style (using env.-modules) -- TODO.
 
==== User Notes ====
 
* [http://www.intel.com/software/products/mkl/docs/linux/WebHelp/userguide.htm User's Guide]
* [http://www.intel.com/software/products/mkl/docs/WebHelp/mklrefman.htm Reference Manual]
* since v10 includes formerly separate Cluster MKL (providing ScaLapack et al.)
* [[Media:MKL userguide.pdf|User Guide (pdf)]] – consult Sec. 5 to resolve linker issues, and 32bit vs. 64bit programming
* provides BLAS, Lapack, ScaLapack, FFTs (including an [http://www.intel.com/software/products/mkl/docs/fftw3xmkl_notes.htm FFTW interface])
* [http://support.intel.com/support/performancetools/libraries/mkl/linux/sb/CS-028423.htm Detailed linking instructions]
* Link example (from <code>/opt/cnm/examples</code>):
  # Sequential:
    -lmkl_intel_lp64 -lmkl_sequential -lmkl_lapack -lmkl_em64t -lmkl_core -lpthread
 
  # alternative:
    -Wl,--start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_lapack -lmkl_em64t -lmkl_core -Wl,--end-group \
    -lpthread
 
  # OpenMP multi-threaded:
    -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_lapack -lmkl_em64t -lmkl_core -Wl,--end-group \
    -lguide -lpthread
* The FFTW interface of the MKL can be linked by ''prefixing'' the above options with one of:
-lfftw2xc_intel
-lfftw2xf_intel
-lfftw3xc_intel
-lfftw3xf_intel
Choose the appropriate version between bindings for FFTW v2 vs. v3 and C vs. Fortran.
*  Avoid specifying absolute library paths in your Makefile unless you wish to select a specific version.  The user environment is normally correctly set up.  To force the library version current at the time of writing, add as first flag to the linker options:
<code>-L/opt/intel/mkl/10.0.2.018/lib/em64t</code> (automatically set in <code>$LD_LIBRARY_PATH</code>).
* see [[HPC/Carbon_Cluster_-_Getting_started#Example_job_file]] for controlling OpenMP execution.
 
=== [http://www.vtk.org/ VTK] ===
* [https://visualization.hpc.mil/wiki/VTK Intro]
* [https://visualization.hpc.mil/wiki/Getting_Started_with_VTK VTK install notes]
 
=== [http://www.mcs.anl.gov/petsc PETSc] ===
* TODO: Portable, Extensible Toolkit for Scientific Computation


==MPI==
==MPI==


===[http://www.open-mpi.org/ OpenMPI]===
=== [http://www.open-mpi.org/ OpenMPI] ===
This is the primary and recommended MPI version.  It supports Ethernet and Infiniband interconnect.
This is the primary and recommended MPI version.  It supports Ethernet and InfiniBand interconnect, with InfiniBand set as default by means of an [http://www.open-mpi.org/faq/?category=openfabrics#ib-btl <code>OMPI_MCA_…</code>] environment variable.
* See also: [http://www.open-mpi.org/faq FAQ]
* [http://www.open-mpi.org/doc/ OpenMPI Documentation]
 
* [http://www.open-mpi.org/faq FAQ]
===[http://www-unix.mcs.anl.gov/mpi/mpich1/ MPICH]/[http://www.mcs.anl.gov/research/projects/mpich2/ MPIC2]===
 
TBD
 
===[http://www.intel.com/cd/software/products/asmo-na/eng/308295.htm#support Intel MPI]===
 
* [http://www.intel.com/cd/software/products/asmo-na/eng/368005.htm Documentation]
 
==Resource Management==


* [http://www.clusterresources.com/pages/products/moab-cluster-suite.php MOAB cluster suite]
=== Intel MPI ===
** [http://www.clusterresources.com/pages/resources/documentation.php Documentation overview]
* See [[HPC/Software/Modules/impi]]
** Torque
** Gold allocation manager


(description of overlap/differences)
===[http://www-unix.mcs.anl.gov/mpi/mpich1/ MPICH]/[http://www.mcs.anl.gov/research/projects/mpich2/ MPICH2]===
* Not installed explicitly, but Intel-MPI is based on MPICH2 and provides an MPICH2 runtime environment for precompiled binaries.


{{Template:Standard Footer}}
[[Category:HPC|Development]]

Latest revision as of 22:22, June 3, 2011

This is an overview of the compilers and MPI libraries available on Carbon. Each package name leads to its documentation.

See also
Module catalog.

Compilers

GNU family

Intel Software Development Products

As of 2011, we have the following tools under support within the Intel® Composer XE 2011 for Linux bundle.

Documentation is installed locally mainly under: 

$ICC_HOME/Documentation

$ ls -F $ICC_HOME/Documentation/en_US
Release_NotesC.pdf  compiler_c/          documentation_f.htm    getting_started_f.pdf  lgpltext
Release_NotesF.pdf  compiler_f/          flicense               idb/                   mkl/
clicense            documentation_c.htm  getting_started_c.pdf  ipp/                   tbb/

No longer maintained on Carbon:

MPI

OpenMPI

This is the primary and recommended MPI version. It supports Ethernet and InfiniBand interconnect, with InfiniBand set as default by means of an OMPI_MCA_… environment variable.

Intel MPI

MPICH/MPICH2

  • Not installed explicitly, but Intel-MPI is based on MPICH2 and provides an MPICH2 runtime environment for precompiled binaries.
Retrieved from "https://wiki.anl.gov/wiki_cnm/index.php?title=HPC/Carbon_Cluster_-_Development_tools&oldid=3376"