|
|
| (20 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| __TOC__ | | __TOC__ |
|
| |
|
| == Compilers and development tools ==
| | This is an overview of the compilers and MPI libraries available on Carbon. |
| | Each package name leads to its documentation. |
|
| |
|
| === [http://www.gnu.org/doc/ GNU] ===
| | ; See also: [[HPC/Software/Modules#Intel development tools | Module catalog]]. |
| | |
| | == Compilers == |
| | |
| | === GNU family === |
| * [http://gcc.gnu.org/onlinedocs/ GCC] | | * [http://gcc.gnu.org/onlinedocs/ GCC] |
| * [http://www.gnu.org/software/gdb/documentation/ GDB] | | * [http://www.gnu.org/software/gdb/documentation/ GDB] |
| Line 9: |
Line 14: |
| * [http://www.gnu.org/manual/manual.html … other manuals] | | * [http://www.gnu.org/manual/manual.html … other manuals] |
|
| |
|
| === [http://www.intel.com/support/performancetools/ Intel] === | | === [http://software.intel.com/en-us/articles/intel-sdp-products/ Intel Software Development Products] === |
| We have supported versions of the following commercial tools, currently installed under <code>/opt/intel</code>:
| | As of 2011, we have the following tools under support within the [http://software.intel.com/en-us/articles/intel-composer-xe/ ''Intel® Composer XE 2011 for Linux''] bundle. |
| * [http://www.intel.com/support/performancetools/fortran/linux/ Fortran]
| |
| ** [http://www.intel.com/cd/software/products/asmo-na/eng/346152.htm Manual for 10.1]
| |
| ** <code>/opt/intel/{fc,fce}/''version''</code>
| |
| * [http://www.intel.com/support/performancetools/c/index.htm C++]
| |
| ** [http://www.intel.com/cd/software/products/asmo-na/eng/346158.htm Manual for 10.1]
| |
| ** <code>/opt/intel/{cc,cce}/''version''</code>
| |
| * [http://www.intel.com/support/performancetools/libraries/mkl/index.htm Intel Math Kernel Library (MKL)], includes Cluster Edition
| |
| ** <code>/opt/intel/mkl/''version''</code>
| |
| * [http://www.intel.com/support/performancetools/vtune/index1.htm VTune analyzer]
| |
| * Intel Debugger
| |
| ** <code>/opt/intel/{idb,idbe}/''version''</code>
| |
| To access the documentation pages, use '''firefox''' on a login node:
| |
| * <code>/opt/intel/fce/''version''/doc/Doc_Index.htm</code>
| |
| * <code>/opt/intel/cce/''version''/doc/Doc_Index.htm</code>
| |
| * <code>/opt/intel/mkl/''version''/doc/Doc_index.htm</code>
| |
| | |
| ==== Note on binary architectures ====
| |
|
| |
|
| CPU architecture labels used by Intel and others can be confusing and have changed over time.
| | * [http://software.intel.com/en-us/articles/intel-c-compiler-professional-edition-for-linux-documentation/ Intel C/C++] |
| Here's a summary of the labels and directory names commonly used:
| | * [http://software.intel.com/en-us/articles/intel-fortran-compiler-professional-edition-for-linux-documentation/ Intel Fortran] |
| ; '''32, ia32, x86''': legacy 32-bit x86 generations up to ''Pentium 4'' and ''[http://en.wikipedia.org/wiki/Intel_Core Intel Core]''
| | * [http://software.intel.com/en-us/articles/intel-math-kernel-library-documentation/ Math Kernel Library] -- see also [[HPC/Software/Modules/mkl]] |
| ; '''em64t''', '''Intel-64''', '''x86_64''', '''AMD-64''': 64-bit generations of the ''[http://en.wikipedia.org/wiki/Intel_Xeon Xeon]'' branded CPUs, and the ''[http://en.wikipedia.org/wiki/Intel_Core_2 Core 2]'' generation, mostly compatible with AMD's ''[http://en.wikipedia.org/wiki/Opteron Opteron]''
| | * [http://software.intel.com/en-us/articles/intel-integrated-performance-primitives-documentation/ Integrated Performance Primitives Documentation (IPP)] |
| ; '''64, ia64''', '''Itanium''': ''[http://en.wikipedia.org/wiki/Itanium Itanium]'' – limited ia32 support, but none for x86_64; ''not used here.''
| | * [http://software.intel.com/en-us/articles/intel-threading-building-blocks-documentation/ Threading Building Blocks Documentation (TBB)] |
| | * [http://software.intel.com/en-us/articles/intel-c-compiler-professional-edition-for-linux-documentation/ Debugger (idb)] |
|
| |
|
| The cce, fce, and idbe directories above provide the 64-bit em64t versions appropriate for the Xeon processors, with ia32 versions being available under cc, fc, idb to compile and run legacy applications. By default, the 64-bit versions are activated in the user's environment (<code>$PATH</code> and others). This is done automatically through <code>/etc/profile.d/{iccvars,ifortvars,mklvarsem64t}.{sh,csh}</code>.
| | Documentation is installed locally mainly under: |
| | $ICC_HOME/Documentation |
| | <pre> |
| | $ ls -F $ICC_HOME/Documentation/en_US |
| | Release_NotesC.pdf compiler_c/ documentation_f.htm getting_started_f.pdf lgpltext |
| | Release_NotesF.pdf compiler_f/ flicense idb/ mkl/ |
| | clicense documentation_c.htm getting_started_c.pdf ipp/ tbb/ |
| | </pre> |
|
| |
|
| In the future, we will leverage the ''[http://modules.sourceforge.net/ environment-modules]'' package for version and architecture selection.
| | No longer maintained on Carbon: |
| | | * [http://www.intel.com/support/performancetools/vtune/index1.htm VTune analyzer] |
| ==Libraries==
| |
| | |
| === [http://www.intel.com/support/performancetools/libraries/mkl/index.htm Intel Math Kernel Library (MKL)] ===
| |
| | |
| ==== Admin Notes ====
| |
| | |
| To install or update, follow these steps:
| |
| | |
| * Dowload to and unpack in <code>mgmt01:/usr/local/src/intel</code>
| |
| * cd, then run ./install.sh, accepting defaults
| |
| * Important: the default procedure installs to /opt, which is symlinked. A correction is needed in the profile files:
| |
| perl -i -pe 's,/drbd/export,,' /opt/intel/mkl/VERSION/tools/environment/mkl*sh
| |
| * to set as default for users ('''Caution - this may break older binaries that are incorrectly linked'''):
| |
| ** old-style:
| |
| cp -p /opt/intel/mkl/VERSION/tools/environment/mkl*sh /opt/teamhpc/node-skel/etc/profile.d/
| |
| skeldistrib -v
| |
| ** new style (using env.-modules) -- TODO.
| |
| | |
| ==== User Notes ====
| |
| | |
| * [http://www.intel.com/software/products/mkl/docs/linux/WebHelp/userguide.htm User's Guide] | |
| * [http://www.intel.com/software/products/mkl/docs/WebHelp/mklrefman.htm Reference Manual]
| |
| * since v10 includes formerly separate Cluster MKL (providing ScaLapack et al.)
| |
| * [[Media:MKL userguide.pdf|User Guide (pdf)]] – consult Sec. 5 to resolve linker issues, and 32bit vs. 64bit programming
| |
| * provides BLAS, Lapack, ScaLapack, FFTs (including an [http://www.intel.com/software/products/mkl/docs/fftw3xmkl_notes.htm FFTW interface])
| |
| * [http://support.intel.com/support/performancetools/libraries/mkl/linux/sb/CS-028423.htm Detailed linking instructions]
| |
| * Link example (from <code>/opt/cnm/examples</code>):
| |
| # Sequential:
| |
| -lmkl_intel_lp64 -lmkl_sequential -lmkl_lapack -lmkl_em64t -lmkl_core -lpthread
| |
| | |
| # alternative:
| |
| -Wl,--start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_lapack -lmkl_em64t -lmkl_core -Wl,--end-group \
| |
| -lpthread
| |
| | |
| # OpenMP multi-threaded:
| |
| -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_lapack -lmkl_em64t -lmkl_core -Wl,--end-group \
| |
| -lguide -lpthread
| |
| * The FFTW interface of the MKL can be linked by ''prefixing'' the above options with one of:
| |
| -lfftw2xc_intel
| |
| -lfftw2xf_intel
| |
| -lfftw3xc_intel
| |
| -lfftw3xf_intel
| |
| Choose the appropriate version between bindings for FFTW v2 vs. v3 and C vs. Fortran.
| |
| * Avoid specifying absolute library paths in your Makefile unless you wish to select a specific version. The user environment is normally correctly set up. To force the library version current at the time of writing, add as first flag to the linker options:
| |
| <code>-L/opt/intel/mkl/10.0.2.018/lib/em64t</code> (automatically set in <code>$LD_LIBRARY_PATH</code>).
| |
| * see [[HPC/Carbon_Cluster_-_Getting_started#Example_job_file]] for controlling OpenMP execution.
| |
| | |
| === [http://www.vtk.org/ VTK] ===
| |
| * [https://visualization.hpc.mil/wiki/VTK Intro]
| |
| * [https://visualization.hpc.mil/wiki/Getting_Started_with_VTK VTK install notes]
| |
| | |
| === [http://www.mcs.anl.gov/petsc PETSc] ===
| |
| * TODO: Portable, Extensible Toolkit for Scientific Computation
| |
|
| |
|
| ==MPI== | | ==MPI== |
|
| |
|
| ===[http://www.open-mpi.org/ OpenMPI]=== | | === [http://www.open-mpi.org/ OpenMPI] === |
| This is the primary and recommended MPI version. It supports Ethernet and Infiniband interconnect. | | This is the primary and recommended MPI version. It supports Ethernet and InfiniBand interconnect, with InfiniBand set as default by means of an [http://www.open-mpi.org/faq/?category=openfabrics#ib-btl <code>OMPI_MCA_…</code>] environment variable. |
| * See also: [http://www.open-mpi.org/faq FAQ]
| | * [http://www.open-mpi.org/doc/ OpenMPI Documentation] |
| | | * [http://www.open-mpi.org/faq FAQ] |
| ===[http://www-unix.mcs.anl.gov/mpi/mpich1/ MPICH]/[http://www.mcs.anl.gov/research/projects/mpich2/ MPIC2]=== | |
| | |
| TBD
| |
| | |
| ===[http://www.intel.com/cd/software/products/asmo-na/eng/308295.htm#support Intel MPI]===
| |
| | |
| * [http://www.intel.com/cd/software/products/asmo-na/eng/368005.htm Documentation] | |
| | |
| ==Resource Management==
| |
|
| |
|
| * [http://www.clusterresources.com/pages/products/moab-cluster-suite.php MOAB cluster suite] | | === Intel MPI === |
| ** [http://www.clusterresources.com/pages/resources/documentation.php Documentation overview]
| | * See [[HPC/Software/Modules/impi]] |
| ** Torque
| |
| ** Gold allocation manager
| |
|
| |
|
| (description of overlap/differences)
| | ===[http://www-unix.mcs.anl.gov/mpi/mpich1/ MPICH]/[http://www.mcs.anl.gov/research/projects/mpich2/ MPICH2]=== |
| | * Not installed explicitly, but Intel-MPI is based on MPICH2 and provides an MPICH2 runtime environment for precompiled binaries. |
|
| |
|
| {{Template:Standard Footer}}
| | [[Category:HPC|Development]] |