HPC/Applications/python: Difference between revisions

From CNM Wiki
Jump to navigation Jump to search
mNo edit summary
Line 2: Line 2:


=== Interpreter-only releases from the Python Software Foundation ===
=== Interpreter-only releases from the Python Software Foundation ===
Releases of just a Python interpreter from the [http://www.python.org python.org] sources:
; Contains:
:* The Python interpreter only, from the [http://www.python.org python.org] sources.
<!-- '''<font color="green">python</font>/<font color="green">''pyMajor.pyMinor''</font>'''/<font color="#c80">''compilerName-compMajor.compMinor''</font>/<font color="#888">''pyMajor.pyMinor.pyPatch''-''carbonBuild''</font> -->
<!-- '''<font color="green">python</font>/<font color="green">''pyMajor.pyMinor''</font>'''/<font color="#c80">''compilerName-compMajor.compMinor''</font>/<font color="#888">''pyMajor.pyMinor.pyPatch''-''carbonBuild''</font> -->
'''python'''/''pyMajor.pyMinor''/''compilerName-compMajor.compMinor''/''pyMajor.pyMinor.pyPatch-moduleBuild''
; Module nomenclature:
: <code>'''python'''/''pyMajor.pyMinor''/''compilerName-compMajor.compMinor''/''pyMajor.pyMinor.pyPatch-moduleBuild''</code>


=== Interpreter releases bundled with the OS ===
=== Interpreter releases bundled with the OS ===
The Python interpreter that comes with the operating system is required for many system-provided and add-on packages.
; Contains:
It normally does not need to be loaded as a module.
:* The Python interpreter that comes with the operating system is required for many system-provided and add-on packages.
It is included here for visibility in the module system, and as a base for (usually older) modules which provide Python packages outside of Conda or Pip. These packages often are tied to the interpreter version under which they were installed.
: Such a module normally does not need to be loaded. It is included here for visibility in the module system, and as a base for (usually older) modules which provide Python packages outside of Conda or Pip. Such packages are often tied to the version of the Python interpreter under which they were installed.
'''python-''osname'''''/''pyMajor.pyMinor''/''compilerName-compMajor.compMinor''/''pyMajor.pyMinor.pyPatch''
; Module nomenclature:
: <code>'''python-''osname'''''/''pyMajor.pyMinor''/''compilerName-compMajor.compMinor''/''pyMajor.pyMinor.pyPatch''</code>


=== Conda-based Python suites ===
=== Conda-based Python suites ===
These packages are not specific applications but rather wide-ranging ''software suites'' which typically contain:
; Contains:
* a Python interpreter
:* Wide-ranging Python-based ''software suites'', rather than specific HPC applications.
* the conda/pip package management systems,
; Module nomenclature:
* a rich set of Python-native packages and add-on executables as selected by the distributor,
: <code>'''python-''distributor'''''/''pyMajor.pyMinor''/''distMajor''/''distributor_defined_version[…]-moduleBuild''</code>
* all underlying libraries like LAPACK and possibly even MPI.
 
** Idle – Python Integrated DeveLopment Environment
 
The suites typically contain:
* A Python interpreter.
* The conda/pip package management systems.
* Idle – Python Integrated DeveLopment Environment
* A rich set of Python-native packages, extensions, and add-on executables, '''as selected by the distributor''', usually including
** iPython and juPyter – Interactive computation system
** Cython – Compiler for a superset of Python and C/C++
* Underlying libraries and supporting binaries, e.g.:
** Tcl – Interpreter for the Tool Command Language
** Tcl – Interpreter for the Tool Command Language
** BLAS/LAPACK – Linear algebra libraries
** MPI – Runtime environment for parallel applications using Message Passing Interface, providing the <code>mpirun</code> or <code>mpiexec</code> commands, plus related binaries and libraries.
** MPI – Runtime environment for parallel applications using Message Passing Interface, providing the <code>mpirun</code> or <code>mpiexec</code> commands, plus related binaries and libraries.


Line 29: Line 41:
Doing so may lead to non-obvious failures with either the bundled parallel Python applications or (worse) unrelated compiled applications on Carbon.
Doing so may lead to non-obvious failures with either the bundled parallel Python applications or (worse) unrelated compiled applications on Carbon.
</div>
</div>
'''python-''distributor'''''/''pyMajor.pyMinor''/''distMajor''/''distributor_defined_version[…]-moduleBuild''


=== Conda environments ===
=== Conda environments ===

Revision as of 23:16, January 9, 2018

Python variants

Interpreter-only releases from the Python Software Foundation

Contains
  • The Python interpreter only, from the python.org sources.
Module nomenclature
python/pyMajor.pyMinor/compilerName-compMajor.compMinor/pyMajor.pyMinor.pyPatch-moduleBuild

Interpreter releases bundled with the OS

Contains
  • The Python interpreter that comes with the operating system is required for many system-provided and add-on packages.
Such a module normally does not need to be loaded. It is included here for visibility in the module system, and as a base for (usually older) modules which provide Python packages outside of Conda or Pip. Such packages are often tied to the version of the Python interpreter under which they were installed.
Module nomenclature
python-osname/pyMajor.pyMinor/compilerName-compMajor.compMinor/pyMajor.pyMinor.pyPatch

Conda-based Python suites

Contains
  • Wide-ranging Python-based software suites, rather than specific HPC applications.
Module nomenclature
python-distributor/pyMajor.pyMinor/distMajor/distributor_defined_version[…]-moduleBuild


The suites typically contain:

  • A Python interpreter.
  • The conda/pip package management systems.
  • Idle – Python Integrated DeveLopment Environment
  • A rich set of Python-native packages, extensions, and add-on executables, as selected by the distributor, usually including
    • iPython and juPyter – Interactive computation system
    • Cython – Compiler for a superset of Python and C/C++
  • Underlying libraries and supporting binaries, e.g.:
    • Tcl – Interpreter for the Tool Command Language
    • BLAS/LAPACK – Linear algebra libraries
    • MPI – Runtime environment for parallel applications using Message Passing Interface, providing the mpirun or mpiexec commands, plus related binaries and libraries.

Caveat: An MPI Runtime within a Python subsystem makes parallel computing easily accessible under Python, but this means a "mere python module" can clash with other MPI implementations on Carbon.

Avoid loading both a Python suite and an MPI implementation at the same time, especially in your dot-files. Doing so may lead to non-obvious failures with either the bundled parallel Python applications or (worse) unrelated compiled applications on Carbon.

Conda environments

These packages are conda environments, derived from one of the conda suites , with package selection customized for Carbon: 

python-env-distributor/pyMajor.pyMinor/distMajor/pyVersion[…]-moduleBuild
# or?
python-env-distributor-Carbon/pyMajor.pyMinor/distMajor/pyVersion[…]-moduleBuild
# Possible extension for alternative package selections:
python-env-distributor-purpose/pyMajor.pyMinor/distMajor/pyVersion[…]-moduleBuild
Retrieved from "https://wiki.anl.gov/wiki_cnm/index.php?title=HPC/Applications/python&oldid=6184"