HPC/Applications/python: Difference between revisions

From CNM Wiki
Jump to navigation Jump to search
Line 2: Line 2:


== Interpreter-only releases from the Python Software Foundation ==
== Interpreter-only releases from the Python Software Foundation ==
; Module nomenclature:
: <code>'''python'''/''pyMajor.pyMinor''/''compilerName-compMajor.compMinor''/''pyMajor.pyMinor.pyPatch-moduleBuild''</code>
; Contains:
; Contains:
:* The Python interpreter only, from the [http://www.python.org python.org] sources.
:* The Python interpreter only, from the [http://www.python.org python.org] sources.
<!-- '''<font color="green">python</font>/<font color="green">''pyMajor.pyMinor''</font>'''/<font color="#c80">''compilerName-compMajor.compMinor''</font>/<font color="#888">''pyMajor.pyMinor.pyPatch''-''carbonBuild''</font> -->
<!-- '''<font color="green">python</font>/<font color="green">''pyMajor.pyMinor''</font>'''/<font color="#c80">''compilerName-compMajor.compMinor''</font>/<font color="#888">''pyMajor.pyMinor.pyPatch''-''carbonBuild''</font> -->
; Module nomenclature:
: <code>'''python'''/''pyMajor.pyMinor''/''compilerName-compMajor.compMinor''/''pyMajor.pyMinor.pyPatch-moduleBuild''</code>


== Python interpreters and packages as bundled with the OS ==
== Python interpreters and packages as bundled with the OS ==
; Module nomenclature:
: <code>'''python-''osname'''''/''pyMajor.pyMinor''/''compilerName-compMajor.compMinor''/''pyMajor.pyMinor.pyPatch''</code>
; Contains:
; Contains:
:* The Python interpreter that comes with the operating system is required for many system-provided and add-on packages.
:* The Python interpreter that comes with the operating system is required for many system-provided and add-on packages.
; Module nomenclature:
: <code>'''python-''osname'''''/''pyMajor.pyMinor''/''compilerName-compMajor.compMinor''/''pyMajor.pyMinor.pyPatch''</code>
; Notes:
; Notes:
: Such a module normally does not need to be loaded. It is included here for visibility in the module system, and as a base for (usually older) modules which provide Python packages outside of Conda or Pip. Such packages are often tied to the version of the Python interpreter under which they were installed.
: Such a module normally does not need to be loaded. It is included here for visibility in the module system, and as a base for (usually older) modules which provide Python packages outside of Conda or Pip. Such packages are often tied to the version of the Python interpreter under which they were installed.


== Conda-based Python suites ==
== Conda-based Python suites ==
; Module nomenclature:
: <code>'''python-''distributor'''''/''pyMajor.pyMinor''/''distMajor''/''distributor_defined_version[…]-moduleBuild''</code>
; Contains:
; Contains:
:* Wide-ranging Python-based software suites (as opposed to more narrowly scoped applications or libraries), with the '''package selection as distributed by the vendor.'''
:* Wide-ranging Python-based software suites (as opposed to more narrowly scoped applications or libraries), with the '''package selection as distributed by the vendor.'''
; Module nomenclature:
: <code>'''python-''distributor'''''/''pyMajor.pyMinor''/''distMajor''/''distributor_defined_version[…]-moduleBuild''</code>




Line 44: Line 44:


== Custom Conda environments ==
== Custom Conda environments ==
; Contains:
:* Conda ''environments'' derived from a vendor-distributed Conda suite, with the '''package selection customized for Carbon.'''
; Module nomenclature:
; Module nomenclature:
: <code>'''python-env-''distributor'''''/''pyMajor.pyMinor''/''distMajor''/''pyVersion[…]-moduleBuild''</code>
: <code>'''python-env-''distributor'''''/''pyMajor.pyMinor''/''distMajor''/''pyVersion[…]-moduleBuild''</code>
:: A default selection of packages deemed useful for jobs on Carbon.
: <code>'''python-env-''distributor''-''purpose'''''/''pyMajor.pyMinor''/''distMajor''/''pyVersion[…]-moduleBuild''</code>
:: Same, with alternative package selections. (TBD)
<!-- # or?
<!-- # or?
  '''python-env-''distributor''-Carbon'''/''pyMajor.pyMinor''/''distMajor''/''pyVersion[…]-moduleBuild''
  '''python-env-''distributor''-Carbon'''/''pyMajor.pyMinor''/''distMajor''/''pyVersion[…]-moduleBuild''
-->
-->
 
; Contains:
Possible extension for alternative package selections:
:* Conda ''environments'' derived from a vendor-distributed Conda suite, with the '''package selection customized for Carbon.'''
'''python-env-''distributor''-''purpose'''''/''pyMajor.pyMinor''/''distMajor''/''pyVersion[…]-moduleBuild''

Revision as of 23:33, January 9, 2018

Python variants

Interpreter-only releases from the Python Software Foundation

Module nomenclature
python/pyMajor.pyMinor/compilerName-compMajor.compMinor/pyMajor.pyMinor.pyPatch-moduleBuild
Contains
  • The Python interpreter only, from the python.org sources.

Python interpreters and packages as bundled with the OS

Module nomenclature
python-osname/pyMajor.pyMinor/compilerName-compMajor.compMinor/pyMajor.pyMinor.pyPatch
Contains
  • The Python interpreter that comes with the operating system is required for many system-provided and add-on packages.
Notes
Such a module normally does not need to be loaded. It is included here for visibility in the module system, and as a base for (usually older) modules which provide Python packages outside of Conda or Pip. Such packages are often tied to the version of the Python interpreter under which they were installed.

Conda-based Python suites

Module nomenclature
python-distributor/pyMajor.pyMinor/distMajor/distributor_defined_version[…]-moduleBuild
Contains
  • Wide-ranging Python-based software suites (as opposed to more narrowly scoped applications or libraries), with the package selection as distributed by the vendor.


The suites typically contain:

  • A Python interpreter as a base.
  • The conda/pip package management systems.
  • A rich set of Python-native packages, extensions, and add-on executables, often including
    • Idle – Python Integrated DeveLopment Environment
    • iPython and juPyter – Interactive computation system
    • Cython – Compiler for a superset of Python and C/C++
  • Underlying libraries and supporting binaries, e.g.:
    • Tcl – Interpreter for the Tool Command Language
    • BLAS/LAPACK – Linear algebra libraries
    • MPI – Runtime environment for parallel applications using Message Passing Interface, providing the mpirun or mpiexec commands, plus related binaries and libraries.

Caveat: An MPI Runtime within a Python suite makes parallel computing easily accessible under Python, but this means a "mere python module" can clash with other MPI implementations used on Carbon.

Avoid loading both a Python suite and an MPI implementation at the same time, especially in your dot-files. Doing so may lead to non-obvious failures with either the bundled parallel Python applications or (worse) unrelated compiled applications on Carbon.

Custom Conda environments

Module nomenclature
python-env-distributor/pyMajor.pyMinor/distMajor/pyVersion[…]-moduleBuild
A default selection of packages deemed useful for jobs on Carbon.
python-env-distributor-purpose/pyMajor.pyMinor/distMajor/pyVersion[…]-moduleBuild
Same, with alternative package selections. (TBD)
Contains
  • Conda environments derived from a vendor-distributed Conda suite, with the package selection customized for Carbon.
Retrieved from "https://wiki.anl.gov/wiki_cnm/index.php?title=HPC/Applications/python&oldid=6192"