HPC/Directories - Revision history

Stern: /* Local RAM disk */

2022-07-18T19:50:58Z

Local RAM disk

← Older revision		Revision as of 19:50, July 18, 2022
Line 116:		Line 116:
	: In contrast to explicit trailing "cp" commands in the job script, this copy will be executed ''even if a job overruns its walltime''. See the [http://www.clusterresources.com/torquedocs21/commands/qsub.shtml#W qsub manual] for further information.		: In contrast to explicit trailing "cp" commands in the job script, this copy will be executed ''even if a job overruns its walltime''. See the [http://www.clusterresources.com/torquedocs21/commands/qsub.shtml#W qsub manual] for further information.

			<!--
	== Local RAM disk ==		== Local RAM disk ==
	(2020-05-21: to be added)		(2020-05-21: to be added)
	~~<!--~~ [https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html XDG Base Directory Specification] <code>$XDG_RUNTIME_DIR</code> – intended for small files. -->		[https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html XDG Base Directory Specification] <code>$XDG_RUNTIME_DIR</code> – intended for small files.
			-->

	<!--		<!--

Stern: /* Global scratch space */

2021-04-30T21:10:24Z

Global scratch space

← Older revision		Revision as of 21:10, April 30, 2021
Line 75:		Line 75:
	These limits are subject to change.		These limits are subject to change.

	The limits are aimed at keeping space available for the intended use by files of unusual size ([http://tvtropes.org/pmwiki/pmwiki.php/Main/RodentsOfUnusualSize F.O.U.S.]) or~~, conversely,~~ small files of unusual count.		The limits are aimed at keeping the space available for the intended use, typically for files of unusual size ([http://tvtropes.org/pmwiki/pmwiki.php/Main/RodentsOfUnusualSize F.O.U.S.]) or for small files of unusual count.

	== Local scratch space ==		== Local scratch space ==

Stern: /* Global scratch space */

2021-04-30T21:09:07Z

Global scratch space

← Older revision		Revision as of 21:09, April 30, 2021
Line 73:		Line 73:
	* Files will be ''deleted automatically'' once they are older than 3 months.		* Files will be ''deleted automatically'' once they are older than 3 months.

	These limits (subject to change) are aimed at keeping space available for the intended use by files of unusual size ([http://tvtropes.org/pmwiki/pmwiki.php/Main/RodentsOfUnusualSize F.O.U.S.]) or, conversely, small files of unusual count.		These limits are subject to change.

			The limits are aimed at keeping space available for the intended use by files of unusual size ([http://tvtropes.org/pmwiki/pmwiki.php/Main/RodentsOfUnusualSize F.O.U.S.]) or, conversely, small files of unusual count.

	== Local scratch space ==		== Local scratch space ==

Stern: /* Global scratch space */

2021-04-30T21:08:30Z

Global scratch space

← Older revision		Revision as of 21:08, April 30, 2021
Line 71:		Line 71:
	* Soft quota are 10 GB and 10,000 files.		* Soft quota are 10 GB and 10,000 files.
	* The grace period for overflowing a soft limit is 3 weeks.		* The grace period for overflowing a soft limit is 3 weeks.
	* Files will be ''deleted automatically'' once they are older than ~~4 weeks~~.		* Files will be ''deleted automatically'' once they are older than 3 months.

	These limits (subject to change) are aimed at keeping space available for the intended use by files of unusual size ([http://tvtropes.org/pmwiki/pmwiki.php/Main/RodentsOfUnusualSize F.O.U.S.]) or, conversely, small files of unusual count.		These limits (subject to change) are aimed at keeping space available for the intended use by files of unusual size ([http://tvtropes.org/pmwiki/pmwiki.php/Main/RodentsOfUnusualSize F.O.U.S.]) or, conversely, small files of unusual count.

Stern: /* Use in job files */

2020-10-16T23:30:27Z

Use in job files

← Older revision		Revision as of 23:30, October 16, 2020
Line 110:		Line 110:
	#PBS -W stageout=<font color="red">$TMPDIR/'''*.bar'''</font>@<font color="green">localhost:$PBS_O_WORKDIR/''subdir''</font>		#PBS -W stageout=<font color="red">$TMPDIR/'''*.bar'''</font>@<font color="green">localhost:$PBS_O_WORKDIR/''subdir''</font>
	* The <font color="red">string on the left</font> of the <code>@</code> character names the ''source'' files in the compute node file system.		* The <font color="red">string on the left</font> of the <code>@</code> character names the ''source'' files in the compute node file system.
	* The <font color="green">string on the right</~~code~~> gives the ''destination'' host and directory, also as seen from the compute node. This means <font color="green">localhost</font> refers to the primary compute node (rank 0 in MPI parlance). <font color="green">$PBS_O_WORKDIR</font> by default stems from what was qsub's current directory on the ''submission'' node, but ''Carbon'' user file systems are mounted on login nodes and compute nodes under the same paths.		* The <font color="green">string on the right</font> gives the ''destination'' host and directory, also as seen from the compute node. This means <font color="green">localhost</font> refers to the primary compute node (rank 0 in MPI parlance). <font color="green">$PBS_O_WORKDIR</font> by default stems from what was qsub's current directory on the ''submission'' node, but ''Carbon's'' user file systems are mounted on login nodes and compute nodes under the same paths.
	* You may give the <code>stageout=…</code> directive multiple times, as shown.		* You may give the <code>stageout=…</code> directive multiple times, as shown.
	: In contrast to explicit trailing "cp" commands in the job script, this copy will be executed ''even if a job overruns its walltime''. See the [http://www.clusterresources.com/torquedocs21/commands/qsub.shtml#W qsub manual] for further information.		: In contrast to explicit trailing "cp" commands in the job script, this copy will be executed ''even if a job overruns its walltime''. See the [http://www.clusterresources.com/torquedocs21/commands/qsub.shtml#W qsub manual] for further information.

Stern: /* Use in job files */

2020-10-16T23:29:26Z

Use in job files

← Older revision		Revision as of 23:29, October 16, 2020
Line 109:		Line 109:
	#PBS -W stageout=<font color="red">$TMPDIR/'''foo.ext'''</font>@<font color="green">localhost:$PBS_O_WORKDIR</font>		#PBS -W stageout=<font color="red">$TMPDIR/'''foo.ext'''</font>@<font color="green">localhost:$PBS_O_WORKDIR</font>
	#PBS -W stageout=<font color="red">$TMPDIR/'''*.bar'''</font>@<font color="green">localhost:$PBS_O_WORKDIR/''subdir''</font>		#PBS -W stageout=<font color="red">$TMPDIR/'''*.bar'''</font>@<font color="green">localhost:$PBS_O_WORKDIR/''subdir''</font>
	** You may use wildcards to specify source <font color="red">files on the compute ~~nodes~~</font>.		* The <font color="red">string on the left</font> of the <code>@</code> character names the ''source'' files in the compute node file system.
	** You may give the <code>stageout=…</code> directive multiple times, as shown.		* The <font color="green">string on the right</code> gives the ''destination'' host and directory, also as seen from the compute node. This means <font color="green">localhost</font> refers to the primary compute node (rank 0 in MPI parlance). <font color="green">$PBS_O_WORKDIR</font> by default stems from what was qsub's current directory on the ''submission'' node, but ''Carbon'' user file systems are mounted on login nodes and compute nodes under the same paths.
			* You may give the <code>stageout=…</code> directive multiple times, as shown.
	: In contrast to explicit trailing "cp" commands in the job script, this copy will be executed ''even if a job overruns its walltime''. See the [http://www.clusterresources.com/torquedocs21/commands/qsub.shtml#W qsub manual] for further information.		: In contrast to explicit trailing "cp" commands in the job script, this copy will be executed ''even if a job overruns its walltime''. See the [http://www.clusterresources.com/torquedocs21/commands/qsub.shtml#W qsub manual] for further information.

Stern: /* Use in job files */

2020-10-16T23:15:16Z

Use in job files

← Older revision		Revision as of 23:15, October 16, 2020
Line 108:		Line 108:
	* copy files back upon job termination:		* copy files back upon job termination:
	#PBS -W stageout=<font color="red">$TMPDIR/'''foo.ext'''</font>@<font color="green">localhost:$PBS_O_WORKDIR</font>		#PBS -W stageout=<font color="red">$TMPDIR/'''foo.ext'''</font>@<font color="green">localhost:$PBS_O_WORKDIR</font>
	#PBS -W stageout=<font color="red">$TMPDIR/'''*.bar'''</font>@<font color="green">localhost:$PBS_O_WORKDIR</font>		#PBS -W stageout=<font color="red">$TMPDIR/'''*.bar'''</font>@<font color="green">localhost:$PBS_O_WORKDIR/''subdir''</font>
	** You may use wildcards to specify source <font color="red">files on the compute nodes</font>.		** You may use wildcards to specify source <font color="red">files on the compute nodes</font>.
	** You may give the <code>stageout=…</code> directive multiple times, as shown.		** You may give the <code>stageout=…</code> directive multiple times, as shown.

Stern: /* Use in job files */

2020-10-16T23:14:29Z

Use in job files

← Older revision		Revision as of 23:14, October 16, 2020
Line 109:		Line 109:
	#PBS -W stageout=<font color="red">$TMPDIR/'''foo.ext'''</font>@<font color="green">localhost:$PBS_O_WORKDIR</font>		#PBS -W stageout=<font color="red">$TMPDIR/'''foo.ext'''</font>@<font color="green">localhost:$PBS_O_WORKDIR</font>
	#PBS -W stageout=<font color="red">$TMPDIR/'''*.bar'''</font>@<font color="green">localhost:$PBS_O_WORKDIR</font>		#PBS -W stageout=<font color="red">$TMPDIR/'''*.bar'''</font>@<font color="green">localhost:$PBS_O_WORKDIR</font>
	: You may ~~specify several of these lines and~~ use wildcards in to specify source <font color="red">files on the compute nodes</font>. In contrast to explicit trailing "cp" commands in the job script, this copy will be executed ''even if a job overruns its walltime''. See the [http://www.clusterresources.com/torquedocs21/commands/qsub.shtml#W qsub manual] for further information.		** You may use wildcards to specify source <font color="red">files on the compute nodes</font>.
			** You may give the <code>stageout=…</code> directive multiple times, as shown.
			: In contrast to explicit trailing "cp" commands in the job script, this copy will be executed ''even if a job overruns its walltime''. See the [http://www.clusterresources.com/torquedocs21/commands/qsub.shtml#W qsub manual] for further information.

	== Local RAM disk ==		== Local RAM disk ==

Stern at 22:06, May 21, 2020

2020-05-21T22:06:11Z

← Older revision		Revision as of 22:06, May 21, 2020
Line 113:		Line 113:
	== Local RAM disk ==		== Local RAM disk ==
	(2020-05-21: to be added)		(2020-05-21: to be added)
	* Advanced users: see [https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html XDG Base Directory Specification]		<!-- [https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html XDG Base Directory Specification] <code>$XDG_RUNTIME_DIR</code> – intended for small files. -->

	<!--		<!--

Stern: /* Local RAM disk */

2020-05-21T22:02:14Z

Local RAM disk

← Older revision		Revision as of 22:02, May 21, 2020
Line 113:		Line 113:
	== Local RAM disk ==		== Local RAM disk ==
	(2020-05-21: to be added)		(2020-05-21: to be added)
			* Advanced users: see [https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html XDG Base Directory Specification]

	<!--		<!--