HPC/Benchmarks/Generation 1 vs 2 - Revision history

Stern: /* Introduction */

2011-05-24T21:55:14Z

Introduction

← Older revision		Revision as of 21:55, May 24, 2011
Line 3:		Line 3:
	\|}		\|}
	== Introduction ==		== Introduction ==
	~~Earlier this year~~, we received 200 additional nodes with two E5540 processors each. The processors have 4 cores each, and support hyperthreading, a feature which allows 2 threads per core. This benchmark investigates the benefit of hyperthreading (HT), and suggests optimal values for the ''nodes'' and ''ppn'' (processors per node) parameters in PBS. The choice is not trivial and involves tradeoffs between various metrics such as execution time or compute-hours charged.		In early 2010, we received 200 additional nodes with two E5540 processors each. The processors have 4 cores each, and support hyperthreading, a feature which allows 2 threads per core. This benchmark investigates the benefit of hyperthreading (HT), and suggests optimal values for the ''nodes'' and ''ppn'' (processors per node) parameters in PBS. The choice is not trivial and involves tradeoffs between various metrics such as execution time or compute-hours charged.

	== Test description ==		== Test description ==

Stern: /* Charge rates */

2011-05-09T12:34:39Z

Charge rates

← Older revision		Revision as of 12:34, May 9, 2011
Line 182:		Line 182:
	=== Charge rates ===		=== Charge rates ===
	Given the above benchmark results, the charge rate on gen1 nodes was set to 0.5 early on (soon after the gen2 nodes were put into production). This means that a task will incur approximately the same charge when run on either gen1 or gen2 nodes. Short tasks even have a certain advantage to run on gen1 nodes, since often the wait time there is lower, while the longer execution time matters less for the user (being small to begin with).		Given the above benchmark results, the charge rate on gen1 nodes was set to 0.5 early on (soon after the gen2 nodes were put into production). This means that a task will incur approximately the same charge when run on either gen1 or gen2 nodes. Short tasks even have a certain advantage to run on gen1 nodes, since often the wait time there is lower, while the longer execution time matters less for the user (being small to begin with).

			Charges in the benchmark above are raw, without this factor, i.e, ''charge cores <math>\times</math> wall time'', where ''charge cores'' is the number of cores reserved by Moab.

	--[[User:Stern\|stern]]		--[[User:Stern\|stern]]

Stern: /* Gen1 nodes */

2011-05-09T12:29:20Z

Gen1 nodes

← Older revision		Revision as of 12:29, May 9, 2011
Line 44:		Line 44:
	{\| class="wikitable sortable" cellpadding="4" style="text-align:center; margin: 1em auto 1em auto;"		{\| class="wikitable sortable" cellpadding="4" style="text-align:center; margin: 1em auto 1em auto;"
	\|- style="background:#eee;"		\|- style="background:#eee;"
	! cores/app<br><br> !! run<br><br> !! nodes<br><br> !! ppn<br><br> !! napps<br><br> !! tmax<br><br> !! charge-<br>cores<br> !! charge<br>(core-h/app)<br> !! perf<br>(ops/s/core)<br> !! combined<br>objective<br> !! HT<br><br>		! cores/app<br><br> !! run<br><br> !! nodes<br><br> !! ppn<br><br> !! napps<br><br> !! tmax<br><br> !! charge-<br>cores<br> !! charge raw<br>(core-h/app)<br> !! perf<br>(ops/s/core)<br> !! combined<br>objective<br> !! HT<br><br>
	\|-		\|-
	\| 4 \|\| 1 \|\| <font color="#0a0">'''1'''</font> \|\| <font color="#0a0">'''4'''</font> \|\| 1 \|\| 1138.83 \|\| 8 \|\| <font color="#0a0">'''2.53'''</font> \|\| <font color="#0a0">'''11.0'''</font> \|\| 28.82 \|\|		\| 4 \|\| 1 \|\| <font color="#0a0">'''1'''</font> \|\| <font color="#0a0">'''4'''</font> \|\| 1 \|\| 1138.83 \|\| 8 \|\| <font color="#0a0">'''2.53'''</font> \|\| <font color="#0a0">'''11.0'''</font> \|\| 28.82 \|\|

Stern: /* Gen2 nodes */

2011-05-09T12:28:59Z

Gen2 nodes

← Older revision		Revision as of 12:28, May 9, 2011
Line 82:		Line 82:
	{\| class="wikitable sortable" cellpadding="4" style="text-align:center; margin: 1em auto 1em auto;"		{\| class="wikitable sortable" cellpadding="4" style="text-align:center; margin: 1em auto 1em auto;"
	\|- style="background:#eee;"		\|- style="background:#eee;"
	! cores/app<br><br> !! run<br><br> !! nodes<br><br> !! ppn<br><br> !! napps<br><br> !! tmax<br><br> !! charge-<br>cores<br> !! charge<br>(core-h/app)<br> !! perf<br>(ops/s/core)<br> !! combined<br>objective<br> !! HT<br><br>		! cores/app<br><br> !! run<br><br> !! nodes<br><br> !! ppn<br><br> !! napps<br><br> !! tmax<br><br> !! charge-<br>cores<br> !! charge raw<br>(core-h/app)<br> !! perf<br>(ops/s/core)<br> !! combined<br>objective<br> !! HT<br><br>
	\|-		\|-
	\| 4 \|\| 2 \|\| 1 \|\| 4 \|\| 1 \|\| 516.69 \|\| 8 \|\| 1.15 \|\| 24.2 \|\| 5.93 \|\|		\| 4 \|\| 2 \|\| 1 \|\| 4 \|\| 1 \|\| 516.69 \|\| 8 \|\| 1.15 \|\| 24.2 \|\| 5.93 \|\|

Stern: /* Update */

2011-05-09T12:27:28Z

Update

← Older revision		Revision as of 12:27, May 9, 2011
Line 173:		Line 173:

	== Update ==		== Update ==
			=== Adjusting VASP input ===
	The [http://cms.mpi.univie.ac.at/vasp/vasp/vasp.html VASP manual] recommends [http://cms.mpi.univie.ac.at/vasp/vasp/node146.html NPAR] to ''number of nodes'' (understood to be number of cores, since at the time single-core CPUs were all the rage). Nowadays, it would be better to set		The [http://cms.mpi.univie.ac.at/vasp/vasp/vasp.html VASP manual] recommends [http://cms.mpi.univie.ac.at/vasp/vasp/node146.html NPAR] to ''number of nodes'' (understood to be number of cores, since at the time single-core CPUs were all the rage). Nowadays, it would be better to set
	NPAR = number of nodes		NPAR = number of nodes
Line 178:		Line 179:

	(Credit: M. Chan).		(Credit: M. Chan).

			=== Charge rates ===
			Given the above benchmark results, the charge rate on gen1 nodes was set to 0.5 early on (soon after the gen2 nodes were put into production). This means that a task will incur approximately the same charge when run on either gen1 or gen2 nodes. Short tasks even have a certain advantage to run on gen1 nodes, since often the wait time there is lower, while the longer execution time matters less for the user (being small to begin with).

	--[[User:Stern\|stern]]		--[[User:Stern\|stern]]

Stern: /* Update */

2010-04-22T18:41:36Z

Update

← Older revision		Revision as of 18:41, April 22, 2010
Line 175:		Line 175:
	The [http://cms.mpi.univie.ac.at/vasp/vasp/vasp.html VASP manual] recommends [http://cms.mpi.univie.ac.at/vasp/vasp/node146.html NPAR] to ''number of nodes'' (understood to be number of cores, since at the time single-core CPUs were all the rage). Nowadays, it would be better to set		The [http://cms.mpi.univie.ac.at/vasp/vasp/vasp.html VASP manual] recommends [http://cms.mpi.univie.ac.at/vasp/vasp/node146.html NPAR] to ''number of nodes'' (understood to be number of cores, since at the time single-core CPUs were all the rage). Nowadays, it would be better to set
	NPAR = number of nodes		NPAR = number of nodes
	meaning the number of SMP nodes, which is the current state of the art.		meaning the ''number of SMP nodes'', which is the current state of the art.

	(Credit: M. Chan).		(Credit: M. Chan).

	--[[User:Stern\|stern]]		--[[User:Stern\|stern]]

Stern: /* Update */

2010-04-22T18:41:11Z

Update

← Older revision		Revision as of 18:41, April 22, 2010
Line 175:		Line 175:
	The [http://cms.mpi.univie.ac.at/vasp/vasp/vasp.html VASP manual] recommends [http://cms.mpi.univie.ac.at/vasp/vasp/node146.html NPAR] to ''number of nodes'' (understood to be number of cores, since at the time single-core CPUs were all the rage). Nowadays, it would be better to set		The [http://cms.mpi.univie.ac.at/vasp/vasp/vasp.html VASP manual] recommends [http://cms.mpi.univie.ac.at/vasp/vasp/node146.html NPAR] to ''number of nodes'' (understood to be number of cores, since at the time single-core CPUs were all the rage). Nowadays, it would be better to set
	NPAR = number of nodes		NPAR = number of nodes
	meaning the number of SMP nodes, which is the state of the art ~~these days~~.		meaning the number of SMP nodes, which is the current state of the art.

	(Credit: M. Chan).		(Credit: M. Chan).

	--[[User:Stern\|stern]]		--[[User:Stern\|stern]]

Stern: /* Update */

2010-04-22T16:43:24Z

Update

← Older revision		Revision as of 16:43, April 22, 2010
Line 176:		Line 176:
	NPAR = number of nodes		NPAR = number of nodes
	meaning the number of SMP nodes, which is the state of the art these days.		meaning the number of SMP nodes, which is the state of the art these days.

			(Credit: M. Chan).

	--[[User:Stern\|stern]]		--[[User:Stern\|stern]]

Stern: NPAR

2010-04-22T16:42:56Z

NPAR

← Older revision		Revision as of 16:42, April 22, 2010
Line 171:		Line 171:
	\|}		\|}
	The last column is an empirically formulated objective to minimize both time and charge. Compared to the minimum-charge runs (Objective columns 2, runs 01 and 22, respectively), adding nodes will reduce the runtime and only slighty increase the charge. The fastest runs (first Objective column) use the same number of cores for calculation, but since the number of nodes is higher, so is charge_cores, and thus the job charge is also higher.		The last column is an empirically formulated objective to minimize both time and charge. Compared to the minimum-charge runs (Objective columns 2, runs 01 and 22, respectively), adding nodes will reduce the runtime and only slighty increase the charge. The fastest runs (first Objective column) use the same number of cores for calculation, but since the number of nodes is higher, so is charge_cores, and thus the job charge is also higher.

			== Update ==
			The [http://cms.mpi.univie.ac.at/vasp/vasp/vasp.html VASP manual] recommends [http://cms.mpi.univie.ac.at/vasp/vasp/node146.html NPAR] to ''number of nodes'' (understood to be number of cores, since at the time single-core CPUs were all the rage). Nowadays, it would be better to set
			NPAR = number of nodes
			meaning the number of SMP nodes, which is the state of the art these days.

	--[[User:Stern\|stern]]		--[[User:Stern\|stern]]

Stern: /* Recommendations */ add gen

2010-03-21T08:19:55Z

Recommendations: add gen

← Older revision		Revision as of 08:19, March 21, 2010
Line 155:		Line 155:
	\| width="200px" \| time × charge → min		\| width="200px" \| time × charge → min
	\|-		\|-
	\| gen1 \|\| <font color="#0a0">'''<code>nodes=4:ppn=3</code>'''</font> \|\| <font color="#0a0">'''<code>nodes=1:ppn=4</code>'''</font> \|\| <font color="#0a0">'''<code>nodes=3:ppn=4</code>'''</font>		\| gen1 \|\| <font color="#0a0">'''<code>nodes=4:ppn=3:gen1</code>'''</font> \|\| <font color="#0a0">'''<code>nodes=1:ppn=4:gen1</code>'''</font> \|\| <font color="#0a0">'''<code>nodes=3:ppn=4:gen1</code>'''</font>
	\|-		\|-
	\|		\|
Line 162:		Line 162:
	\| ''run=33<br>tmax=544.74<br>charge=3.63''		\| ''run=33<br>tmax=544.74<br>charge=3.63''
	\|-		\|-
	\| gen2 \|\| <font color="#0a0">'''<code>nodes=4:ppn=4</code>'''</font> \|\| <font color="#0a0">'''<code>nodes=1:ppn=8</code>'''</font> \|\| <font color="#0a0">'''<code>nodes=2:ppn=8</code>'''</font>		\| gen2 \|\| <font color="#0a0">'''<code>nodes=4:ppn=4:gen2</code>'''</font> \|\| <font color="#0a0">'''<code>nodes=1:ppn=8:gen2</code>'''</font> \|\| <font color="#0a0">'''<code>nodes=2:ppn=8:gen2</code>'''</font>
	\|-		\|-
	\|		\|