HPC/Hardware Details: Difference between revisions

From CNM Wiki
< HPC
Jump to navigation Jump to search
Line 8: Line 8:
* Carbon has two hardware node types, named <code>gen1</code> and <code>gen2</code>.
* Carbon has two hardware node types, named <code>gen1</code> and <code>gen2</code>.
* node properties
* node properties
{| class="wikitable" cellpadding=8 style="margin: 1em auto 1em 2em;"
{{Template:Table of node types}}
|- style="background:#eee;"
! Features !! Count !! Processor !! Clock<br>(GHz) !! Memory<br>(GB) !! Memory/core<br>(GB) !! Disk (GB)
|-  align="center"
| gen1 || 144 || Xeon X5355 || 2.67 || 16 || 2 || 160
|-  align="center"
| gen2 bigmem || 38 || Xeon E5540 || 2.53 || 48 || 6 || 250
|-  align="center"
| gen2 || 166 || Xeon E5540 || 2.53 || 24 || 3 || 250
|-  align="center" style="background:#f8f8f8;"
| login1 || 1 || Xeon X5355 || 2.67 || 16 || 2 || 250
|-  align="center" style="background:#f8f8f8;"
| login5/login6 || 2 || Xeon E5540 || 2.53 || 24 || 3 || 250
|}
* all nodes are dual-socket (8 cores/node).
* all nodes are dual-socket (8 cores/node).
* gen1 nodes are charged at 50% discount of walltime. Depending on cores used and memory throughput demanded, they may be about ''on par'' with gen2 (low memory throughput) or up to about 2–3 times slower.
* gen1 nodes are charged at 50% discount of walltime. Depending on cores used and memory throughput demanded, they may be about ''on par'' with gen2 (low memory throughput) or up to about 2–3 times slower.

Revision as of 19:03, November 18, 2010


HPC-Main-external.jpg

Carbon Cluster
User Information


User nodes

1U Twin node chassis (Supermicro).
HPC Compute Rack-up.png
  • Carbon has two hardware node types, named gen1 and gen2.
  • node properties
Node
names, types 		Node
generation 		Node
extra
properties 		Node
count 		Cores
per node
(max. ppn) 		Cores total,
by type 		Account
charge
rate 		CPU
model 		CPUs
per node 		CPU
nominal
clock
(GHz) 		Mem.
per node
(GB) 		Mem.
per core
(GB) 		GPU
model 		GPU
per node 		VRAM
per GPU
(GB) 		Disk
per node
(GB) 		Year
added 		Note
Login
login5…6 gen7a gpus=2 2 16 32 1.0 Xeon Silver 4125 2 2.50 192 12 Tesla V100 2 32 250 2019
Compute
n421…460 gen5 40 16 640 1.0 Xeon E5-2650 v4 2 2.10 128 8 250 2017
n461…476 gen6 16 16 256 1.0 Xeon Silver 4110 2 2.10 96 6 1000 2018
n477…512 gen6 36 16 576 1.0 Xeon Silver 4110 2 2.10 192 12 1000 2018
n513…534 gen7 gpus=2 22 32 704 1.5 Xeon Gold 6226R 2 2.90 192 6 Tesla V100S 2 32 250 2020
n541…580 gen8 20 64 2560 1.0 Xeon Gold 6430 2 2.10 1024 16 420 2024
Total 134 4736 48
  • all nodes are dual-socket (8 cores/node).
  • gen1 nodes are charged at 50% discount of walltime. Depending on cores used and memory throughput demanded, they may be about on par with gen2 (low memory throughput) or up to about 2–3 times slower.

Infrastructure nodes

  • 2 Management nodes
  • 2 Lustre MDS
  • 4 Lustre OSS
  • dual socket, quad core (Intel Xeon E5345, 2.33 GHz)
  • pairwise failover

Storage

  • Lustre parallel file system
  • 42 TB effective (84 TB raw RAID-10)
  • 2 NexSAN SATAbeast
  • 160–250 GB local disk per compute node
  • NFS
    • for user applications and cluster management
    • highly-available server based on DRBD
HPC Infiniband-blue.png

Interconnect

  • Infiniband 4x DDR (Mellanox, onboard ) – MPI and Lustre
  • Ethernet 1 GB/s – node access and management
  • Ethernet 10 Gb/s – crosslinks and uplink
  • FibreChannel – storage backends

Power

  • UPS (2) – carries infrastructure nodes and network switches
  • PDUs – switched and metered
  • Power consumption at typical load: 118 kW
Retrieved from "https://wiki.anl.gov/wiki_cnm/index.php?title=HPC/Hardware_Details&oldid=2825"