HPC/Hardware Details: Difference between revisions

From CNM Wiki
< HPC
Jump to navigation Jump to search
Line 6: Line 6:
[[Image:HPC Compute Rack-up.png|thumb|right|200px|]]
[[Image:HPC Compute Rack-up.png|thumb|right|200px|]]
<!-- * [http://www.supermicro.com/products/nfo/1UTwin.cfm "1U Twin"] by [http://www.supermicro.com/ Supermicro] -->
<!-- * [http://www.supermicro.com/products/nfo/1UTwin.cfm "1U Twin"] by [http://www.supermicro.com/ Supermicro] -->
 
* Carbon has two hardware node types, named <code>gen1</code> and <code>gen2</code>.
{| class="wikitable" cellpadding=8 style="text-align:center;  margin: 1em auto 1em auto;"
* node properties
{| class="wikitable" cellpadding=8 style="margin: 1em auto 1em 2em;"
|- style="background:#eee;"
|- style="background:#eee;"
! Features !! Count !! Processor !! Cores !! Clock<br>(GHz) !! Memory<br>(GB) !! Memory/core<br>(GB) !! Disk (GB)
! Features !! Count !! Processor !! Clock<br>(GHz) !! Memory<br>(GB) !! Memory/core<br>(GB) !! Disk (GB)
|-  align="center"
|-  align="center"
| login1 || 1 || Xeon X5355 || 8 || 2.67 || 16 || 2 || 250
| gen1 || 144 || Xeon X5355 || 2.67 || 16 || 2 || 160
|-  align="center"
|-  align="center"
| login5/login6 || 2 || Xeon E5540 || 8 || 2.53 || 24 || 3 || 250
| gen2 bigmem || 38 || Xeon E5540 || 2.53 || 48 || 6 || 250
|-  align="center"
|-  align="center"
| gen1 || 144 || Xeon X5355 || 8 || 2.67 || 16 || 2 || 160
| gen2 || 166 || Xeon E5540 || 2.53 || 24 || 3 || 250
|-  align="center"
|-  align="center" style="background:#f8f8f8;"
| gen2 bigmem || 38 || Xeon E5540 || 8 || 2.53 || 48 || 6 || 250
| login1 || 1 || Xeon X5355 || 2.67 || 16 || 2 || 250
|-  align="center"
|-  align="center" style="background:#f8f8f8;"
| gen2 || 166 || Xeon E5540 || 8 || 2.53 || 24 || 3 || 250
| login5/login6 || 2 || Xeon E5540 || 2.53 || 24 || 3 || 250
|}
|}
* all nodes are dual-socket (8 cores/node).
* gen1 nodes are charged at 50% discount of walltime. Depending on cores used and memory throughput demanded, they may be about ''on par'' with gen2 (low memory throughput) or up to about 2–3 times slower.


== Infrastructure nodes ==
== Infrastructure nodes ==

Revision as of 15:57, October 27, 2010


HPC-Main-external.jpg

Carbon Cluster
User Information


User nodes

1U Twin node chassis (Supermicro).
HPC Compute Rack-up.png
  • Carbon has two hardware node types, named gen1 and gen2.
  • node properties
Features Count Processor Clock
(GHz)		 Memory
(GB)		 Memory/core
(GB)		 Disk (GB)
gen1 144 Xeon X5355 2.67 16 2 160
gen2 bigmem 38 Xeon E5540 2.53 48 6 250
gen2 166 Xeon E5540 2.53 24 3 250
login1 1 Xeon X5355 2.67 16 2 250
login5/login6 2 Xeon E5540 2.53 24 3 250
  • all nodes are dual-socket (8 cores/node).
  • gen1 nodes are charged at 50% discount of walltime. Depending on cores used and memory throughput demanded, they may be about on par with gen2 (low memory throughput) or up to about 2–3 times slower.

Infrastructure nodes

  • 2 Management nodes
  • 2 Lustre MDS
  • 4 Lustre OSS
  • dual socket, quad core (Intel Xeon E5345, 2.33 GHz)
  • pairwise failover

Storage

  • Lustre parallel file system
  • 42 TB effective (84 TB raw RAID-10)
  • 2 NexSAN SATAbeast
  • 160–250 GB local disk per compute node
  • NFS
    • for user applications and cluster management
    • highly-available server based on DRBD
HPC Infiniband-blue.png

Interconnect

  • Infiniband 4x DDR (Mellanox, onboard ) – MPI and Lustre
  • Ethernet 1 GB/s – node access and management
  • Ethernet 10 Gb/s – crosslinks and uplink
  • FibreChannel – storage backends

Power

  • UPS (2) – carries infrastructure nodes and network switches
  • PDUs – switched and metered
  • Power consumption at typical load: 118 kW
Retrieved from "https://wiki.anl.gov/wiki_cnm/index.php?title=HPC/Hardware_Details&oldid=2780"