The Deepthought2 cluster

The Deepthought2 HPC cluster

The Deepthought High-Performance Computing (HPC) cluster is the newest of the HPC clusters maintained by the Division of Information Technology as a general campus resource. It is much more homogeneous in its composition, making it more suitable for large, parallel jobs. It also includes a substantial number of GPU-enabled nodes.

Hardware

The following table lists the hardware on the Deepthought2 cluster:

Description Processor Number
of nodes
Cores/node Memory/node
GB
Memory/core
GB
Scratch space
per node, GB
GPUs/node Interconnect
C8220 Ivy Bridge, 2.8 GHz 444 20 128 6.4 750 0 FDR Infiniband
C8220X Ivy Bridge, 2.8 GHz 40 20 128 6.4 750 2 FDR Infiniband
Poweredge R820 Ivy Bridge, 2.2 GHz 4 40 1024 25.6 750 0 FDR Infiniband

All of the nodes except for the 1 TB RAM nodes have dual Intel Ivy Bridge E5-2680v2 processors running at 2.80 GHz. Memory is DDR3 at 1866 MHz.

The 1 TB RAM nodes require a different processor in order to support that much memory. These have quad Intel Xeon Ivy Bridge E5-4640v2 processors running at 2.20 GHz. The memory is DDR3 at 1333 MHz in these nodes.

The nodes containing GPUs have dual Nvidia Tesla K20m (GK110GL) GPUs.

The cluster has over 1 PB of file storage, and has FDR infiniband interconnects between the nodes with a theoretical maximum throughput of about 56 Gb/s.

Performance

Deepthought2 debuted on the June 2014 list of the world's most powerful supercomputers, as produced by TOP500. At number 345 in the list for all systems, this makes it in the top 10 among public universities in the United States.

Our benchmark for this list shows over 160 TFlop/s based on actual linpack runs. Because of restrictions for inclusion on the list, the results are based on benchmarks using the CPUs only, no GPUs. The theoretical maximum performance based on only using the CPUs is about 200 TFlop/s. Using the GPUs would add about another 100 TFlop/s to this theoretical maximum, for a total theoretical maximum of about 300 TFlop/s.

The theoretical maximums assume ideal conditions, in which the calculations are able to keep all the CPUs and GPUs fully utilized, which of course does not happen in practice. But these numbers are easy to compute and useful for rough comparisons. The linpack benchmark is based on actual calculations, and although still a somewhat optimal case, it is a bit more realistic.

To put in comparison, a typical laptop might have a theoretical peak of around 25-50 GFlop/s, so Deepthought2's computing power could be compared to the combined power of roughly 10,000 typical laptops. A typical laptop disk size is 500 GB, or 1/2000 the storage of Deepthought2. Typical broadband speeds over fiber or cable modems can sometimes reach 1 Gbps, or 1/50 of the speed of Deepthought2's internetwork speed.