In order to evaluate the performance of the different HPC systems in the Teraflop Workbench NEC and HLRS work together to run different standard HPC benchmarks. The applied benchmarks so far have been the HPC Challenge benchmark and the SPEC OMP benchmark.

SPEC OMP benchmark

SPEC OMP (OpenMP Benchmark Suite) is SPEC‘s benchmark suite for evaluating performance based on OpenMP (http://www.openmp.org) applications. The benchmark consists of 11 different scientific and engineering applications. All codes are parallel using OpenMP. These benchmarks place heavy demands on systems and shared memory. Two levels of workload (OMPM and OMPL) characterize the performance of medium and large sized systems. On the time of submission the NEC TX7 delivered the best performance for its class of CPUs (Itanium II, 1.5 GHz). For more details see http://www.spec.org/omp.

HPC Challenge benchmark

The HPC Challenge benchmark, which is supported by the DARPA, the US Ministry of Defence‘s research department and the American National Science Foundation, comprises a total of 23 tests to measure performance. It enables more efficient evaluation of the actual performance capability of high performance computers.

The 23 individual tests in the HPC Challenge benchmark (http://icl.cs.utk.edu/hpcc/index.html) do not measure the theoretical peak performance of a computer. Rather, they provide information on the performance of the computer in real applications. The tests do not measure processor performance but criteria that are decisive for the user such as the rate of transfer of data from the processor to the memory, the speed of communication between two processors in a supercomputer, the response times and data capacity of a network. Since the tests measure various aspects of a system, the results are not stated in the form of one single value. In their entirety, the measurements enable an assessment of how effectively the system performs on high performance computing applications.

Webpage plots (256 processors)
Intel MPI Benchmark Suite (IMB)


Thomas Bönisch
Höchstleistungsrechenzentrum Universität Stuttgart
Noberlstr. 19
70569 Stuttgart
E-Mail: boenisch@hlrs.de

Sunil Reddy Tiyyagura
Höchstleistungsrechenzentrum Universität Stuttgart
Nobelstrasse 19
70569 Stuttgart
Phone: ++49-711-685-5799
E-Mail: sunil@hlrs.de

Holger Berger
NEC High Performance Computing Europe
Phone: ++49-711-687-7035
E-Mail: hberger@hpce.nec.com