Classic Benchmarks

The Classic Benchmarks are the first programs that set standards of performance for computers. As for many other benchmarks in the collection, the executables were produced via the Watcom C/C++ compiler. The Classics are:

Whetstone ...... Floating point benchmark for minicomputers
Dhrystone ....... Integer benchmark for UNIX systems
Linpack ........... Floating point benchmark for workstations
Livermore ........ Numeric benchmark for supercomputers

At the time of writing, the benchmarks, results and source code could be obtained from my Virgin FreeSpace Internet Home Roy Longbottom's PC Benchmark Colection. The new Internet Home is via the link www.roylongbottom.org.uk

Livermore Kernels (Livermore Loops)

This supercomputer benchmark was first introduced in 1970, initially comprising 14 kernels of numerical application, written in Fortran. The number of kernels was increased to 24 in the 1980's. Performance measurements are in terms of Millions of Floating Point Operations Per Second or MFLOPS. The program also checks the results for computational accuracy. One main aim was to avoid producing single number performance comparisons, the 24 kernels being executed three times at different Do-loop spans to produce short, medium and long vector performance measurements. If overall averages are quoted, the benchmark reference below indicates that the geometric mean may be interpreted as a characteristic rate of computation but it would be more realistic to retain the range of statistics in terms of geometric, harmonic and arithmetic means, minimum and maximum.

For results on PCs see Livermore Loops Results.htm.

Livermore Reference

F.H. McMahon, The Livermore Fortran Kernels: A Computer Test Of The Numerical Performance Range, Lawrence Livermore National Laboratory, Livermore, California, UCRL-53745, December 1986.

Go Back

Whetstone Benchmark

The Whetstone benchmark was written by Harold Curnow of CCTA, the British government computer procurement agency, based on work by Brian Wichmann of the National Physical Laboratory. An Algol version of the benchmark was released in November 1972 and Fortran single and double precision varieties in April 1973. The Fortran codes became the first general purpose benchmarks that set industry standards of performance.

The benchmark produced speed ratings in terms of Thousands of Whetstone Instructions Per Second (KWIPS). In 1978, self timing versions (by Roy Longbottom also of CCTA) produced speed ratings in MOPS (Millions of Operations Per Second) and MFLOPS (Floating Point) and overall rating in MWIPS

Whetstone benchmark source code can also be downloaded in programming languages Fortran, C, Basic, Java, Visual Basic, Excel Spreadsheet Visual Basic. These are all of the same sort of format with self timing for PCs. Performance from these is included in the benchmark results file. Please excuse all code looking like Fortran.

Older compilers were not very efficient. To demonstrate how bad performance can be, a number of benchmarks compiled via old 16 bit compilers can be downloaded - Whetstone Basic, Whetstone Fortran (2), Whetstone C and Dhrystone 1 C. Performance from these is also included in the benchmark results file.

For results on PCs see Whetstone Results.htm. See also Whetstone Benchmark History and Results, which includes performance ratings of computers produced in the 1960's to today's PCs.

Whetstone Reference

H J Curnow and B A Wichmann, "A Synthetic Benchmark", Computer Journal Vol 19, No 1 1976

Go Back

Dhrystone Benchmarks

The Dhrystone "C" benchmark, a sort of Whetstone without floating point, became the key standard benchmark, from 1984, with the growth of Unix systems. The first version was produced by Reinhold P. Weicker in ADA and translated to "C" by Rick Richardson.

Two versions are available Dhrystone versions 1.1 and 2.1. The second version was produced to avoid over-optimization problems encountered with version 1. Although it is recommended that advanced optimization levels should be avoided with the latter, it is clear from published results that the recommendation is usually ignored. The default option in the Watcom compiler produces high levels of optimization and omits some constant calculations from the timing loop. Version 2 is compiled from three source files.

Original versions of the benchmark gave performance ratings in terms of Dhrystones per second. This was later changed to VAX MIPS by dividing Dhrystones per second by 1757, the DEC VAX 11/780 result.

For results on PCs see Dhrystone Results.htm.

Dhrystone Reference

Reinhold P. Weicker, CACM Vol 27, No 10, 10/84,pg.1013

Go Back

Linpack Benchmark

This benchmark was produced by Jack Dongarra from the "LINPACK" package of linear algebra routines. It became the primary benchmark for scientific applications from the mid 1980's with a slant towards supercomputer performance.

The original version was produced in Fortran but a "C" version appeared later. The standard "C" version operates on 100x100 matrices in double precision with rolled/unrolled and single/double precision options. The pre-compiled versions are double precision, rolled. Other versions are available with different sizes of matrices.

Performance rating is in terms of MFLOPS.

For results on PCs see Linpack Results.htm.

Linpack Reference

Jack Dongarra, Performance of Various Computers Using Standard Linear Algebra Software in a Fortran Environment (includes numerous results) and can be downloaded from Here.

Go Back