Classic Benchmarks
The Classic Benchmarks are the first programs that set
standards of performance for computers. As for many other
benchmarks in the collection, the executables were produced via
the Watcom C/C++ compiler. The Classics are:
Whetstone ...... Floating
point benchmark for minicomputers
Dhrystone ....... Integer
benchmark for UNIX systems
Linpack ........... Floating
point benchmark for workstations
Livermore ........ Numeric
benchmark for supercomputers
At the time of writing, the benchmarks, results and source code could be
obtained from my Virgin FreeSpace Internet Home Roy Longbottom's PC Benchmark Colection. The new Internet Home is via the link
www.roylongbottom.org.uk
Livermore Kernels (Livermore Loops)
This supercomputer benchmark was first introduced in 1970,
initially comprising 14 kernels of numerical application, written
in Fortran. The number of kernels was increased to 24 in the
1980's. Performance measurements are in terms of Millions of
Floating Point Operations Per Second or MFLOPS. The program also
checks the results for computational accuracy. One main aim was
to avoid producing single number performance comparisons, the 24
kernels being executed three times at different Do-loop spans to
produce short, medium and long vector performance measurements.
If overall averages are quoted, the benchmark reference below
indicates that the geometric mean may be interpreted as a
characteristic rate of computation but it would be more realistic
to retain the range of statistics in terms of geometric, harmonic
and arithmetic means, minimum and maximum.
For results on PCs see Livermore Loops Results.htm.
Livermore Reference
F.H. McMahon, The Livermore Fortran Kernels: A Computer Test
Of The Numerical Performance Range, Lawrence Livermore National
Laboratory, Livermore, California, UCRL-53745, December 1986.
Go Back
Whetstone Benchmark
The Whetstone benchmark was written by Harold Curnow of CCTA,
the British government computer procurement agency, based on work
by Brian Wichmann of the National Physical Laboratory. An Algol
version of the benchmark was released in November 1972 and
Fortran single and double precision varieties in April 1973. The
Fortran codes became the first general purpose benchmarks that
set industry standards of performance.
The benchmark produced speed ratings in terms of Thousands of
Whetstone Instructions Per Second (KWIPS). In 1978, self timing
versions (by Roy Longbottom also of CCTA) produced speed ratings
in MOPS (Millions of Operations Per Second) and MFLOPS (Floating
Point) and overall rating in MWIPS
Whetstone benchmark source code can also be downloaded in
programming languages Fortran, C, Basic, Java, Visual Basic,
Excel Spreadsheet Visual Basic. These are all of the same sort of
format with self timing for PCs. Performance from these is
included in the benchmark results file. Please excuse all code
looking like Fortran.
Older compilers were not very efficient. To demonstrate how
bad performance can be, a number of benchmarks compiled via old
16 bit compilers can be downloaded - Whetstone Basic, Whetstone
Fortran (2), Whetstone C and Dhrystone 1 C. Performance from
these is also included in the benchmark results file.
For results on PCs see Whetstone Results.htm.
See also Whetstone Benchmark History and Results, which
includes performance ratings of computers produced in the 1960's to today's PCs.
Whetstone Reference
H J Curnow and B A Wichmann, "A Synthetic
Benchmark", Computer Journal Vol 19, No 1 1976
Go Back
Dhrystone Benchmarks
The Dhrystone "C" benchmark, a sort of Whetstone
without floating point, became the key standard benchmark, from
1984, with the growth of Unix systems. The first version was
produced by Reinhold P. Weicker in ADA and translated to
"C" by Rick Richardson.
Two versions are available Dhrystone versions 1.1 and 2.1. The
second version was produced to avoid over-optimization problems
encountered with version 1. Although it is recommended that
advanced optimization levels should be avoided with the latter,
it is clear from published results that the recommendation is
usually ignored. The default option in the Watcom compiler
produces high levels of optimization and omits some constant
calculations from the timing loop. Version 2 is compiled from
three source files.
Original versions of the benchmark gave performance ratings in
terms of Dhrystones per second. This was later changed to VAX
MIPS by dividing Dhrystones per second by 1757, the DEC VAX
11/780 result.
For results on PCs see Dhrystone Results.htm.
Dhrystone Reference
Reinhold P. Weicker, CACM Vol 27, No 10, 10/84,pg.1013
Go Back
Linpack Benchmark
This benchmark was produced by Jack Dongarra from the
"LINPACK" package of linear algebra routines. It became
the primary benchmark for scientific applications from the mid
1980's with a slant towards supercomputer performance.
The original version was produced in Fortran but a
"C" version appeared later. The standard "C"
version operates on 100x100 matrices in double precision with
rolled/unrolled and single/double precision options. The
pre-compiled versions are double precision, rolled. Other
versions are available with different sizes of matrices.
Performance rating is in terms of MFLOPS.
For results on PCs see Linpack Results.htm.
Linpack Reference
Jack Dongarra, Performance of Various Computers Using Standard
Linear Algebra Software in a Fortran Environment (includes numerous results) and can be downloaded from
Here.
Go Back
|