Benchmarks for PDP-8 Emulators

In an alt.sys.pdp8 posting, Klemens Krause suggested the following benchmarks for PDP-8 emulators:

  1. Benchmarks without I/O

    A program which calculates 1500 functions (squareroot, sine and logarithm) and stores the results in a two-dimensional array. Times are stopped between the "START" and "END" message on the screen, without the compile time and the 17 lines at the end of the programs. The program is translated to three languages:

    1. BASIC, with 23-bit accuracy and use of the EAE
    2. FORTRAN-IV with 27-bit
    3. PASCAL with 35-bit (plus 1-bit for the sign)
       
  2. Benchmarks with file I/O (on a RK05 disk)

    The PASCAL.PA file was the last file on DSK: and I squished the disk before assembling or editing, so that the head movement is approximately constant.

    1. Assembling the PASCAL-S compiler (source file of 9.000 lines and 144 kB) with PAL-8 without listing: 
      .R PAL8
*PASCAL<PASCAL/K
    2. Assembling the same compiler, with producing a 328 kB list file: 
      .R PAL8
*PASCAL,PASCAL<PASCAL/K
    3. Reading and writing the PASCAL.PA source file with the OS/8-editor EDIT.SV. 
      .ED PASCAL.PA
#R
#E
.

These are the hand measured times (in seconds) given by Klemens Krause for a hardware PDP-8/E and his PDP-8/E simulator running on PC hardware of the year 1994:

1.a 1.b 1.c 2.a 2.b 2.c
Hardware PDP-8/E  16 18 44 75 192 91
486/33 (EISA)  6 8 15 28 79 45
486/33 (EISA) with RAMDISK  6 8 15 25 55 35
486DLC/40  5 7 15 30 105 55
486DLC/40 with RAMDISK  5 7 15 24 57 36
386/28  26 23 81 n/a n/a n/a
386/40  8 9 20 37 120 65
386/40 with RAMDISK  8 9 20 32 73 50
386SX/25  30 36 78 122 279 183

The benchmarks were performed with the Macintosh PDP-8/E Simulator, and the following hand measured times (in seconds) were perceived. The first column specifies the Macintosh and in parenthesis the system software version, the runtime environment and the version of the simulator used for the test:

1.a1 1.b 1.c 2.a 2.b 2.c
Macintosh Plus (2.0.1/68K/1.5)  n/a 1070 2385 n/a n/a n/a
Colour Classic (7.1/68K/1.1)2 n/a 327 637 930 2060 1265
Colour Classic (7.1/68K/1.2)5 n/a 410 840 1150 2550 1635
SE/30 (6.0.4/68K/1.1)  n/a 173 633 540 825 740
SE/30 (7.1/68K/1.1)  n/a 180 633 553 1246 780
IISi (6.0.8/68K/1.1)  n/a 140 315 440 992 615
IISi (7.1/68K/1.1)  n/a 150 401 615 1180 760
Quadra 650 (7.5.1/68K/1.0)  n/a 783 80 115 200 160
Quadra 650 (7.5.1/68K/1.1)  n/a 25 52 76 170 106
PowerMac 7100 66 MHz (7.5.1/68K/1.1)  n/a 87 187 258 580 348
PowerMac 7100 66 MHz (7.5.1/PPC/1.1)  n/a 11 22 32 71 45
Performa 6200 75 MHz (7.5.1/PPC/1.1)  n/a 11 21 33 73 46
PowerMac 5400 180 MHz (7.5.5/PPC/1.1)  n/a 4 8 10 23 14
PowerMac 5400 180 MHz (8.0/68K/1.2)  n/a 14 24 37 82 52
PowerMac 5400 180 MHz (8.5.1/PPC/1.2)4 n/a 4 (13)7 8 (26) 10 (41) 23 (85) 15 (55)
PowerMac 5400 180 MHz (8.6/PPC/1.3)6 n/a 18 44 79 191 90
PowerMac G3 300 MHz (8.6/PPC/1.2)  n/a 2 4 5 11 7
PowerMac G4 400 MHz (9.0.4/PPC/1.4)  n/a 1 2 4 8 (10) 5 (6)
PowerMac G4 Cube 450 MHz (9.0.4/PPC/1.4)8 n/a 1 2 4 (6) 7 (13) 5 (8)
PowerMac G4 Cube 450 MHz (X PB/Carbon/1.4)  n/a 1 2 (3) 4 (6) 7 (12) 5 (8)
PowerMac G4 Cube 450 MHz (10.1/Carbon/1.4)9 n/a 1 (3) 2 (9) 4 (14) 8 (26) 5 (15)
iBook G3 500 MHz (10.1/Carbon/1.4)  n/a 2 (11) 3 (21) 5 (30) 10 (33) 7 (45)
MacBook Pro 2006 Core Duo 2.16 GHz (10.4/Rosetta+Carbon/1.5)10 1(1) 2(2) 2(3) 4(4) 7(9) 4(5)
MacBook Pro 2009 Core 2 Duo 2.53 GHz (10.11/x64/2.3)11 0.27 0.29 0.67 1.2012 2.1112 1.9712
iMac 2019 i5 3.7 GHz (11/x64/2.3)11 0.08 0.09 0.17 0.31 0.52 0.44
MacBook Air M1 (11/Rosetta2/2.3)11 0.09 0.10 0.22 0.35 0.63 0.50
MacBook Air M1 (11/arm64/2.3)11 0.05 0.06 0.14 0.21 0.37 0.32

For many years now, benchmarks that compare the PDP-8/E with current hardware no longer make sense. The last lines of the table show that the simulated PDP-8/E can run 300 to 500 times faster than the original on hardware of the year 2020. They also show that Macs with the Apple M1 CPU are remarkably faster than current Macs with Intel CPUs.


Footnotes

  1. The BASIC benchmarks could not be performed due to malfunctioning of the OS/8 BASIC interpreter.
  2. The initial release of the PDP-8/E Simulator was developed on a Colour Classic. (Since July 1997, the Power Macintosh 5400 was used.)
  3. This test was performed without use of the EAE because a bug in the EAE implementation of version 1.0 of the simulator prevented the functioning of the FORTRAN mathematical library in EAE mode.
  4. This benchmark was performed using the PowerPC versions of the I/O device plug-ins. When using the 68K versions of the plug-ins, the measured times were nearly the same (less than one second longer). The KC8-EA Console plug-in was not loaded for this benchmark. With the KC8-EA Console plug-in loaded and locked (display lights off), the times are nealy the same (less than one second longer).
  5. A reason for the significant slowdown of version 1.2 of the simulator compared to version 1.1 could not be found. The I/O device plug-in interface itself causes nearly no overhead, only the plug-ins IOT instructions have to handle the A4/A5 world switches and access of the PDP-8 data structure over a pointer. But the .FT and .PS benchmarks involve no I/O operations. Maybe the 68K code optimization of the newer C compiler is not as good as in previous versions.
  6. With the CPU and all I/O devices running with real time speed. Note that these are nearly the same times Klemens Krause measured with a hardware PDP-8/E.
  7. In brackets: times with KC8-EA unlocked (display lights turned on). All other times are measured without KC8-EA or with locked KC8-EA (display lights turned off).
  8. See footnote 7. Since 13. September 2000 (Mac OS X Public Beta release date), the PDP-8/E Simulator is hosted on the Cube. The PDP-8/E with console lights turned on might be slower on the 450 MHz G4 Cube than on a 400 MHz G4 because the Apple 17" Studio Display has a higher screen refresh rate (up to 154 Hz) and colour depth (32 bits) than the VGA monitor of the G4 400 MHz, and the KC8-EA refreshs the lights with the screen refresh rate.
  9. The remarkably slower times with console lights turned on measured with Mac OS X final compared to the public beta may be caused by the fact that screen double buffering was not implemented in Mac OS X Public Beta.
  10. Note that on the Intel-based MacBook Pro, the PDP-8/E Simulator itself is emulated by the “Rosetta” PowerPC emulator of Mac OS X, so the PDP-8/E is “double-emulated” on the Intel Core Duo processor (only one of the two cores is used).
  11. Benchmarks 1.a – 1.c were executed 100 times in sequence with an additional loop around the START END block. Benchmarks 2.a and 2.b were executed 100 times in sequence using OS/8 BATCH. Benchmark 2.c was executed by pasting the 100 ED commands (i. e. 300 lines) into the TTY window. The KC8-EA and the Memory Content window were enabled, the TTY was running as fast as possible.
  12. Without Memory Content plug-in that slows down the simulation too much.
Bernhard Baehr 29.01.2022 22:07:00