Revelation pyconuk2016

Simulating a CPU
with Python
or: surprising programs
you might have thought
were better written in C
Sarah Mount @snim2

Data from Stanford VLSI Group CPUdb

Left: late 1980s INMOS Transputer (image © Konstantin
Lanzet - CPU collection. Licensed under CC BY 3.0 via
Wikimedia Commons)
Right: a 2011 Parallella board, the spiritual successor to
the INMOS Transputer (image © Adapteva Inc)

Adapteva Epiphany III chip
A new style of energy-efficient manycore
computer?
● 16 / 64 cores, theoretical maximum of 4095 cores
● 1 GHz
● <2 Watts
● Whole Parallella board: 5 Watts
● 19 GFLOPS / Watt
● 32 kB RAM / core

Packing code into that small space
● Most 32-bit Epiphany core instructions have
a 16-bit equivalent
● Compiler chooses the 16-bit instruction
whenever possible

Derek Lockhart, Berkin Ilbeyi, and Christopher Batten. (2015) Pydgin: Generating Fast Instruction Set
Simulators from Simple Architecture Descriptions with Meta-Tracing JIT Compilers. IEEE International
Symposium on Performance Analysis of Systems and Software (ISPASS).

Revelation: a new Epiphany sim
● Written using the Pydgin framework in
RPython (statically typed Python)
● All instructions implemented except some
multicore
● Small code base: ~1k Source Lines of Code
● http://www.revelation-sim.org/
Still a work in progress though (i.e. don’t expect
it to work!)

● Clone pypy:
$ hg clone https://bitbucket.org/pypy/pypy
● Clone Pydgin:
$ git clone ...github.com/cornell-brg/pydgin.git
● Put Pydgin on your PYTHONPATH:
$ export PYTHONPATH=${PYTHONPATH}:.../pydgin/:
● Write a simple simulator!
Write your own Pydgin simulator!

Decode and implementation (add32)

Some oddities about the Epiphany
● The chip has a flat memory map. Everything
(registers, flags) is a location in RAM.
● Each core has 32kB local memory,
addressed as 0x0…0xFFFFF
● The same memory can be globally
addressed, and accessed by other cores, as
(COREID << 20) | local address, e.g.
0xF0408 (the PC) is globally 0x808F0408
for chip 0x808 (on row 32, column 8 of the
chip.

More oddities
● There are many instructions, almost all of
which have 16-bit and 32-bit versions. This
could lead to duplicate code.
● Thankfully, we can use closures to create all
the versions of an instruction implementation
that we need in one go:

Implementation of JUMP instructions

Test, compile, iterate
To compile your simulator (with a JIT):
$ .../rpython/bin/rpython -Ojit sim.py
To compile with the ability to print an instruction
trace:
$ .../rpython/bin/rpython -Ojit sim.py
--debug

Test, compile, iterate
To run your simulator:
$ ./pydgin-sim-jit --help
Then iterate!

But you said TEST, compile, iterate!
● Compiling is slow, so you want to avoid
compiling every time you make a change to
the code base
● That means you want to test your RPython
code dynamically (untranslated)
● Because RPython is a subset of Python, you
can use Python test tools, such as py.test,
which means you can unit tests your
untranslated simulator

Unit tests
● “Proper” unit tests, which test small program
units (such as a single instruction)
● The Epiphany Architecture Reference
Manual has many examples (1 or more for
each instruction). Each of these is written
into an assembler file and used as the basis
of a (more complex) unit test.
● Currently >500 unit and integration tests in
Revelation, testing a very tiny part of the
state space!

A unit test to go with the assembler

Integration tests
● Integration tests typically load and execute a
full ELF file (compiled from C code). They
then check one or more of:
○ The machine state
○ The number of instructions executed
○ Output on STDOUT
● ELF files need to be cross-compiled with the
Epiphany SDK (e-gcc, etc.) and so are
checked into version control

“Hello, world!” the integration test

Using a test oracle
● A test oracle is a source of truth, for example
an existing, complete simulator
● Adapteva e-sim:
○ https://github.com/adapteva/epiphany-cgen
○ https://github.com/adapteva/epiphany-gdb

e-sim trace file (4.5MB, 46265 lines)
0x000000 b.l 0x0000000000000100 - pc <-
0x100
0x000100 mov.l r3,0x138 - registers <-
0x138
0x000104 movt r3,0x0 - registers <-
0x138
0x000108 jalr r3 - registers <-
0x10a, pc <- 0x138
0x000138 --- _epiphany_star mov.l sp,0x7ff0 - registers <-
0x7ff0
0x00013c --- _epiphany_star movt sp,0x0 - registers <-
0x7ff0
0x000140 --- _epiphany_star mov.l fp,0x0 - registers <-
0x0
0x000144 --- _epiphany_star mov.l r0,0x58 - registers <-
0x58
0x000148 --- _epiphany_star movt r0,0x0 - registers <-
0x58

Revelation trace (6.7MB, 46096 lines)
0 000080e8 bcond32 0 AN=False AZ=False
AC=False AV=False AVS=False BN=False BZ=False BIS=False
BUS=False BV=False BVS=False
100 0012670b movimm32 1 :: WR.RF[3 ] = 00000138
104 1002600b movtimm32 2 :: RD.RF[3 ] = 00000138
:: WR.RF[3 ] = 00000138
108 00000d52 jalr16 3 :: WR.RF[14] = 0000010a
:: RD.RF[3 ] = 00000138
138 27f2be0b movimm32 4 :: WR.RF[13] = 00007ff0
13c 3002a00b movtimm32 5 :: RD.RF[13] = 00007ff0
:: WR.RF[13] = 00007ff0
140 2002e00b movimm32 6 :: WR.RF[15] = 00000000
144 00020b0b movimm32 7 :: WR.RF[0 ] = 00000058
148 1002000b movtimm32 8 :: RD.RF[0 ] = 00000058
:: WR.RF[0 ] = 00000058

Example trace diff (4.MB, 51845 lines)
Semantics of instruction at 0x44e differ:
Revelation: 44e 2f8b0417 fsub16 35097 :: RD.RF[0
] = 3f800000 :: RD.RF[1 ] = 5f800000 :: RD.RF[0 ] = 3f800000 ::
WR.RF[0 ] = 5f800000 :: RD.RF[0 ] = 5f800000 :: RD.RF[0 ] =
5f800000 :: RD.RF[1 ] = 5f800000 :: RD.RF[0 ] = 5f800000
AN=False AZ=False AC=False AV=False AVS=True BN=False BZ=False
BIS=False BUS=False BV=False BVS=False
e-sim: 0x00044e --- main fsub r0,r1,r0 -
bzbit <- 0x0, bnbit <- 0x0, bvbit <- 0x0, bvsbit <- 0x0, busbit
<- 0x0, bisbit <- 0x0, registers <- 0x5f7fffff
Registers differ. Revelation: rf<-0x5f800000 e-sim: rf<-0x5f7fffff
Semantics of instruction at 0x8e006d1e differ:
Revelation: 8e006d4c 0652268b movimm32 35845 :: WR.RF[1
] = 00006534
e-sim: 0x8e006d1e mov.l r1,0x6534
- registers <- 0x6534
Program counters differ. Revelation: 0x8e006d4c, e-sim: 0x8e006d1e

A specious difference between traces
Semantics of instruction at 0x8e009980 differ:
Revelation: 8e009980 6dda1fe2 trap16 990 :: RD.RF[3
] = 00000005 :: RD.RF[0 ] = 00000001 :: RD.RF[1 ] = 8f000008 ::
RD.RF[2 ] = 00000001 :: WR.RF[0 ] = 000
00001 :: WR.RF[3 ] = 00000000
e-sim: 0x8e009980 trap 0x7
- registers <- 0x1
Registers differ. Revelation: rf<-0x0 rf<-0x1 e-sim: rf<-0x1

Things I haven’t tried but might...
● American Fuzzy Lop:
○ http://lcamtuf.coredump.cx/afl/
● Hypothesis
○ http://hypothesis.works/
● ...any other form of automated testing

Revelation pyconuk2016

Recommended

More Related Content

What's hot (20)

Similar to Revelation pyconuk2016 (20)

More from Sarah Mount (7)

Recently uploaded (20)

Revelation pyconuk2016