L – A Common Lisp for Embedded Systems

Rodney A. Brooks∗ and Charles Rosenberg

IS Robotics
Twin City Office Center, Suite 6
22 McGrath Highway
Somerville, MA 02143, USA

Abstract these systems is performed on a host computer and code

is then installed on the target system. IS Robotics uses
A commercially available system has been developed the behavior control approach (Brooks 1986) to program
which allows for the use of Common Lisp in real time its robots. In this approach the control of the robot is
embedded control systems. The backbone of this sys- broken up into many small parallel processes each of
tem is a language called L. L is a subset of Common which operates on some sensor data and can control
Lisp with multi-processing extensions. It is ideal for actuators on the robot. In earlier work we used what
use in embedded systems with small computers. The we called the Behavior Language (Brooks 1990). It
system has a minimal memory footprint and can run is written in Common Lisp and produces a complete
on small processors. L contains both a runtime envi- ROM image for a particular instance of a users program
ronment and an interpreter which runs on the target which is then installed (or downloaded to non-volatile
system and a cross compiler which runs on the host sys- RAM) on the robot. We decided we needed more flexi-
tem. Making use of Common Lisp’s macro facilities, a bility and dynamic debugging however, so we decided to
special purpose language has been constructed on top implement Common Lisp (Steele Jr. 1990) directly on
of L, called MARS for Multiple Agency Reactiv- the robots. We wanted a system which allows us to dy-
ity System. This language is uniquely tailored for real namically redefine functions, test new code interactively,
time control applications. It allows for the spawning of start and kill processes, and easily create extensions to
many small processes which communicate with one an- the language, and tools which would reduce develop-
other via a message passing approach. A graphical user ment time. To realize these goals we created a new
interface debugging tool was also developed which al- system composed of a dialect of Common Lisp (which
lows real time monitoring and modification of values of we call L), a set of language extensions specific to real
variables on the target system by the host system. Un- time control, and a special operating system.
derlying this system is an efficient real time operating
system, called VENUS. The operating system is writ-
ten in C for portability. The interface between L code 2 VENUS
and the hardware takes place via operating system calls.
L accesses the C operating system calls via its foreign VENUS is a compact, efficient real time operating
function interface. The first application for this system system. VENUS provides the low level interface to
has been the programming and control of autonomous the target system hardware necessary to support L.
mobile robots. These robotic systems must process and VENUS is written primarily in C for portability.
react to their sensor systems in real time. Only a small portion, the processor dependent part, of
the operating system is written in assembly language.
The low level target system support includes basic in-
put/output and timing functions. The input/output
1 Introduction functionality implements a protocol which utilizes a sin-
gle bi-directional serial line to support five virtual bi-
This development effort was motivated by IS Robotics
directional serial lines. A front end processor on the host
main business, the manufacture and programming of
decodes this protocol and exchanges packets with the
small autonomous robots for real world tasks. These
embedded system. Currently these virtual serial lines
robots have relatively small processors, a 16 MHz Mo-
are used to allow the embedded system to: access files
torola 68332 with 1 megabyte of RAM. Development for
on the host, provide a TTY interface for communica-
∗ Also a member of the faculty at the MIT Artificial Intelligence tion with the embedded system, and provide a means for
Laboratory. data exchange with graphic debugging tools. The tim-
ing functions implemented by VENUS provide access The L system is designed to be retargetable. So far we
to the processor periodic interrupt timer for multithread have run it on Macintoshes and on single chip Motorola
scheduling. 68332’s. To retarget it is necessary to write a back-
VENUS is designed to be compact and modular; end for the assembler, provide code templates for about
only those pieces of the operating system required for 100 primitive operations, code sequences for about 20
a specific application need be included in the run time relatively large but primtive operations (such as cons),
kernel. For our typical Motorola 68332 implementation, code sequences for about 10 operations such as spread-
the entire operating system occupies less than 32K bytes ing multiple values on the stack and search the stack for
of memory. VENUS is also a dynamically linkable and a throw tag, and LAP macros for about 25 things such
modifiable system. The C functions which make up the as procedure call, building catch frames, etc.
most basic parts of the operating system can be changed
dynamically, without rebuilding the entire system, as is 3.1 The Compiler
the case with many other operating systems.
The VENUS system includes a low level monitor pro- The compiler is an intellectual descendant of the S-1
gram which provides low level debugging support and compiler (Brooks, Gabriel & Steele Jr. 1982), and more
the ability to modify the VENUS run time kernel. This recently the Lucid compiler (Brooks, Posner, McDonald,
monitor includes support for examining and modifying White, Benson & Gabriel 1986). It shares no code with
memory, linking in new OS kernel code, and debugging either of these previous compilers however.
support for ‘hard’ crashes. The design goals for the L compiler were

• it should produce reasonably efficient code

3 L • compilation speed should be fast

L is a completely new implementation of a subset of • it should be target independent

Common Lisp. It is carefully designed so that very • it should be easy to maintain
small kernel subsets can be used to build Lisp images,
e.g., ones without eval, read, etc., for small dedicated The compiler is written in Common Lisp, of course,
applications. We have successfully run L programs in and will probably run in L itself although that has not
load images as small as 10K bytes. More typically our been tried at this point.
full implementation of L now has a load image of almost The compiler is rather simple and does not do reg-
200K bytes, including robot specific libraries. ister allocation for variables. It does open code many
The main design goal in L has been simplicity. For Common Lisp primitive operations however, and when
this reason we have left out a number of aspects of Com- compiling a tree of primitive operations will use registers
mon Lisp, including its bulk. to store multiple intermediate results.
Specifically we have excluded all the sequence func- The compiler has three phases: alphatization, analy-
tions, overloaded aritmetic, supplied-p arguments, ex- sis, and generation.
tendable data structures (we do include automatically During alphatization everything is syntactically
growing hash tables), the EQ/EQL distinction, flet and checked and a number of source to source transforma-
labels, parts of format, large parts of packages, pre- tions are made. These include quotation of self eval-
serving whitespace aspects of read, CLOS, the type sys- uating literals, macro expansions, and compiler macro
tem, the error condition system, and rationals, bignums, expansions. Most of the Common Lisp control forms
and trigonometry. are implemented as compiler macros which take precen-
On the other hand L does include multiple values, dence over macros defined in the host Lisp—in this way
strings, characters, arrays, a simplified but compatible the user can use the host Lisp macro system and the
package system, all the “ordinary” aspects of format, compiler can still use its own definitions of macro expan-
backquote and comma, setf etc., almost full Common sions for such things as dotimes. During alphatization
Lisp lambda lists including optionals and keyword argu- the code tree is walked exactly once.
ments, macros, an inspector, a debugger, defstruct (in- During the analysis phase a number of further source
tegrated with the inspector), block, catch, and throw, to source transformations are made that are target
etc., full dynamic closures, a full lexical interpreter, specific—for instance simple vector references and struc-
floating point, fast garbage collection, and so on. The ture references might all get mapped to a common un-
compiler runs in time linear in the size of an input ex- derlying reference mechanism at this point, or not, de-
pression, except in the presence of lexical closures. It pending on the particular memory model being used for
nevertheless produces highly optimized code in most the target. Other transformations carried out here in-
cases. clude constant folding in arithmetic expressions, and
even reordering of arithmetic operations. All these :rep :flag
transforms take alphatized source and return alphatized :loc ’eq)
source. At the same time as the code tree is being
walked, in a single pass, for these transformations an (deflprim car
analysis tree is formed. It notes side effects, read/write :args ((x areg))
status of lexical variables, register usage requirements, :loc ‘(ref-cons ,x 0))
stack height, pointers up the tree to lexical variable,
block, and tag owners, etc. Also during this phase, anal- (deflprim cdr
ysis of closures is carried out, and source to source trans- :args ((x areg))
forms are made so that shared lexical variables refer to :loc ‘(ref-cons ,x 4))
the right data structures, with optimizations for read
only lexical variables, etc. If closures are present it is (deflprim set-car
sometimes necessary to re-invoke analysis on a subtree :args ((x areg) (y any))
for a second time. :code ‘((move ,y (ref-cons ,x 0)))
As analysis proceeds, the compiler looks at the re- :effects (make-rplac-effects)
quirements for open coding primitive procedures di- :loc y)
rectly in machine code. The following are the descrip-
tions of fixnum + and - for the Motorola 68332. (deflprim second
:args ((x areg))
(defun code-68k-adder (x y ops) :code ‘((move (ref-cons ,x 4) ,x))
(cond ((and (consp y) :loc ‘(ref-cons ,x 0))
(eq (first y) ’quote))
(if (<= 1 (second y) 2) Note that some operations produce a value, while oth-
‘((,(third ops) ers, like eq produce a condition code. A coercion process
,(ash (second y) 2) ,x)) in generation must convert such representations if they
‘((,(second ops) ,y ,x)))) are not appropriate for the particular place in which the
(t ‘((,(first ops) ,y ,x))))) operation is used. Note also that set-car has its side
effects noted—this means that generation can not op-
(deflprim + timize it away if its value is not used. Also car and
:args ((x dreg) (y any)) cdr produce no code at all. They describe how given
:code (code-68k-adder x y ’(add addi addq)) a cons cell in an address register, the car and cdr can
:loc x) be accessed by an addressing mode—expressed with the
LAP (Lisp assembler program) macro ref-cons shown
(deflprim - below:
:args ((x dreg) (y any))
:code (code-68k-adder x y ’(sub subi subq)) (deflapoperand ref-cons (reg offset)
:loc x) ‘(areg-indirect-disp
,reg
Note that these descriptions include requirements on ,(+ offset
where the arguments should be (the first argument (ltarget (- 8 *lptr-cons*)))))
should be in a data register), and the actual code gen-
erator does some optimizations when possible. If the The last phase of the compiler is code generation. For
compiler has a tree of primitive operations that require special forms, procedure calls, etc., it produces a series
more than the available number of registers, it may in- of calls to LAP macros which essentially describe an ab-
troduce a lexical variable to save an intermediate value stract machine. For primitive operations it uses the code
and rebuild the analysis tree and source appropriately. templates as described above to generate specific ma-
The descriptions above assume that earlier phases of the chine instructions to implement them. Some primitive
compiler have guaranteed that all calls to these two pro- operations, such as floating point arithmetic, or storage
cedures are dyadic, and that constants have been pushed allocation are too long to generate each time they are
rightward wherever possible. invoked. So there is a library of fastops, handcoded ma-
The following are additional primitive operations, out chine code subroutines to implement these, and the com-
of a total of about 100 altogether. piler generates simple subroutine calls to these where
appropriate.
(deflprim eq Except in the case of closure analysis, all three phases
:args ((x areg) (y any)) of the compiler are single pass, and thus rather efficient.
:code ‘((cmpa ,y ,x)) The compiler has a simple form and is not weighed down
by special purpose optimizations. The code it produces so on in a depth first search are checked until a sym-
for standard sorts of non-arithmetic things when run bol with the same name is found. If none are found,
on a Macintosh is much more efficient than the code then a new symbol is interned in the original package.
produced by the Macintosh Common Lisp compiler This system is very easy to implement. By restricting
(it would be unfair to compare arithmetic operations as the total number of packages to be no more than 16,384
L does not handle generic arithmetic). the total storage requirement per symbol is 14 bits, plus
one 32 bit entry in a special package hash table which
simply records the presence or absence of a symbol.
3.2 Arithmetic
When L starts up there are five packages present:
There are two types of arithmetic supported in L. "SYSTEM", "KEYWORD", "LIB", "L", and "USER". The
Fixnum arithmetic and small floating point. "SYSTEM" package contains all the system code. The
The arithmetic operators +, *, etc., only support "USER" package is empty—both these packages use both
fixnums. The Motorola 68332 implementation of the "L", and "LIB" packages. The "L" package is where
fixnums is 30 bits signed. The fixnum operators act just all the Common Lisp symbols reside. The "LIB" pack-
like Common Lisp arithmetic operators given fixnums, age includes all entry points to robot specific libraries
except that in L there can never be an overflow into that have been built on top of L and C for the particu-
bignums. Any overflows are ignored and arithmetic is es- lar robot.
sentially mod230 . Thus fixnum arithmetic never causes
any consing in L.
Besides normal arithmetic, the usual logical operators 3.4 Garbage Collection and Storage
are also supported on fixnums. Management
To use floating point, other operators +$, *$, etc., The garbage collection system used in L is a very simple
must be used. This is similar to the situation in and elegant stop and copy collector. This interferes with
Maclisp except that L does not include any additional real time requirements placed on L, so the rest of L is
generic operators. The Motorola 68332 implementa- designed so that for real time programming there is no
tion of floating point numbers is 28 bits long, including need to invoke cons. This includes the design of the
8 bits of signed exponent in excess 128 format, 19 bits arithmetic system and the design of MARS. The idea
of mantissa (plus a hidden bit), and 1 bit for the sign of is that the user loads everything (or has it autoloaded)
the number. This floating point format is designed to during which time consing takes place, but thereafter,
fit in an immediate data pointer, so there is never any unless in debugging mode, there is no need for further
consing caused by floating point arithmetic. The draw- consing.
back is that floating point arithmetic is implemented in
The storage arrangement in L consists of six spaces:
software (as fastops), so they are slow, but there is no
pure code ROM
floating point hardware on the Motorola 68332 in any
pure data ROM
case.
static data RAM GC scans
Floating point underflow causes an error, but a macro user code RAM
without-floating-underflow-traps is provided to from space RAM
suppress these if desired1 . to space RAM GC scans
The system code is all in the pure code space. String
3.3 Packages names of system symbols, keyword symbols themselves
used by system code, the closure objects (including
The package system in L is simplicity itself, but never- pointers to all literals) for all system procedures, and
theless it provides the standard functionality of provid- minor other constant data structures are kept in pure
ing provate namespaces and it appears compatible with data. It has no pointers to from or to spaces, but may
Common Lisp packages as long as one does not poke have pointers within itself and to the static space. The
too hard. static space provides pre-allocated data structures which
In L every symbol in every package is external and ev- are guaranteed to remain in place. These include all
ery symbol is in precisely one package. Packages can use other structures that are present when L starts up, such
other packages. When it comes time to resolve the name as symbols, packages, etc. The multithreading system
of a symbol in a package (e.g., in the package bound by allocates all stacks in this space so that the garbage col-
*package* during read), that package is checked first. lector does not have to deal with relocating an active
If the name is found there then that is the result. Oth- stack. Users can also request that data be allocated
erwise all the packages used by the first package, and here by using the macro with-static-consing. The
1 Note that this was proposed to X3J13 in June 1989 to be trade off in using this space is that the garbage collector
added to Common Lisp but the proposal was rejected. never has to copy things from here, but it does have to
scan them even if they do become garbage, and they are VENUS saying how much time is left in the slice of
never collected. the current thread. When it is zero or less the thread
User loaded machine code goes into the user code relinquuishes control to the scheduler which reschedules
space. This space is not garbage collected and can po- it some small time interval later. Threads can also vol-
tentially fill up just from repeatedly reloading the same untarily reschedule themselves by calling sleep, or vari-
compiled file. ants thereon.
The from and to spaces are the two half spaces for Context switching of heavy weight processes can be
user data. Their roles are swapped at each garbage col- expensive if they are suspended with special variables
lection. rebound—their stack needs to be traversed to undo
these binds before switching to another thread, and then
3.5 Multithreading be traversed again to put them back when restarting this
thread.
The multithreading system in L is unique—it is not a
subset or superset of anything in Common Lisp. It
is a polling pre-emptive scheduler with the abiltity for 3.6 Debugging
processes to put themselves to sleep for a given duration. The standard debugging tools provided in L are a text
Since the pre-emptive scheduler uses polling, it based inspector and a text based stack walker.
only pre-empts if code was specifically compiled for The inspector lets the user examine the full internals
multithreading—this is the default and only about 20 of any data structure. It is integrated with defstruct
critical procedures in the whole L system are not com- so that slot names are displayed for structures whose
piled for polling. defstruct has been loaded into the current environ-
A thread only polls, and thus is only ever suspended, ment.
on procedure entry right after all argument parsing has The stackwalker lets the user move up and down any
been done. This includes catching cases of self tail re- suspended stack (including those of other threads that
cursive calls. Many threads have their own stack, so on have signalled errors), examing the stack frames and in-
entry to a multithreading compiled procedure, a check voking the inspector on any slots. There are also facili-
is done to see if it looks like there is enough stack left, ties for looking at raw machine memory which is some-
and if not an error is signalled. times useful in debugging sensors and actuators.
When L starts up it is not in multithreaded Machine traps are caught in the VENUS operating
mode. But it is possible to evaluate a procedure system and handed back to L so that they can be dis-
called start-multi-processing, which spawns a new played within the context of the Lisp computation which
read-eval-print loop, and schedules it to be run re- caused the problem.
peatedly. The scheduler runs in the normal Lisp stack. Other useful tools are apropos and who-refers?.
The scheduler is based on an internal clock, gotten The first is standard Common Lisp and uses
with get-internal-real-time. It uses an efficient data do-all-symbols to find symbols whose name matches
structure to reschedule when to next run each thread as a partial string. The second is an L construct which
it requests as it is suspended. searchs all the procedure objects in the system to find
A procedure called spawn can be used to start up all literal references to an argument symbol.
additional threads. There are two sorts of threads.

3.5.1 Light weight threads 3.7 The Motorola 68332 Implementa-

tion
Lightweight threads do not have their own stack and
are run from the normal Lisp stack. Whenever such a For 68000 based architectures we have used a lowtag
thread is invoked, some specific procedure is called and scheme to represent lisp objects. Here the bottom three
it runs until it returns—never polling. Its return value bits determine the gross type of a pointer according to
is how many internal time units before it wants to be the following scheme:
run again. Bits Pointer
000 even fixnum
3.5.2 Heavy weight threads 001 cons cell
010 gc forwarding pointer
Heavy weight threads have their own stack. They are 011 object with header
spawned as a procedure to be called, but it is expected 100 odd fixnum
that that procedure will never return—the thread is 101 symbol
killed if it does. During every procedure entry within 110 reserved
the thread the system polls a register maintained by 111 immediate data item
 All of the even data types can also be pointers to The graphical user interface is supported via one
objects created by C, and there only needs to be a little of the extra virtual serial lines implemented in L and
extra checking in the garbage collector to disambiguate VENUS. A packet protocol is built on top of this se-
forwarding pointers. rial line (which itself is built on top of a packet protocol
Besides fixnums, immediate data objects include char- which is built on top of an actual serial line). Besides
acters, floating point numbers, unbound markers, vari- handling graphics traffic this system is also used to im-
ous stack markers for special binds, etc., and headers for plement other interactions between the front end (host
objects in the heap. machine), and L.
All objects in the heap are aligned at eight byte Great care has been taken in the implementation of
boundaries, and pointers to them point to within the both MARS itself and the graphical user interface to
previous eight bytes. Symbols and cons cells have no make debugging easy. Everything is dynamically linked
headers—all their elements are simply scanned by the so that new versions of subsystems can be recompiled
garbage collector. Other objects such as strings, arrays, and reloaded onto an existing working system, without
procedure objects, etc., have headers which say what any interruptions. All the graphic windows resize them-
their size is and how the garbage collector should treat selves should it be necessary when this happens, and
them as it scans through the heap. things that have had monitors attached to them stay
monitored even when they are completely redefined by
overloading a new version of the code–even to the point
4 MARS of handling changes in underlying representations in the
middle of display option choice back on the host com-
The Multiple Agency Reactivity System, or puter. All this is a marked improvement over the earlier
MARS, is a language for programming multiple con- Behavior Language.
current agents. It is embeded in L, and as such, all of
that subset of Common Lisp is available for use any-
where within programs. 5 Conclusion
MARS is a method for defining many asynchronous
Our goal was to create a development system which
parallel processes which run in a single L heap. Groups
would allow us to have all of the advantages of Common
of these processes are called assemblages which share a
Lisp on a small embedded computer system. The L lan-
lexical environment and some number of ports which are
guage was developed to serve this purpose. It is a highly
message passing interfaces between assemblages. Ports
efficient subset of Common Lisp with multithreading
are bi-directional, but a port from one assemblage can
extensions and a cross compiler. It can run on the rela-
only be connected uni-directionally to a port of an-
tively small processors often used in embedded systems.
other assemblage. Connections of ports involve one deep
To improve programming efficiency, a language, named
buffers and message arrival flags. New instances of as-
MARS, was built on top of L. This package greatly
semblages can be spawned dynamically at runtime. As-
simplifies the creation and debugging of large numbers
semblages also can be dynamically killed. Connections
of inter-communicating processes. IS Robotics is com-
can be dynamically made and broken.
mitted to the continuing development and improvement
There is also a graphical user interface which allows
of the L system and its commercial viability. We firmly
for dynamic monitoring and altering of all port values.
believe L to be a superior solution for creating complex
These monitors can be added dynamically at run time to
real time embedded control systems.
any port. The graphics run on a host machine, such as
a Macintosh—there is a standard interface envionment
that runs in Macintosh Common Lisp.
References
MARS is somewhat like the earlier Behavior Lan-
guage, a method for layering behavior producing pro- Brooks, R. A. (1986), ‘A Robust Layered Control Sys-
grams, one after the other, in a system connecting sen- tem for a Mobile Robot’, IEEE Journal of Robotics
sors to actuators. In fact there is a fairly straightfor- and Automation RA-2, 14–23.
ward translation from Behavior Language programs
to MARS programs. But MARS programs can be Brooks, R. A. (1990), The Behavior Language User’s
much richer as they have access to all of Common Lisp. Guide, Memo 1227, Massachusetts Institute of
In addition, all linking is completely dynamic, and all TechnologyArtificial Intelligence Lab, Cambridge,
housekeeping is invisibly taken care of so that assem- Massachusetts.
blages (roughly corresponding to behaviors in Behavior
Language) can be redefined at run time, new assem- Brooks, R. A., Gabriel, R. P. & Steele Jr., G. L. (1982),
blages can be created, and old ones killed. An Optimizing Compiler for Lexically Scoped Lisp,
 in ‘Proceedings of the 1982 Symposium on Com-
piler Construction, ACM SIGPLAN’, Boston, Mas-
sachusetts, pp. 261–275. Published as ACM SIG-
PLAN Notices 17, 6 (June 1982).
Brooks, R. A., Posner, D. B., McDonald, J. L., White,
J. L., Benson, E. & Gabriel, R. P. (1986), Design
of An Optimizing Dynamically Retargetable Com-
piler for Common Lisp, in ‘Proceedings of the 1986
ACM Symposium on Lisp and Functional Program-
ming’, Cambridge, Massachusetts, pp. 67–85.

Steele Jr., G. L. (1990), Common Lisp, The Language,

second edn, Digital Press.