Mach: A Foundation for Open Systems

A Position Paper
Richard Rashid, Robert Baron, Alessandro Forin, David Golub,
Michael Jones, Daniel Julin, Douglas On, Richard Sanzi

School of Compuer Science

Carnegie Mellon University
Pittsburgh, PeMsylvania 15213

1. Introduction for proprietary systems -- each at considerable expense. Bal-

Operating systems have become one of the most hotly con- looning software development costs translate into increased
tested battlegrounds for "open system standards". Various cost for the user and delays in the introduction of new features
national. international and industry groups are attcmpting to and ncw applications.
define. implement and ultimately convince users to buy new
"open" computing environments. Most of these efforts have 2. An Alternative Approach to OS
centered around versions of the UNTX' [9] operating system. Organization
but there is no consensus among industrial groups as to which An altemative approach to building an entire operating sys-
"version" of UNIX is ultimately the corrcct basis for a open tcm is to separate those parts of the operating system which
system standard. Already two major industrial organizations, control the basic hardware resources -- often called the
the Open Software Foundation and UNIX Intemational. have operating system "kemel" -- from those parts of the operating
endorsed two rather different UNIX implementations: IBM's system which determine the unique characteristics of an
AIX and AT&T's System V.4. Non-UNIX systems such as operating systcm environment, for example a particular file
the Macintosh OS, MS-DOS and OSR are also seen as system interface. The advantage of this scheme is that it can
"standax&" by various hardware and software developers. allow more than one operating system environment to be
This divergence of systems and standards raises fundamental implemented on the same hardware/software base so that
issues about the strategy which should be employed in the machine dependent software need be written only once for
development of open operating systems. It has become in- each new architecture.
creasingly important to provide support withiin a single com- The Camegie Mellon University Mach kemel [l] is an ex-
puting environment for "multiple standards", i.e. multiple ample of this laycred approach to operating system design
operating system environments which may be tailored to dif- Mach is a multiprocessor kemel that incorporates in one sys-
ferent vendor or user needs. Moreover, manufacturcrs fre- tem a number of kcy facilities that allow the efficient im-
quently need to provide customers with continuing access to plementation of those functions necessary to support binary
proprietary operating systems developed during the 1960's compatibility with existing operating system environments.
and 1970's (e.g. VMS, MVS and MS-DOS). These mechanisms arc intended not simply as extensions to
Traditional systems such as UNIX or VMS are implemented normal operating system facilities but as a foundation upon
"all in one piece" with knowledge about the basic system which UNIX and other operating system facilities can be
stmcture spread throughout. Thcy are poorly suited to the built.
compatible support of multiple operating system cnviron- We bclicve that such a design is very well suited to the
ments. As a result, large computer manufacturers are com- implementation of modem systems, and that it can yield sig-
monly forced to support several completely distinct operating nificant improvements in terms of structure, without sacrific-
system groups: one for OSR. one for UNIX and one or more ing pcrformance. Furthermore, it also allows, at least poten-
tially, for more than one operating system environment to be
'UNM is I tndemak of AT&T Bell Laboratories
supported in "native mode" simultaneously on the same
hardware. To demonstrate the practical applicability of this
concept, and to evaluate its snengths and weaknesses, we are
This I.cseprch was sponsored by the Defense Advanced Research Pqecu currcntly working on three separate approaches to implement-
Agmcy (DOD). ARPA Order No. 4864, monitored by Ihe Space and Naval ing a 4.3 BSD [3] cnvironment within the Mach framework.
Warfare Systems Command under contract N00039-85-C-1034.

3. Mach Features Supporting OS Emulation

Mach provides an unusually flexible execution environment
for both system and user applications. It exposes the manage-
ment of CPU. communication, virtual memory and secondary

0 1989 IEEE
89TH02814/89/oooO/0109$01.00 109
storage resources in a way that allows systcm applications processes using Mach's flexible virtual memory management
such as database management facilities to use those resources facilities. The parent process that established this shared
efficiently. library can then tell the Mach kemel to redirect system call
The key features of Mach in its rolc as a system software traps from the child into the s h a d library in the address
kemel are: space of that child. This allows any embedded system call
0 support for multiple threads of control within a
traps in a program binary to be interpreted outside the kernel
single address space, and either handled directly or converted into a message to be
sent to a system server. There is an override facility that
0 extensible and secure interproccss com- allows thc transparent library code to redirect a call to the
munication facility (IPC) [ 101. kemel if necessary, to simplify development and debugging
of the transparent library itself. This facility can be used for a
0 architecture independcnt virtual mcmory manage-
variety of purposes, such as:
ment (VM) [71,
0 binary compatibility with non-Mach OS environ-
0 integrated IPCIVM supporf including: copy-on- ments,
write message passing, copy-on-reference nct-
work communication and extensible memory ob- 0 support for multiple OS environments (e.g. UNIX
jects, 4.3 BSD, UNIX V.4).

hooks for transparent shared libraries, to provide 0 debugging and monitoring and
binary compatibility with existing operating sys- 0 network redirection of OS traps.
tem environments.
The Mach kemel provides software equivalents of the key
elements of uniprocessor and multiprocessor architectures.
4. In-kernel OS Emulation
In the first implementation of a 4.3 BSD emulation on top of
The Mach thread mechanism, for example, is a kind of
Mach, the Mach kernel is used as the lower layer of a two-tier
software processor. By allowing multiple threads to run
operating system implementation. In such a scheme the Mach
within the same program, Mach permits a systcm or applica-
kemel provides support for key functions such as virtual
tion programmer to directly manage multiple CPUs in a mul-
memory, scheduling, interprocess communication and device
tiprocessor. Mach's interprocess communication facility
access. The target operating system can then be implemented
UPC) provides the kind of I/O channel between threads that
using these functions. In this approach the entire system.
may exist in a multiprocessor with a mcssagc-passing bus or
kemel and OS environment, is packaged as a unit and run in a
between workstations on a network.
privilegcd state just as in a traditional OS design. In many
Interprocess communication and mcmory managcmcnt in respects it continues to resemble the more traditional operat-
Mach are tightly integrated. Memory managcmcnt techniques ing systems it replaces. One advantage to this approach,
(such as the use of memory re-mapping to avoid data however, is that more than one OS environment can be im-
copying) are employed whencver largc amounts of data are plemented using the same kernel interface -- reducing the
sent in a message from one program to another. This allows software effort required to bring a new architecture to market
the transmission of megabytes of data at very low cost. with several supported operating system. Another advantage
One of the most unusual and important facilities Mach is that the basic kemel could be made freely available to all
provides is the notion of a mcmory object which an applica- without compromising the proprietary added value of the par-
tion program may create and manage. The memory object is ticular operating system environment l a y e d above it. This
like a file or data container which can be mapped into the approach would allow companies to share the costs of porting
address space of a program. Unlike traditional systems in the kemel to a new architecture.
which the operating system has complete control of "paging" Commercial versions of Mach available today are, in fack
data to and from such a data object, Mach allows the applica- examples of 4.3 BSD UNIX layered above Mach kernel
tion which creates the mcmory object to act as though it were
primitives and packaged together with the Mach code. Al-
the disk storage or "pager" for that objcct. Mach virtual
though one might assume that this layered approach to UNIX
memory objects are representcd as communication channcls.
implementation would be a performance disadvantage,
On a page fault, the kemel sends a message to thc backing measurements of Mach versus traditional UNIX implemen-
storage communication channel of a mcmory object to get the tations indicate otherwise. Simple compilation benchmarks
data contained in the faultcd page. This provides the on SUN 3/60 workstations, for example, run nearly 40%
flexibility necessary to implcmcnt cfficicntly such system ap-
faster under Mach than they do under Sun Microsystems own
plications as file systems, databascs. dynamic cncryption or SunOS 4.0 version of UNIX. Times for UNIX "fork and
compression of data on access or evcn network shared
"cxec" operations are also nearly a factor of two faster under
memory. Mach than SunOS.
Finally, the Mach transparcnt library facility allows a code
library to be loaded into the addrcss space of a program
without its knowledge, which can intcrccpt systcm calls made
by h a t program. Transparcnt shared librarics arc loadcd by a
parent process and transparently inhcritcd by its child

I IO
5.Out-of-kernel OS Emulation: Two could allow vendors with proprietary OS environ-
Approaches ments to more quickly take advantage of Mach as
A second approach to building a layered operating system a basis for their systems.
environment has even greater potential for open system In practice. this single task Unix server works well and
development. The kemel can be packaged by itself as a "pure" demonstrates the feasibility of such an approach. Its im-
kernel with no operating system environment. In this ap- plementation was completed in less than sixth months, and
proach, only the kernel runs in privileged state. The rest of the can already be used for self-development. It currently NILS on
operating system environment nms,in effect, as one or more VAX and Sun 3 platforms and is functionally interchangeable
programs (or, more precisely, one or more server processes) with existing versions of 4.3 BSD/Mach on those machines.
on top of the kernel. User applications run as before, but We expect to extend this implementation to the other
instead of making direct calls on the operating system via hardware platforms which run Mach and to put this version
system calls traps, the kernel's communication and memory into production use within CMU over the next few months.
management facilities are employed to communicate infor- Initial performance measurements are encouraging. A com-
mation between the application and operating system pilation benchmark which takes takes 29 seconds to complete .
processes. The reason this implementation seategy is so at- on a Sun 3/60 running in-kernel 4.3BSD/Mach takes 34
tractive for open systems, is that it can allow more than one seconds with out-of-kernel BSD support and 49 seconds run-
operating system environment to be supported on the same ning under SunOS 4.0.
machine, on the same kernel. at the same time. Systems such
as UNM or OS/2 could potentially co-exist in their native 7. Multiserver UNIX
form. The kernel becomes a kind of universal "socket" into This system divides responsibility for UNIX supprt among
which more than one operating system environment can be a collection of libraries and servers responsible for particular
plugged, insulating that software from the hardware itself and OS functions such as naming, authentication and file data
greatly simplifying its design and maintenance. access. Wherever possible, the interfaces between the various
This approach is currently being put to the test at Camegie system components. and those components themselves, are
Mellon in the development of two rather different user-state designed to be independent of the target environment. This
implementations of Berkeley UNIX 4.3 BSD: the approach presents two major advantages:
Multithreaded System and the Multiserver System. Both im- 0 access to various system resources can be shared
plementations run unmodified 4.3 BSD binarics. by multiple independent operating system en-
vironments. communicating over a network or
6. The Multithreaded UNIX Server concurrently executing on the same machine.
This system consists of transparent library support aug-
mented by a multithreaded UNIX server. This server. con- individual components can easily be reused for
tained in a single task, is typically invoked via a Mach mes- the implementation of different operating system
sage exchange for each system call issued by application environments.
processes. In addition to managing system call emulation for On the negative side, this approach requires sophisticated
Unix processes, the Unix server acts as an external pager for synchronization between servers to achieve precise UNIX
Unix inodes. It is implemented using Mach's C-Threads semantics, and very careful design of the standardized inter-
package with each incoming request handled by a cthread faces. The major interfaces defined for that system organiza-
allocated from a pool of waiting threads. tion are:
A single task operating system emulation of this kind is 0 a standard access protocol defining the authen-

attractive for several reasons: tication, access control and naming procedures
The server is solely responsible for performing used for access to all system objects such as files.
the emulation of all OS environment semantics. devices, processes. etc.
The structure of the server is, in fact, similar to 0 a standard 1/0 protocol for the transfer of data
that of an in-kernel implementation; it has global between h e producers and consumers of that
knowledge of all the information needed for the data.
emulation. Internal context switching ktwccn
threads can be extremely fast. a standard exccption protocol to handle excep-
tions happening during client-server interactions.
0 The OS server is completely pageablc and can in and to report asynchronous events to clients.
fact make more efficient use of memory (by shar- The following sections describc the major components of
ing data structures and stack space) than can a this system.
multiple server implementation.
7.1. Mach Object Programming Facility
It can be relatively straightforward to transform The development of the multiserver UNIX system is aided
an existing in-kernel OS implcmcntation into by a C-based object-oriented programming package called
such a server, because most of the code can be MuchObjects, which has been integrated wifh the Mach inter-
simply carried over. This can make it easy to process communication facility. This package allows:
preserve both existing code and semantics. This
 dynamic classlmethod spccification. protocols used for network access (TCP/IF', OSI.
etch
class/superclass hierarchy,
the UNIX File Server, which manages WIX Fie
multiple inheritance through delegation, systems on permanent storage, but uses the stan-
automatic remote delegation (through IPC), dard naming and I/O protocols, so that it is acces-
sible from all environments.
user-specifiable method lookup to implement
other forms of inheritance, the NFS Server, which translates requests from
the standard access and U 0 interface into the
automatic dispatching of method invocations to NFS protocol, allowing access to remote. NFS f i e
multiple threads of control, systems,
reference count garbage collection of objects and the UNIX TTY Server, a front-end for access to
automatic object locking. terminal lines and pseudo terminals, implement-
ing the line disciplines and
To simplify the organization of the various components,
libraries of standard MachObjects classes are used. that im- the UNIX Pipe Server, implementing traditional
plement the standard system interfaces. In many cases, a UNIX pipes, using FIFO buffers in shared
special MachObject mechanism is used to allow a server to memory.
dynamically select the class of an object to be instantiated in
its client's address space. When this approach is used, only 8. Related Work
the client-side object must implement the standard interfaces. Several other research groups are also investigating the
Each server may use a different, specialized protocol to com- issues involved with OS emulation, particularly with respect
municate with the client-side objects that it returns. to the UNIX environment. CMU's Accent operating
7.2. Transparent Library system [6] was used as the base for a System III Unix emula-
The transparent library is responsible for translating the tion called QNIX. The Amoeba system[8] uses port
UNIX 4.3 BSD system calls from an application process into capabilities in a manner very similar to that of Mach to
invocations on the appropriate system servers, via the cor- implement a fast server-based system. The CHORUS system
responding MachObjects elements. [2] has adopted an object-oriented approach to build a com-
An important aspect of the system organization is that many plete UNIX emulation; its use of memory protection is
emulated system calls, for example read and wrife, can be however rather different from that adopted for the Mach OS
implemented within the transparent library with no messages emulators. The Taos system [4] provides an emulation of
exchanged with servers. This is possible because many data Ultrix in the Topaz environment. Several of the concepts used
objects can be represented as Mach memory objects and for the Mach OS emulators are inspired from ideas presented
mapped into the addrcss space of the transparent library after with the Sprite and V systems [SI [ll].
a open call is made. The read and wrife calls thus translate
into simple memory references into this mapped area. 9. Mach availability
The portability of Mach has been demonstrated by the range
7 3 . System Servers of uniprocessor and multiprocessor systems on which it is
The various system servers cooperate to implcmcnt the func- available. Mach has been ported to the VAX architecture
tions needed for thc emulation. As indicated, many of these
uniprocessors and multiprocessors, the SUN 3 family, the
servers axe in fact independent of the spccific UNIX environ-
IBM RT PC family, the DecStation 3100, the 64-processor
ment, and only the fact that they are invoked from the 4.3 IBM RP3, the 8-processor IBM ACE multiprocessor worksta-
BSD transparent library produces 4.3 BSD scniantics from h e tion, the Sequent Balance. the Macintosh II. the IBM 370. the
point of view of the application proccsses. The major servers
SUN 4, the Intel 386 and the Intel i860. Implementations for
are or will be: other MIPS R2000 and R 3 0 machines are nearing comple-
*the Name Server, implementing a hierarchical tion and several implementations for the Motorola 88000 are
name space with only directories. symbolic links underway. Commercial versions of Mach are available &om
and mount points. which can bc used to tie BBN Advanced Computers, Evans and Sutherland Computer
together several other name spaces and represent Division, Encore Computers and NeXT. In addition to these
the "root" of the UNIX name space, vendor releases of Mach, Mt Xinu, Inc. has announced that it
the Task Manager, responsible for keeping track will develop commercial end-user releases of Mach for a
of all the application processes participating in an variety of machine architectures. Finally, Prime. Intel,
operating system environment, Olivetti, Convergent and AT&T have recently announced a
joint research project to build a multiprocessor version of
the Authentication Server. used to verify the System V.4 using the Mach kemel.
credentials of processes pcrforming operations on All software implemented by the Mach project is licensed
behalf of the authorized users of the systcm. and distributed to universities. research laboratories and cor-
the Network Server. implementing the transport porations at no cost by Carnegie Mellon.

I I2
 References
1. Acaetta. M.J., Baron, R.V.. Bolosky. W.. Golub. D.B.. Rashid.
R.F., Tevanian, A.. and Young,M.W. Mach: A New Kemel Foun-
dation for UNIX Dwelopnent Proceedings of Summcr Usenix.
July, 1986.
2. Francois Armand and Michel Gien and Marc Guillcmont and
Pierre Leonard. Towards a Distributed UNIX System - Thc
CHORUS Approach. Proceedings of the Europcan UNIX Systems
User Group Conference, September, 1986.
3. Joy, W., e t al. 4.2BSD System Manual. Technical repon ,
Computer Systems Research Group. Computer Science Division,
University of Califomia, Berkeley. July, 1963.
4. Paul R. McJones and Garret F. Swart. Evolving the UNM System
Inrerface to Support Multithreaded Programs. Research Repon 21.
DEC Systems Research Center. September. 1987.
5. John K. Ousterhout and Andrew R. Chcrenson and Frederick
Douglis and Michael N. Nelson and Brent B. Wclch. "The Sprite
Network Operating System". IEEE Compufer21.2 (February 1988).
23-36.
6. Rashid, R. F. and Robertson, G. Acccnt: A Communication
Oriented Network Operating System Kcmel. Proc. 81hSymposium
on Operating Systems Principlcs. December. 1981, pp. 64-75.
7. Rashid, R.F.. Tevanian. A., Young, M.W.. Golub, D.B.. I?aron.
R.V.. Black, D.L., Bolosky. W., and Chew, J.J. Machinc-
Independent Virtual Memory Management for Paged Uniprocessor
and Multiprocessor Architectures. Proceedings of thc 2nd Sym-
posium on Architectural Support for Programming Languages and
Operating Systems, ACM. October. 1987.
8. Robbert van Renesse and Hans van Staveren and Andrcw
S. Tanenbaum. "Performance of the World's Fastcst Distributed
Operating System". ACM Operating Sysiems Review 22.4 (October
1988). 23-34.
9. D.M. Ritchie and K. Thompson. "l'hc UNIX time-sharing
system". Bell System Technical JOWMI (July 1978).
10. Sansom. R.D., Julin. D.P. and Rashid R.F. Extcnding a
Capability Based System into a Nerwork hvironmcnt. Procecdings
of the ACM SIGCOMM 86 Symposium on Communications Ar-
chitectures and Protocols. August, 86, pp. 265-274. Also available as
Technical Repon CMU-CS-86-115.
11. WUy Zwaenepoel. "Implementation and Performance of Pipes
in the V-System". IEEE Tram. on Compu. C-34.12 (Deccmber
1985). 99-106.

I I3