Table Of ContentMechanisms for Reliable
Distributed Real-Time
Operating Systems
The Alpha Kernel
J. Duane Northcutt
Deportment of Computer Science
Cornegie-Mellon University
Pittsburgh, Pennsylvonio
ACADEMIC PRESS, INC.
Horcourt Broce Jovonovich, Publishers
Boston Orlando San Diego
New York Austin London Sydney
Tokyo Toronto
Copyright © 1987 by Academic Press, Inc.
All rights reserved.
No part of this publication may be reproduced or
transmitted in any form or by any means, electronic
or mechanical, including photocopy, recording, or
any information storage and retrieval system, without
permission in writing from the publisher.
ACADEMIC PRESS, INC.
Orlando, Florida 32887
United Kingdom Edition published by
ACADEMIC PRESS INC. (LONDON) LTD.
24-28 Oval Road, London NW1 7DX
Library of Congress Cataloging-in-Publication Data
Northcutt, J. Duane.
Mechanisms for reliable distributed real-time operating
systems.
(Perspectives in computing; vol. 16)
Bibliography: p.
1. Operating systems (Computers) 2. Real-time data
processing. 3. Electronic data processing — Distributed
processing. I. Title. II. Series.
QA76.76.063N67 1987 004'.33 87-Ί421
ISBN 0-12-521690-4 (alk. paper)
87 88 89 90 9 8 7 6 5 4 3 2 1
Printed in the United States of America
To my brother,
Richard Vernon Northcutt,
1961-1979
Preface
This research monograph describes the Alpha kernel — a set of mechanisms that support the con-
struction of reliable, modular, decentralized operating systems for real-time control applications.
This work, performed in the course of the author's Ph.D. thesis research, is part of the Archons
project at Carnegie-Mellon University, led by Prof. E. Douglas Jensen.
The Alpha effort is aimed at providing fresh experience with operating system kernel design in two
senses. First, in the context of operating systems per se, many current beliefs and practices have
evolved from earlier implementations and have limited validity in contemporary settings. Given that
there are few technology-independent lessons to be learned in computer systems research, it is impor-
tant to frequently re-evaluate some of the basic premises and their system implications. Second, in
the context of real-time control systems, operating systems are substantially behind the state of the
art, making many deployed systems much less cost-effective than they could be. This creates an
attractive opportunity for the infusion of new technology into this area.
The Alpha kernel incorporates a number of carefully integrated, modern techniques, such as:
decentralized management of global system (not just local node) resources; kernel level support for
atomic transactions and replication; object orientation; judicious exploitation of hardware support;
and strict exclusion of policy from the kernel mechanisms. To these, Alpha also adds the use of
application-derived time constraints (e.g., deadlines) and relative importance attributes for managing
resources to achieve optimal responsiveness and utility for its clients.
The design of Alpha was based on fifteen years of industrial experience with distributed real-time
supervisory command and control systems, together with nearly a decade of Archons academic
research into the fundamental issues of this class of system. The perspective and scope of the Alpha
research is the entire computer system, so it is being created entirely from the bare hardware up. Its
focus extends from the operating system concepts and interface abstractions, down through all the
detailed engineering tradeoffs necessary to achieve cost-effective implementations. The most ap-
propriate forms of synergy between the hardware and kernel designs have been, and continue to be, a
driving factor in this project.
Hie Alpha kernel is not an end in itself; it is a vehicle for exploring new ideas and the foundation for
the Alpha decentralized operating system. This monograph provides an initial snapshot of the kernel
design and implementation part of this relatively large scale research effort. The Alpha kernel design
is complete, and as of this writing, mostly implemented. A major application survey and system
evaluation task is under way which will provide detailed feedback to augment the experience already
gained. The first version of the Alpha real-time decentralized operating system being constructed on
the Alpha kernel is scheduled for completion in 1988. A commercial-quality product version of the
Alpha kernel is being constructed by Kendall Square Research Corporation, targeted for their own
machine, as well as those of other manufacturers' (beginning with Sun Microsystems workstations).
Acknowledgments
I would like to thank my advisor, Doug Jensen, for allowing me the opportunity to do this work and
for providing me with such an fine example of how to do computer systems research. Doug is a good
friend and an outstanding researcher, and one could not hope for more from a thesis advisor. Special
thanks is due to Martin McKendry, who provided the starting impetus for this effort and whose
initial concepts helped shape the work presented here. It was Martin who dragged me, kicking and
screaming, into the world of objects and convinced me of their necessity in implementing practical,
reliable distributed systems, and for this I am deeply grateful. Martin is missed both technically and
personally. Also, Rick Rashid deserves thanks for providing me with much needed guidance
throughout the course of this work. Rick's advice has always proved to be sound and I wish that I
had taken more of it sooner. It has been my good fortune to work with and learn from these people,
who are all members of that small community who put their research concepts into practice by
building interesting and significant computer systems.
I would like to thank Ray Clark for the assistance he provided while working with me on the kernel
and this document I doubt it would have been possible for me to complete this work without his
help. Sam Shipman also deserves thanks for his contributions to this work. Thanks is also due to Jon
Bentley for his guidance over the years and for all the effort he put into reading this and making it a
better document
Thanks is also due to the many people at Sun Microsystems Inc. who helped support this project over
the last few years. Their support took the form of equipment donations, agreements to allow us to
purchase hardware not in their current product line, access to source code, and access to proprietary
technical information about their products. I particularly appreciate the fact that they took the time
to help us, despite the fact that they were quite busy creating a successful new company, and our
requests frequently fell outside of the mainstream of their market interests.
I would like to thank Dan Siewiorek for providing me with the opportunity to publish my work in
this way. Also, I would like to acknowledge and thank Huay-Yong Wang, Chuck Kollar, Bruce
Taylor and Dan Reiner for all of their work in support of Alpha. I^rry Slomcr also deserves thanks
for his help in getting this document printed, as well as for the help he has given to the Archons
project in the past. Finally, ί would like to thank the Office of Naval Research for taking up our
burden as their own and relieving us of the greatest impediment to our research.
This research was sponsored in part by the USAF Rome Air Development Center under contract
number F30602-85-C-0274, the US Naval Ocean Systems Center under contract number N66001-83-
C-0305, and the IBM Federal Systems Division under university agreement number YA-278067. The
views and conclusions contained in this document are those of the author and should not be inter-
preted as representing the official policies, either expressed or implied, of RADC, NOSC, IBM, or
the U.S. Government
List of Figures
Figure 2-1: Alpha Kernel Example 32
Figure 2-2: Example Queue Object 34
Figure 2-3: Example Source Code for an Alpha Object 39
Figure 2-4: Example of ITiread Coordination 52
Figure 2-5: Example of Locking Within Objects 56
Figure 4-1: Object Structure 79
Figure 4-2: Basic Operation Invocation 85
Figure 4-3: Parameter Passing on Invocation 87
Figure 4-4: Remote Invocation 90
Figure 4-5: Thread Structure 94
Figure 5-1: Logical View of the Implementation Structure 121
Figure 5-2: Alphabit Control Blocks 128
Figure 5-3: Virtual Memory Hierarchy 130
Figure 5-4: Virtual Memory Data Structures 134
Figure 5-5: The Communications Virtual Machine 138
Figure 5-6: Example Protocol Specification 140
Figure 5-7: Load Module Physical Memory Layout 143
Figure 5-8: Virtual Address Space Layout 144
Figure 5-9: Kernel Region Layout 145
Figure 5-10: Client Thread Layout 147
Figure 5-11: Client Object Layout 148
Figure 5-12: Logical View of Secondary Storage in Alpha 155
Figure 5-13: The Operation Invocation Facility 163
Figure 6-1: The Current Ardions Testbed Facility 176
Figure 6-2: The Structure of an Alpha Processing Node 178
Figure 6-3: Sun Microsystems Version 1.5 Processor Board 179
Figure 6-4: Sun Microsystems Version 1.5 Memory Management Unit 180
Figure 6-5: Sun Microsystems Version 2.0 Processor Board 181
Figure 6-6: Sun Microsystems Version 2.0 Memory Management Unit 182
List of Tables
Tabic 2-1: Lock Compatibility Table
1
Introduction
This book documents a set of kernel-level mechanisms that support the construction of modular,
reliable, decentralized operating systems for real-time control applications. The perspective and
scope of this research is the entire computer system, rather than the more narrow focus on a subsys-
tem or algorithm. Consequently, its major contributions extend from programming abstractions and
an operating system kernel interface down through the detailed engineering tradeoffs required to
create, implement, and cleanly integrate the internal mechanisms. Furthermore, this system also
illustrates how the judicious use of hardware support can make possible kernel mechanisms and
abstractions that might otherwise be impractical. This is an initial report of on-going research by the
Archons project, and much work remains to be done in the use, analysis, optimization, and extension
of the kernel. However, many of the initial system objectives have already been validated by the
successful design and implementation of the Alpha kernel directly on distributed system hardware.
1.1. Background
The operating system kernel described herein is named Alpha and represents the initial phase of the
first implementation effort of the Archons project at Carnegie-Mellon University [Jensen 84]. This
project, led by E. Douglas Jensen, is performing research on new concepts and techniques of
decentralized computer resource management (both in operating systems and architectures) for real-
time command and control applications. The Archons project has been active since 1979, sponsored
primarily by the USAF Rome Air Development Center, with additional funding from the USN
Ocean Systems Center, the IBM Corporation Federal Systems Division, and the Fort Worth Division
of General Dynamics Corporation.
The other areas of research in the Archons project that influenced the design of Alpha include:
setwise-serializable transactions [Sha 85a], interprocessor communications [Applewhite 81], time-
driven scheduling [Locke 86], and the object programming model [McKendry 84a].
