Endless Ebook, One Click Away – Start Downloading at ebookname.

com

Distributed Computing Fundamentals Simulations and

Advanced Topics 2nd Edition Hagit Attiya

https://ebookname.com/product/distributed-computing-
fundamentals-simulations-and-advanced-topics-2nd-edition-
hagit-attiya/

OR CLICK HERE

DOWLOAD EBOOK

Browse and Get More Ebook Downloads Instantly at https://ebookname.com

Click here to visit ebookname.com and download ebook now
Instant digital products (PDF, ePub, MOBI) available
Download now and explore formats that suit you...

Distributed Computing Fundamentals Simulations and

Advanced Topics Second Edition Hagit Attiya

https://ebookname.com/product/distributed-computing-fundamentals-
simulations-and-advanced-topics-second-edition-hagit-attiya/

Concurrent and Distributed Computing in Java 1st

Edition Vijay K. Garg

https://ebookname.com/product/concurrent-and-distributed-
computing-in-java-1st-edition-vijay-k-garg/

Parallel Distributed and Pervasive Computing 1st

Edition Marvin Zelkowitz Ph.D. Ms Bs.

https://ebookname.com/product/parallel-distributed-and-pervasive-
computing-1st-edition-marvin-zelkowitz-ph-d-ms-bs/

Beginning python from novice to professional 2nd

Edition Magnus Lie Hetland

https://ebookname.com/product/beginning-python-from-novice-to-
professional-2nd-edition-magnus-lie-hetland/
Advancing the Human Right to Health 1st Edition Jose M.
Zuniga

https://ebookname.com/product/advancing-the-human-right-to-
health-1st-edition-jose-m-zuniga/

The Temple of Jerusalem Wonders of the World Simon

Goldhill

https://ebookname.com/product/the-temple-of-jerusalem-wonders-of-
the-world-simon-goldhill/

The Qur an s Reformation of Judaism and Christianity

Return to the Origins Holger M. Zellentin

https://ebookname.com/product/the-qur-an-s-reformation-of-
judaism-and-christianity-return-to-the-origins-holger-m-
zellentin/

Random Walk in Random and Non Random Environments 2nd

ed Edition Pal Revesz

https://ebookname.com/product/random-walk-in-random-and-non-
random-environments-2nd-ed-edition-pal-revesz/

Militarism Sport Europe War Without Weapons The

European Sports History Review Volume 5 1st Edition J.
A. Mangan

https://ebookname.com/product/militarism-sport-europe-war-
without-weapons-the-european-sports-history-review-volume-5-1st-
edition-j-a-mangan/
Atlas of Nuclear Cardiology Imaging Companion to
Braunwald s Heart Disease Expert Consult Online and
Print Imaging Techniques to Braunwald s Heart Disease
1st Edition Ami E. Iskandrian Md Macc Faha Fasnc
https://ebookname.com/product/atlas-of-nuclear-cardiology-
imaging-companion-to-braunwald-s-heart-disease-expert-consult-
online-and-print-imaging-techniques-to-braunwald-s-heart-
disease-1st-edition-ami-e-iskandrian-md-macc-faha/
Distributed Computing
WILEY SERIES ON PARALLEL
AND DISTRIBUTED COMPUTING

Editor: Albert Y. Zomaya

A complete list of titles in this series appears at the end of this volume.
Distributed Computing
Fundamentals, Simulations
and Advanced Topics

Second Edition

Hagit Attiya
Jennifer Welch

ffiWILEY­
�INTERSCIENCE
A JOHN WILEY & SONS, INC., PUBLICATION
This text is printed on acid-free paper. @)
Copyright © 2004 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form
or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as
permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior
written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to
the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax
(978) 646-8600, or on the web at www.copyright.com. Requests to the Publisher for permission should
be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ
07030, (201) 748-6011, fax (201) 748-6008.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in
preparing this book, they make no representations or warranties with respect to the accuracy or
completeness of the contents of this book and specifically disclaim any implied warranties of
merchantability or fitness for a particular purpose. No warranty may be created or extended by sales
representatives or written sales materials. The advice and strategies contained herein may not be
suitable for your situation. You should consult with a professional where appropriate. Neither the
publisher nor author shall be liable for any Joss of profit or any other commercial damages, including
but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services please contact our Customer Care
Department within the U.S. at 877-762-2974, outside the U.S. at 317-572-3993 or fax 317-572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print,
however, may not be available in electronic format.

Library of Congress Cataloging-in-Publication Data is available.

ISBN 0-471-45324-2

Printed in the United States of America.

10 9 8 7 6 5 4 3 2 1
 Preface

The explosive growth of distributed computing systems makes understanding them

imperative. Yet achieving such understanding is notoriously difficult, because of the
uncertainties introduced by asynchrony, limited local knowledge, and partial failures.
The field of distributed computing provides the theoretical underpinning for the design
and analysis of many distributed systems: from wide-area communication networks,
through local-area clusters of workstations to shared-memory multiprocessors.
This book aims to provide a coherent view of the theory of distributed computing,
highlighting common themes and basic techniques. It introduces the reader to the
fundamental issues underlying the design of distributed systems-communication,
coordination, synchronization, and uncertainty-and to the fundamental algorithmic
ideas and lower bound techniques. Mastering these techniques will help the reader
design correct distributed applications.
This book covers the main elements of the theory of distributed computing, in
a unifying approach that emphasizes the similarities between different models and
explains inherent discrepancies between them. The book presents up-to-date results
in a precise, and detailed, yet accessible manner. The emphasis is on fundamental
ideas, not optimizations. More difficult results are typically presented as a series
of increasingly complex solutions. The exposition highlights techniques and results
that are applicable in several places throughout the text. This approach exposes the
inherent similarities in solutions to seemingly diverse problems.
The text contains many accompanying figures and examples. A set of exercises,
ranging in difficulty, accompany each chapter. The notes at the end of each chapter

v
vi PREFACE

provide a bibliographic history of the ideas and discuss their practical applications in
existing systems.
Distributed Computing is intended as a textbook for graduate students and ad­
vanced undergraduates and as a reference for researchers and professionals. It should
be useful to anyone interested in learning fundamental principles concerning how to
make distributed systems work, and why they sometimes fail to work. The expected
prerequisite knowledge is equivalent to an undergraduate course in analysis of (se­
quential) algorithms. Knowledge of distributed systems is helpful for appreciating
the applications of the results, but it is not necessary.
This book presents the major models of distributed computing, varying by the
mode of communication (message passing and shared memory), by the synchrony
assumptions (synchronous, asynchronous, and clocked), and by the failure type (crash
and Byzantine). The relationships between the various models are demonstrated by
simulations showing that algorithms designed for one model can be run in another
model. The book covers a variety of problem domains within the models, including
leader election, mutual exclusion, consensus, and clock synchronization. It presents
several recent developments, including fast mutual exclusion algorithms, queue locks,
distributed shared memory, the wait-free hierarchy, and failure detectors.
Part I of the book introduces the major issues-message passing and shared
memory communication, synchronous and asynchronous timing models, failures,
proofs of correctness, and lower bounds-in the context of three canonical problems:
leader election, mutual exclusion, and consensus. It also presents the key notions of
causality of events and clock synchronization.
Part II addresses the central theme of simulation between models of distributed
computing. It consists of a series of such simulations and their applications, in­
cluding more powerful interprocess communication from less powerful interprocess
communication, shared memory from message passing, more synchrony from less
synchrony, and more benign kinds of faults from less benign kinds of faults.
Part III samples advanced topics that have been the focus of recent research, in­
cluding randomization, the wait-free hierarchy, asynchronous solvability, and failure
detectors.
An introductory course based in this book could cover Chapters 2 through 1 0,
omitting Section 1 0.3. A more theoretical course could cover Chapters 2, 3, 4, 5,
Section 14.3, and Chapters 1 0, 15, 1 1 and 1 7 . Other courses based on this book are
possible; consider the chapter dependencies on the next page. The book could also
be used as a supplemental text in a more practically oriented course, to flesh out the
treatment of logical and vector clocks (Chapter 6), clock synchronization (Chapters 6
and 1 3), fault tolerance (Chapters 5 and 8), distributed shared memory (Chapter 9),
and failure detectors (Chapter 1 7).
Changes in the second edition: We have made the following changes:

• We added a new chapter (Chapter 17) on failure detectors and their application
to solving consensus in asynchronous systems and deleted two chapters, those
on bounded timestamps (formerly Chapter 1 6) and sparse network covers
(formerly Chapter 1 8).
 PREFACE vii

Chapter l
Introduction

Chapter 9
Distributed Shared Memory

Chapter 10
Read/Write Objects

Chapter 1 7 Chapter 1 6
Eventually Stable Consensus Asynchronous Solvability
Chapter dependencies.

• We added new material to the existing chapters on fast mutual exclusion and
queue locks (Chapter 4), practical clock synchronization (Chapters 6 and 1 3),
and the processor lower bound for simulating shared memory with message
passing (Chapter 1 0).

• We corrected errors and improved the presentation throughout. Improvements

include a simpler proof of the round lower bound for consensus (Chapter 5)
and a simpler randomized consensus algorithm in Chapter 14.
Acknowledgments for the first edition: Many people contributed to our view of
distributed computing i n general, and to this book in particular. Danny Dolev and
Nancy Lynch i ntroduced us to this subject area. Fred Schneider provided moral
support in continuing the book and ideas for organization. Oded Goldreich and Marc
Snir inspired a general scientific attitude.
Our graduate students helped in the development of the book. At the Technion,
Ophir Rachman contributed to the lecture notes written in 1 993-94, which were the
origin of this book. Jennifer Walter read many versions of the book very carefully
viii PREFACE

at TAMU, as did Leonid Fouren, who was the teaching assistant for the class at the
Technion.
The students in our classes suffered through confusing versions of the material and
provided a lot of feedback; in particular, we thank Eyal Dagan, Eli Stein (Technion,
Spring 1 993), Saad Biaz, Utkarsh Dhond, Ravishankar Iyer, Peter Nuernberg, Jingyu
Zhang (TAMU, Fall 1996), Alla Gorbach, Noam Rinetskey, Asaf Shatil, and Ronit
Teplixke (Technion, Fall 1997).
Technical discussions with Yehuda Afek, Brian Coan, Eli Gafni, and Maurice
Herlihy helped us a lot. Several people contributed to specific chapters (in alphabetic
order): Jim Anderson (Chapter 4), Rida Bazzi (Chapter 1 2), Ran Cannetti (Chap­
ter 14) , Soma Chaudhuri (Chapter 3), Shlomi Dolev (Chapters 2 and 3),Roy Friedman
(Chapters 8 and 9), Sibsankar Haldar (Chapters 1 0), Martha Kosa (Chapter 9), Eyal
Kushilevitz (Chapter 14), Dahlia Malkhi (Chapter 8), Mark Moir (Chapter 4), Gil
Neiger (Chapter 5), Boaz Patt-Shamir (Chapters 6, 7 and 1 1), Sergio Rajsbaum
(Chapter 6), and Krishnamurthy Vidyasankar (Chapters 10).
Acknowledgmentsfor the second edition: We appreciate the time that many people
spent using the first edition and giving us feedback. We benefited from many of Eli
Gafni 's ideas. Panagiota Fatourou provided us with a thoughtful review. Evelyn
Pierce carefully read Chapter 10. We received error reports and suggestions from Uri
Abraham, James Aspnes, Soma Chaudhuri, Jian Chen, Lucia Dale, Faith Fich, Roy
Friedman, Mark Handy, Maurice Herlihy, Ted Herman, Lisa Higham, lyad Kanj,
ldit Keidar, Neeraj Kou!, Ajay Kshemkalyani, Marios Mavronicolas, Erich Mikk,
Krzysztof Parzyszej , Antonio Romano, Eric Ruppert, Cheng Shao, T.N. Srikanta,
Jennifer Walter, and Jian Xu.
Several people affiliated with John Wiley & Sons deserve our thanks. We are
grateful to Albert Zomaya, the editor-in-chief of the Wiley Series on Parallel and
Distributed Computing for his support. Our editor Val Moliere and program coordi­
nator Kirsten Rohstedt answered our questions and helped keep us on track.
Writing this book was a long project, and we could not have lasted without the love
and support of our families. Hagit thanks Osnat, Rotem and Eyal, and her parents.
Jennifer thanks George, Glenn, Sam, and her parents.
The following web site contains supplementary material relating to this book,
including pointers to courses using the book and information on exercise solutions
and lecture notes for a sample course:
http://www.cs.technion.ac.il/""hagiVOC/

Dedicated to our parents:

Malka and David Attiya
Judith and Ernest Lundelius
 Contents

1 Introduction 1
1.1 Distributed Systems 1
1.2 T heory of Distributed Computing 2
1.3 Over view 3
1.4 Relationship of T heory to Practice 4

Part I F undamentals

2 Basic Algorithms in Message-Passing Systems 9

2.1 Formal Models for Message Passing Systems 9
2.1.l Systems 9
2.1.2 Complexity Measures 13
2.1.3 Pseudocode Conventions 14
2.2 Broadcast and Convergecast on a Spanning Tree 15
2.3 F looding and Building a Spanning Tree 19
2.4 Constructing a Depth-First Search Spanning Tree for
a Specified Root 23
2.5 Constructing a Depth-First Search Spanning Tree
without a Specified Root 25

ix
x CONTENTS

3 Leader Election in Rings 31

3.1 The Leader Election Problem 31
3.2 Anonymous Rings 32
3.3 Asynchronous Rings 34
3.3.1 An O(n 2 ) Algorithm 34
3.3.2 An O(niog n) Algorithm 35
3.3.3 An fi(n log n) Lower Bound 38
3.4 Synchronous Rings 42
3.4.1 An O(n) Upper Bound 43
3.4.2 An fi(n log n) Lower Bound for Restricted
Algorithms 48

4 Mutual Exclusion in Shared Memory 59

4.1 Formal Model for Shared Memory Systems 60
4.1.1 Systems 60
4.1.2 Complexity Measures 62
4.1.3 Pseudocode Conventions 62
4.2 The Mutual Exclusion Problem 63
4.3 Mutual Exclusion Using Poweiful Primitives 65
4.3.1 Binary Test&Set Registers 65
4.3.2 Read-Modify-Write Registers 66
4.3.3 Lower Bound on the Number of Memory States 69
4.4 Mutual Exclusion Using Read/Write Registers 71
4.4.1 The Bakery Algorithm 71
4.4.2 A Bounded Mutual Exclusion Algorithm for
Two Processors 73
4.4.3 A Bounded Mutual Exclusion Algorithm for n
Processors 77
4.4.4 Lower Bound on the Number of Read/Write
Registers 80
4.4.5 Fast Mutual Exclusion 84

5 Fault-Tolerant Consensus 91
5.1 Synchronous Systems with Crash Failures 92
5.1.1 Formal Model 92
5.1.2 The Consensus Problem 93
5.1.3 A Simple Algorithm 93
5.1.4 Lower Bound on the Number of Rounds 95
5.2 Synchronous Systems with Byzantine Failures 99
 CONTENTS xi

5.2.J Formal Model JOO

5.2.2 The Consensus Problem Revisited JOO
5.2.3 Lower Bound on the Ratio of Faulty Processors JOJ
5.2.4 An Exponential Algorithm J 03
5.2.5 A Polynomial Algorithm J06
5.3 Impossibility in Asynchronous Systems J08
5.3.J Shared Memory-The Wait-Free Case J09
5.3.2 Shared Memory-The General Case J JJ
5.3.3 Message Passing J J9

6 Causality and Time J25

6.J Capturing Causality J26
6.J.J The Happens-Before Relation J26
6.J.2 Logical Clocks J28
6.J.3 Vector Clocks J29
6.J.4 Shared Memory Systems 133
6.2 Examples of Using Causality J33
6.2.J Consistent Cuts J34
6.2.2 A Limitation of the Happens-Before Relation:
The Session Problem J37
6.3 Clock Synchronization J40
6.3.J Modeling Physical Clocks J40
6.3.2 The Clock Synchronization Problem J43
6.3.3 The Two Processors Case J44
6.3.4 An Upper Bound J46
6.3.5 A Lower Bound J48
6.3.6 Practical Clock Synchronization: Estimating
Clock Difef rences J49

Part II Simulations

7 A Formal Model for Simulations J57

7.J Problem Specifications J57
7.2 Communication Systems J58
7.2.J Asynchronous Point-to-Point Message Passing J59
7.2.2 Asynchronous Broadcast J59
7.3 Processes J60
7.4 Admissibility J63
xii CONTENTS

7.5 Simulations 164

7.6 Pseudocode Conventions 165

8 Broadcast and Multicast 167

8.1 Specification of Broadcast Services 168
8.1.1 The Basic Service Specification 168
8.1.2 Broadcast Service Qualities 169
8.2 Implementing a Broadcast Service 171
8.2.1 Basic Broadcast Service 172
8.2.2 Single-Source F IF O Ordering 172
8.2.3 Totally Ordered Broadcast 172
8.2.4 Causality 175
8.2.5 Reliability 177
8.3 Multicast in Groups 179
8.3.1 Specification 180
8.3.2 Implementation 181
8.4 An Application: Replication 183
8.4.1 Replicated Database 183
8.4.2 The State Machine Approach 183

9 Distributed Shared Memory 189

9.1 Linearizable Shared Memory 190
9.2 Sequentially Consistent Shared Memory 192
9.3 Algorithms 193
9.3.1 Linearizability 193
9.3.2 Sequential Consistency 194
9.4 Lower Bounds 198
9.4.1 Adding Time and Clocks to the Layered Model 198
9.4.2 Sequential Consistency 199
9.4.3 Linearizability 199

10 Fault-Tolerant Simulations of Read/Write Objects 207

JO.I Fault-Tolerant Shared Memory Simulations 208
10.2 Simple Read/Write Register Simulations 209
10.2.1 Multi-Valued from Binary 210
10.2.2 Multi-Reader from Single-Reader 215
10.2.3 Multi-Writer from Single-Writer 219
10.3 Atomic Snapshot Objects 222
 CONTENTS xiii

10.3.J Handshaking Procedures 223

10.3.2 A Bounded Memory Simulation 225
10.4 Simulating Shared Registers in Message-Passing
Systems 229

11 Simulating Synchrony 239

11.1 Synchronous Message-Passing Specification 240
11.2 Simulating Synchronous Processors 241
11.3 Simulating Synchronous Processors and Synchronous
Communication 243
11.3.1 A Simple Synchronizer 243
11.3.2 Application: Constructing a Breadth-First
Search Tree 247
11.4 Local vs. Global Simulations 247

12 Improving the Fault Tolerance of Algorithms 251

12.1 Overview 251
12.2 Modeling Synchronous Processors and Byzantine
fu�� 2"
12.3 Simulating Identical Byzantine Failures on Top of
Byzantine Failures 255
12.3.1 Definition of Identical Byzantine 255
12.3.2 Simulating Identical Byzantine 256
12.4 Simulating Omission Failures on Top of Identical
Byzantine Failures 258
12.4.1 Definition of Omission 259
12.4.2 Simulating Omission 259
12.5 Simulating Crash Failures on Top of Omission Failures 264
12.5.l Definition of Crash 264
12.5.2 Simulating Crash 265
12.6 Application: Consensus in the Presence of Byzantine
fu�� 2�
12.7 Asynchronous Identical Byzantine on Top of Byzantine
fu�� 2@
12.7.1 Definition of Asynchronous Identical
Byzantine 269
12.7.2 Definition of Asynchronous Byzantine 270
12.7.3 Simulating Asynchronous Identical Byzantine 2 70

13 Fault-Tolerant Clock Synchronization 277

xiv CONTENTS

13.1 ProblemDefinition 277

13.2 The Ratio of FaultyProcessors 279
13.3 A Clock Synchronization Algorithm 284
13.3.1 Timing Failures 284
13.3.2 Byzantine Failures 290
13.4 Practical Clock Synchronization: Identifying Faulty
Clocks 291

Part III Advanced Topics

14 Randomization 297
14.1 Leader Election: A Case Study 297
14.1.1 Weakening the ProblemDefinition 297
14.1.2 Synchronous One-Shot Algorithm 299
14.1.3 Synchronous Iterated Algorithm and
Expectation 300
14.1.4 Asynchronous Systems and Adversaries 302
14.1.5 Impossibility of Uniform Algorithms 303
14.1.6 SummaryofProbabilisticDefinitions 303
14.2 Mutual Exclusion with Small Shared Variables 305
14.3 Consensus 308
14.3.1 The General Algorithm Scheme 309
14.3.2 A Common Coin with Constant Bias 314
14.3.3 Tolerating Byzantine Failures 315
14.3.4 Shared Memory Systems 316

15 Wait-Free Simulations of ArbitraryObjects 321

15.1 Example: A F IFO Queue 322
15.2 The Wait-Free Hierarchy 326
15.3 Universality 327
15.3.1 A Nonblocking Simulation Using
Compare&Swap 328
15.3.2 A Nonblocking Algorithm Using Consensus
Objects 329
15.3.3 A Wait-Free Algorithm Using Consensus
Objects 332
15.3.4 Bounding the MemoryRequirements 335
15.3.5 Handling Nondeterminism 337
 Another Random Document on
Scribd Without Any Related Topics
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
back
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookname.com