Distributed Optimization Game And Learning

Algorithms Theory And Applications In Smart Grid

Systems 1st Edition Huiwei Wang download

https://ebookbell.com/product/distributed-optimization-game-and-
learning-algorithms-theory-and-applications-in-smart-grid-
systems-1st-edition-huiwei-wang-36373782

Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Gametheoretic Learning And Distributed Optimization In Memoryless

Multiagent Systems Tatarenko

https://ebookbell.com/product/gametheoretic-learning-and-distributed-
optimization-in-memoryless-multiagent-systems-tatarenko-6753550

Distributed Optimization In Networked Systems Algorithms And

Applications Qingguo L

https://ebookbell.com/product/distributed-optimization-in-networked-
systems-algorithms-and-applications-qingguo-l-47735708

Distributed Optimization Advances In Theories Methods And Applications

1st Ed Huaqing Li

https://ebookbell.com/product/distributed-optimization-advances-in-
theories-methods-and-applications-1st-ed-huaqing-li-22476542

Distributed Optimizationbased Control Of Multiagent Networks In

Complex Environments 1st Edition Minghui Zhu

https://ebookbell.com/product/distributed-optimizationbased-control-
of-multiagent-networks-in-complex-environments-1st-edition-minghui-
zhu-5141560
Distributed Optimization And Learning A Controltheoretic Perspective
1st Edition Zhongguo Li

https://ebookbell.com/product/distributed-optimization-and-learning-a-
controltheoretic-perspective-1st-edition-zhongguo-li-231392748

Largescale And Distributed Optimization 1st Ed Pontus Giselsson

https://ebookbell.com/product/largescale-and-distributed-
optimization-1st-ed-pontus-giselsson-7324614

Speeding Up Distributed Constraint Optimization Search Algorithms

2014th Edition William Yeoh

https://ebookbell.com/product/speeding-up-distributed-constraint-
optimization-search-algorithms-2014th-edition-william-yeoh-59069740

A Class Of Algorithms For Distributed Constraint Optimization Adrian

Petcu

https://ebookbell.com/product/a-class-of-algorithms-for-distributed-
constraint-optimization-adrian-petcu-36514906

Distributed Control And Optimization Of Networked Microgrids A

Multiagent System Based Approach Power Systems Lei Ding

https://ebookbell.com/product/distributed-control-and-optimization-of-
networked-microgrids-a-multiagent-system-based-approach-power-systems-
lei-ding-38378670
Huiwei Wang
Huaqing Li
Bo Zhou

Distributed
Optimization,
Game and
Learning
Algorithms
Theory and Applications in Smart Grid
Systems
Distributed Optimization, Game and Learning
Algorithms
Huiwei Wang Huaqing Li Bo Zhou
• •

Distributed Optimization,
Game and Learning
Algorithms
Theory and Applications in Smart Grid
Systems

123
Huiwei Wang Huaqing Li
Southwest University Southwest University
Chongqing, China Chongqing, China

Bo Zhou
Chongqing Jiaotong University
Chongqing, China

ISBN 978-981-33-4527-0 ISBN 978-981-33-4528-7 (eBook)

https://doi.org/10.1007/978-981-33-4528-7

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature
Singapore Pte Ltd. 2021
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
herein or for any errors or omissions that may have been made. The publisher remains neutral with regard
to jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
To My Family
Huiwei Wang
To My Family
Huaqing Li
To My Family
Bo Zhou
Preface

Advances in wireless technology and computing power have necessitated the

development of theory, models, and tools to cope with the new challenges posed by
large-scale control and optimization problems over networks. The classical opti-
mization methodology works under the premise that all problem data are available
to a centralized server. However, this premise does not apply to large networked
systems in the distributed environment motivated by applications including power
systems, sensor networks, smart buildings, and smart manufacturing. In such an
environment, each node (agent) performs local computation based on its own data
(information) and information received from its neighboring agents through the
underlying communication network, so that the large-scale control and optimization
problem can be solved in a distributed manner. Eventually, the centralized opti-
mization methodology must surely slide into decline, bringing a new distributed
optimization type into being, which considers effective coordination between
multiple agents, i.e., all agents cooperate to minimize a global function which is a
sum of local objective functions.
This book investigates several standard hot topics in recent distributed opti-
mization problems, such as the unconstrained optimization, constrained optimiza-
tion, distributed game, and distributed/decentralized learning, etc. To emphasize the
role of the distributed optimization in these topics, we focus on a simple primal
(sub)gradient method, but we also provide an overview of other distributed methods
for optimization in networks. Applications of the distributed optimization frame-
work to the control of power systems are also presented. This book mainly includes,
naturally, three parts. The first part deals with the distributed optimization algorithm
theory, which consists of four chapters: (1) Cooperative distributed optimization in
multi-agent networks with delays; (2) Constrained consensus of multi-agent sys-
tems with time-varying topology; (3) Distributed optimization under inequality
constraints and random projections; and (4) Accelerated distributed optimization
over digraphs with stochastic matrices. The second part, as a transition, is con-
cerned with the distributed optimization algorithm theory and their applications in
dynamic economic dispatch problems of smart grid systems, which includes two
chapters: (5) Linear convergence for constrained optimization over time-varying

vii
viii Preface

digraphs; and (6) Stochastic gradient-push for economic dispatch on time-varying

digraphs. The analysis and synthesis of distributed optimization, game, and learning
algorithm theory are treated in the third part, all algorithms in this part are designed
for the targeted case scenarios within smart grid systems. This part consists of three
chapters: (7) Reinforcement learning in energy trading game among smart micro-
grids; (8) Reinforcement learning for constrained games with incomplete infor-
mation; and (9) Reinforcement learning for PHEV route choice based on congestion
game. Among the topics, simulation results including practical application exam-
ples are presented to illustrate the effectiveness and the practicability of the opti-
mization, game, and learning algorithms proposed in the previous parts.
This book is appropriate as a college course textbook for undergraduate and
graduate students majoring in computer science, automation, artificial intelligence,
electric engineering, etc., and as a reference material for researchers and technol-
ogists in related fields.

Chongqing, China Huiwei Wang

Huaqing Li
Bo Zhou
Acknowledgements

This book was supported in part by the National Natural Science Foundation of
China under Grants 61773321 and 61803056, in part by the Fundamental Research
Funds for the Central Universities under Grant XDJK2018B013, in part by China
Postdoctoral Science Foundation under Grant 2017M620374, and in part by the
National Natural Science Foundation of Chongqing under Grant cstc2020jcyj-
msxmX0057. We would like to begin by acknowledging Ziyu Sheng, Xing Liu,
Lianjiao Wang, Yifan Yu, Junjie Lv, Yang Yun, and Yan Zuo who have unselfishly
given their valuable time in arranging raw materials. Their assistance has been
invaluable for the completion of this book. The authors are especially grateful to
their families for their encouragement and never-ending support when it was most
required. Finally, we would like to thank the editors at Springer for their profes-
sional and efficient handling of this book.

ix
Contents

1 Cooperative Distributed Optimization in Multiagent Networks

with Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Preliminary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.6 Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2 Constrained Consensus of Multi-agent Systems with Time-Varying
Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.1 Model Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.2 State Transition Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.3 Constrained Consensus . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.4 Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.4.1 Constrained Consensus of a Simple Directed Multi-agent
System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 30
2.4.2 Constrained Consensus of a Large-Scale Directed
Multi-agent System . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 33
2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 36
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 36

xi
xii Contents

3 Distributed Optimization Under Inequality Constraints

and Random Projections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.2 Problem Formulation and Assumptions . . . . . . . . . . . . . . . . . . . . 41
3.2.1 Network Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2.3 Lagrangian Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2.4 Distributed Primal-Dual Random Projection Subgradient
Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.3 Basic Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.4 Convergence Results for Diminishing Stepsize . . . . . . . . . . . . . . . 50
3.5 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4 Accelerated Distributed Optimization over Digraphs
with Stochastic Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.2 Preliminaries and Problem Formulation . . . . . . . . . . . . . . . . . . . . 69
4.3 Algorithm Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.4 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.4.1 Auxiliary Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.4.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.5 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5 Linear Convergence for Constrained Optimization
Over Time-Varying Digraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.2.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.2.2 Communication Network . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.2.3 Necessary Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.3 Main Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.3.1 Centralized Primal-Dual Method . . . . . . . . . . . . . . . . . . . . 88
5.3.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.3.3 Distributed Primal-Dual Push-DIGing Algorithm . . . . . . . . 91
5.3.4 Supporting Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.4 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.5 Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.5.1 Case Study 1: Simulation for Five-Agents Network . . . . . . 101
5.5.2 Case Study 2: Simulation for Large-Scale Network . . . . . . 103
5.5.3 Case Study 3: Performance Comparison . . . . . . . . . . . . . . 105
Contents xiii

5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6 Stochastic Gradient-Push for Economic Dispatch on Time-Varying
Digraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6.2.1 Graph Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6.2.2 Economic Dispatch Problem . . . . . . . . . . . . . . . . . . . . . . . 112
6.2.3 Centralized Lagrangian-Based Method . . . . . . . . . . . . . . . 113
6.3 Algorithm and Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6.3.1 State-of-the-Art Method for EDP . . . . . . . . . . . . . . . . . . . 114
6.3.2 Distributed Stochastic Gradient-Push Algorithm . . . . . . . . 115
6.3.3 Necessary Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 117
6.3.4 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
6.4 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.4.1 Supporting Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.4.2 Proof of Theorem 6.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
6.4.3 Proof of Theorem 6.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
6.4.4 Proof of Theorem 6.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
6.5 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.5.1 Without Time Delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.5.2 With Time Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
7 Reinforcement Learning in Energy Trading Game Among Smart
Microgrids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
7.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.2.1 Smart Grid Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.2.2 Specifical Utility Functions . . . . . . . . . . . . . . . . . . . . . . . 141
7.2.3 Stackelberg Game Scheme . . . . . . . . . . . . . . . . . . . . . . . . 142
7.3 Utility Maximization and Best Strategy . . . . . . . . . . . . . . . . . . . . 143
7.3.1 Utility Maximization and the Best Action . . . . . . . . . . . . . 144
7.3.2 Learning the Best Response . . . . . . . . . . . . . . . . . . . . . . . 145
7.3.3 Properties of Stackelberg Equilibrium . . . . . . . . . . . . . . . . 147
7.4 Learning Algorithms Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.4.1 Finite Action Learning Automaton (FALA) . . . . . . . . . . . . 147
7.4.2 Continuous Action Learning Automaton (CALA) . . . . . . . 149
7.5 Numerical Case Studies and Discussion . . . . . . . . . . . . . . . . . . . . 150
7.5.1 Case Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
7.5.2 Buyer Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
xiv Contents

7.5.3 Seller Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

7.5.4 Stackelberg Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
7.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
8 Reinforcement Learning for Constrained Games with Incomplete
Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
8.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
8.2.1 Smart Grid Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
8.2.2 Double Auction Mechanism . . . . . . . . . . . . . . . . . . . . . . . 165
8.3 Learning-Based Game to Energy Trading . . . . . . . . . . . . . . . . . . . 166
8.3.1 Strategy Updating as Learning Automaton . . . . . . . . . . . . 167
8.3.2 Repeated Game with Incomplete Information . . . . . . . . . . 167
8.4 Mixed-Strategy NE and Learning Games . . . . . . . . . . . . . . . . . . . 169
8.4.1 Mixed Strategy and Nash Equilibrium . . . . . . . . . . . . . . . 169
8.4.2 Stationary Strategies and Convergence Analysis . . . . . . . . 170
8.5 Properties of Nash Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . 172
8.5.1 Existence of NE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
8.5.2 Diagonal Concavity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
8.5.3 Lagrange Multiplier and Uniqueness of NE . . . . . . . . . . . . 178
8.6 Learning Algorithm Design and Analysis . . . . . . . . . . . . . . . . . . . 179
8.7 Numerical Case Studies and Discussion . . . . . . . . . . . . . . . . . . . . 182
8.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
9 Reinforcement Learning for PHEV Route Choice Based
on Congestion Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
9.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
9.2.1 Route Choice Problem in Traffic Networks . . . . . . . . . . . . 194
9.2.2 Learning Automaton . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
9.3 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
9.3.1 Existence and Uniqueness of Optimal Route Choice
Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
9.3.2 Reinforcement Learning Scheme . . . . . . . . . . . . . . . . . . . . 199
9.4 Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
9.4.1 Experiment Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 207
9.4.2 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
9.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
List of Figures

Fig. 1.1 Plots of function error versus number of iterations for a random
geometric graph with N ¼ 5 nodes . . . . . . . . . . . . . . . . . . . . . .. 15
Fig. 2.1 Asynchronous communication topologies of a simple
multi-agent system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 31
Fig. 2.2 State evolution of the agents in a simple multi-agent system . .. 32
Fig. 2.3 State evolution of the agents in a simple multi-agent system . .. 32
Fig. 2.4 Asynchronous communication topologies of relatively large
multi-agent systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 34
Fig. 2.5 The union of the communication topologies . . . . . . . . . . . . . . .. 34
Fig. 2.6 Evolutions of the states of the agents of relatively
large multi-agent systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 35
Fig. 2.7 Evolutions of the states of the agents of relatively
large multi-agent systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 35
Fig. 3.1 Communication topology of the multiagent systems
with 4 agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Fig. 3.2 Estimates of variable x1 of DPDRPS algorithm . . . . . . . . . . . . . 62
Fig. 3.3 Estimates of variable x2 of DPDRPS algorithm . . . . . . . . . . . . . 62
Fig. 4.1 Performance comparison across different algorithms . . . . . . . . . . 80
Fig. 5.1 A time-varying directed unbalanced network GðkÞ that
switches among three different topologies G1 , G2 and G3 , i.e.,
Gð3kÞ ¼ G1 , Gð3k þ 1Þ ¼ G2 and Gð3k þ 2Þ ¼ G3
for all k  0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Fig. 5.2 The estimates of optimal power allocation for Case Study 1 . . . 102
Fig. 5.3 The estimates of optimal Lagrange multipliers for Case
Study 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Fig. 5.4 The estimate of optimal state allocation x for Case Study 2 . . . . 104
Fig. 5.5 The estimate of optimal Lagrange multipliers for Case
Study 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
Fig. 5.6 Performance comparison over time-varying directed
unbalanced networks for Case Study 3 . . . . . . . . . . . . . . . . . . . . 106
Fig. 6.1 Time-varying directed communication networks . . . . . . . . . . . . . 127

xv
xvi List of Figures

Fig. 6.2 Simulation results of the 4-bus system with synchronous

step-size but without noisy gradient . . . . . . . . . . . . . . . . . . . . . . 129
Fig. 6.3 Simulation results of the 4-bus system with asynchronous
step-sizes and noisy gradient. . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Fig. 6.4 Simulation results of the 4-bus system with synchronous
step-size but without noisy gradient under time delays . . . . . . . . 131
Fig. 6.5 Simulation results of the 4-bus system with asynchronous
step-sizes and noisy gradient under time delays . . . . . . . . . . . . . 132
Fig. 7.1 Energy trading among microgrids . . . . . . . . . . . . . . . . . . . . . . . . 140
Fig. 7.2 Microgird test system based on IEEE 37-bus test feeder . . . . . . 151
Fig. 7.3 DGs output and load over 24 h . . . . . . . . . . . . . . . . . . . . . . . . . 152
Fig. 7.4 Probability of price bid action and power injection versus
iteration number, c ¼ 0:1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Fig. 7.5 Average action and average utility versus iteration number,
c ¼ 0:1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
Fig. 7.6 Performance of the CALA-based learning algorithm versus
iteration number, c ¼ 106 , C ¼ 1 and rL ¼ 103 . . . . . . . . . . . 155
Fig. 7.7 Probability of proportion action and power abortion versus
iteration number, c ¼ 0:1 and k ¼ 104 . . . . . . . . . . . . . . . . . . . 156
Fig. 7.8 Evolution of payoff radius in comparison with probability
of action versus actions, c ¼ 0:1, t ¼ 102 . . . . . . . . . . . . . . . . . . 157
Fig. 7.9 Performance of the CALA-based learning algorithm versus
iteration number, c ¼ 106 , C ¼ 1 and rL ¼ 103 . . . . . . . . . . . 157
Fig. 7.10 Average action and average utility of buyers versus round
number, t ¼ 103 and 100 experiments . . . . . . . . . . . . . . . . . . . . 158
Fig. 7.11 Average action and average utility of sellers versus round
number, t ¼ 103 and 100 experiments . . . . . . . . . . . . . . . . . . . . 158
Fig. 8.1 Dynamic pricing mechanism via double auction . . . . . . . . . . . . . 165
Fig. 8.2 A sketch of the expected utility . . . . . . . . . . . . . . . . . . . . . . . . . 174
Fig. 8.3 Energy trading test system based on IEEE 13-bus test
feeder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Fig. 8.4 The initialized reservation price and the evolution
of the trading price . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Fig. 8.5 The evolution of the probability distribution of actions
for each seller i ¼ f1; 2; 3; 4g in turn . . . . . . . . . . . . . . . . . . . . . 184
Fig. 8.6 Real-time power abortion versus power supply. . . . . . . . . . . . . . 185
Fig. 8.7 Real-time power injection versus power demand . . . . . . . . . . . . 186
Fig. 8.8 Averaged trading quantity and utility of each seller . . . . . . . . . . 187
Fig. 9.1 The Nguyen–Dupuis network . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
Fig. 9.2 Traffic flows at equilibrium on each route . . . . . . . . . . . . . . . . . 208
Fig. 9.3 The evolutions trajectories of route choice strategies
of the PHEVs on Route 2 for different solution methods . . . . . . 209
Fig. 9.4 The evolutions trajectories of route choice strategies
of the PHEVs on Route 4 for different solution methods . . . . . . 210
List of Figures xvii

Fig. 9.5 The evolutions trajectories of route choice strategies

of the PHEVs on Route 14 for different solution methods . . . . . 211
Fig. 9.6 The evolutions trajectories of route choice strategies
of the PHEVs on Route 19 for different solution methods . . . . . 212
Fig. 9.7 The evolutions of the total travel times of the PHEVs in traffic
networks for the four solution methods. The red line, green
line, blue line and dark line present the evolutions of the total
travel times with the B–M RL-based solution method, the
multinomial logit choice model, the improved learning
automata and the modified Q-learning algorithm,
respectively . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Fig. 9.8 The proportion of PHEVs who choose the routes for Case
I-Case IV, in which the red bar, the blue bar, the yellow bar
and the dark bar represent the proportions for Case I, Case II,
Case III and Case IV, respectively . . . . . . . . . . . . . . . . . . . . . . . 214
Fig. 9.9 The running time of the four different solution methods
with respect to the four cases. The red bar, green bar, blue bar
and dark bar present the running time of the B–M RL-based
solution method, the multinomial logit choice model,
the improved learning automata and modified Q-learning
algorithm, respectively . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
Chapter 1
Cooperative Distributed Optimization in
Multiagent Networks with Delays

Abstract In this chapter, we consider a distributed cooperative optimization prob-

lem encountered in a computational multi-agent network with delay, where each
agent has local access to its convex cost function, and jointly minimizes the cost
function over the whole network. To solve this problem, we develop an algorithm
that is based on dual averaging updates and delayed subgradient information, and
analyze its convergence properties for a diminishing step-size by utilizing Bregman-
distance functions. Moreover, we provide sharp bounds on the convergence rates as
a function of the network size and topology embodied in the inverse spectral gap.
Finally, we present a numerical example to evaluate our algorithm and compare its
performance with several similar algorithms.

Keywords Delayed subgradient · Distributed optimization · Multi-agent

networks · Spectral gap

1.1 Introduction

The distributed computation of the average of datasets and the coordination of a set
of agents in optimization tasks, i.e., driving all the agents collectively to optimize a
global objective, are ubiquitous tasks in current engineering problems. Distributed
computation over networks has been extensively studied in systems and control the-
ory starting with the pioneering work of Tsitsiklis [1]. Two decades later, an excellent
work [2] introducing the theoretical framework for posing and solving consensus
problems has recently attracted much attention [3–6]. Exploiting the consensus idea
or computing exact averages of the initial values of the agents, recent works [7–14]
have investigated a distributed model for optimization over a network. For more
details, the readers may refer to the recent book [15] and the references therein.
Primary Motivations: The distributed optimization algorithms presented in earlier
literatures [7, 9–12] assume that at any time, each agent has access to estimate the
states of its immediate neighbors. As pointed out in [15], this may not be possible in
communication networks where delays exist in the transmission of agent estimates
over a communication channel. Establishing the convergence rate properties of the
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 1
H. Wang et al., Distributed Optimization, Game and Learning Algorithms,
https://doi.org/10.1007/978-981-33-4528-7_1
2 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

update rule is essential in understanding the robustness of the optimization algorithm

to delays and dynamics associated with information exchange over finite bandwidth
communication channels. In this brief, we will deal with this challenging scenario.
Literature Review: Distributed algorithms possess the inherent advantage in solv-
ing large scale optimization and machine learning problems, which are attracting
more and more researchers to this field. In a necessarily incomplete list of references
including [7, 8, 15], the authors described their approaches that project the original
update iteration onto the local constrain sets by using distributed subgradient algo-
rithms to deal with a non-differentiable convex objective function. Further theoretical
extensions of this work were presented in [9] with a look toward treatment of iden-
tical local constraint sets. Building on some earlier works, a distributed primal-dual
subgradient method was studied in [10, 11] by characterizing the primal and dual
optimal solutions as the saddle points of the Lagrangian function associated with the
problem. In addition, the primal and dual optimal networks were also investigated
for solving the assignment problem [16, 17]. Recently, a distributed dual averaging
(DDA) algorithm based on standard dual averaging [18] was proposed in [12] for
distributed optimization to investigate the effects of network size and topology on the
convergence rate. Further extensions of this work were presented in [13, 14, 19] to
handle distributed optimization problem over a network with communication delays
or quantized communication. In addition, the convergence of decentralized network
optimization algorithms that are based on their updates and delayed information
projection were analyzed in [20, 21].
Statement of Contributions: The main contributions of this chapter are twofold.
The first contribution is the extension of the DDA algorithm to tackle the distributed
non-smooth minimization problem, where the procedure receives out-of-date gra-
dients instead of current gradients. By establishing several iteration properties for
delayed subgradients, we prove that the algorithm asymptotically achieves to the
optimal value as the iteration approaches infinity in spite of delays. The main advan-
tage of this method is to preserve the performance benefits of the stale gradient
information, which will help the optimization error achieve the given accuracy faster
in many situations of practical interest. The second main contribution is a concise
analysis that investigates the effects of the delay, network size and the underlying
spectral properties of the network on the convergence rate. By utilizing the Bregman
divergence and a novel bounding technique, we provide an elegant illustration of the
role played by the delay in convergence rates: if the subgradients are
√ computed with
a delay of τ , then the convergence rate of the procedure is O(τ 2 / T ). It is easy to
observe that the optimization error caused by delay has become a second-order term:
the delay penalty is asymptotically negligible.
1.2 Related Work 3

Table 1.1 Extra costs due to delays

Literature Storage for nodes Computation for edges
[13, 14] τ |E | τ |E |
This chapter τN –

1.2 Related Work

In this section, we survey some previous work with the aim of giving a clear under-
standing of how our algorithm and results relate to and, in many cases, improve upon
it. Our work is closest to [12–14] building on the dual averaging method [18].
We present the algorithm similar to the initial discovery [13, 14], but the dif-
ference is the position of the communication delays in the DDA algorithm. In [13,
14], the authors proposed the DDA algorithm with delayed dual variable updates
to handle the distributed optimization problem over a network with communication
delays. It suffices to add finitely many virtual agents and communication links instead
of delayed transmission, then the original delayed multi-agent model is reduced to
a new augmented model without delays. Although the method in [13, 14] is ele-
gant, it leads to computational and storage burdens due to the redundant virtual
agents and communication links. Unlike [13, 14], the gradient projection steps in
this chapter are determined by the out-of-date subgradients. Compared with their
works, our algorithm directly handles the communication delay, which has tapped
the maximum potential to the utmost from the delayed gradient information with less
computational and storage requirement, as shown in Table 1.1, where τ represents
the communication delay and |E| denotes the cardinality of the edge set E. In general,
|E| = Θ(N log(N )) holds for a random geometric graph with N nodes.
As discussed in the introduction, we extend the algorithm in [12] to handle com-
munication delays, but the real-time gradient information of the objective function
is not available due to delays. In order to overcome this challenge, we bound the
inner product including delayed subgradients to meaningfully offset the effect of
delays by some novel techniques. These are primarily different from previous tech-
niques [12] and even [13, 14]. As a result, the error caused by the delays becomes
a second-order term, which means the delay penalty is asymptotically negligible. In
addition, despite of using different intermediate techniques and bounding methods,
we theoretically improve the convergence rate presented in [12] by a logarithmic
factor log(T ). Meanwhile, for a well connected network such as a bounded degree
expander, we also improve the result in [8] that provides an -optimal solution to
be independent of the network topology which can be reached in O(N 3 /2 ) time.
Finally, it is worth pointing out that our algorithm can preserve the performance gain
of the stale gradient information, which might help the optimization error to quickly
achieve the given accuracy in many situations of practical interest.
4 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

1.3 Preliminary

Consider a network of N agents, indexed by i = 1, . . . , N . The communication

topology is modeled by an undirected graph G = (V, E) over the vertex set V with
edge set E ⊂ V × V. Here, we assume the underlying communication graph G is
connected and without self loops. The objective of N agents is to cooperatively
solve the following optimization problem:

1 
N
minn f i (x) s.t. x ∈ ∩i=1
N
Xi (1.1)
x∈R N i=1

where f i : Rd → R is the objective function associated with agent i, x is a global

decision vector and Xi ⊆ Rd is the nonempty compact and convex set. Each agent i
maintains its own parameter vector xi ∈ Rd and has local access only to the objective
function f i and can only communicate directly with its immediate neighbors j ∈
N (i)  { j ∈ V|(i, j) ∈ E}. Assume that each function f i is convex and hence sub-
N
differentiable, but it needs not be smooth. We denote f (x)  (1/N ) i=1 f i (x)
and X  ∩i=1 Xi . Thus, f is convex and X is compact. To build intuition for the
N

algorithms we analyze, we begin by listing the notations and giving useful definitions
and assumptions.
Notations: The inner product of two vectors is denoted as x, y = x T y, and the
standard Euclidean norm is x = x, x 1/2 . The dual norm · ∗ to the standard
Euclidean norm is defined by v ∗  sup u =1 v, u .

Definition 1.1 A vector ∇h(x) is a subgradient of a function h at a point x in the

domain of h (denoted by dom h) if the following inequality holds

∇h(x)T (y − x) ≤ h(y) − h(x) for all y ∈ dom h. (1.2)

Definition 1.2 The function h(x) is m-strongly convex in that there exists a constant
m > 0 such that for any pair of points x ∈ X and y ∈ X , the following inequality
holds m
h(y) ≥ h(x) + ∇h(x)T (y − x) + y − x 2. (1.3)
2

Definition 1.3 ([22]) For a given compact, proper convex functional ψ(·) : Rd → R,
the Bregman divergence is defined as

Dψ (x, y) = ψ(x) − ψ(y) − ∇ψ(y), x − y . (1.4)

Assumption 1.1 For any x, y ∈ X ,

(a) Each function f i is L-Lipschitz with respect to the same norm · , i.e.,

| f i (x) − f i (y)| ≤ L x − y .
1.3 Preliminary 5

(b) The subgradient ∇ f i is G-Lipschitz continuous with respect to the norm · ,

i.e.,
G
f i (y) ≤ f i (x) + ∇ f i (x), y − x + x − y 2.
2
We note that a convex function f i is L-Lipschitz if and only if the following
supx∈X ∂ f i (x(t)) ∗ ≤ L holds. This means that, for any x ∈ X and any subgra-
dient gi (t) ∈ ∂ f i (x(t)), we have

gi (t) ∗ ≤ L. (1.5)

Assumption 1.2 For x∗ ∈ arg minx∈X f (x) and R ∈ R, the bounds ψ(x∗ ) ≤ R 2 and
Dψ (x∗ , x) ≤ R 2 for any x ∈ X are both valid.

We are now in a position to describe the DDA algorithm based on delayed subgra-
dients for solving the multi-agent cooperative optimization problem. The algorithm
is based on a proximal function ψ(x) : Rd → R. There is no loss of generality to
assume that ψ(x) is 1-strongly convex and ψ(x) ≥ 0 for all x ∈ X and that ψ(0) = 0.
At time step t, the delayed subgradient information is received, and the sequence
{z i (t), xi (t)}∞
t=0 is updated via the following iterations:


N
z i (t + 1) = Wi j z j (t) + gi (t − τ ), (1.6)
j=1

ψ
xi (t + 1) = ΠX (z i (t + 1), α(t + 1)), (1.7)

where Wi j ≥ 0 is a weight that agent i assigns to its neighbor j ∈ Ni , gi (t − τ ) ∈

∂ f i (xi (t − τ )), τ is a positive integer denoting the constant delay. Moreover,
 
ψ 1
ΠX (z i (t + 1), α(t + 1)) = arg min z i (t + 1), x + ψ(x)
x∈X α(t + 1)

is a type of projection with a non-increasing sequence of positive step sizes {α(t)}∞

t=0 .
In order to prove the main results, we show a property of the algorithm. For a
doubly stochastic matrix W , the distribution W t ei − 1/N 1 satisfies the following
inequality:
√
W t ei − 1/N 1 ≤ N W t ei − 1/N 2
√ √
= N W t (ei − 1/N ) 2 ≤ N [σ2 (W )]t ei − 1/N 2
√
≤ N [σ2 (W )]t , (1.8)

where ei denotes the ith standard basis vector, and σ2 (W ) is the second largest
singular value of W .
6 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

1.4 Main Results

section, we will state the main results of our work. By denoting 

In this  xi (T ) 
T
(1/T ) t=1 xi (t), we have the following theorems.

Theorem 1.4 Suppose that both Assumptions 1.1 and 1.2 hold. Let the sequence
{xi (t)}∞
t=0 be generated by the updates (1.6) and (1.7). Then

xi (T )) − f (x ∗ ) ≤ OT + NT,
f (

where

2(τ + 1)2 G R 2 
T
2(τ + 1)L R R2
OT = + + [α(t)]2 , and
T T α(T ) T t=1

√
6 L2  38(τ + 1)2 N G L 2 
T T
N
NT = α(t) + [α(t)]2 .
1 − σ2 (W ) T t=1 [1 − σ2 (W )]2 T t=1

It should be noticed that Theorem 1.4 gives a common result accounting for
the effects of the network communication cost (NT) and the optimization term
(OT)on the convergence rate. By choosing a proper step size sequence satisfy-
ing ∞ t=0 α(t) = ∞ and lim t→∞ α(t) = 0, it is shown that {x i (t)} asymptotically
approaches
∞ a common value for any i, which no longer needs to satisfy the condition
t=0 [α(t)] 2
< ∞ stated in [7]. Based on this fact and Theorem 1.4, we obtain the
following result for demonstrating that the convergence rate is closely related to the
spectral gap 1 − σ2 (W ) of the iteration matrix W . It should be pointed out that by
constructing a proper doubly stochastic matrix W , the spectral gap is equivalent to the
algebraic connectivity of graph G, say, the second smallest eigenvalue of Laplacian
matrix.

Theorem 1.5 Suppose that Assumptions 1.1 and 1.2 are valid. With step size choice
√ 2 (W
α(t) = R[1−σ √ ,
)]
9 NL t

√
∗ N (τ + 1)2
xi (T )) − f (x ) ≤ Ξ
f ( √ ,
1 − σ2 (W ) T

where Ξ = 13R L + G R 2 + (G R 4 /L 2 ).

This theorem establishes a tight connection between the convergence rate of the
DDA algorithm and the spectral properties of the underlying networks. Based on the
above analysis, it is easy to obtain the convergence rates for some special network
type as shown in Table 1.2. The detailed analysis can be found in [12]. In order to
understand the relationship among the convergence rate, network size and topology,
we replace the left-hand side of the inequality in Theorem 1.5 with a desired error
1.4 Main Results 7

Table 1.2 Convergence rate for special network topologies

Network type xi (T )) − f (x ∗ )
f (
 √ 
O Ξ (τ√+1) N k 2 N
2 2
k-connected paths and cycles
T
√ √  √ 
O Ξ (τ√+1) N k 2 N
2
k-connected N × N grids
T
 √ 
O Ξ (τ√+1) Nlog NN
2
Random geometric graphsa
T
 2√

Expandersb O Ξ (τ√+1) N
T
√
a Random geometric graphs are assumed with connectivity radius r =Ω log1+ N / N for any
>0
b Expanders are assumed with bounded ratio of minimum to maximum node degree

accuracy , and then obtain the upper bound of the number of iterations TG (; N ) as
follows:
(a) For the single cycle graph, Tcy (; N ) = O(N 5 /2 );
(b) For the two-dimensional grid, Tgr (; N ) = O(N 3 /2 );
(c) For a bounded expander, Tex (; N ) = O(N /2 ).
In general, Theorem 1.5 implies that at most

N 1
TG (; N ) = O (1.9)
2 [1 − σ2 (W[G] )]2

iterations are required to achieve an -accurate solution. It is generally known that any
subgradient methods (even for centralized optimization algorithms) require at least
Ω(1/2 ) iterations to achieve -accuracy [23], so that the 1/2 term is unavoidable.
Meanwhile, the square of the inverse spectral gap term is also difficult to improve
due to bounding the interval of the decision variable.

1.5 Convergence Analysis

In this section, we investigate the convergence properties of the

distributed algorithm.
N
We start by defining two auxiliary sequences z̄(t)  (1/N ) i=1 z i (t) and y(t) 
ψ
ΠX (−z̄(t), α(t)). Since W is doubly stochastic, we have

1  1 
N N N
z̄(t + 1) = Wi j (t)z j (t) + gi (t − τ )
N i=1 j=1 N i=1

1 
N
=z̄(t) + gi (t − τ ). (1.10)
N i=1
8 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

We introduce a subgradient error vector ei (t) = gi (t) − gi (t − τ − 1) and the transi-

tion matrix (t, s) = W t−s+1 . These notations and the following lemmas are useful
for the proof of Theorem 1.4.
Lemma 1.6 ([12]) For an arbitrary pair u, v ∈ Rd , we have
ψ ψ
ΠX (u, α) − ΠX (v, α) ≤ α u − v ∗ . (1.11)

Lemma 1.7 ([24]) Let x + minimize z, x + Aψ(x) for all x ∈ X . Then for any
x ∈ X,
z, x + Aψ(x) ≥ z, x + + Aψ(x + ) + ADψ (x, x + ).

Now we state a general property of the DDA algorithm, which allows us to bound
the difference between xi (t) and xi (t + 1).
Lemma 1.8 For any i ∈ V and any x ∗ ∈ X ,

xi (t) − xi (t + 1) ≤ α(t) z i (t) − z̄(t) ∗

+ α(t + 1) z i (t + 1) − z̄(t + 1) ∗ + α(t)[L + ψ(x ∗ )].

Proof For any x ∗ ∈ X , applying Lemma 1.6 yields

xi (t) − xi (t + 1)
≤ xi (t) − y(t) + y(t + 1) − xi (t + 1)
+ y(t) − y(t + 1)
≤ xi (t) − y(t) + y(t + 1) − xi (t + 1)
ψ
+ y(t) − ΠX (−z̄(t + 1), α(t)) + [α(t) − α(t + 1)]ψ(x ∗ )
≤α(t) z i (t) − z̄(t) ∗ + α(t + 1) z i (t + 1) − z̄(t + 1) ∗
+ α(t) z̄(t) − z̄(t + 1) ∗ + [α(t) − α(t + 1)]ψ(x ∗ )
<α(t) z i (t) − z̄(t) ∗ + α(t + 1) z i (t + 1) − z̄(t + 1) ∗
∗
+ α(t)L + α(t)ψ(x ).

The lastinequality is derived from the relationships that z̄(t + 1) − z̄(t) ∗ =

N
(1/N ) i=1 gi (t − τ ) ∗ ≤ L and ψ(·) ≥ 0. Hence, we complete the proof.

From Lemma 1.8, it is easy to observe that the decision variable sequence {xi (t)}
is convergent if limt→∞ α(t) = 0 and the sequence { z i (t) − z̄(t) ∗ } is bounded for
any time step t. The next lemma is useful in deriving the bound of the sequence
{ z i (t) − z̄(t) ∗ } for all i. A similar version of this lemma can be found in [12]. The
only difference is that we develop a constant bound independent of T , which is more
convenient in analyzing the complexity of the algorithm.

Lemma 1.9 Suppose that Assumptions 1.1 and 1.2 hold. Then, for the iterations of
the algorithms (1.6) and (1.7) we have
1.5 Convergence Analysis 9
√
3 NL
z i (t) − z̄(t) ∗ < .
1 − σ2 (W )

Proof From the proof of Theorem 2 [12], we can obtain

t−1 
 
 
z i (t) − z̄(t) ≤L [(t − 1, )]i − 1  + 2 L .
∗  N 1
=1

√
From (1.8), [(t − 1, )]i − 1/N 1 ≤ N [σ2 (W )]t− and noting that 0 < σ2 (W )
< 1, a simple computation yields

√ σ2 (W ) − [σ2 (W )]t
z i (t) − z̄(t) ∗ ≤ NL +2L
1 − σ2 (W )
√ √
( N + 2)L 3 NL
< ≤ .
1 − σ2 (W ) 1 − σ2 (W )

We then obtain the desired result and complete the proof.

In order to simplify the proof, the following two lemmas are separated from the
proof of main results as two independent properties of the distributed delayed dual
averaging algorithm.
Lemma 1.10 For any x ∗ ∈ X ,

1 
T N
1
g j (t − τ − 1), y(t) − x ∗ ≤ ψ(x ∗ ).
N t=1 j=1 α(T )

Proof From Lemma 1.7 and the expression (1.7), we have

−z̄(t − 1), y(t) ≤ − z̄(t − 1), y(t − 1)

1
+ [ψ(y(t)) − ψ(y(t − 1))]
α(t − 1)
1
− Dψ (y(t), y(t − 1)). (1.12)
α(t − 1)
N
By utilizing the iteration z̄(t) = z̄(t − 1) + N1 i=1 gi (t − τ − 1) from (1.10) and
the non-negativity of Bregman divergence, i.e., Dψ (y(t), y(t − 1)) ≥ (1/2) y(t −
1) − y(t) 2 ≥ 0, it follows from (1.12) that
10 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

1 
N
g j (t − τ − 1), y(t) − x ∗
N j=1
=z̄(t), y(t) − x ∗ − z̄(t − 1), y(t) − x ∗
≤z̄(t), y(t) − x ∗ − z̄(t − 1), y(t − 1) − x ∗
1
+ [ψ(y(t)) − ψ(y(t − 1))]. (1.13)
α(t − 1)

Summing over the expression (1.13), we have

1 
T N
g j (t − τ − 1), y(t) − x ∗
N t=1 j=1
1
≤z̄(T ), y(T ) − x ∗ + ψ(y(T ))
α(T − 1)
T  
1 1
+ − ψ(y(t − 1))
t=2
α(t − 2) α(t − 1)
1
=z̄(T ), y(T ) + ψ(y(T ))
α(T )
1
− z̄(T ), x ∗ − ψ(x ∗ )
α(T )
 
1 1 1
+ ψ(x ∗ ) + − ψ(y(T ))
α(T ) α(T − 1) α(T )
T  
1 1
+ − ψ(y(t − 1)).
t=2
α(t − 2) α(t − 1)

Since y(T ) minimizes z̄(T ), y + 1

α(T )
ψ(y) for any y ∈ X , it can be derived that

1 
T N
g j (t − τ − 1), y(t) − x ∗
N t=1 j=1
1
≤y(T ) − y(T ) + ψ(x ∗ )
α(T )
 
1 1
+ − ψ(y(T ))
α(T − 1) α(T )
 T  
1 1
+ − ψ(y(t − 1))
t=2
α(t − 2) α(t − 1)
1
≤ ψ(x ∗ ).
α(T )
1.5 Convergence Analysis 11

The last inequality follows by the fact that {α(t)}∞

t=1 is a non-increasing sequence
and ψ(·) ≥ 0. The proof is completed.

Lemma 1.11 For x ∗ ∈ arg min x∈X f j (x),

1  
T N T
e j (t), x j (t) − x ∗ ≤ 2(τ + 1)L R + Θ [α(t)]2
N t=1 j=1 t=1

where
N G L2
Θ = 38(τ + 1)2 + 2(τ + 1)2 G[ψ(x ∗ )]2 .
[1 − σ2 (W )]2

Proof Recalling the subgradient error vector and the four term equality of a Bregman
divergence, we have

e j (t), x j (t) − x ∗
= g j (t) − g j (t − τ − 1), x j (t) − x ∗
= D f j (x ∗ , x j (t)) − D f j (x ∗ , x j (t − τ − 1))
− D f j (x j (t), x j (t)) + D f j (x j (t), xi (t − τ − 1)). (1.14)

By the G-Lipschitz continuity of ∇ f j , one obtains

f j (x j (t)) ≤ f j (x j (t − τ − 1))
+ g j (t − τ − 1), x j (t) − x j (t − τ − 1)
G
+ x j (t − τ − 1) − x j (t) 2 . (1.15)
2

which implies that D f j (x j (t), x j (t − τ − 1)) ≤ (G/2) x j (t − τ − 1) − x j (t) 2 .

Summing over the expression (1.14) yields that

1 
T N
e j (t), x j (t) − x ∗
N t=1 j=1

1  
T N
≤ D f (x ∗ , x j (t))
N t=T −τ j=1 j

1  G
T N
+ x j (t − τ − 1) − x j (t) 2 . (1.16)
N t=1 j=1 2

It follows from Assumption 1.2 that for x ∗ ∈ arg min x∈X f j (x), x ∗ − x j (t) 2 ≤
2Dψ (x ∗ , x j (t)) ≤ 2R 2 . Combining this relationship with the optimality of x ∗ implies
that
12 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

D f j (x ∗ , x j (t)) = f j (x ∗ ) − f j (x j (t)) − g j (t), x ∗ − x j (t)

≤ g j (t) ∗ x ∗ − x j (t) ≤ 2 L R. (1.17)

Moreover, the convexity leads to

x j (t − τ − 1) − x j (t) 2
 τ 2
  
=  [x j (t − s − 1) − x j (t − s)]
s=0
τ
 1
≤(τ + 1) 2
x j (t − s − 1) − x j (t − s) 2 . (1.18)
s=0
τ +1

Substituting inequalities (1.17) and (1.18) into (1.16), and making use of Lemma 1.8
and the relation (A + B + C + D)2 ≤ 4[A2 + B 2 + C 2 + D 2 ], we can get

1 
T N
e j (t), x j (t) − x ∗ ≤ 2(τ + 1)L R
N t=1 j=1
 √ 2 T τ
3 NL 
+2(τ + 1)G [α(t − s − 1)]2
1 − σ2 (W ) t=1 s=0
 √ 2 T τ
3 NL 
+2(τ + 1)G [α(t − s)]2
1 − σ2 (W ) t=1 s=0
  τ
T 
+2(τ + 1)G L 2 + [ψ(x ∗ )]2 [α(t − s − 1)]2 .
t=1 s=0

Since {α(t)}∞t=1 is a non-increasing sequence, after some simple computations, we

obtain the desired result and complete the proof.

We are now ready to prove our basic convergence result, which essentially relates
xi (T )) − f (x ∗ ) and the disagreement z i (t) − z̄(t) ∗ , and further
the error bound f (
demonstrates the effects of network topology (say the spectral gap of the weight
matrix W ) on the convergence rate.

T (Proof of Theorem∗ 1.4) By using the convexity of f , i.e., f (

Proof xi (T )) ≤ (1/T )
t=1 f (x i (t)), for any x ∈ X , we have

1  
T
∗
f (
xi (T )) − f (x ) ≤ f (y(t)) − f (x ∗ )
T t=1

1  
T
+ f (xi (t)) − f (y(t)) . (1.19)
T t=1
1.5 Convergence Analysis 13

Since f i is L-Lipschitz for any i ∈ V, combining this with Lemma 1.6 yields that

f i (xi (t)) − f i (y(t)) ≤L xi (t) − y(t)

≤Lα(t) z i (t) − z̄(t) ∗ . (1.20)

It should be noticed that condition (1.20) is also satisfied for f . Since g j (t) ∈
∂ f j (x j (t)) is a subgradient of f j at x j (t), using the convexity of f j and (1.20),
we can derive that

1 
N
f (y(t)) − f (x ∗ ) = f j (x j (t)) − f (x ∗ )
N j=1

1  
N
+ f j (y(t)) − f j (x j (t))
N j=1

1 
N
≤ g j (t), x j (t) − x ∗
N j=1

L 
N
+ α(t) z j (t) − z̄(t) ∗ . (1.21)
N j=1

Recalling the subgradient error vector, we have

g j (t), x j (t) − x ∗
= g j (t − τ − 1), x j (t) − x ∗ + e j (t), x j (t) − x ∗
= g j (t − τ − 1), y(t) − x ∗ + g j (t − τ − 1), x j (t) − y(t)
+ e j (t), x j (t) − x ∗ . (1.22)

By Hölder’s inequality, after using Lemma 1.6 and Assumption 1.2, we can derive
that

g j (t − τ − 1), x j (t) − y(t) ≤ g j (t − τ − 1) ∗ x j (t) − y(t)

≤Lα(t) z j (t) − z̄(t) ∗ . (1.23)

Substituting (1.20)–(1.23) into (1.19), we can get

14 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

1 
T N
xi (T )) − f (x ∗ ) ≤
f ( g j (t − τ − 1), y(t) − x ∗
N T t=1 j=1

1 
T N
+ e j (t), x j (t) − x ∗
N T t=1 j=1

L 
T N
+ α(t) z j (t) − z̄(t) ∗
N T t=1 j=1

L
T
+ α(t) z i (t) − z̄(t) ∗ . (1.24)
T t=1

By Lemmas 1.8–1.11, we obtain the result of Theorem 1.4 and complete the proof.

By choosing a proper step size sequence, we now turn to the proof of Theorem 1.5.

Proof (Proof of Theorem 1.5) Choosing a proper step size as given by Theorem 1.5,
it is easy to see that

T  T √
1 1
√ ≤1+ √ dt ≤ 2 T ,
t=1
t 1 t
T  T
1 1
≤1+ dt = log T + 1.
t=1
t 1 t

√
By some
T simple calculations,
√ it can be derived that log T + 1 ≤ 2 T , which means
that t=1 [α(t)]2 ≤ 2 T . By Theorem 1.4, we have
√
∗ LR 9 N LR
f (
xi (T )) − f (x ) ≤2(τ + 1) + √
T 1 − σ2 (W ) T
LR (τ + 1)2 G R 2
+ 2[1 − σ2 (W )] √ + √
T 1 − σ2 (W ) T
(τ + 1)2 R 2 [1 − σ2 (W )] G R 2
+ √
N L2 T
  √
G R4 N (τ + 1)2
< 13R L + G R 2 + 2 √ .
L 1 − σ2 (W ) T

This completes the proof.

1.6 Numerical Examples 15

0
10
DDA
DDA−DSG
DDA−DDV
max |fi (xi (t)) − f ∗ |

−1
10

−2
10

0 200 400 600 800 1000 1200 1400

Iteration t

Fig. 1.1 Plots of function error versus number of iterations for a random geometric graph with
N = 5 nodes

1.6 Numerical Examples

In this section, we report the experimental results on the cooperative distributed

minimization of a sum of the regularization items and hinge loss functions as well
as making comparison with the methods in [12–14]. This optimization problem is
widely employed in the support vector machine (SVM) method for classification
and regression analysis. For a given graph size N , a random instance of a SVM
classification problem is formulated as follows. We first generate N pairs of the
form (bi , yi ) on a unit hypersphere, where bi ∈ Rd corresponds to a feature vector,
yi ∈ {−1, +1} is the associated label indicating the class to which the point bi belongs
and is separated into two categories by adding 5% of noise. The goal is to use these
samples to estimate a linear classifier based on some weight vectors x ∈ Rd that are
chosen by minimizing the global objective given by

N  
1  λi
f (x) = x 2
2 + max{0, 1 − yi (bi , x + c)} ,
N i=1 2

where λi ∈ R+ corresponds to the regularization parameter and c ∈ R is the bias term

which plays a significant role in solving the classification problem on imbalanced
data sets. It should be noticed that f is L-Lipschitz and non-smooth at any point
with bi , x + c = yi . In this case, we set L = maxi {λi + bi 2 } and impose the
box constraint on the minimization problem, i.e., X = {x ∈ Rd | x 2 ≤ 5}.
16 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

In order to study the effect of communication delays, we perform simulations

with comparison to the DDA algorithm in [12], the DDA algorithm with delayed
dual variable (DDA-DDV) studied in [13, 14] and the DDA algorithm with delayed
subgradient (DDA-DSG) proposed in this chapter. For a given random geometric
graph, we run the optimization algorithm using the optimal setting of the step size α.
Then, the plots of the function error maxi [ f (xi (T ) − f (x ∗ )] versus the number of
iterations are drawn in Fig. 1.1. It is observed that the function error of our method
converges rapidly to the given accuracy  compared to [12–14] due to the preserved
performance gain.
These results illustrate the excellent agreement of the empirical behavior with our
theoretical predictions and improved algorithmic performance.

1.7 Conclusions

In this chapter, we investigated the DDA algorithm for solving cooperative opti-
mization problems in delayed multi-agent networks. We provided a sharp bound on
the convergence rate. It is shown that the effectiveness of the algorithm is highly
dependent on the properties of the optimization algorithm itself and the underlying
connectivity structure of the network. Moreover, we would like to point out that our
result can be easily extended to the case of stochastic communication protocol and
stochastic optimization with noisy gradients. For future work, we will investigate
the mirror descent algorithms for distributed cooperative optimization in delayed
settings.

References

1. Tsitsiklis, J.N.: Problems in decentralized decision making and computation. Ph.D. thesis,
M.I.T. (1984)
2. Olfati-Saber, R., Murray, R.: Consensus problems in networks of agents with switching topol-
ogy and time-delays. IEEE Trans. Autom. Control 49(9), 1520–1533 (2004)
3. Liu, B., Chen, T.: Consensus in networks of multiagents with cooperation and competition via
stochastically switching topologies. IEEE Trans. Neural Netw. Learn. Syst. 19(11), 1967–1973
(2008)
4. Yu, W., Chen, G., Cao, M.: Some necessary and sufficient conditions for second-order consensus
in multi-agent dynamical systems. Automatica 46(6), 1089–1095 (2010)
5. Li, Z., Duan, Z., Chen, G., Huang, L.: Consensus of multiagent systems and synchronization
of complex networks: a unified viewpoint. IEEE Trans. Circuits Syst. I: Regul. Pap. 57(1),
213–224 (2010)
6. Xiong, W., Ho, D.W.C., Wang, Z.: Consensus analysis of multiagent networks via aggregated
and pinning approaches. IEEE Trans. Neural Netw. Learn. Syst. 22(8), 1231–1240 (2011)
7. Nedi, A., Ozdaglar, A., Parrilo, P.A.: Constraint consensus and optimization in multi-agent
networks. IEEE Trans. Autom. Control 55(4), 922–938 (2010)
8. Ram, S.S., Nedi, A., Veeravalli, V.V.: Distributed stochastic subgradient projection algorithms
for convex optimization. J. Optim. Theory Appl. 147(3), 516–545 (2010)
References 17

9. Johansson, B., Rabi, M., Johansson, M.: A randomized incremental subgradient method for
distributed optimization in networked systems. SIAM J. Optim. 20(3), 1157–1170 (2009)
10. Zhu, M., Martnez, S.: On distributed convex optimization under inequality and equality con-
straints. IEEE Trans. Autom. Control 57(1), 151–164 (2012)
11. Yuan, D., Xu, S., Zhao, H.: Distributed primal-dual subgradient method for multiagent opti-
mization via consensus algorithms. IEEE Trans. Syst. Man Cybern.: Syst. 41(6), 1715–1724
(2011)
12. Duchi, J.C., Agarwal, A., Wainwright, M.J.: Dual averaging for distributed optimization: con-
vergence analysis and network scaling. IEEE Trans. Autom. Control 57(3), 592–606 (2012)
13. Tsianos, K.I., Rabbat, M.G.: Distributed consensus and optimization under communication
delays. In: 49th Annual Allerton Conference, Illinois, USA, pp. 974–982 (2011)
14. Tsianos, K.I., Rabbat, M.G.: Distributed dual averaging for convex optimization under commu-
nication delays. In: American Control Conference, Montreal, Canada, pp. 1067–1072 (2012)
15. Nedi, A., Ozdaglar, A.: Cooperative distributed multi-agent optimization. In: Eldar, Y., Palo-
mar, D. (eds.) Convex Optimization in Signal Processing and Communications, pp. 340–386.
Cambridge University Press, Cambridge (2010)
16. Wang, J.: Primal and dual assignment networks. IEEE Trans. Neural Netw. Learn. Syst. 8(3),
784–790 (1997)
17. Hu, X., Wang, J.: Solving the assignment problem using continuous-time and discrete-time
improved dual networks. IEEE Trans. Neural Netw. Learn. Syst. 23(5), 821–827 (2012)
18. Nesterov, Y.: Primal-dual subgradient methods for convex problems. Math. Program. 120(1),
261–283 (2009)
19. Yuan, D., Xu, S., Zhao, H., Rong, L.: Distributed dual averaging method for multiagent opti-
mization with quantized communication. Syst. Control Lett. 61(11), 1053–1061 (2012)
20. Agarwal, A., Duchi, J.: Distributed delayed stochastic optimization (2011).
arXiv:abs/1104.5525
21. Cheng, L., Hou, Z.G., Tan, M.: A delayed projection neural network for solving linear varia-
tional inequalities. IEEE Trans. Neural Netw. Learn. Syst. 20(6), 915–925 (2009)
22. Bregman, L.: The relaxation method of finding the common point of sets and its application
to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 7(3),
200–217 (1967)
23. Nemirovski, A., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. Wiley,
New York (1983)
24. Tseng, P.: On accelerated proximal gradient methods for convex-concave optimization. Tech-
nical Reports, Department of Mathematics, University of Washington (2008)
Chapter 2
Constrained Consensus of Multi-agent
Systems with Time-Varying Topology

Abstract In this chapter, the constrained consensus problem is studied for the asyn-
chronous discrete-time multi-agent system, where each agent needs to lie in a closed
convex constraint set. The communication graphs of this asynchronous system are
assumed to be directed, unbalanced, dynamically changing. In addition, their union
graph is assumed to be strongly connected within a certain interval of finite length. To
deal with the asynchronous communication issue among agents, the original asyn-
chronous system is equivalently transformed to a synchronous one by adding some
new agents. By employing the properties of the projection on the convex sets, the
distance between the states of the agent in the newly constructed system and the
intersection set of all constraint sets for agents is estimated. Based on this estima-
tion, the original system is proven to reach consensus by showing that the linear
parts of the newly constructed system converge and the nonlinear parts vanish over
time. Finally, two numerical examples are provided to show the effectiveness of the
method and the correctness of theoretical analysis.

Keywords Constrained consensus · Multi-agent systems · Asynchronous

communication · Distributed control

2.1 Introduction

In recent years, the collective behaviors in the network of autonomous agents have
received an increasing interest due to their promising potential for widely application
areas, such as formation control [1], flocking control [2], filtering in sensor networks
[3], swarming [4], unmanned autonomous vehicles (UAV) formations [5, 6], etc.
Most of the existing works are concerned with continuous-time dynamics [7–
17]. In real world applications, the information transmission among agents may
not be continuous due to the unreliability of communications channels or the limited
sensing ability of agents. Thus, discrete-time multi-agent systems have found widely
applications and their dynamics have attracted a lot of research interests [18–27],
in which each agent synchronously receives its neighbors’ information at discrete
time instants, where the synchrony means that all agents update their states using
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 19
H. Wang et al., Distributed Optimization, Game and Learning Algorithms,
https://doi.org/10.1007/978-981-33-4528-7_2
20 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

the information of its neighboring agents at the same time. However, considering a
central synchronizing clock may not be available and the communication topology
is dynamically changing [28]. In [28], the authors studied the stationary consensus
of the asynchronous second-order multi-agent system under switching topologies,
where the asynchrony means that each agent’s update action is independent of the
others’.
In some practical applications, the states of agents may need to reach a consensus
but the state must lie in certain constraint sets. A related application of the constrained
consensus is the dynamic coalitional TU games [29]. The distributed bargaining
protocol of the coalitional TU games can be formulated as follows:
⎡ ⎤

N
xi (t + 1) = PXi (t) ⎣ ai j (t)x j (t)⎦ ,
j=1

where {1, 2, . . . , N } is the set of the players. PXi [·] is the projection operator onto
the local constraint set Xi . Xi (t) is the bounding set of player i at time t. The authors
in [29] proved the convergence of the bargaining process to a random allocation that
lines in the core of the dynamical TU game, which means that the players finally
reach an agreement via the distributed bargaining protocol with probability 1. To
the best of our knowledge, few work has considered the constrained consensus of
discrete-time multi-agent systems except in [30], where the communication delays
was considered. It should be mentioned that the communication topology consid-
ered in [30] is unbalanced and jointly connected which is less conservative than the
communication topology considered in [29], where the communication topologies
are assumed to be balanced.
Motivated by the above discussions, we study the constrained consensus of the
asynchronous multi-agent systems, where each agent is required to lie in a closed con-
vex constraint set while reaching consensus. The communication graphs are assumed
to be directed, dynamically changing, and not necessarily balanced. In addition, their
union graph is assumed to be strongly connected. Symmetry would be lost in the
presence of unbalanced graphs and asynchronous communications, rendering the
approaches in [29] are not yet applicable. The existence of the constraint sets would
introduce nonlinearity, that is, the projection error, rendering the approaches in [28]
not applicable either. To deal with the projection errors under the unbalanced com-
munication topology, we divide the proof into the following two procedures. First,
we present a transformation method, built on the seminal work [31], which equiv-
alently transforms the original asynchronous network to the synchronous one by
adding new agents. Few existing work has presented such an idea. The analysis is
then performed for the newly constructed synchronous system. Second, we employ
the properties of the projection on the convex sets, the distance from the states of the
agents to the intersection set of all agents’ constraint sets is estimated. Based on the
estimation, the original system is then proven to reach consensus by showing that the
linear parts of the newly constructed systems converge and the nonlinear parts vanish
2.1 Introduction 21

over time. Finally, two numerical examples are provided to show the effectiveness
of the theoretical results.
Notations: A vector is viewed as a column. For a vector x, x denotes the
Euclidean norm. For a vector x and a closed convex set X , P X [x] denotes projection
of x onto X , i.e., P X [x] = argmin y∈X x − y. R denotes the real number set, N
denotes the nature number set, and Rn = R × ·· · × R. For matrix A, [A]i j denotes
n
the (i, j)th component of matrix A. The superscript “T” denotes the transpose of the
vector and a matrix.

2.2 Preliminaries

The communication topology of a network of N agents is represented using a directed

graph G = (V, E, A) with the vertex set V = {1, 2, . . . , N }, the edge set E ⊂ V × V,
and the weighted adjacency matrix A. If the edge set E and the weighted adjacency
matrix A change over time, then such a time-varying graph is called as the time-
varying communication topology for networks. Consider a discrete-time multi-agent
system consisting of N agents. Each agent is regarded as a vertex in a time-varying
directed graph G(k), whose edge set is E(k) and adjacency matrix is A(k) with
respect to k. ai j (k) is the (i, j)th entry of the adjacency matrix A(k). Each edge
( j, i) ∈ E(k) is an available communication channel, through which, each agent can
receive information from its neighbors. The neighboring set of agent i at time k is
denoted by Ni (k).
The state of each agent, denoted by xi (k) ∈ Rn , is constrained to lie in a nonempty
N
closed convex set Xi ⊆ Rn , which is private for agent i. Define X = i=1 Xi , which
is unknown for all agents. Due to the privacy of the constraint sets and the asyn-
chronous clocks of each agent, we adopt the distributed asynchronous projection
algorithm, in which, at the updating time, each agent first computes the weighted
average among his own state and its neighbors’ states, then projects the averaged
value to its constraint set Xi . For all i = 1, 2, . . . , N , the algorithm is stated as
follows:
⎡ ⎤

xi (tk+1
i
) = PXi ⎣ ai j (tki )x j (tki )⎦ , (2.1)
j∈Ni (tki )

with initial value xi (0) ∈ Xi , where PXi [·] is the projection operator onto Xi . Time
t0i , t1i , . . . , tki , . . . with t0i = 0 for all i = 1, 2, . . . , N , are positive integers, at which,
agent i receives its neighbors’ states. For simplicity, we denote the updating time
t0i , t1i , . . . , tki , . . . for agent i by {tki }. We further assume that for all i = 1, 2, . . . , N
and k ∈ N, {tki } satisfies the following condition:

T ≤ tk+1
i
− tki ≤ T , (2.2)
22 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

where T and T are positive numbers that satisfy T ≤ T . Hereafter, we denote τki =
i
tk+1 − tki > 0.
Definition 2.1 ([28]) Discrete-time multi-agent system is said to be asynchronous
if for all i, j = 1, 2, . . . , N , the time series {tki } are mutually independent, i.e., {tki }
j
is independent of {tk }, i = j.
We set ai j (tki ) = 0 for all j ∈
/ Ni (tki ). Then, the asynchronous discrete-time multi-
agent system (2.1) can be rewritten as follows
⎡ ⎤

N
xi (tk+1
i
) = PXi ⎣ ai j (tki )x j (tki )⎦ . (2.3)
j=1

Proposition 2.2 ([29]) For the projection operator P on a closed convex set X ⊆
Rn , for any ω ∈ Rn and any x ∈ X ,

PX [ω] − x2 ≤ ω − x2 − PX [ω] − ω2

Proposition 2.2 establishes a relation between the projection error vector and the
feasible directions of the convex set X at the projection vector.
Assumption 2.1 (Weight Rule) There exists a constant 0 < η < 1, such that for all
i = 1, 2, . . . , N
(a) aii (tki ) > η for all k ≥ 0.
(b) ai j (tki ) > η for all k ≥ 0 and all agents j communicate directly with agent i at
time tki .
(c) ai j (tki ) > 0, j∈Ni (k) ai j (tki ) = 1 for all i, j = 1, 2, . . . , N and k ∈ N.
Assumption 2.1 says that every agent assigns a substantial weight to the infor-
mation received from its neighbors. This guarantees that the information from each
agent influences the information of each other agent persistently in time. In other
words, this assumption guarantees that the agent information is mixing at a nondi-
minishing rate in time. Without this assumption, information from some of agents
may become less influential in time, and in the limit, resulting in loss of information
from these agents.
Remark 2.3 In [30], the authors studied the constrained consensus in multi-agent
networks under the assumption that the positive edge weights are chosen from a finite
set (Assumption 2 in [30]). In this chapter, the weights can be chosen infinitely from
a bounded set, i.e., ai j (tki ) ∈ [0, 1] whenever j ∈ Ni (k) for all i = 1, 2, . . . , N and
k ∈ N.
Remark 2.4 In [29], the constrained consensus is studied under the assumption that
N
the adjacency matrix A(k) is doubly stochastic for all k ∈ N, i.e, i=1 ai j (k) = 1,
N
j=1 ai j (k) = 1 for all i, j = 1, 2, . . . , N . In this chapter, this constrained assump-
tion is removed.
2.2 Preliminaries 23

Definition 2.5 The constrained consensus problem of the multi-agent system (2.3)
is said to be reached if for some z̃ ∈ X and all i = 1, 2, . . . , N ,

lim xi (tki ) − z̃ = 0.

k→∞

Before presenting the main results, we need to give the following assumption on
the connectivity of the communication graph.
+∞

Assumption 2.2 There exists an infinite sequence of time indices ki |i=0 , where
k0 = 0,0 < km+1
 − km < M, m ∈ Z, M ∈ Z+ , such that the union of the directed
km+1 −1
graphs G(ti ) i=k m
is strongly connected.

Remark 2.6 The assumption on the strong connectivity of communication networks

is crucial for the constrained consensus of multi-agent systems [29]. It ensures that
the agents’ states are sufficiently fused over the communication network, and avoids
that certain agents remain stationary states in themselves constraint set, if so, the
network will not reach consensus due to the different local constraint sets. This
assumption is quite different from the unconstrained consensus, i.e., Xi = Rn for all
i = 1, 2, . . . , N , where the communication network only needs to contain a directed
spanning tree [22, 25, 28].

2.3 Main Results

In this section, we will investigate the constrained consensus problem of the multi-
agent system (2.3). The motivation is mainly from [28, 30]. We should first merge the
time sequence that each agent receives its neighbors’ states into a single ordered time
sequence T and then the original asynchronous discrete-time system is casted into
an equivalent augmented synchronous discrete-time multi-agent which evolves over
time sequence T . Then, we employ the mixed tools from graph theory, nonnegative
matrix theory and the infinite product of stochastic matrices to obtain the main results.

2.3.1 Model Transformation

Here, we reduce the original system with asynchronous communication to a new

system with synchronous communication under bounded communication interval.
We introduce an enlarged system that is obtained by adding new agents into the
original system in order to deal with asynchronous communication. It suffices to
add (m̂ − 1)N agents into the original systems,  where m̂ be the upper bound for the
number of elements in set t j : t j ∈ tki , tk+1
i
, j ∈ N for all i = 1, 2, . . . , N and
 
k ∈ N. By Lemma 1 in [28], we have m̂ = T /T + 1 (N − 1) + 1 where T and
T are defined in (2.2) and T /T is the largest integer no less than T /T .
24 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

We label the newly added agent corresponding to agent j by j + N , . . . , j + (m̂ −

1)N . For t with 1 ≤ t ≤ τki , agent j + (t − 1)N transmits its state to agent j + t N
and agent j + t N transmits the same state to agent j + (t + 1)N if t + 1 < τki or to
agent i with a product of weight ai j (tki ) if t + 1 = τki . Then, the weight ãi h (t) of the
enlarged system between agent i and agent h is defined as follows:
⎧
⎨ ai j (tki ), h = j + t N , t = τki ,
ãi h (t) = 1, h = j + t N , 1 ≤ t ≤ τki − 1, (2.4)
⎩
0, otherwise,
 
for k ≥ 0. Hereafter, we denote for all t ≥ 0, Ã(t) = ãi j (t) m̂ N ×m̂ N be the adja-
cency matrix and G̃(t) for all t ≥ 0 be the underlying graph of the enlarged system,
respectively. By (2.4), it is easy to see that Ã(t) is a row-stochastic matrix, but not a
column-stochastic one. On the other hand, we let R n be the constrained set of agent
i = N + 1, N + 2, . . . , m̂ N .
Using the weights rule (2.4), the discrete-time asynchronous multi-agent system
(2.3) can be written as follows:
⎡ ⎤

m̂ N
x̃i (t + 1) = PXi ⎣ ãi j (t)x̃ j (t)⎦ , (2.5)
j=1

for all i = 1, 2, . . . , m̂ N , whose initial values are given by

x̃i (0) = xi (0) for all i = 1, 2, . . . , N ,

x̃i (0) = 0 for all i = N + 1, N + 2, . . . , m̂ N .

In the following, we provide a useful lemma without proof, please refer to Lemma
3 in [28] for a similar proof.
+∞

Lemma 2.7 Let ki |i=0 be an infinite sequence of time indices, where k0 =
0, 0 < km+1 − km < M, m ∈ Z, M ∈ Z+ , such that the union of the directed
km+1 −1
graphs {G(ti )}i=k is strongly connected. Then, the union of enlarged graphs
  m
km+1 −1
G̃(ti ) i=km is also strongly connected.

Lemma 2.7 says that if the union of the original graphs is strongly connected in
certain finite time interval, then the union of the enlarged graphs is also strongly con-
nected in the same time interval. This means that the model transformation method
does not change the connectivity of the original graphs, which guarantees that infor-
mation is mixed sufficiently in the enlarged multi-agent system. In the following, we
separate the linear and nonlinear parts of the constrained multi-agent system (2.4) to
reveal the effect of projection errors for constrained consensus.
2.3 Main Results 25

x̃i (t + 1) = ωi (t) + ei (t), for all i = 1, 2, . . . , m̂ N , (2.6)

m̂ N
where ωi (t) = j=1 ãi j (t)x̃ j (t), ei (t) = PXi [ωi (t)] − ωi (t).

2.3.2 State Transition Matrix

Define the state transition matrix of (2.6) be

Φ(t, s) = Ã(t)Ã(t − 1) · · · Ã(s). (2.7)

By simple calculation, we have


m̂ N
x̃i (t + 1) = [Φ(t, s)]i j x̃ j (s)
j=1
⎛ ⎞

t m̂ N
+ ⎝ [Φ(t, r )]i j e j (r − 1)⎠ + ei (k). (2.8)
r =s+1 j=1

Clearly, the state transition matrix plays a key role in the evolution of the agents’
states. In the following, we give some properties of the state transition matrix, which
include the row-stochasticity of Φ(t, s) and its limiting matrix, the convex properties
of each column of Φ(t, s), and the estimate of the convergence rate of the transition
matrix based on the norm properties. Their proofs are omitted, because the proofs
can be found in Lemma 3 in [22] and Lemma 5 in [30].
Lemma 2.8 Under Assumptions 2.1 and 2.2.
(a) Φ(t, s) is a row-stochastic matrix for all t ≥ s ≥ 0.
(b) The limit Φ(s) = limt→∞ Φ(t, s) exists for all s ≥ 0.
(c) Φ(s) has identical rows and the rows are stochastic, i.e.,

Φ(s) = 1φ T (s),

where φ(s) ∈ Rm̂ N is a stochastic vector for each s and 1 is a vector whose
elements are 1.
(d) For all s ≥ 0, i, j = 1, 2, . . . , N ,
 
Φ(s + (m̂ N − 1)M − 1, s) i j ≥ η(m̂ N −1)M ,

where M is defined in Assumption 2.2.

(e) For all j = 1, 2, . . . , m̂ N , the entries [Φ(t, s)]i j , i = 1, 2, . . . , m̂ N , converge
to the same limit φ j (s) as t → ∞ with a geometric rate, i.e., for all j =
1, 2, . . . , m̂ N and s ≥ 0,
26 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

  −B   t−s
[Φ(t, s)]i j − φ j (s) ≤ 2 1 + η 1 − η B B
1 − ηB

for all t ≥ s and i = 1, 2, . . . , N , where B = (N − 1)M + m̂ N .

2.3.3 Constrained Consensus

In the following, we derive the main result in this chapter. First, we prove that
under Assumptions 2.1 and 2.2, the projection errors ei (t) → 0 as t → ∞ for all
i = 1, 2, . . . , m̂ N . Second, we build an auxiliary variable y(t), then prove that x̃i (t)
and ωi (t) converge to y(t) as t → ∞ for all i = 1, 2, . . . , N . Finally, we prove the
constrained consensus of the multi-agent system (2.3) can be reached.
We start by proving that the projection errors of agents vanish over time under
Assumptions 2.1 and 2.2.
Proposition 2.9 Under Assumptions 2.1 and 2.2, limt→∞ ei (t) = 0 for all i =
1, 2, . . . , m̂ N .

Proof Let z̃ be any element in X . By the strict non-expansion property of the

Euclidean projection on a closed convex set Xi , we have for all i = 1, 2, . . . , N
and t ≥ 0,

x̃i (t + 1) − z̃2 ≤ ωi (t) − z̃2 − ei (t)2 . (2.9)

Actually, by (2.6), one has x̃i (t + 1) = PXi [ωi (t)]. Thus, it follows by Proposi-
tion 2.2 that
 2
x̃i (t + 1) − z̃2 = PXi [ωi (t)] − z̃ 
 2
≤ ωi (t) − z̃2 − PXi [ωi (t)] − ωi (t)
≤ ωi (t) − z̃2 − ei (t)2 .

By (2.9), we have

x̃i (t + 1) − z̃2 ≤ ωi (t) − z̃2 . (2.10)

Thus,

max {x̃i (t + 1) − z̃} ≤ max {ωi (t) − z̃} (2.11)

i=1,2,...,N i=1,2,...,N

On the other hand, it is easy to have that

2.3 Main Results 27
 2
 m̂ N 
  
ωi (t) − z̃2 = 
 ãij (t) x̃ j (t) − z̃ 

 j=1 

m̂ N
 2
≤ ãi j (t) x̃ j (t) − z̃  , (2.12)
j=1

where the last inequality is obtained by the convexity of the operator  · 2 and the
row-stochasticity of Ã(t), i.e., m̂j=1
N
ãi j (t) = 1. Then, by (2.12), we get

  2
m̂ N
 
ωi (t) − z̃2 ≤ ãi j (t) max x̃ j (t) − z̃ 
j=1,2,...,N
j=1
 2
= max x̃i (t) − z̃ ,
i=1,2,...,N

which implies that

max ωi (t) − z̃ ≤ max x̃i (t) − z̃ . (2.13)

i=1,2,...,N i=1,2,...,N

Let D1 (t) = maxi=1,2,...,N x̃i (t) − z̃ and D2 (t) = maxi=1,2,...,N ωi (t) − z̃. Then,
by (2.10) and (2.12), we have D1 (t) and D2 (t) are monotone decreasing. Since D1 (t)
and D2 (t) are bounded, they converge to the same non-negative value d1 at t → ∞.
Again from (2.9), (2.11) and (2.13), in order to prove limt→∞ ei (t) = 0, we
only need to prove limt→∞ x̃i (t) − z̃ = d1. By contradiction, if it does not hold,
there must exist an i 0 such that the limit of x̃i0 (t)−  z̃  does not equal to d1 . We
 
suppose there exists a subsequence of x̃i0 (t) − z̃  , denoted by x̃i0 (ts ) − z̃  ,
has the limit d2 < d1 . Then, there existpositive numbers 1 , 2 and s(1 , 2 ) with
d2 + 2 < d1 − 1 , such that d2 − 2 < x̃i0 (ts ) − z̃  < d2 + 2 for all ts > s(1 , 2 )
and d1 − 1 < D1 (t) < d1 + 1 for all t > s(1 , 2 ).
Since D1 (t) < d1 + 1 for all t > s(1 , 2 ), x̃i (t) − z̃ < d1 + 1 for all t >
s(1 , 2 ) and i = 1, 2, . . . , N . Let s0 be the s0 th point defined in Assumption 2.2
and satisfy ts0 ≥ s(1 , 2 ). Then, we have
 
x̃i (s(1 , 2 ) + 1) − z̃ 
 0

≤ ωi0 (s(1 , 2 )) − z̃ 
  
m̂ N
 
≤ ãi0 i0 x̃i0 (s(1 , 2 )) − z̃  + ãi0 j x̃ j (s(1 , 2 )) − z̃ 
j=1, j=i 0
≤ η(d2 + 2 ) + (1 − η)(d1 + 1 ), (2.14)
m̂ N
where the last inequality is obtained by Assumption 2.1 and j=1 ãi j = 1 for all
i = 1, 2, . . . , N .
28 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

Similar to (2.14), it is easy to derive that

 
x̃i (s(1 , 2 ) + 2) − z̃ 
0

  
m̂ N
 
≤ ãi0 i0 x̃i0 (s(1 , 2 ) + 1) − z̃  + ãi0 j x̃ j (s(1 , 2 ) + 1) − z̃ 
j=1, j=i 0
≤ η (η(d2 + 2 ) + (1 − η)(d1 + 1 )) + (1 − η)(d1 + 1 )
≤ η2 (d2 + 2 ) + (1 − η2 )(d1 + 1 ).

Recursively, for all s(1 , 2 ) ≤ t ≤ ts0 , we have

 
x̃i (t) − z̃  ≤ η M (d2 + 2 ) + (1 − η M )(d1 + 1 ), (2.15)
0

where the last inequality is obtained by 0 < η < 1 and ts0 − s(1 , 2 ) ≤ M.
By Lemma 2.8(a), Φ(t, s) defined in (2.7) is a row-stochastic matrix
 for all t ≥ s ≥
0. By Lemma 2.8(d), we obtain that Φ(s1 + (m̂ N −1)M −1, s1 ) ji0 ≥ η(m̂ N −1)M for
 
all j = 1, 2, . . . , N and s1 ≥ 0. Also, it follows from (2.15) that x̃i0 (ts0 ) − z̃  ≤
η M (d2 + 2 ) + (1 − η M )(d1 + 1 ). Thus, we have for all i = 1, 2, . . . , N
 
x̃i (ts + (m̂ N − 1)M) − z̃ 
 0
 
 
≤  Φ(ts0 + (m̂ N − 1)M − 1, ts0 ) ii0 x̃i0 (ts0 ) − z̃ 
 
 m̂ N 
    
+  Φ(ts0 + (m̂ N − 1)M − 1, ts0 ) i j x̃ j (ts0 ) − z̃ 

 j=1, j=i0 
   
≤ Φ(ts0 + (m̂ N − 1)M − 1, ts0 ) ii0 x̃i0 (ts0 ) − z̃ 
  !
+ 1 − Φ(ts0 + (m̂ N − 1)M − 1, ts0 ) ii0 (d1 + 1 )
 
≤ η(m̂ N −1)M η M (d2 + 2 ) + (1 − η M )(d1 + 1 )
!
+ 1 − η(m̂ N −1)M (d1 + 1 ), (2.16)

where the last inequality is obtained by Lemma 2.8(d).

Let 0 < 1 < η2(2−η (d1 −d2 ) ηm̂ N (d1 −d2 )
m̂ N
m̂ N ) and 0 < 2 < 2
. Then, we can check that d2 +
2 < d1 + 1 . Then, by simple calculation, we obtain from (2.16) that
 
x̃i (ts + (m̂ N − 1)M) − z̃  < d1 − 1 , (2.17)
0

which contradicts to D1 (t) > d1 − 1 for all t > s(1 , 2 ). Hence, we have
limt→∞ x̃i (k) − z̃ = d1 holds for all i = 1, 2, . . . , N , i.e., limt→∞ ei (k) = 0
for all i = 1, 2, . . . , N .

Define an auxiliary variable y(t) as

2.3 Main Results 29


m̂ N 
t 
m̂ N
y(t) = φi (s)x̃i (s) + φ j (r )e j (r − 1) + ei (t), (2.18)
i=1 r =s+1 j=1

where φi (s) is the ith element of φ(s) defined in Lemma 2.8. By (2.8), we have


m̂ N
x̃i (t + 1) = [Φ(t, s)]i j x̃ j (s)
j=1
⎛ ⎞

t 
m̂ N
+ ⎝ [Φ(t, r )]i j e j (r − 1)⎠ + ei (k). (2.19)
r =s+1 j=1

The following property shows that ωi (t) and x̃i (t) converge to y(t) as t → ∞.
Proposition 2.10 Under Assumptions 2.1 and 2.2, limt→∞ ωi (t) − y(t) = 0 and
limt→∞ x̃i (t) − y(t) = 0.

Proof By Proposition 2.9, we have limt→∞ ei (t) = 0 for all i = 1, 2, . . . , N .

Then, for any 3 > 0 and all i = 1, 2, . . . , N , there exists a positive integer s2 , such
that whenever t ≥ s > s2 , ei (t) < 3 .
By Lemma 2.8, Eqs. (2.18) and (2.19), we get

x̃i (t + 1) − y(t + 1)


m̂ N
  
≤ [Φ(t, s)]i j − φ j (s) x̃ j (s)
j=1


t 
m̂ N
  
+ [Φ(t, r )]i j − φ j (r ) e j (r − 1)
r =s+1 j=1

1 + η−B   t−s 
m̂ N
 
≤2 1 − η B B x̃ j (s)
1−η B
j=1

1 + η−B    t−r 
t m̂ N
+ 1 − η B B
e j (r − 1)
1 − η B r =s+1 j=1

1 + η−B   t−s 
m̂ N
 
≤2 1 − η B B x̃ j (s)
1 − ηB j=1
  t−s−1  − 1
1 + η−B 1 − ηB B − 1 − ηB B
+ m̂ N 3  − 1 . (2.20)
1 − ηB 1 − ηB B

In view of arbitrary choice of 3 , taking the limits of both sides of (2.20) as t → ∞,

we have for all i = 1, 2, . . . , N ,
30 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

lim x̃i (t) − y(t) = 0.

t→∞

 
Then, it follows ωi (t) − y(t) ≤ m̂j=1 N
[Φ(k, k)]i j x̃ j (t) − y(t) ≤ m̂ N
  j=1
x̃ j (t) − y(t). Then, we have for all i = 1, 2, . . . , N ,

lim ωi (t) − y(t) = 0,

t→∞

which completes the proof.

Now, we are in the position to get the main results of this chapter, which guarantees
the constrained consensus of the multi-agent system (2.3) can be finally reached.
Theorem 2.11 Under Assumptions 2.1 and 2.2. Constrained consensus of multi-
agent system (2.3) can be reached.
Proof Proposition 2.9 has proven that for any z̃ ∈ X sequence {x̃i (t) − z̃} is con-
vergent for all i = 1, 2, . . . , N . By Proposition 2.10, we have that limt→∞ x̃i (t) −
y(t) = 0 for all i = 1, 2, . . . , N . Thus, from y(t)−z̃ ≤ x̃i (t) − y(t) + x̃i (t) −
z̃, we have the sequence {y(t) − z̃} is convergent for any z̃ ∈ X . Thus, y(t) must
converge to a certain point x̃ ∈ X . That is, for x̃ ∈ X , there exists a subsequence
{y(tm )} of {y(t)}, such that limm→∞ y(tm ) − x̃ = 0, i.e., for any 4 > 0, there
exists a constant t0 , whenever tm > t0 , we have y(tm ) − z̃ < 21 1 . Note from Propo-
sition 2.10 that limt→∞ x̃i (t) − y(t) = 0. That is, for 4 > 0, there exists a con-
stant t1 , whenever t > t1 , we have xi (t) − y(t) < 21 4 . Set t2 = max{t0 , t1 }, we
have for all tm > t2 , xi (tm ) − x̃ ≤ xi (tm ) − y(tm ) + y(tm ) − x̃ ≤ 4 . Thus,
for all t > tm > t2 and i = 1, 2, . . . , N , we have xi (t) − x̃ ≤ 4 , which means
that limt→∞ xi (t) − x̃ = 0 for all i = 1, 2, . . . , N , implying that the constrained
consensus for the multi-agent system (2.3) is reached.

2.4 Numerical Examples

In this section, two numerical examples are given to illustrate the effectiveness of
the method and the correctness of theoretical analysis.

2.4.1 Constrained Consensus of a Simple Directed

Multi-agent System

We first illustrate the constrained consensus of a simple multi-agent system with 4

agents, whose states are two-dimensional vectors, and the communication graphs
are shown in Fig. 2.1. The agents in the multi-agent system evolve in the following
asynchronous way:
2.4 Numerical Examples 31

Fig. 2.1 Asynchronous communication topologies of a simple multi-agent system

(1) Agent 2 receives the state information from agent 1 at update times 4k + 1,
k ∈ N, whose communication graph G1 is shown in Fig. 2.1a;
(2) Agent 3 receives the state information from agent 2 at update times 4k + 2,
k ∈ N, whose communication graph G2 is shown in Fig. 2.1b;
(3) Agent 4 receives the state information from agent 3 at update times 4k + 3,
k ∈ N, whose communication graph G3 is shown in Fig. 2.1c;
(4) Agent 1 receives the state information from agent 4 at update times 4k + 4,
k ∈ N, whose communication graph G4 is shown in Fig. 2.1d.

Apparently, none of these graphs is strongly connected, but their union is. Thus,
Assumption 2.2 is satisfied. The edge weights are set to be a21 = 0.4, a22 = 0.6 in
G1 ; a32 = 0.5, a33 = 0.5 in G2 ; a43 = 0.3, a33 = 0.7 in G3 ; a14 = 0.5, a11 = 0.5 in
G4 . The agents’ initial values are randomly chosen from their local constraint sets.
The simulation results are shown in Figs. 2.2 and 2.3. Figures 2.2 and 2.3 provide
plots of the component-wise evolutions of the agents’ states xi1 , xi2 , i = 1, 2, 3, 4,
respectively. It can be seen from Figs. 2.2 and 2.3 that the agents’ states converge to
(1, 1)T ∈ X , which means that the constrained consensus has been finally reached.
The numerical results are consistent with that of theoretic analysis, it therefore con-
cludes that the correctness of the theoretical analysis for a simple multi-agent system.
32 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

Agent 1
0
Agent 2
Agent 3
xi (t) (i=1,2,3,4)

−1 Agent 4

−2
1

−3

−4

−5
0 20 40 60 80 100
Time t

Fig. 2.2 State evolution of the agents in a simple multi-agent system

7
Agent 1
6 Agent 2
Agent 3
5 Agent 4

4
xi (t) (i=1,2,3,4)

1
2

−1

−2

−3
0 20 40 60 80 100
Time t

Fig. 2.3 State evolution of the agents in a simple multi-agent system

2.4 Numerical Examples 33

2.4.2 Constrained Consensus of a Large-Scale Directed

Multi-agent System

We then consider a relatively large multi-agent system with 100 agents, whose states
are two-dimensional vectors. A set of N = 100 agents uniformly distribute on the
unit square [0, 1] × [0, 1], and the agents establish bidirectional links to each other
obeying the following rules:
(1) In Fig. 2.4a, the
√ Euclidean distances amongst them are smaller than the connec-
tivity radius log N /N /2;
(2) In Fig.√2.4b, the Euclidean√
distances among them are between the connectivity
radius log N /N /2 and 3 log N /N /5;
(3) In Fig. 2.4c,
√ the Euclidean distances
√ amongst them are between the connectivity
radius 3 log N /N /5 and 7 log N /N /10;
(4) In Fig. 2.4d,
√ the Euclidean distances
√ amongst them are between the connectivity
radius 7 log N /N /10 and 4 log N /N /5.
Obviously, as illustrated in Fig. 2.5, the union of Fig. 2.4a, b, c, d is strongly con-
nected. Then, we can construct the following asynchronous communication rules:
(1) At update times 4k + 1, k ∈ N, the communication graph G1 is shown in Fig. 2.4a;
(2) At update times 4k + 2, k ∈ N, the communication graph G2 is shown in
Fig. 2.4b;
(3) At update times 4k + 3, k ∈ N, the communication graph G3 is shown in Fig. 2.4c;
(4) At update times 4k + 4, k ∈ N, the communication graph G4 is shown in
Fig. 2.4d.
For each communication graph G j , j = 1, 2, 3, 4, we choose the weight on each
connection be 1/n ij , where n ij − 1 is the in-degree of agent i on the communication
graph G j , i = 1, 2, . . . , 100; j = 1, 2, 3, 4. Thus, Assumptions 2.1 and 2.2 are satis-
fied. Moreover, the local constraint set for each agent i = 1, 2, . . . , 100 is assumed
to be Xi = [1, 5] × [−5, 10] if i ≡ 1 mod 4; Xi = [−2, 6] × [−5, 9] if i ≡ 2 mod 4;
Xi = [−10, 5] × [4, 10] if i ≡ 3 mod 4; Xi = [−10, 3] × [−5, 7] if i ≡ 0 mod 4.
Obviously, the intersection set X of the constraint sets Xi , i = 1, 2, . . . , 4 is
X = [1, 3] × [4, 7]. Each agent is initialized as uniformly distributed random values
from their local constraint sets.
The simulation results are shown in Figs. 2.6 and 2.7. Figures 2.6 and 2.7 provide
plots of the component-wise evolutions of the agents’ states xi1 , xi2 , i = 1, 2, . . . , 100,
respectively. It can be seen from Figs. 2.6 and 2.7 that the states of the agents con-
verge to (3, 4)T ∈ X , which means that the constrained consensus has been finally
reached. The numerical results are consistent with that of theoretic analysis, it there-
fore concludes that the effectiveness of the theoretical analysis for a relatively large
multi-agent system.
34 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 0.5 1 0 0.5 1
(a) (b)
1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 0.5 1 0 0.5 1
(c) (d)

Fig. 2.4 Asynchronous communication topologies of relatively large multi-agent systems

Fig. 2.5 The union of the 1

communication topologies
0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 0.2 0.4 0.6 0.8 1
2.4 Numerical Examples 35

6
x1(t) (i=1,2,…,100)
4

0
i

−2

−4
0 20 40 60 80 100
Time t

Fig. 2.6 Evolutions of the states of the agents of relatively large multi-agent systems

12

10

8
x2(t) (i=1,2,…,100)

4
i

−2
0 20 40 60 80 100
Time t

Fig. 2.7 Evolutions of the states of the agents of relatively large multi-agent systems
36 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

2.5 Conclusion

In this chapter, the constrained consensus problem is studied for the asynchronous
multi-agent system, where each agent needs to lie in a closed convex constraint set.
The communication graphs are directed, dynamically changing and the union graph
is assumed to be strongly connected among each interval of finite length. In addition,
their adjacency matrix is not necessarily doubly stochastic. We propose a transfor-
mation method that equivalently transforms the original asynchronous network to the
synchronous one by adding some new agents. The analysis is then performed based
on the newly constructed system. The considered algorithm is separated as the linear
parts and the projection error parts. By analyzing, we show that the linear parts of the
newly constructed system converge and the nonlinear parts, i.e., projection errors,
vanish over time, which means that the original system reaches consensus. Finally,
two numerical examples are provided to show the effectiveness of the method and
the correctness of theoretical analysis.

References

1. Fax, A., Murray, R.: Information flow and cooperative control of vehicle formations. IEEE
Trans. Autom. Control 49(9), 1453–1464 (2004)
2. Jadbabaie, A., Lin, J., Morse, A.S.: Coordination of groups of mobile autonomous agents using
nearest neighbor rules. IEEE Trans. Autom. Control 48(6), 998–1001 (2003)
3. Yu, W., Chen, G., Wang, Z., Yang, W.: Distributed consensus filtering in sensor networks. IEEE
Trans. Syst. Man Cybern. Part B Cybern. 39(6), 1568–1577 (2009)
4. Gazi, V., Passino, K.M.: Stability analysis of swarms. IEEE Trans. Autom. Control 48(4),
692–697 (2003)
5. Anderson, B., Fidan, B., Walle, D.: UAV formation control: theory and application. Lecture
Notes in Control and Information Sciences, vol. 371, pp. 15–33 (2008)
6. Vanke, B., Peni, T., Bokor, J., Balas, G.: Practical approach to real-time trajectory tracking of
UAV formation. In: American Control Conference, Portland, USA, pp. 122–127 (2005)
7. Chen, S., Zhao, L., Zhang, W., Shi, P.: Consensus on compact Riemannian manifolds. Inf. Sci.
268, 220–230 (2014)
8. Hong, Y., Hu, J., Gao, L.: Tracking control for multi-agent consensus with an active leader and
variable topology. Automatica 42, 1177–1182 (2006)
9. Ji, L., Liu, Q., Liao, X.: On reaching group consensus for linearly coupled multi-agent networks.
Inf. Sci. 287, 1–12 (2014)
10. Jin, X., Park, J.H.: Adaptive sliding-mode insensitive control of a class of non-ideal complex
networked systems. Inf. Sci. 274, 273–285 (2014)
11. Mo, L., Niu, Y., Pan, T.: Consensus of heterogeneous multi-agent systems with switching
jointly-connected interconnection. Phys. A: Stat. Mech. Its Appl. 427, 132–140 (2015)
12. Li, H., Gao, H., Shi, P., Zhao, X.: Fault-tolerant control of Markovian jump stochastic systems
via the augmented sliding mode observer approach. Automatica 50, 1825–1834 (2014)
13. Li, H., Jiang, X., Karimi, H.R.: Output-feedback-based H∞ Control for vehicle suspension
systems with control delay. IEEE Trans. Ind. Electron. 61(1), 436–446 (2014)
14. Li, H., Yu, J., Hilton, C., Liu, H.: Adaptive sliding-mode control for nonlinear active suspension
vehicle systems using T-S fuzzy approach. IEEE Trans. Ind. Electron. 60(8), 3328–3338 (2013)
15. Olfati-Saber, R., Murray, R.: Consensus problems in networks of agents with switching topol-
ogy and time-delays. IEEE Trans. Autom. Control 49(9), 1520–1533 (2004)
Another Random Scribd Document
with Unrelated Content
rays of light which seemed to come nearer. As their light grew more
distinct that of the toad began to die out, and the creature itself
commenced shifting uneasily on its seat. The Lord of Fear seemed
more alarmed than ever. His teeth chattered like castanets—he had
to make three attempts before he could speak.
“You do ill to deny my power; all these who surround me have
acknowledged it!”
“They are ashamed of it now,” cried Mitaine; and then turning to
them, she shouted, “Can you submit to such a lord? You have only
to make one step towards him, and you will drive him and his
wretched race from the face of the earth. Your hands are not dead,
they are but benumbed for a while. Make one more effort. Fling
yourselves on the tyrant. I will show you the way!”
At these words the dead let fall their winding-sheets, and
discovered to view a legion of knights in rusty armour with their
swords drawn. Alarm gave a shriek, which was answered by screams
from Fear himself, from Dame Coward, from Consternation, Fright,
Terror, Dismay, Apprehension, Trepidation, Timidity, Pusillanimity,
Poltroonery, and Dastardy.
Then was seen a strange sight. The bas-reliefs began to start into
life, and continued their wild dance along the pillars, to the
accompaniment of alarming shrieks. The thunder rolled, and
yawning fissures opened in the walls and ceiling. The earth gaped
amid deafening clamours, and Mitaine found herself in the dark. She
did not remain long thus, for the galleries sank by degrees, and day
came on apace. Its first rays glittered on her arms; the cheery voice
of Chanticleer resounded, and, as if it had but waited the signal, the
Fortress of Fear vanished into air!
Mitaine was mute with astonishment. How fair appeared the
country to her! how beautiful the sun! and how softly did the breeze
of morning woo her cheek! She fell on her knees, and uttered a
heartfelt prayer.
The fields were variegated with a thousand colours, as though
they contained specimens of every kind of flower that blows. The
birds joined in—never had they chanted a more joyous welcome to
dawn. There was nothing left of the castle but the recollection, and
that was already growing indistinct.
When Mitaine had finished her orisons and rose to her feet, she
beheld an old man and a young woman gazing at her with an
affectionate expression. They were but a few steps from her, yet she
could scarcely see them, for they were enveloped in a faint mist.
“Who are you?” she asked.
“Your grateful friends. You have delivered us from Fear, who used
to hold us captive. For a long time we have ceased to breathe, but,
thanks to you, we are about to see once more those from whom we
were so hastily snatched away. To-day is the Feast of the Dead, and
heaven allows us to pass the day on earth. All those whom you have
delivered are going to escort you to Charlemagne’s camp to testify to
your great courage and noble bearing.”
Then Mitaine saw gathered around her from all quarters a number
of knights clad in armour that was eaten up with rust. They were of
all ages and of all countries, the greater part being mounted. A few
women and children followed the procession. The footfalls were
unheard, and left no mark behind them. The figures were
transparent, bathed in a strange mist, to which the sun gave an
opalescent gleam.
Having ranged themselves in column, they began to march
onward, and Mitaine retraversed the places which had seemed to
her so terrific on the preceding night: the stone bridge across the
torrent, the wall covered with creepers, and almost hidden by acacia
boughs, the forest of naked stems—everything, in short, appeared
full of gaiety now that the sun was shining.
She called to her the old man who had lately addressed her, and
bade him tell her what were the tortures, the sight of which had so
roused her.
“Those,” said the dead man, “were the halls of nightmare, my
child. The Lord of Fear gives his victims no rest. He and Sleep, who
delivers them into his hand, understand each other. Incubi, demons,
vampires, and ghoules form his terrible executioners, and preside
over the punishments. You have seen them at their task, I need not
attempt to describe them!”
“What can defend us against them?”
“A clear conscience and a good digestion.”
In a quarter of an hour Mitaine perceived the hut at which she had
stopped on the night before. The peasant was seated on the ground
among his little ones.
“Thank heaven!” said she. “Poor old man, your feeble sight will
not, doubtless, allow you to distinguish your son as yet. But you, his
wife, you can no doubt perceive him.”
“We have never ceased to see him since we parted,” said she.
“There is neither limit nor let to the vision of the dead.”
The peasant turned his head, saw the procession approaching,
recognised Mitaine, and, with a shout of surprise, at once ran to
meet her.
Ere he had reached half-way, his glance fell on his father and his
wife, and, overcome with joy, he sank on his knees, stretching out
his arms towards them. He would fain have spoken, but could find
no language to express in fitting terms the joy he experienced. He
scarce dared to move, lest he should put to flight the beloved group
he saw before him. When he had ascertained that he was not
suffering from an illusion, tears filled his eyes, and, clasping his
hands, he fell on his face, saying, “Kind Heaven, I am indeed
grateful for this!”
I will not attempt to describe to you the joy of these three, whom
death had, for a while, no power to separate. The mother covered
her babes with kisses. The peasant, now as aged as his father had
been, could not tear himself from his arms. Their white beards
mingled at each embrace. The first outburst of joy over, they all
three turned to Mitaine, and kissed her hands.
“Who could have forewarned you of all this happiness, my son?”
said the father.
 “Do you not know, then? My child, who, some years ago lost his
reason, has become the cleverest of the family since daybreak this
morning. Henceforth there are no saints in the calendar I shall
revere as I do you!” said he to Mitaine, who had no small difficulty in
freeing herself from the demonstrations of gratitude of which she
was the object. She called for the horse which she had left with her
host of the previous night, and rode away at full gallop, followed by
her fantastic escort.
In about six hours she saw the camp of Charlemagne. The
sentinels on outpost duty, seeing a cloud of dust in the distance
coming along towards them with such speed, fell back and gave the
alarm.
“What is it?” said the Emperor. “Who are these that thus fall into
our hands? Go, Miton; mount your horse, take an escort, and inspect
these new comers.”
In a moment the whole camp was alive. Every one put on his
corslet, laced his helm, seized his lance, and sprang to saddle. Miton
chose thirty mounted knights and led them out.
“By my faith!” said he, “these be strange folks. To judge from their
size they ought not to be far off, and yet I can hardly make them
out. Can you see them better, Red John?” he asked one of his men.
“Not I! My wonder is as great as yours. But is not that a page in
the imperial livery who is riding at their head?”
“By my life, it is Mitaine!” And Miton spurred forward at such
speed that in three minutes he was in his daughter’s arms. The
ghostly squadron halted, and the thirty knights halted likewise,
striving to pacify their startled horses, which were snuffing the air,
snorting with dilated nostrils, pawing the ground, and neighing as if
ready to die of terror.
“Who are these whom you are leading?”
“Those whom I have liberated.”
“Liberated! How?”
 “I will tell you all in the Emperor’s presence. The sun is low
already, and we have no time to spare.”
Miton and his thirty knights, and Mitaine with her strange
followers, rode towards the camp. Charlemagne, surrounded by his
peers, came out to meet them.
“By St. James! these people look as if they didn’t belong to this
world. And if I am not stupidly mistaken, it is my godchild who
commands them.”
Mitaine dismounted, and approached her royal godsire, who asked
her, “Well, little one, what is this strange array? Do you know that I
have a mind to punish you, and yet I haven’t the heart to scold you,
I am so rejoiced to see you again, and so anxious to learn who these
are that accompany you.”
“My prisoners, sire!” And the spectres lowered their lances to show
their submission to her.
“But whence come they? Have you been to seek them in another
world?”
“By my faith, sire, I could almost believe I passed last night
there;” and she related her adventures briefly to Charlemagne in the
presence of his peers and knights.
“Come, let me embrace you, my darling. So it appears I have
promised you something. What is it?”
“You promised me, sire, to ask Roland to take me into his service
as a squire.”
“It is Roland whom I reward by giving him such a treasure. What
say you, nephew mine?”
The only answer Roland gave was to clasp Mitaine in his arms.
The little heroine, ruddy with joy, turned to her escort to thank
them. They had disappeared! On seeing this, Charlemagne sank 011
his knees; his example was followed by all the rest, and Turpin
recited the prayers for the dead.
Thus ended the adventure undertaken by Mitaine.
 I wish I could tell you, my friends, that the Fortress of Fear was
destroyed for good and all. I am compelled, as a veracious
chronicler, to confess that it was rebuilt the same evening.
You will some day or other, my young friends, most assuredly fall
in with the Lord of Fear. Call to mind Mitaine whenever you do meet
him, and remember that the monster can boast no weapons save
those you surrender to him—no power save that which you give him
—no courage save that which you lose.

Original Size -- Medium-Size

END OF THE THIRD BOOK

 EPILOGUE — RONCESVALLES — A.
D. 778.

Original Size -- Medium-Size

YOU have, I hope, not forgotten, my dear readers, that

Charlemagne had dispatched Ganelon to Aquitaine. For the shame
and injury of France, the Count of Mayence had turned this trip to
good account, by establishing a perfect understanding between
himself and our old and little-respected friend, Wolf. They decided
on the destruction of Charlemagne and his peers; but as for
attacking them openly, they did not dream of that!
“I will undertake,” said Ganelon, “to lead them into the mountains,
if you will only place some twenty thousand Navarrese and Gascons
on the heights that I will show you. Then we shall be able in perfect
safety to crush beneath the rocks this haughty and hated brood.”
About the same period Marsillus had called his warriors together,
and was conversing with them, reposing in the shade on the white
marble steps of his palace.
“My friends, since this accursed Charles has set foot in Spain, we
have never had a moment’s peace. Great as has been the bravery
we have displayed, we have been everywhere worsted. We can do
no more, for each has done his best. I suppose you are none of you
less desirous than I to yield this beautiful Spain to these Northern
barbarians. Aid me, therefore, by your counsels to avenge our
disasters.”
Blancandrin, the wisest and most crafty of the Pagans, was the
first to speak.
“The fox often passes where the lion cannot. Well, then, since we
fail as lions, let us assume the part of foxes, instead of wasting our
time in idle laments, and our resources in vain endeavours. Charles
is very proud; and when pride is warder, the city is ill-watched.
Profess a respectful regard for this crowned bully; tell him you desire
to be baptised, and appoint a meeting with him in his own
dominions. Promise to meet him there by Michaelmas, with your
principal nobles, to do homage to him, and to acknowledge the
Christian faith. Add further, that you will make him a present of three
hundred mules, laden with gold and silver; a hundred chariots, filled
with a countless stock of rare stuffs; numberless war-steeds; three
hundred trained falcons, lions, and leopards broken-in for the chase;
besides five hundred fair Saracen damsels, if such be his good
pleasure. The invaders have been a long time from home, and have
left their estates in the charge of their wives. There is not one of
them who would not be glad of a rest. As soon as they have divided
the booty, they will all be pressing the king to return, and when once
they get home again, he will have no easy task to prevail on them to
stir a second time.”
“The advice is good, possibly, but Charles is not the man to be
satisfied with simple promises.”
“Send him hostages—ten, twenty, thirty, if he asks for them.
Would it not be better to lose a few women and children than the
whole of Spain? I offer to give my son as a hostage, at the risk of his
life.”
This counsel was considered sound, and was approved by all.
“Go, then,” said Marsillus to Blancandrin. “I promise you a splendid
escort when you set out, and boundless rewards on your return.
Exchange the sword for the olive branch, and be not sparing in
promises.”
 The envoys were accordingly mounted on white mules, with
trappings and bells of gold and silver, and before long set out for the
camp of Charlemagne.
When the envoys arrived, no time was lost in introducing them
into the Emperor’s presence. His Majesty of the snowy beard was
sitting in his orchard surrounded by his bravest warriors. The
younger ones were practising the use of arms; the elder were talking
or playing at chess.
Blancandrin, after having saluted Charles with dignified courtesy,
delivered his message so cunningly, that the nobles began to shout,
“Hurrah, now we shall speedily return home!” Charlemagne,
however, remained lost in meditation. It was not his habit to give
way readily either to astonishment or disappointment. At last he rose
and said, “The news you bring me causes me great pleasure. It King
Marsillus is really desirous of securing his soul’s safety, let him meet
me at Aix-la-Chapelle, and I will welcome him there as a brother.”
A tent was prepared for Blancandrin and his suite, on whom every
attention and boundless generosity were lavished.
The next day, after mass and matins, the Emperor wisely called
together his peers to learn what they thought of the speech of the
envoy from the Court of the King of Saragossa. Naymes of Bavaria, a
knight of great renown, and one of the king’s best counsellors, rose
and spoke:—“Sire, you have beaten the enemy wherever he has
dared to offer battle. Of his fortresses, not one stone rests on
another; his cities have been burnt; his troops have been either
killed or converted. You have raised the cross wherever it had been
formerly overthrown; what can you desire more? You are offered a
ransom for the kingdom in which you will hold the sovereign power;
a nation of unbelievers demands baptism at your hands, and offers
you hostages. It would be a sin to continue a warfare which has no
longer any object. Such is my opinion!”
After this speech, Roland was not slow to spring to his feet.
“So it is you, then, Naymes, whom I hear? and can you give such
counsel? Marsillus is your enemy, sire, and you have scarcely treated
him in a way to make him very anxious to embrace you. Do not turn
your back upon Spain until your undertaking is accomplished; we
have been here longer already than was necessary for its
completion. Send home those brave soldiers who have tired of war
before they have completed their conquest, and I venture to say
that with those who remain you shall plant the cross within sight of
Africa, if such be your good pleasure. How can you trust the words
of a Pagan? Have you already forgotten the fate to which Marsillus
condemned two of your nobles, the Counts Basan and Basilic? They
went on an embassy from you to the King of Saragossa, and he had
them beheaded on Mount Hautille. It was your honour, sire, which
that day fell beneath the infidel axe. Will you let them trample it
under foot because a few prudent warriors would be glad to
abandon this undertaking? Go, then, but I must remain! I shall stay
here to make my death so glorious that you will all envy me.”
During this speech Charles knitted his brows and tugged his long
moustache; seeing which, Ganelon rose in his turn.
“These be proud words, forsooth! I could not but ask myself when
I heard them whether we live in the reign of Roland or of
Charlemagne. This sort of thing is easily spoken, and sounds
remarkably well, like everything that’s hollow. We are told to retreat;
are we in the habit of doing so? Does it not look as if Roland had
been conquering Spain while we followed at a respectful distance?
Forgive my anger, sire, but I cannot help speaking somewhat freely.
Take no one’s counsel but your own, sire, and you will do right.”
Thereupon Charlemagne asked his knights which of them would
like to carry his message to Marsillus. All rose and offered to go,
Roland being more importunate than all the others.
“You’ll deafen me, nephew,” said the Emperor. “I shall certainly not
send you on a mission you have just condemned. My friend Ganelon
shall carry my wishes to the King of Saragossa. To him will I entrust
the gauntlet and truncheon.”
“That is indeed a wise choice,” said Roland, laughing. “You will
nowhere find a more cautious ambassador.”
 “Enough said! By my beard, nephew mine, you will provoke me
too far presently. Be seated, and wait until I bid you speak.”
“Sire,” said Ganelon, “from such a mission one does not always
return. I recommend to your care my son Baldwin, who will one day
be a brave warrior.”
Charlemagne handed the gauntlet to the Count of Mayence, who
let it fall on the ground. “A bad omen!” said the Franks, seeing it.
“Roland may be right after all!”
“You will hear of me before long, gentlemen,” said Ganelon, with
an ill-favoured smile.
Then, furnished with truncheon and letter, he made ready to set
out on his mission.
Ganelon and Blancandrin, followed by the Saracen body-guard,
journeyed for three days side by side. The Pagan was not slow to
perceive in a moment the hatred entertained by the Count of
Mayence for Roland, and he rejoiced to see it. Let us hear what they
are talking about.
“Whence comes it,” said Blancandrin, “that your sovereign, instead
of seeking an alliance with us, made war on us so fiercely?”
“It is Roland who is always egging him on. But for him, we should
long since have returned to France.”
They reached the camp of Marsillus. Fifty thousand Saracens
surrounded the King of Saragossa, but they maintained perfect
silence, for fear of losing a syllable of what was going to be said.
“May Allah and Mahomet preserve you, beloved sovereign! We
have borne your message, and we bring back to you one of the
noblest peers of the Court of France, to decide with you on peace or
war.”
“I am prepared to give him an immediate audience.”
Marsillus and Ganelon remained shut up together for two hours—
two hours, which laid the foundation of ages of regret. When the
tent was o o re-opened the ‘King of Saragossa came out, leaning on
the arm of the French envoy. Had Roland come instead of Ganelon,
that would never have happened.
“Gentlemen,” said Marsillus to his nobles, “welcome the preserver
of Spain! This lord, although a Christian, is a true friend to us, and I
desire that he be treated as such.”

Original Size -- Medium-Size

A Saracen advanced, drew his sword from its sheath, and

presented it to Ganelon.
“This weapon is the best in the world,” said he. “Its jewelled hilt
alone is estimated at thirty thousand bezants, at the lowest, and yet
the blade is even more valuable. Accept it, and may it serve you well
against Roland.”
 Original Size -- Medium-Size

“I will put it to the test,” said the Count of Mayence, coolly; and
the traitor and unbeliever kissed each other.
The queen passed by Marsillus stopped her cortege, and bade her
dismount, saying, “This is our best of friends. You owe it to him that
we shall remain under that Spanish sky which you love so much.
Embrace him for the love of us all.”
“With all my heart,” said the Sultana. “I wish you also, Sir
Ganelon, to bear to your wife from me these bracelets, which are
the finest in my possession. Neither the Pope at Rome nor the
Emperor at Aix-la-Chapelle can boast anything to equal them among
all their treasures.”
All vied in paying the Count attention, and in loading him with the
most precious gifts. .
The same evening Ganelon returned to the French camp,
accompanied by presents and hostages for the Emperor.
Three days later, at early dawn, Ganelon and his escort arrived at
Charlemagne’s quarters.
“So you have returned,” said Charles. “Have you sped well with
your mission?”
“Sire, you have nothing more to do here! The gallant King
Marsillus is altogether your devoted liegeman. Behold the treasures
he sends you, as a guarantee of others yet more valuable. See, too,
the hostages whom I have chosen, thirty in number, all of them of
the noblest rank. In a month the King of the Saracens will visit you
at the French Court to receive baptism, together with all his nobles
and knights.”
“You could not bring me more welcome news, and I rejoice greatly
that I chose you for the mission. Before long you will have reason to
rejoice at it too!”
His audience concluded, Ganelon retired with his nephew Pinabel,
to whom he wished to reveal the real state of the case. It happened
that Mitaine preceded them into the stable, towards which the traitor
took his way, and knowing the hate the count bore to Roland, her
friend, she was curious to hear him speak openly. She therefore
crept up in the manger, and hid herself among the hay in the rack.

Original Size -- Medium-Size

This second Judas, going up to his horse, began to talk as follows:

—
“Marsillus, who had treated me distantly enough in the morning,
apologised at night for so doing, and, as a slight reparation,
presented me with some valuable sables. I gave him to understand
the dreadful fate that awaited him, and assured him that Roland was
the only obstacle in the way of our return to France. ‘Hope for no
mercy,’ said I, ‘while Roland lives.’ ‘How can we kill him?’ said he.
Whereupon I answered, I would undertake to do it with his
assistance. ‘What can I do?’ he asked. ‘I will tell you what I have
planned,’ said I. ‘Before long we shall be on the march for France.
The most dangerous post is the rear-guard, and that Roland will
claim. When he reaches the pass of Roncesvalles, surrounded by the
flower of our chivalry, twenty thousand Navarrese and Gascons,
posted there by me, will hurl down a very shower of rocks. Take
advantage of the surprise, and with two hundred thousand men fall
on them in the rear. I won’t guarantee your men’s lives, but you
must carry on the battle incessantly, and at last Roland must be
slain.’ ‘It is very well said,’ answered the king; ‘this counsel is worth
ten mules, laden with gold pieces, and I will pay you that sum yearly
as long as I live.’”
At this point Pinabel, observing that Ganelon’s horse, although it
had just come off a long journey, only smelt at the rack without
touching its contents, took a pitchfork, and in order to find out what
hindered the animal from eating, thrust it into the hay. One of the
prongs pierced Mitaine’s thigh, but she nevertheless remained silent,
determined not to lose for a cry the advantage of the conversation
she had overheard.
“There’s nothing there,” said Pinabel.
“What did you think there would be? Don’t you know that a good
horse never eats much in the morning?” And with that the worthy
couple quitted the stable.
Mitaine had great difficulty in crawling back to Miton’s tent. She
dressed her wound with a celebrated ointment, which is still in great
use—the “Balm of Miton-Mitaine”—and was able to present herself
the same evening before Roland.
The Count of Mans listened to what his squire had to tell.
“This is good news you bring me, little one; and, with the aid of
Heaven, I will find a way thereby to rid the world of this traitor
Ganelon.”
“What!” said Mitaine; “will you not alter your line of march?”
“Remember this: he who finds a snake in his path has two
alternatives to choose between. He can either make a détour, and
continue his route, by doing which he leaves an enemy in his rear;
or he can go straight to the monster and kill it, which is the safer
course. There is, by the way, a third solution of the matter—flight;
but, of course, no one would dream of that. I shall take care not to
neglect the opportunity which is offered me. In the meantime, swear
to keep strict silence on this point!”
The trumpets resounded through the camp of Marsillus. The
unbelievers placed themselves in ambush beside the French line of
march, and waited for the next morning.
The clarions rang out through the camp of Charlemagne. The hour
of departure had come. Charles rode proudly amid his gallant
knights.
“Who will lead the rear-guard through the passes of Cisaire?”
asked the Emperor of his nobles.
 “Count Roland,” suggested Ganelon, “since he is the bravest. Does
not the place of danger belong to him?”
“Count of Mayence, some evil intention influences that speech.”
“Why so, sire?” interposed Roland. “Sir Ganelon is right. The task
is mine—I claim it.”
“So be it,” said the Emperor. “My peers shall accompany you with
twenty-five thousand horsemen.”
“The Saracens will have a hot day’s work,” said Ganelon to himself.
The Saracens were concealed in the forests at the entrance of the
pass. The Navarrese and Gascons (everlasting shame upon them!)
were lying in ambush on the heights, ready to hurl death upon their
brother Christians.
The vanguard, consisting of twenty thousand men, led by Ogier
the Dane, was first to present itself. But it was not they who were
wanted—-they were allowed to pass.
Charlemagne came next, with Ganelon in attendance upon him.
For six hours the troops, the wagons, the booty, were slowly
marching through the defile. There was an abundance of wealth; but
who dared touch it? They were suffered to pass. Finally came the
rear-guard, led by Roland. Then the Pagans began to be on the
move, the Gascons prepared for action. The great carnage was
about to begin.
Marsillus was on horseback at the head of his troops. Buriabel,
King of Alexandria, came swaggering up to him.
“Sire, I have brought you thirty thousand soldiers, fully armed. I
have not hesitated to risk my life in your service. In return for this, I
only ask one thing—the honour of despatching Roland. If I meet
him, he dies!”
“You forget, it appears to me,” said the King of Saragossa, in a
severe tone, “that I am here. I am not in the habit of handing over
difficult tasks to others; Roland belongs to me! You will have enough
to do with the rest.”
Then, armed to the teeth, they rode forward in serried ranks.
 The Franks entered the pass. Roland halted them, and spoke:
—“Brothers in arms! We are going to have a tough day’s work. But
few of us will ever again behold fair France. Ganelon, the traitor, has
brought us to this evil pass! He has sold us to the Saracens. In a few
minutes these rocks will be hurled down upon us, and we shall hear
the Saracen trumpets sounding. They do not know that we are
forewarned, and the sound of our bugles will be the signal. Let those
who are in doubt about our safety, therefore, leave us to join the
main body. But let those, who desire wounds more awful than death
—those who are ready to sacrifice their lives, in order to be
revenged on Ganelon—let those remain with me!” Not a single
knight quitted the ranks.
“If any one of us escapes, his life must be devoted to the
extermination of Ganelon, and all his race. For my part, I swear to
do this!”
All repeated the oath. Roland heard behind him a voice, shriller
than any of the others, cry, “By the Shrine of St. Landri, death to the
Count of Mayence!”
He turned, and saw Mitaine.
“Ah, unhappy child, what are you doing here? You know well what
fate awaits us. Is this a place for babes-in-arms?”
“You do wrong to blame me, sir knight. You will, perhaps, have
reason to be sorry for your words before sunset.”
Mitaine was on the summit of a peak. She gazed around on all
sides, and soon discovered the enemy. The sun was shining brightly,
and glistened on corslet and casque, spear and pennon. At the same
moment the neighing of horses reached her ear.
 Original Size -- Medium-Size

“The Saracens are coming from the Spanish side. They are so
many in number, it is difficult to understand how any troops can be
left to guard the cities. If we had to encounter so large a Christian
army, the result would be doubtful. But these are Pagans, and
Heaven will not fail us.”
“If that be the case,” said Oliver, “you had better sound your horn,
friend Roland. Charlemagne has not gone far and will return at once
to our aid on hearing it.”
“We must wield swords, not horns, here. The way is open, if you
fear the adventure is too arduous.”
“Trust me, comrade; in a few moments it will be too late. Wind
your horn!”
“You give me base counsel! It shall never be told that Roland
quitted his grasp of Durandal to wind his horn for aid against
Pagans!”
“So be it,” said Oliver. “We will not quarrel about it.”
Roland turned to Gautier de Luz, and said to him—
“Dismount, Gautier, and let two thousand of our knights do the
same. You will take the command of them, climb the mountain, and
take these accursed Gascons in the rear before we enter the pass.
Cut them up without mercy, like dogs as they are, and then, when
you have accomplished the task, sound on your horn. We shall then
draw on the Saracens in pursuit, and when I give the signal, do you
roll down on them the rocks prepared for our destruction.”
 “Well conceived,” said Hoel of Nantes. “An excellent jest. I would
not exchange my place here for anything in the whole world!”
Two thousand knights dismounted, and with Gautier de Luz at
their head, commenced the ascent. Mitaine, more active and lighter
than the others, went first to reconnoitre. Roland followed them with
his eyes until they disappeared behind the rocks.
In about a quarter of an hour, which, I can assure you, seemed
long enough to those below, a great uproar broke out, and the
Navarrese and Gascons appeared in disorder on the cliffs. They were
close pressed, and those who were not put to the sword on the spot,
were flung down into the ravine, in which there was soon an almost
insurmountable heap of dead bodies. There was hardly a bush that
was not adorned with some bleeding fragment or other.
Presently was heard the bugle note which announced that the
heights were taken, and Roland, followed by some thousands of
knights, rode out to meet the Saracens.
“What is the meaning of this?” said Marsillus, on beholding the
Christians issuing from the pass. “It strikes me these brave warriors
are afraid to attempt the pass. But we know how to compel them to
do so. Their graves are dug there, and there they must sleep this
night—and nowhere else!”
Thirty thousand Saracens spurred forward in haste, and grew
doubly courageous on beholding the Christians turn to retreat.
“What have they been telling us about the courage of these
people?” said Arroth, the nephew of Marsillus. “So far, there has
been more of the chase than the combat. We need hardly have
come in such numbers.”
“Your words are wanting in sense,” said Turgis of Toulouse. “Pray
Heaven to allow your brains to grow old enough to perceive the
folly.”
The Saracens entered the defile in pursuit of the Franks, who had
surmounted all the obstacles in the pass. Their pursuers, however,
halted in wonder before the heap of dead bodies that barred their
passage. Roland took advantage of their hesitation and gave the
signal, on hearing Avhich Gautier de Luz set to work. Huge blocks of
stone crashed down from overhead, involving horses and men;
living, dead, and wounded; Saracens, Gascons, and Navarrese, in
one common destruction. The pass was completely blocked up.

Original Size -- Medium-Size

“Truly,” said Roland, “Ganelon contrived this trap very cleverly. But
one cannot foresee everything in this world, and in this instance it is
the hare that is hunting; the hounds!”
The Pagans who returned to the King of Saragossa were barely
eight thousand, including the wounded who had escaped
destruction. They had flung away their banners and their arms in
order to facilitate their flight.
“Is this what you promised us?” they cried, threateningly, to
Marsillus. “We have just fallen into a snare laid for us by Ganelon.
Ah, dastard of a Roland, treacherous Count of Mayence, coward of
an Emperor, you shall hear more of us yet! By Mahomet, our
vengeance shall be something to speak of, rascals!”
A hundred thousand Saracen knights pricked forward at full speed,
taking a different road, which permitted them to cut off the retreat
of the Franks. In the meantime Gautier de Luz and Mitaine had
rejoined Roland.
 Archbishop Turpin had ridden to a slight eminence. The twenty
thousand knights were on their knees around him.
“Prepare to perish nobly, my brothers-in-arms,” said he to them.
“The heroes who do not shrink from the fight will sleep in Paradise
by sunset. All your past sins shall be atoned for by cuts or thrusts of
sword or lance. I absolve you all from this moment!”
He gave them his blessing, and they rose, comforted and
encouraged.
Presently the sound of the enemies’ horses was heard, and before
long the two armies had encountered each other. Lances were
shattered—the field wras covered with fragments of arms and
armour. Death had made a speedy harvest, and riderless horses
were galloping hither and thither, amid the groans and cries of the
wounded.
Everywhere destruction was being dealt out.
At the head of the Saracens rode Arroth, nephew of Marsillus.
“By Allah! Charlemagne must be childish to give the command of
the rear-guard of his forces to Roland.”
The Count of Mans heard him, but answered not. Lance in rest, he
rode down on him. Good heavens! what a thrust!—nothing could
resist it. It clave the shield of the nephew of the King of Saragossa,
pierced his chest, broke his spine, and pinned him to the earth.
Fauseron, brother of King Marsillus, beheld Miton, and shouted to
him—“Your Emperor, Charlemagne, must be sorely jealous of the
fame of his knights, to send them to be slaughtered thus.”
Miton dashed at him with uplifted blade, and dealt him three
terrific wounds: a partridge might have flown through any one of
them with ease.
“You lie, knave!” cried the father of Mitaine; “our Charles is the
bravest of the brave, and whoever questions it shall die the death of
a dog—as you die!”
Anseis charged at Turgis of Toulouse, and ran him through with
his lance. The white pennon was stained crimson with the thrust.
 But I should never finish if I told you all the wonderful blows they
interchanged. At last the spear of Roland shivered. He drew
Durandal and rushed into the thickest of the fight, slicing off heads
with his sword as easily as a pigeon severs the heads of millet with
its sharp beak.
The fury of the combat was redoubled. The Franks performed
prodigies of valour, but the Saracens seemed never to tire of being
slaughtered. No sooner were thirty thousand Pagans stretched on
the earth than thirty thousand more offered themselves for
slaughter. The swords were blunted with repeated blows, but the
strength of the heroes wearied not. How many Christians had
received the crown of martyrdom! Yonder they lay, trampled under
the horses’ hoofs, while their mothers, their wives, their daughters
were, perchance, singing cheerily as they awaited their return.
At length came a time when there were no more Saracens left to
kill. Of a hundred thousand Pagans but two survived.
“Mountjoy St. Denis!” resounded over the field. But lo! King
Marsillus arrived with the main body.
They had only encountered the advanced guard!
“Brethren,” said Turpin, pointing to the Saracens with his mace,
“yonder comes our death-struggle. Let us be polite, and go meet it;
we shall only be in Paradise the sooner!” and he rode off as swiftly
as if he bestrode a swallow.
“Shame, false friend, to outstrip me!” cried Roland, spurring
Veillantif. “Bishop, do not perish without e!”
Once more the contest raged furiously. Turpin perceived Abyme,
the most unbelieving Pagan of them all.
“What deity do you serve?” cried the bishop.
“None,” said the heretic; whereupon, with three mighty blows of
his mace, Turpin scattered over the field the amethysts, topazes, and
carbuncles that covered the Pagan’s shield. At the third blow the soul
of Abyme fled to the regions below.
 Climborin smote down Angelier of Gascony, but he did not live
more than ten seconds to enjoy his conquest. Miton had seen the
deed, lowered his lance, and pierced the Pagan’s throat.
“There, dog! you may go boast of your victory!” said he, as he
rode off.
Oliver had rested but little all this while; he drove right and left at
the ranks of the enemy, brandishing Hauteclaire, mowing the
Saracens down like stubble.
His shield was of gold, charged with a red cross.
“That is a foul blazon,” said Valdabron, striking the shield with his
lance.
“Nevertheless, you shall bow to it,” answered the brother of Aude,
and with one back-stroke he beheaded the paynim.
The Duke Sanche was slain: it was Haucuidant who struck the
fatal blow; by his hand, too, perished Gerin and Anseis, Beranger
and Guy de St. Antoine. But Roland rode right at the Pagan, and
with the hilt of Durandal crushed his face in, and flung him, an
unrecognisable corpse under his horse’s hoofs.
“It is truly sad that we can only kill once a hound who has done so
much mischief.”
Then the knight stood up in his stirrups, and gazed around him.
Merciful heavens, what a sight! Out of the twenty thousand Franks
who had come there, but sixty remained alive.
“By my hopes of Heaven!” said Roland, “I should die the happier if
I could but bear Marsillus with me to the grave. But how can I find
him amid such a mêlée?”
Mitaine heard him.
“I will show him to you, if you will follow me;” and she began to
strip off her armour piecemeal. Roland caught her by the arm to
stop her—?
“What proof of madness are you going to give us now?”
“You take wisdom for folly, my lord. Do you think I should be
suffered to pass, wearing your colours? My mother used to scold me
for spoiling my clothes; they might get damaged now.”
“And you think I am going to let you perish like this?”
“Is it not absurd to make all this difficulty about it? Have we not
come here to die?”
And Mitaine freed herself from his grasp, and sprang on a Saracen
horse that she caught as it went riderless by. She was naked to the
waist, and her golden hair floated around her shoulders. She
seemed like the spirit of youth. Death fled from the presence of such
lofty courage.
“Come and seek me, dastard of a Croquemitaine!” she cried. “Here
I am well protected from thee.”
Roland followed her; his eyes were blinded with tears.
“Merciful heaven! what will they say of me for all these deaths? I
shall scarce dare to show myself to-night in Paradise.”
Mitaine had caught sight of the King of Saragossa, and made
direct for him, without looking right or left. Miton, whose headlong
courage had carried him into the ranks of the foe, was beside her,
surrounded by the Saracens. He was striking out right and left at
random, thinking only to hack and hew the bodies of Pagans. Alas
for the double misfortune! Mitaine drew near him and her father’s
sword traced a gory slash across her shoulder. She turned, and
father and child recognised each other.
“Is it you my father? It was a good stroke, but ‘tis wasted!”
Horrified at the sight, Miton for a second forgot to defend himself.
In another moment poor Mita was a widow!
Meanwhile Mitaine had ridden close up to Marsillus, and rising up
in her stirrups, to make sure Roland should see her, smote him on
the face, crying, as loud as she was able—“Behold the King of
Saragossa! Mountjoy for Charlemagne!”
She could say no more. Marganice, King of Carthage, and uncle of
Marsillus, dealt her a blow on the chest that was far heavier than
was needed. The poor girl sank, insensible, and rolled under the
horse’s hoofs, with blood gushing from her lips and nostrils.
 When Roland saw this, his rage overpowered him. He drew near
Oliver, and said, “Brother, shall we go slay that boastful Marsillus
yonder?”
“It shall be done,” said the other.
They dashed forward, followed by a few of the Franks still
remaining on the field—Beuve, Lord of Beaune and Dijon, whose
death was a sore loss to Charles—Yve, and Yvoire, and Gerard of
Roussillon. Roland and Oliver penetrated farthest into the infidel
ranks; at last they came within a few paces of Marsillus.
“Is it you, then, whom they call the King Marsillus?” said Roland.
“It is a name the Franks will not forget.”
“I am called Roland. If you never knew me before you shall know
me to-day;” and with that he smote off the King’s right hand as he
raised it to strike.
The Saracens shouted in alarm, “Mahomet preserve us!” and fled
like doves before an eagle. If they had found legs to bring them
thither, they had found wings to take them away.
There remained on the field only a thousand Ethiopians, the forces
of Marganice. They were drawn up at a distance, and seemed
undecided whether to advance. Roland put his horn to his lips, and
blew a blast so powerful that it echoed and re-echoed for twenty
leagues around.
“What are you doing?” said Oliver. “Have you lost all shame, and
do you no longer fear to sound for help against Pagans?”
“These are cruel words, comrade!”
“Why disturb Charlemagne for such a trifle? We are three yet. If
you had been less brave we should not have bequeathed this defeat
to our country. If you sound the bugle on my behalf, do not trouble
yourself—henceforth I do not desire to live. If for Turpin, our friend
only survives by a miracle, and will be dead before any one can
come to his aid. If you sound, it is for yourself; and, by Heaven’s
truth! you will be a brave man to face Charlemagne.”
 “Truly,” said Turpin, “you might do better than quarrel now. Wind
your horn, Roland, not for our sakes, but for the honour of France.
We shall be avenged, and our bones will be laid in consecrated soil.
Wind your horn, Roland!”
The Count of Mans lifted his bugle to his lips, and blew so loud
and long, that the veins in his temples stood up like ropes, and the
blood flowed from his mouth.
The Emperor reined up his steed.
“Did you hear, as I did, the bugle of Roland?”
The Count of Mayence trembled, but he answered, “‘Tis some
goatherd calling together his flock.”
“Do you think I’ve grown childish, that I should mistake a horn for
a pipe? It was Roland’s horn, past a doubt.”
“Well, sire, he sounds his bugle for nothing often; perchance he is
chasing some wild animal.”
“By your leave, sire, the horn has a mournful sound,” said Naymes
of Bavaria, “and it is but due to your peers to go and see what has
befallen them.”
“You are right, friend. Ganelon, you will remain here;” and Charles
called for Besgue, his head cook, and entrusted to him the custody
of the Count of Mayence.
“It is the duty of your scullions to guard this criminal. Have you
any stout rope to put him to the question with?”
“I have, sire, the rope, saving your presence, with which I tie up
the pigs when I stick them.”
“That will do well! And now, my comrades, let us hasten to
Roland.”
“There is no need to hurry,” said Ganelon, with a grin; “Roland
does not ring the bell until mass is over.”
“Even so, renegade,” said the Emperor, “we may arrive in time for
vespers, and so much the worse for the Pagans.”
Roland was the only one left alive on the plains of Roncesvalles.
To the shouts and yells of conflict had succeeded a silence infinitely
more terrible.
Dismayed at their success, the Saracens had fled. The work was
accomplished; the vultures would fitly succeed them. Insatiable
parasites of the King of Saragossa, these new comers seldom had
time to wipe their beaks between the banquets.
Roland dismounted for the first time in the four-and-twenty hours.
The brave knight could scarcely stand. Leaning his brow on his
horse’s saddle, he cried like a child—he had poured out all his blood,
and he had nothing left to shed but tears!

Original Size -- Medium-Size

His wounds seemed nothing to him. It was despair that was killing
him. In his grief he knelt beside the body of Oliver, and clasped it in
his arms. He laid it on the turf, unlaced the helmet, kissed the cold
brow, stripped off the armour, and examined it all over, unable to
believe that he had really lost such a friend and companion in arms.
He did the same for Turpin, Miton, and Gautier de Luz. But of
what avail was it to lavish cares upon the lifeless clay? Their spirits
were in heaven.
Roland raised his head. He fancied he heard a faint but sweet
voice pronounce his name. What happiness if there yet survived
some one!
 “Do you not know me, my dear lord? Come hither and bid me
farewell!”
Pale, stretched on the field among the slain, lay the godchild of
Charlemagne.
“Heaven be praised, my pretty one! To see you still alive makes
me almost fancy Heaven smiles upon me. You will not die—I would
not be the cause of your death! Charles will be here soon, and will
bear you back to our own beloved France.”
“You deceive yourself, Roland. I shall never again behold the great
Emperor—never again my native land! Before long I shall meet my
father once more. But tell me, have the Saracens retreated?”
“They have retreated into Spain.”
“Then the victory belongs to us two! By the shrine of St. Landri! I
am happier than I ever dreamed of being.”
Roland knelt down, took off one of his great gold spurs, and fixed
it on Mitaine’s heel.
“There, brave little hero, none ever better merited the rank of
knight!” and he buckled it on. The two little feet of the squire would
have both fitted easily into the single spur.
In an ecstacy of joy, Mitaine raised herself, and flung her arms
round Roland’s neck.
“Quick, quick, my beloved lord! give me the accolade, for I feel I
am dying!”
And Mitaine sank back on the turf, plucked with a last effort two
blades of grass, which she fashioned into a cross, and expired while
kissing it with fervour.
 Original Size -- Medium-Size

Roland felt very solitary now. Feeling the shades of death

gathering round him, he stole up to Veillantif.
“My brave charger, your mouth is not meant for the bit of the
Saracen, nor your sides for the Pagan spur.”
And Roland, having kissed its soft muzzle, killed his favourite steed
with one blow of Durandal.
“Now, my treasured Durandal, what shall I do with thee? Thy hilt
encloses one of the teeth of St. Peter, and a hair from the beard of
St. Denis. Neither must thou fall into the hands of unbelievers!”
He called up all his strength, and struck his sword upon the
granite. It clave the rock, without denting its blade. Three times he
essayed again, but with no better success.
His sight was failing him. A cold chill seized him. He sank down
beside a granite peak, stretched upon his invincible sword, that
people might know well that he died a conqueror.
Roland had just ceased to breathe when Charlemagne arrived on
the field.
You will imagine, my young friends, that the Emperor made the
Saracens pay dearly for the loss of his knights. It was not until he
had utterly destroyed the infidel army that Charles would consent to
dismount from his horse on the plains of Roncesvalles. Alas! the
butchery of Saracens could not restore life to Roland or his
companions.
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookbell.com