Download the Full Ebook and Access More Features - ebooknice.

com

(Ebook) Distributed Optimization, Game and

Learning Algorithms: Theory and Applications in
Smart Grid Systems by Huiwei Wang, Huaqing Li, Bo
Zhou ISBN 9789813345270, 9813345276
https://ebooknice.com/product/distributed-optimization-game-
and-learning-algorithms-theory-and-applications-in-smart-
grid-systems-36373782

OR CLICK HERE

DOWLOAD EBOOK

Download more ebook instantly today at https://ebooknice.com

 Instant digital products (PDF, ePub, MOBI) ready for you
Download now and discover formats that fit your needs...

Start reading on any device today!

(Ebook) Distributed Optimization, Game and Learning

Algorithms: Theory and Applications in Smart Grid Systems
by Huiwei Wang, Huaqing Li, Bo Zhou ISBN 9789813345270,
9813345276
https://ebooknice.com/product/distributed-optimization-game-and-
learning-algorithms-theory-and-applications-in-smart-grid-
systems-36373784
ebooknice.com

(Ebook) Biota Grow 2C gather 2C cook by Loucas, Jason;

Viles, James ISBN 9781459699816, 9781743365571,
9781925268492, 1459699815, 1743365578, 1925268497
https://ebooknice.com/product/biota-grow-2c-gather-2c-cook-6661374

ebooknice.com

(Ebook) SAT II Success MATH 1C and 2C 2002 (Peterson's SAT

II Success) by Peterson's ISBN 9780768906677, 0768906679

https://ebooknice.com/product/sat-ii-success-
math-1c-and-2c-2002-peterson-s-sat-ii-success-1722018

ebooknice.com

(Ebook) Matematik 5000+ Kurs 2c Lärobok by Lena

Alfredsson, Hans Heikne, Sanna Bodemyr ISBN 9789127456600,
9127456609
https://ebooknice.com/product/matematik-5000-kurs-2c-larobok-23848312

ebooknice.com
(Ebook) Master SAT II Math 1c and 2c 4th ed (Arco Master
the SAT Subject Test: Math Levels 1 & 2) by Arco ISBN
9780768923049, 0768923042
https://ebooknice.com/product/master-sat-ii-math-1c-and-2c-4th-ed-
arco-master-the-sat-subject-test-math-levels-1-2-2326094

ebooknice.com

(Ebook) Cambridge IGCSE and O Level History Workbook 2C -

Depth Study: the United States, 1919-41 2nd Edition by
Benjamin Harrison ISBN 9781398375147, 9781398375048,
1398375144, 1398375047
https://ebooknice.com/product/cambridge-igcse-and-o-level-history-
workbook-2c-depth-study-the-united-states-1919-41-2nd-edition-53538044

ebooknice.com

(Ebook) Second-Order Consensus of Continuous-Time Multi-

Agent Systems by Huaqing Li, Dawen Xia, Qingguo Lu, Zheng
Wang, Xiangzhao Wu, Huiwei Wang, Lianghao Ji ISBN
9780323901314, 032390131X
https://ebooknice.com/product/second-order-consensus-of-continuous-
time-multi-agent-systems-34533510

ebooknice.com

(Ebook) Distributed Control and Optimization Technologies

in Smart Grid Systems by Fanghong Guo, Changyun Wen, Yong-
Duan Song ISBN 9781138088597, 9781315109732,
9781351613972, 1138088595, 1315109735, 1351613979
https://ebooknice.com/product/distributed-control-and-optimization-
technologies-in-smart-grid-systems-6837710

ebooknice.com

(Ebook) Engineering Applications of Discrete Element

Method: Operation Analysis and Optimization Design of Coal
and Agricultural Machinery by Xuewen Wang, Bo Li, Rui Xia,
Haozhou Ma ISBN 9789811579769, 9789811579776, 9811579768,
https://ebooknice.com/product/engineering-applications-of-discrete-
9811579776
element-method-operation-analysis-and-optimization-design-of-coal-and-
agricultural-machinery-22473792
ebooknice.com
Huiwei Wang
Huaqing Li
Bo Zhou

Distributed
Optimization,
Game and
Learning
Algorithms
Theory and Applications in Smart Grid
Systems
Distributed Optimization, Game and Learning
Algorithms
Huiwei Wang Huaqing Li Bo Zhou
• •

Distributed Optimization,
Game and Learning
Algorithms
Theory and Applications in Smart Grid
Systems

123
Huiwei Wang Huaqing Li
Southwest University Southwest University
Chongqing, China Chongqing, China

Bo Zhou
Chongqing Jiaotong University
Chongqing, China

ISBN 978-981-33-4527-0 ISBN 978-981-33-4528-7 (eBook)

https://doi.org/10.1007/978-981-33-4528-7

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature
Singapore Pte Ltd. 2021
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
herein or for any errors or omissions that may have been made. The publisher remains neutral with regard
to jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
To My Family
Huiwei Wang
To My Family
Huaqing Li
To My Family
Bo Zhou
Preface

Advances in wireless technology and computing power have necessitated the

development of theory, models, and tools to cope with the new challenges posed by
large-scale control and optimization problems over networks. The classical opti-
mization methodology works under the premise that all problem data are available
to a centralized server. However, this premise does not apply to large networked
systems in the distributed environment motivated by applications including power
systems, sensor networks, smart buildings, and smart manufacturing. In such an
environment, each node (agent) performs local computation based on its own data
(information) and information received from its neighboring agents through the
underlying communication network, so that the large-scale control and optimization
problem can be solved in a distributed manner. Eventually, the centralized opti-
mization methodology must surely slide into decline, bringing a new distributed
optimization type into being, which considers effective coordination between
multiple agents, i.e., all agents cooperate to minimize a global function which is a
sum of local objective functions.
This book investigates several standard hot topics in recent distributed opti-
mization problems, such as the unconstrained optimization, constrained optimiza-
tion, distributed game, and distributed/decentralized learning, etc. To emphasize the
role of the distributed optimization in these topics, we focus on a simple primal
(sub)gradient method, but we also provide an overview of other distributed methods
for optimization in networks. Applications of the distributed optimization frame-
work to the control of power systems are also presented. This book mainly includes,
naturally, three parts. The first part deals with the distributed optimization algorithm
theory, which consists of four chapters: (1) Cooperative distributed optimization in
multi-agent networks with delays; (2) Constrained consensus of multi-agent sys-
tems with time-varying topology; (3) Distributed optimization under inequality
constraints and random projections; and (4) Accelerated distributed optimization
over digraphs with stochastic matrices. The second part, as a transition, is con-
cerned with the distributed optimization algorithm theory and their applications in
dynamic economic dispatch problems of smart grid systems, which includes two
chapters: (5) Linear convergence for constrained optimization over time-varying

vii
viii Preface

digraphs; and (6) Stochastic gradient-push for economic dispatch on time-varying

digraphs. The analysis and synthesis of distributed optimization, game, and learning
algorithm theory are treated in the third part, all algorithms in this part are designed
for the targeted case scenarios within smart grid systems. This part consists of three
chapters: (7) Reinforcement learning in energy trading game among smart micro-
grids; (8) Reinforcement learning for constrained games with incomplete infor-
mation; and (9) Reinforcement learning for PHEV route choice based on congestion
game. Among the topics, simulation results including practical application exam-
ples are presented to illustrate the effectiveness and the practicability of the opti-
mization, game, and learning algorithms proposed in the previous parts.
This book is appropriate as a college course textbook for undergraduate and
graduate students majoring in computer science, automation, artificial intelligence,
electric engineering, etc., and as a reference material for researchers and technol-
ogists in related fields.

Chongqing, China Huiwei Wang

Huaqing Li
Bo Zhou
Acknowledgements

This book was supported in part by the National Natural Science Foundation of
China under Grants 61773321 and 61803056, in part by the Fundamental Research
Funds for the Central Universities under Grant XDJK2018B013, in part by China
Postdoctoral Science Foundation under Grant 2017M620374, and in part by the
National Natural Science Foundation of Chongqing under Grant cstc2020jcyj-
msxmX0057. We would like to begin by acknowledging Ziyu Sheng, Xing Liu,
Lianjiao Wang, Yifan Yu, Junjie Lv, Yang Yun, and Yan Zuo who have unselfishly
given their valuable time in arranging raw materials. Their assistance has been
invaluable for the completion of this book. The authors are especially grateful to
their families for their encouragement and never-ending support when it was most
required. Finally, we would like to thank the editors at Springer for their profes-
sional and efficient handling of this book.

ix
Contents

1 Cooperative Distributed Optimization in Multiagent Networks

with Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Preliminary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.6 Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2 Constrained Consensus of Multi-agent Systems with Time-Varying
Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.1 Model Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.2 State Transition Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.3 Constrained Consensus . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.4 Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.4.1 Constrained Consensus of a Simple Directed Multi-agent
System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 30
2.4.2 Constrained Consensus of a Large-Scale Directed
Multi-agent System . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 33
2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 36
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 36

xi
xii Contents

3 Distributed Optimization Under Inequality Constraints

and Random Projections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.2 Problem Formulation and Assumptions . . . . . . . . . . . . . . . . . . . . 41
3.2.1 Network Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2.3 Lagrangian Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2.4 Distributed Primal-Dual Random Projection Subgradient
Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.3 Basic Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.4 Convergence Results for Diminishing Stepsize . . . . . . . . . . . . . . . 50
3.5 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4 Accelerated Distributed Optimization over Digraphs
with Stochastic Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.2 Preliminaries and Problem Formulation . . . . . . . . . . . . . . . . . . . . 69
4.3 Algorithm Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.4 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.4.1 Auxiliary Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.4.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.5 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5 Linear Convergence for Constrained Optimization
Over Time-Varying Digraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.2.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.2.2 Communication Network . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.2.3 Necessary Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.3 Main Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.3.1 Centralized Primal-Dual Method . . . . . . . . . . . . . . . . . . . . 88
5.3.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.3.3 Distributed Primal-Dual Push-DIGing Algorithm . . . . . . . . 91
5.3.4 Supporting Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.4 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.5 Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.5.1 Case Study 1: Simulation for Five-Agents Network . . . . . . 101
5.5.2 Case Study 2: Simulation for Large-Scale Network . . . . . . 103
5.5.3 Case Study 3: Performance Comparison . . . . . . . . . . . . . . 105
Contents xiii

5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6 Stochastic Gradient-Push for Economic Dispatch on Time-Varying
Digraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6.2.1 Graph Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6.2.2 Economic Dispatch Problem . . . . . . . . . . . . . . . . . . . . . . . 112
6.2.3 Centralized Lagrangian-Based Method . . . . . . . . . . . . . . . 113
6.3 Algorithm and Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
6.3.1 State-of-the-Art Method for EDP . . . . . . . . . . . . . . . . . . . 114
6.3.2 Distributed Stochastic Gradient-Push Algorithm . . . . . . . . 115
6.3.3 Necessary Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 117
6.3.4 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
6.4 Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.4.1 Supporting Lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.4.2 Proof of Theorem 6.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
6.4.3 Proof of Theorem 6.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
6.4.4 Proof of Theorem 6.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
6.5 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.5.1 Without Time Delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.5.2 With Time Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
7 Reinforcement Learning in Energy Trading Game Among Smart
Microgrids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
7.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.2.1 Smart Grid Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.2.2 Specifical Utility Functions . . . . . . . . . . . . . . . . . . . . . . . 141
7.2.3 Stackelberg Game Scheme . . . . . . . . . . . . . . . . . . . . . . . . 142
7.3 Utility Maximization and Best Strategy . . . . . . . . . . . . . . . . . . . . 143
7.3.1 Utility Maximization and the Best Action . . . . . . . . . . . . . 144
7.3.2 Learning the Best Response . . . . . . . . . . . . . . . . . . . . . . . 145
7.3.3 Properties of Stackelberg Equilibrium . . . . . . . . . . . . . . . . 147
7.4 Learning Algorithms Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.4.1 Finite Action Learning Automaton (FALA) . . . . . . . . . . . . 147
7.4.2 Continuous Action Learning Automaton (CALA) . . . . . . . 149
7.5 Numerical Case Studies and Discussion . . . . . . . . . . . . . . . . . . . . 150
7.5.1 Case Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
7.5.2 Buyer Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
xiv Contents

7.5.3 Seller Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

7.5.4 Stackelberg Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
7.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
8 Reinforcement Learning for Constrained Games with Incomplete
Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
8.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
8.2.1 Smart Grid Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
8.2.2 Double Auction Mechanism . . . . . . . . . . . . . . . . . . . . . . . 165
8.3 Learning-Based Game to Energy Trading . . . . . . . . . . . . . . . . . . . 166
8.3.1 Strategy Updating as Learning Automaton . . . . . . . . . . . . 167
8.3.2 Repeated Game with Incomplete Information . . . . . . . . . . 167
8.4 Mixed-Strategy NE and Learning Games . . . . . . . . . . . . . . . . . . . 169
8.4.1 Mixed Strategy and Nash Equilibrium . . . . . . . . . . . . . . . 169
8.4.2 Stationary Strategies and Convergence Analysis . . . . . . . . 170
8.5 Properties of Nash Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . 172
8.5.1 Existence of NE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
8.5.2 Diagonal Concavity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
8.5.3 Lagrange Multiplier and Uniqueness of NE . . . . . . . . . . . . 178
8.6 Learning Algorithm Design and Analysis . . . . . . . . . . . . . . . . . . . 179
8.7 Numerical Case Studies and Discussion . . . . . . . . . . . . . . . . . . . . 182
8.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
9 Reinforcement Learning for PHEV Route Choice Based
on Congestion Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
9.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
9.2.1 Route Choice Problem in Traffic Networks . . . . . . . . . . . . 194
9.2.2 Learning Automaton . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
9.3 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
9.3.1 Existence and Uniqueness of Optimal Route Choice
Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
9.3.2 Reinforcement Learning Scheme . . . . . . . . . . . . . . . . . . . . 199
9.4 Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
9.4.1 Experiment Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 207
9.4.2 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
9.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
List of Figures

Fig. 1.1 Plots of function error versus number of iterations for a random
geometric graph with N ¼ 5 nodes . . . . . . . . . . . . . . . . . . . . . .. 15
Fig. 2.1 Asynchronous communication topologies of a simple
multi-agent system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 31
Fig. 2.2 State evolution of the agents in a simple multi-agent system . .. 32
Fig. 2.3 State evolution of the agents in a simple multi-agent system . .. 32
Fig. 2.4 Asynchronous communication topologies of relatively large
multi-agent systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 34
Fig. 2.5 The union of the communication topologies . . . . . . . . . . . . . . .. 34
Fig. 2.6 Evolutions of the states of the agents of relatively
large multi-agent systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 35
Fig. 2.7 Evolutions of the states of the agents of relatively
large multi-agent systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 35
Fig. 3.1 Communication topology of the multiagent systems
with 4 agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Fig. 3.2 Estimates of variable x1 of DPDRPS algorithm . . . . . . . . . . . . . 62
Fig. 3.3 Estimates of variable x2 of DPDRPS algorithm . . . . . . . . . . . . . 62
Fig. 4.1 Performance comparison across different algorithms . . . . . . . . . . 80
Fig. 5.1 A time-varying directed unbalanced network GðkÞ that
switches among three different topologies G1 , G2 and G3 , i.e.,
Gð3kÞ ¼ G1 , Gð3k þ 1Þ ¼ G2 and Gð3k þ 2Þ ¼ G3
for all k  0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Fig. 5.2 The estimates of optimal power allocation for Case Study 1 . . . 102
Fig. 5.3 The estimates of optimal Lagrange multipliers for Case
Study 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Fig. 5.4 The estimate of optimal state allocation x for Case Study 2 . . . . 104
Fig. 5.5 The estimate of optimal Lagrange multipliers for Case
Study 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
Fig. 5.6 Performance comparison over time-varying directed
unbalanced networks for Case Study 3 . . . . . . . . . . . . . . . . . . . . 106
Fig. 6.1 Time-varying directed communication networks . . . . . . . . . . . . . 127

xv
xvi List of Figures

Fig. 6.2 Simulation results of the 4-bus system with synchronous

step-size but without noisy gradient . . . . . . . . . . . . . . . . . . . . . . 129
Fig. 6.3 Simulation results of the 4-bus system with asynchronous
step-sizes and noisy gradient. . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Fig. 6.4 Simulation results of the 4-bus system with synchronous
step-size but without noisy gradient under time delays . . . . . . . . 131
Fig. 6.5 Simulation results of the 4-bus system with asynchronous
step-sizes and noisy gradient under time delays . . . . . . . . . . . . . 132
Fig. 7.1 Energy trading among microgrids . . . . . . . . . . . . . . . . . . . . . . . . 140
Fig. 7.2 Microgird test system based on IEEE 37-bus test feeder . . . . . . 151
Fig. 7.3 DGs output and load over 24 h . . . . . . . . . . . . . . . . . . . . . . . . . 152
Fig. 7.4 Probability of price bid action and power injection versus
iteration number, c ¼ 0:1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Fig. 7.5 Average action and average utility versus iteration number,
c ¼ 0:1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
Fig. 7.6 Performance of the CALA-based learning algorithm versus
iteration number, c ¼ 106 , C ¼ 1 and rL ¼ 103 . . . . . . . . . . . 155
Fig. 7.7 Probability of proportion action and power abortion versus
iteration number, c ¼ 0:1 and k ¼ 104 . . . . . . . . . . . . . . . . . . . 156
Fig. 7.8 Evolution of payoff radius in comparison with probability
of action versus actions, c ¼ 0:1, t ¼ 102 . . . . . . . . . . . . . . . . . . 157
Fig. 7.9 Performance of the CALA-based learning algorithm versus
iteration number, c ¼ 106 , C ¼ 1 and rL ¼ 103 . . . . . . . . . . . 157
Fig. 7.10 Average action and average utility of buyers versus round
number, t ¼ 103 and 100 experiments . . . . . . . . . . . . . . . . . . . . 158
Fig. 7.11 Average action and average utility of sellers versus round
number, t ¼ 103 and 100 experiments . . . . . . . . . . . . . . . . . . . . 158
Fig. 8.1 Dynamic pricing mechanism via double auction . . . . . . . . . . . . . 165
Fig. 8.2 A sketch of the expected utility . . . . . . . . . . . . . . . . . . . . . . . . . 174
Fig. 8.3 Energy trading test system based on IEEE 13-bus test
feeder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Fig. 8.4 The initialized reservation price and the evolution
of the trading price . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Fig. 8.5 The evolution of the probability distribution of actions
for each seller i ¼ f1; 2; 3; 4g in turn . . . . . . . . . . . . . . . . . . . . . 184
Fig. 8.6 Real-time power abortion versus power supply. . . . . . . . . . . . . . 185
Fig. 8.7 Real-time power injection versus power demand . . . . . . . . . . . . 186
Fig. 8.8 Averaged trading quantity and utility of each seller . . . . . . . . . . 187
Fig. 9.1 The Nguyen–Dupuis network . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
Fig. 9.2 Traffic flows at equilibrium on each route . . . . . . . . . . . . . . . . . 208
Fig. 9.3 The evolutions trajectories of route choice strategies
of the PHEVs on Route 2 for different solution methods . . . . . . 209
Fig. 9.4 The evolutions trajectories of route choice strategies
of the PHEVs on Route 4 for different solution methods . . . . . . 210
List of Figures xvii

Fig. 9.5 The evolutions trajectories of route choice strategies

of the PHEVs on Route 14 for different solution methods . . . . . 211
Fig. 9.6 The evolutions trajectories of route choice strategies
of the PHEVs on Route 19 for different solution methods . . . . . 212
Fig. 9.7 The evolutions of the total travel times of the PHEVs in traffic
networks for the four solution methods. The red line, green
line, blue line and dark line present the evolutions of the total
travel times with the B–M RL-based solution method, the
multinomial logit choice model, the improved learning
automata and the modified Q-learning algorithm,
respectively . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Fig. 9.8 The proportion of PHEVs who choose the routes for Case
I-Case IV, in which the red bar, the blue bar, the yellow bar
and the dark bar represent the proportions for Case I, Case II,
Case III and Case IV, respectively . . . . . . . . . . . . . . . . . . . . . . . 214
Fig. 9.9 The running time of the four different solution methods
with respect to the four cases. The red bar, green bar, blue bar
and dark bar present the running time of the B–M RL-based
solution method, the multinomial logit choice model,
the improved learning automata and modified Q-learning
algorithm, respectively . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
Chapter 1
Cooperative Distributed Optimization in
Multiagent Networks with Delays

Abstract In this chapter, we consider a distributed cooperative optimization prob-

lem encountered in a computational multi-agent network with delay, where each
agent has local access to its convex cost function, and jointly minimizes the cost
function over the whole network. To solve this problem, we develop an algorithm
that is based on dual averaging updates and delayed subgradient information, and
analyze its convergence properties for a diminishing step-size by utilizing Bregman-
distance functions. Moreover, we provide sharp bounds on the convergence rates as
a function of the network size and topology embodied in the inverse spectral gap.
Finally, we present a numerical example to evaluate our algorithm and compare its
performance with several similar algorithms.

Keywords Delayed subgradient · Distributed optimization · Multi-agent

networks · Spectral gap

1.1 Introduction

The distributed computation of the average of datasets and the coordination of a set
of agents in optimization tasks, i.e., driving all the agents collectively to optimize a
global objective, are ubiquitous tasks in current engineering problems. Distributed
computation over networks has been extensively studied in systems and control the-
ory starting with the pioneering work of Tsitsiklis [1]. Two decades later, an excellent
work [2] introducing the theoretical framework for posing and solving consensus
problems has recently attracted much attention [3–6]. Exploiting the consensus idea
or computing exact averages of the initial values of the agents, recent works [7–14]
have investigated a distributed model for optimization over a network. For more
details, the readers may refer to the recent book [15] and the references therein.
Primary Motivations: The distributed optimization algorithms presented in earlier
literatures [7, 9–12] assume that at any time, each agent has access to estimate the
states of its immediate neighbors. As pointed out in [15], this may not be possible in
communication networks where delays exist in the transmission of agent estimates
over a communication channel. Establishing the convergence rate properties of the
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 1
H. Wang et al., Distributed Optimization, Game and Learning Algorithms,
https://doi.org/10.1007/978-981-33-4528-7_1
2 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

update rule is essential in understanding the robustness of the optimization algorithm

to delays and dynamics associated with information exchange over finite bandwidth
communication channels. In this brief, we will deal with this challenging scenario.
Literature Review: Distributed algorithms possess the inherent advantage in solv-
ing large scale optimization and machine learning problems, which are attracting
more and more researchers to this field. In a necessarily incomplete list of references
including [7, 8, 15], the authors described their approaches that project the original
update iteration onto the local constrain sets by using distributed subgradient algo-
rithms to deal with a non-differentiable convex objective function. Further theoretical
extensions of this work were presented in [9] with a look toward treatment of iden-
tical local constraint sets. Building on some earlier works, a distributed primal-dual
subgradient method was studied in [10, 11] by characterizing the primal and dual
optimal solutions as the saddle points of the Lagrangian function associated with the
problem. In addition, the primal and dual optimal networks were also investigated
for solving the assignment problem [16, 17]. Recently, a distributed dual averaging
(DDA) algorithm based on standard dual averaging [18] was proposed in [12] for
distributed optimization to investigate the effects of network size and topology on the
convergence rate. Further extensions of this work were presented in [13, 14, 19] to
handle distributed optimization problem over a network with communication delays
or quantized communication. In addition, the convergence of decentralized network
optimization algorithms that are based on their updates and delayed information
projection were analyzed in [20, 21].
Statement of Contributions: The main contributions of this chapter are twofold.
The first contribution is the extension of the DDA algorithm to tackle the distributed
non-smooth minimization problem, where the procedure receives out-of-date gra-
dients instead of current gradients. By establishing several iteration properties for
delayed subgradients, we prove that the algorithm asymptotically achieves to the
optimal value as the iteration approaches infinity in spite of delays. The main advan-
tage of this method is to preserve the performance benefits of the stale gradient
information, which will help the optimization error achieve the given accuracy faster
in many situations of practical interest. The second main contribution is a concise
analysis that investigates the effects of the delay, network size and the underlying
spectral properties of the network on the convergence rate. By utilizing the Bregman
divergence and a novel bounding technique, we provide an elegant illustration of the
role played by the delay in convergence rates: if the subgradients are
√ computed with
a delay of τ , then the convergence rate of the procedure is O(τ 2 / T ). It is easy to
observe that the optimization error caused by delay has become a second-order term:
the delay penalty is asymptotically negligible.
1.2 Related Work 3

Table 1.1 Extra costs due to delays

Literature Storage for nodes Computation for edges
[13, 14] τ |E | τ |E |
This chapter τN –

1.2 Related Work

In this section, we survey some previous work with the aim of giving a clear under-
standing of how our algorithm and results relate to and, in many cases, improve upon
it. Our work is closest to [12–14] building on the dual averaging method [18].
We present the algorithm similar to the initial discovery [13, 14], but the dif-
ference is the position of the communication delays in the DDA algorithm. In [13,
14], the authors proposed the DDA algorithm with delayed dual variable updates
to handle the distributed optimization problem over a network with communication
delays. It suffices to add finitely many virtual agents and communication links instead
of delayed transmission, then the original delayed multi-agent model is reduced to
a new augmented model without delays. Although the method in [13, 14] is ele-
gant, it leads to computational and storage burdens due to the redundant virtual
agents and communication links. Unlike [13, 14], the gradient projection steps in
this chapter are determined by the out-of-date subgradients. Compared with their
works, our algorithm directly handles the communication delay, which has tapped
the maximum potential to the utmost from the delayed gradient information with less
computational and storage requirement, as shown in Table 1.1, where τ represents
the communication delay and |E| denotes the cardinality of the edge set E. In general,
|E| = Θ(N log(N )) holds for a random geometric graph with N nodes.
As discussed in the introduction, we extend the algorithm in [12] to handle com-
munication delays, but the real-time gradient information of the objective function
is not available due to delays. In order to overcome this challenge, we bound the
inner product including delayed subgradients to meaningfully offset the effect of
delays by some novel techniques. These are primarily different from previous tech-
niques [12] and even [13, 14]. As a result, the error caused by the delays becomes
a second-order term, which means the delay penalty is asymptotically negligible. In
addition, despite of using different intermediate techniques and bounding methods,
we theoretically improve the convergence rate presented in [12] by a logarithmic
factor log(T ). Meanwhile, for a well connected network such as a bounded degree
expander, we also improve the result in [8] that provides an -optimal solution to
be independent of the network topology which can be reached in O(N 3 /2 ) time.
Finally, it is worth pointing out that our algorithm can preserve the performance gain
of the stale gradient information, which might help the optimization error to quickly
achieve the given accuracy in many situations of practical interest.
4 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

1.3 Preliminary

Consider a network of N agents, indexed by i = 1, . . . , N . The communication

topology is modeled by an undirected graph G = (V, E) over the vertex set V with
edge set E ⊂ V × V. Here, we assume the underlying communication graph G is
connected and without self loops. The objective of N agents is to cooperatively
solve the following optimization problem:

1 
N
minn f i (x) s.t. x ∈ ∩i=1
N
Xi (1.1)
x∈R N i=1

where f i : Rd → R is the objective function associated with agent i, x is a global

decision vector and Xi ⊆ Rd is the nonempty compact and convex set. Each agent i
maintains its own parameter vector xi ∈ Rd and has local access only to the objective
function f i and can only communicate directly with its immediate neighbors j ∈
N (i)  { j ∈ V|(i, j) ∈ E}. Assume that each function f i is convex and hence sub-
N
differentiable, but it needs not be smooth. We denote f (x)  (1/N ) i=1 f i (x)
and X  ∩i=1 Xi . Thus, f is convex and X is compact. To build intuition for the
N

algorithms we analyze, we begin by listing the notations and giving useful definitions
and assumptions.
Notations: The inner product of two vectors is denoted as x, y = x T y, and the
standard Euclidean norm is x = x, x 1/2 . The dual norm · ∗ to the standard
Euclidean norm is defined by v ∗  sup u =1 v, u .

Definition 1.1 A vector ∇h(x) is a subgradient of a function h at a point x in the

domain of h (denoted by dom h) if the following inequality holds

∇h(x)T (y − x) ≤ h(y) − h(x) for all y ∈ dom h. (1.2)

Definition 1.2 The function h(x) is m-strongly convex in that there exists a constant
m > 0 such that for any pair of points x ∈ X and y ∈ X , the following inequality
holds m
h(y) ≥ h(x) + ∇h(x)T (y − x) + y − x 2. (1.3)
2

Definition 1.3 ([22]) For a given compact, proper convex functional ψ(·) : Rd → R,
the Bregman divergence is defined as

Dψ (x, y) = ψ(x) − ψ(y) − ∇ψ(y), x − y . (1.4)

Assumption 1.1 For any x, y ∈ X ,

(a) Each function f i is L-Lipschitz with respect to the same norm · , i.e.,

| f i (x) − f i (y)| ≤ L x − y .
1.3 Preliminary 5

(b) The subgradient ∇ f i is G-Lipschitz continuous with respect to the norm · ,

i.e.,
G
f i (y) ≤ f i (x) + ∇ f i (x), y − x + x − y 2.
2
We note that a convex function f i is L-Lipschitz if and only if the following
supx∈X ∂ f i (x(t)) ∗ ≤ L holds. This means that, for any x ∈ X and any subgra-
dient gi (t) ∈ ∂ f i (x(t)), we have

gi (t) ∗ ≤ L. (1.5)

Assumption 1.2 For x∗ ∈ arg minx∈X f (x) and R ∈ R, the bounds ψ(x∗ ) ≤ R 2 and
Dψ (x∗ , x) ≤ R 2 for any x ∈ X are both valid.

We are now in a position to describe the DDA algorithm based on delayed subgra-
dients for solving the multi-agent cooperative optimization problem. The algorithm
is based on a proximal function ψ(x) : Rd → R. There is no loss of generality to
assume that ψ(x) is 1-strongly convex and ψ(x) ≥ 0 for all x ∈ X and that ψ(0) = 0.
At time step t, the delayed subgradient information is received, and the sequence
{z i (t), xi (t)}∞
t=0 is updated via the following iterations:


N
z i (t + 1) = Wi j z j (t) + gi (t − τ ), (1.6)
j=1

ψ
xi (t + 1) = ΠX (z i (t + 1), α(t + 1)), (1.7)

where Wi j ≥ 0 is a weight that agent i assigns to its neighbor j ∈ Ni , gi (t − τ ) ∈

∂ f i (xi (t − τ )), τ is a positive integer denoting the constant delay. Moreover,
 
ψ 1
ΠX (z i (t + 1), α(t + 1)) = arg min z i (t + 1), x + ψ(x)
x∈X α(t + 1)

is a type of projection with a non-increasing sequence of positive step sizes {α(t)}∞

t=0 .
In order to prove the main results, we show a property of the algorithm. For a
doubly stochastic matrix W , the distribution W t ei − 1/N 1 satisfies the following
inequality:
√
W t ei − 1/N 1 ≤ N W t ei − 1/N 2
√ √
= N W t (ei − 1/N ) 2 ≤ N [σ2 (W )]t ei − 1/N 2
√
≤ N [σ2 (W )]t , (1.8)

where ei denotes the ith standard basis vector, and σ2 (W ) is the second largest
singular value of W .
6 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

1.4 Main Results

section, we will state the main results of our work. By denoting 

In this  xi (T ) 
T
(1/T ) t=1 xi (t), we have the following theorems.

Theorem 1.4 Suppose that both Assumptions 1.1 and 1.2 hold. Let the sequence
{xi (t)}∞
t=0 be generated by the updates (1.6) and (1.7). Then

xi (T )) − f (x ∗ ) ≤ OT + NT,
f (

where

2(τ + 1)2 G R 2 
T
2(τ + 1)L R R2
OT = + + [α(t)]2 , and
T T α(T ) T t=1

√
6 L2  38(τ + 1)2 N G L 2 
T T
N
NT = α(t) + [α(t)]2 .
1 − σ2 (W ) T t=1 [1 − σ2 (W )]2 T t=1

It should be noticed that Theorem 1.4 gives a common result accounting for
the effects of the network communication cost (NT) and the optimization term
(OT)on the convergence rate. By choosing a proper step size sequence satisfy-
ing ∞ t=0 α(t) = ∞ and lim t→∞ α(t) = 0, it is shown that {x i (t)} asymptotically
approaches
∞ a common value for any i, which no longer needs to satisfy the condition
t=0 [α(t)] 2
< ∞ stated in [7]. Based on this fact and Theorem 1.4, we obtain the
following result for demonstrating that the convergence rate is closely related to the
spectral gap 1 − σ2 (W ) of the iteration matrix W . It should be pointed out that by
constructing a proper doubly stochastic matrix W , the spectral gap is equivalent to the
algebraic connectivity of graph G, say, the second smallest eigenvalue of Laplacian
matrix.

Theorem 1.5 Suppose that Assumptions 1.1 and 1.2 are valid. With step size choice
√ 2 (W
α(t) = R[1−σ √ ,
)]
9 NL t

√
∗ N (τ + 1)2
xi (T )) − f (x ) ≤ Ξ
f ( √ ,
1 − σ2 (W ) T

where Ξ = 13R L + G R 2 + (G R 4 /L 2 ).

This theorem establishes a tight connection between the convergence rate of the
DDA algorithm and the spectral properties of the underlying networks. Based on the
above analysis, it is easy to obtain the convergence rates for some special network
type as shown in Table 1.2. The detailed analysis can be found in [12]. In order to
understand the relationship among the convergence rate, network size and topology,
we replace the left-hand side of the inequality in Theorem 1.5 with a desired error
1.4 Main Results 7

Table 1.2 Convergence rate for special network topologies

Network type xi (T )) − f (x ∗ )
f (
 √ 
O Ξ (τ√+1) N k 2 N
2 2
k-connected paths and cycles
T
√ √  √ 
O Ξ (τ√+1) N k 2 N
2
k-connected N × N grids
T
 √ 
O Ξ (τ√+1) Nlog NN
2
Random geometric graphsa
T
 2√

Expandersb O Ξ (τ√+1) N
T
√
a Random geometric graphs are assumed with connectivity radius r =Ω log1+ N / N for any
>0
b Expanders are assumed with bounded ratio of minimum to maximum node degree

accuracy , and then obtain the upper bound of the number of iterations TG (; N ) as
follows:
(a) For the single cycle graph, Tcy (; N ) = O(N 5 /2 );
(b) For the two-dimensional grid, Tgr (; N ) = O(N 3 /2 );
(c) For a bounded expander, Tex (; N ) = O(N /2 ).
In general, Theorem 1.5 implies that at most

N 1
TG (; N ) = O (1.9)
2 [1 − σ2 (W[G] )]2

iterations are required to achieve an -accurate solution. It is generally known that any
subgradient methods (even for centralized optimization algorithms) require at least
Ω(1/2 ) iterations to achieve -accuracy [23], so that the 1/2 term is unavoidable.
Meanwhile, the square of the inverse spectral gap term is also difficult to improve
due to bounding the interval of the decision variable.

1.5 Convergence Analysis

In this section, we investigate the convergence properties of the

distributed algorithm.
N
We start by defining two auxiliary sequences z̄(t)  (1/N ) i=1 z i (t) and y(t) 
ψ
ΠX (−z̄(t), α(t)). Since W is doubly stochastic, we have

1  1 
N N N
z̄(t + 1) = Wi j (t)z j (t) + gi (t − τ )
N i=1 j=1 N i=1

1 
N
=z̄(t) + gi (t − τ ). (1.10)
N i=1
8 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

We introduce a subgradient error vector ei (t) = gi (t) − gi (t − τ − 1) and the transi-

tion matrix (t, s) = W t−s+1 . These notations and the following lemmas are useful
for the proof of Theorem 1.4.
Lemma 1.6 ([12]) For an arbitrary pair u, v ∈ Rd , we have
ψ ψ
ΠX (u, α) − ΠX (v, α) ≤ α u − v ∗ . (1.11)

Lemma 1.7 ([24]) Let x + minimize z, x + Aψ(x) for all x ∈ X . Then for any
x ∈ X,
z, x + Aψ(x) ≥ z, x + + Aψ(x + ) + ADψ (x, x + ).

Now we state a general property of the DDA algorithm, which allows us to bound
the difference between xi (t) and xi (t + 1).
Lemma 1.8 For any i ∈ V and any x ∗ ∈ X ,

xi (t) − xi (t + 1) ≤ α(t) z i (t) − z̄(t) ∗

+ α(t + 1) z i (t + 1) − z̄(t + 1) ∗ + α(t)[L + ψ(x ∗ )].

Proof For any x ∗ ∈ X , applying Lemma 1.6 yields

xi (t) − xi (t + 1)
≤ xi (t) − y(t) + y(t + 1) − xi (t + 1)
+ y(t) − y(t + 1)
≤ xi (t) − y(t) + y(t + 1) − xi (t + 1)
ψ
+ y(t) − ΠX (−z̄(t + 1), α(t)) + [α(t) − α(t + 1)]ψ(x ∗ )
≤α(t) z i (t) − z̄(t) ∗ + α(t + 1) z i (t + 1) − z̄(t + 1) ∗
+ α(t) z̄(t) − z̄(t + 1) ∗ + [α(t) − α(t + 1)]ψ(x ∗ )
<α(t) z i (t) − z̄(t) ∗ + α(t + 1) z i (t + 1) − z̄(t + 1) ∗
∗
+ α(t)L + α(t)ψ(x ).

The lastinequality is derived from the relationships that z̄(t + 1) − z̄(t) ∗ =

N
(1/N ) i=1 gi (t − τ ) ∗ ≤ L and ψ(·) ≥ 0. Hence, we complete the proof.

From Lemma 1.8, it is easy to observe that the decision variable sequence {xi (t)}
is convergent if limt→∞ α(t) = 0 and the sequence { z i (t) − z̄(t) ∗ } is bounded for
any time step t. The next lemma is useful in deriving the bound of the sequence
{ z i (t) − z̄(t) ∗ } for all i. A similar version of this lemma can be found in [12]. The
only difference is that we develop a constant bound independent of T , which is more
convenient in analyzing the complexity of the algorithm.

Lemma 1.9 Suppose that Assumptions 1.1 and 1.2 hold. Then, for the iterations of
the algorithms (1.6) and (1.7) we have
1.5 Convergence Analysis 9
√
3 NL
z i (t) − z̄(t) ∗ < .
1 − σ2 (W )

Proof From the proof of Theorem 2 [12], we can obtain

t−1 
 
 
z i (t) − z̄(t) ≤L [(t − 1, )]i − 1  + 2 L .
∗  N 1
=1

√
From (1.8), [(t − 1, )]i − 1/N 1 ≤ N [σ2 (W )]t− and noting that 0 < σ2 (W )
< 1, a simple computation yields

√ σ2 (W ) − [σ2 (W )]t
z i (t) − z̄(t) ∗ ≤ NL +2L
1 − σ2 (W )
√ √
( N + 2)L 3 NL
< ≤ .
1 − σ2 (W ) 1 − σ2 (W )

We then obtain the desired result and complete the proof.

In order to simplify the proof, the following two lemmas are separated from the
proof of main results as two independent properties of the distributed delayed dual
averaging algorithm.
Lemma 1.10 For any x ∗ ∈ X ,

1 
T N
1
g j (t − τ − 1), y(t) − x ∗ ≤ ψ(x ∗ ).
N t=1 j=1 α(T )

Proof From Lemma 1.7 and the expression (1.7), we have

−z̄(t − 1), y(t) ≤ − z̄(t − 1), y(t − 1)

1
+ [ψ(y(t)) − ψ(y(t − 1))]
α(t − 1)
1
− Dψ (y(t), y(t − 1)). (1.12)
α(t − 1)
N
By utilizing the iteration z̄(t) = z̄(t − 1) + N1 i=1 gi (t − τ − 1) from (1.10) and
the non-negativity of Bregman divergence, i.e., Dψ (y(t), y(t − 1)) ≥ (1/2) y(t −
1) − y(t) 2 ≥ 0, it follows from (1.12) that
10 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

1 
N
g j (t − τ − 1), y(t) − x ∗
N j=1
=z̄(t), y(t) − x ∗ − z̄(t − 1), y(t) − x ∗
≤z̄(t), y(t) − x ∗ − z̄(t − 1), y(t − 1) − x ∗
1
+ [ψ(y(t)) − ψ(y(t − 1))]. (1.13)
α(t − 1)

Summing over the expression (1.13), we have

1 
T N
g j (t − τ − 1), y(t) − x ∗
N t=1 j=1
1
≤z̄(T ), y(T ) − x ∗ + ψ(y(T ))
α(T − 1)
T  
1 1
+ − ψ(y(t − 1))
t=2
α(t − 2) α(t − 1)
1
=z̄(T ), y(T ) + ψ(y(T ))
α(T )
1
− z̄(T ), x ∗ − ψ(x ∗ )
α(T )
 
1 1 1
+ ψ(x ∗ ) + − ψ(y(T ))
α(T ) α(T − 1) α(T )
T  
1 1
+ − ψ(y(t − 1)).
t=2
α(t − 2) α(t − 1)

Since y(T ) minimizes z̄(T ), y + 1

α(T )
ψ(y) for any y ∈ X , it can be derived that

1 
T N
g j (t − τ − 1), y(t) − x ∗
N t=1 j=1
1
≤y(T ) − y(T ) + ψ(x ∗ )
α(T )
 
1 1
+ − ψ(y(T ))
α(T − 1) α(T )
 T  
1 1
+ − ψ(y(t − 1))
t=2
α(t − 2) α(t − 1)
1
≤ ψ(x ∗ ).
α(T )
1.5 Convergence Analysis 11

The last inequality follows by the fact that {α(t)}∞

t=1 is a non-increasing sequence
and ψ(·) ≥ 0. The proof is completed.

Lemma 1.11 For x ∗ ∈ arg min x∈X f j (x),

1  
T N T
e j (t), x j (t) − x ∗ ≤ 2(τ + 1)L R + Θ [α(t)]2
N t=1 j=1 t=1

where
N G L2
Θ = 38(τ + 1)2 + 2(τ + 1)2 G[ψ(x ∗ )]2 .
[1 − σ2 (W )]2

Proof Recalling the subgradient error vector and the four term equality of a Bregman
divergence, we have

e j (t), x j (t) − x ∗
= g j (t) − g j (t − τ − 1), x j (t) − x ∗
= D f j (x ∗ , x j (t)) − D f j (x ∗ , x j (t − τ − 1))
− D f j (x j (t), x j (t)) + D f j (x j (t), xi (t − τ − 1)). (1.14)

By the G-Lipschitz continuity of ∇ f j , one obtains

f j (x j (t)) ≤ f j (x j (t − τ − 1))
+ g j (t − τ − 1), x j (t) − x j (t − τ − 1)
G
+ x j (t − τ − 1) − x j (t) 2 . (1.15)
2

which implies that D f j (x j (t), x j (t − τ − 1)) ≤ (G/2) x j (t − τ − 1) − x j (t) 2 .

Summing over the expression (1.14) yields that

1 
T N
e j (t), x j (t) − x ∗
N t=1 j=1

1  
T N
≤ D f (x ∗ , x j (t))
N t=T −τ j=1 j

1  G
T N
+ x j (t − τ − 1) − x j (t) 2 . (1.16)
N t=1 j=1 2

It follows from Assumption 1.2 that for x ∗ ∈ arg min x∈X f j (x), x ∗ − x j (t) 2 ≤
2Dψ (x ∗ , x j (t)) ≤ 2R 2 . Combining this relationship with the optimality of x ∗ implies
that
12 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

D f j (x ∗ , x j (t)) = f j (x ∗ ) − f j (x j (t)) − g j (t), x ∗ − x j (t)

≤ g j (t) ∗ x ∗ − x j (t) ≤ 2 L R. (1.17)

Moreover, the convexity leads to

x j (t − τ − 1) − x j (t) 2
 τ 2
  
=  [x j (t − s − 1) − x j (t − s)]
s=0
τ
 1
≤(τ + 1) 2
x j (t − s − 1) − x j (t − s) 2 . (1.18)
s=0
τ +1

Substituting inequalities (1.17) and (1.18) into (1.16), and making use of Lemma 1.8
and the relation (A + B + C + D)2 ≤ 4[A2 + B 2 + C 2 + D 2 ], we can get

1 
T N
e j (t), x j (t) − x ∗ ≤ 2(τ + 1)L R
N t=1 j=1
 √ 2 T τ
3 NL 
+2(τ + 1)G [α(t − s − 1)]2
1 − σ2 (W ) t=1 s=0
 √ 2 T τ
3 NL 
+2(τ + 1)G [α(t − s)]2
1 − σ2 (W ) t=1 s=0
  τ
T 
+2(τ + 1)G L 2 + [ψ(x ∗ )]2 [α(t − s − 1)]2 .
t=1 s=0

Since {α(t)}∞t=1 is a non-increasing sequence, after some simple computations, we

obtain the desired result and complete the proof.

We are now ready to prove our basic convergence result, which essentially relates
xi (T )) − f (x ∗ ) and the disagreement z i (t) − z̄(t) ∗ , and further
the error bound f (
demonstrates the effects of network topology (say the spectral gap of the weight
matrix W ) on the convergence rate.

T (Proof of Theorem∗ 1.4) By using the convexity of f , i.e., f (

Proof xi (T )) ≤ (1/T )
t=1 f (x i (t)), for any x ∈ X , we have

1  
T
∗
f (
xi (T )) − f (x ) ≤ f (y(t)) − f (x ∗ )
T t=1

1  
T
+ f (xi (t)) − f (y(t)) . (1.19)
T t=1
1.5 Convergence Analysis 13

Since f i is L-Lipschitz for any i ∈ V, combining this with Lemma 1.6 yields that

f i (xi (t)) − f i (y(t)) ≤L xi (t) − y(t)

≤Lα(t) z i (t) − z̄(t) ∗ . (1.20)

It should be noticed that condition (1.20) is also satisfied for f . Since g j (t) ∈
∂ f j (x j (t)) is a subgradient of f j at x j (t), using the convexity of f j and (1.20),
we can derive that

1 
N
f (y(t)) − f (x ∗ ) = f j (x j (t)) − f (x ∗ )
N j=1

1  
N
+ f j (y(t)) − f j (x j (t))
N j=1

1 
N
≤ g j (t), x j (t) − x ∗
N j=1

L 
N
+ α(t) z j (t) − z̄(t) ∗ . (1.21)
N j=1

Recalling the subgradient error vector, we have

g j (t), x j (t) − x ∗
= g j (t − τ − 1), x j (t) − x ∗ + e j (t), x j (t) − x ∗
= g j (t − τ − 1), y(t) − x ∗ + g j (t − τ − 1), x j (t) − y(t)
+ e j (t), x j (t) − x ∗ . (1.22)

By Hölder’s inequality, after using Lemma 1.6 and Assumption 1.2, we can derive
that

g j (t − τ − 1), x j (t) − y(t) ≤ g j (t − τ − 1) ∗ x j (t) − y(t)

≤Lα(t) z j (t) − z̄(t) ∗ . (1.23)

Substituting (1.20)–(1.23) into (1.19), we can get

14 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

1 
T N
xi (T )) − f (x ∗ ) ≤
f ( g j (t − τ − 1), y(t) − x ∗
N T t=1 j=1

1 
T N
+ e j (t), x j (t) − x ∗
N T t=1 j=1

L 
T N
+ α(t) z j (t) − z̄(t) ∗
N T t=1 j=1

L
T
+ α(t) z i (t) − z̄(t) ∗ . (1.24)
T t=1

By Lemmas 1.8–1.11, we obtain the result of Theorem 1.4 and complete the proof.

By choosing a proper step size sequence, we now turn to the proof of Theorem 1.5.

Proof (Proof of Theorem 1.5) Choosing a proper step size as given by Theorem 1.5,
it is easy to see that

T  T √
1 1
√ ≤1+ √ dt ≤ 2 T ,
t=1
t 1 t
T  T
1 1
≤1+ dt = log T + 1.
t=1
t 1 t

√
By some
T simple calculations,
√ it can be derived that log T + 1 ≤ 2 T , which means
that t=1 [α(t)]2 ≤ 2 T . By Theorem 1.4, we have
√
∗ LR 9 N LR
f (
xi (T )) − f (x ) ≤2(τ + 1) + √
T 1 − σ2 (W ) T
LR (τ + 1)2 G R 2
+ 2[1 − σ2 (W )] √ + √
T 1 − σ2 (W ) T
(τ + 1)2 R 2 [1 − σ2 (W )] G R 2
+ √
N L2 T
  √
G R4 N (τ + 1)2
< 13R L + G R 2 + 2 √ .
L 1 − σ2 (W ) T

This completes the proof.

1.6 Numerical Examples 15

0
10
DDA
DDA−DSG
DDA−DDV
max |fi (xi (t)) − f ∗ |

−1
10

−2
10

0 200 400 600 800 1000 1200 1400

Iteration t

Fig. 1.1 Plots of function error versus number of iterations for a random geometric graph with
N = 5 nodes

1.6 Numerical Examples

In this section, we report the experimental results on the cooperative distributed

minimization of a sum of the regularization items and hinge loss functions as well
as making comparison with the methods in [12–14]. This optimization problem is
widely employed in the support vector machine (SVM) method for classification
and regression analysis. For a given graph size N , a random instance of a SVM
classification problem is formulated as follows. We first generate N pairs of the
form (bi , yi ) on a unit hypersphere, where bi ∈ Rd corresponds to a feature vector,
yi ∈ {−1, +1} is the associated label indicating the class to which the point bi belongs
and is separated into two categories by adding 5% of noise. The goal is to use these
samples to estimate a linear classifier based on some weight vectors x ∈ Rd that are
chosen by minimizing the global objective given by

N  
1  λi
f (x) = x 2
2 + max{0, 1 − yi (bi , x + c)} ,
N i=1 2

where λi ∈ R+ corresponds to the regularization parameter and c ∈ R is the bias term

which plays a significant role in solving the classification problem on imbalanced
data sets. It should be noticed that f is L-Lipschitz and non-smooth at any point
with bi , x + c = yi . In this case, we set L = maxi {λi + bi 2 } and impose the
box constraint on the minimization problem, i.e., X = {x ∈ Rd | x 2 ≤ 5}.
16 1 Cooperative Distributed Optimization in Multiagent Networks with Delays

In order to study the effect of communication delays, we perform simulations

with comparison to the DDA algorithm in [12], the DDA algorithm with delayed
dual variable (DDA-DDV) studied in [13, 14] and the DDA algorithm with delayed
subgradient (DDA-DSG) proposed in this chapter. For a given random geometric
graph, we run the optimization algorithm using the optimal setting of the step size α.
Then, the plots of the function error maxi [ f (xi (T ) − f (x ∗ )] versus the number of
iterations are drawn in Fig. 1.1. It is observed that the function error of our method
converges rapidly to the given accuracy  compared to [12–14] due to the preserved
performance gain.
These results illustrate the excellent agreement of the empirical behavior with our
theoretical predictions and improved algorithmic performance.

1.7 Conclusions

In this chapter, we investigated the DDA algorithm for solving cooperative opti-
mization problems in delayed multi-agent networks. We provided a sharp bound on
the convergence rate. It is shown that the effectiveness of the algorithm is highly
dependent on the properties of the optimization algorithm itself and the underlying
connectivity structure of the network. Moreover, we would like to point out that our
result can be easily extended to the case of stochastic communication protocol and
stochastic optimization with noisy gradients. For future work, we will investigate
the mirror descent algorithms for distributed cooperative optimization in delayed
settings.

References

1. Tsitsiklis, J.N.: Problems in decentralized decision making and computation. Ph.D. thesis,
M.I.T. (1984)
2. Olfati-Saber, R., Murray, R.: Consensus problems in networks of agents with switching topol-
ogy and time-delays. IEEE Trans. Autom. Control 49(9), 1520–1533 (2004)
3. Liu, B., Chen, T.: Consensus in networks of multiagents with cooperation and competition via
stochastically switching topologies. IEEE Trans. Neural Netw. Learn. Syst. 19(11), 1967–1973
(2008)
4. Yu, W., Chen, G., Cao, M.: Some necessary and sufficient conditions for second-order consensus
in multi-agent dynamical systems. Automatica 46(6), 1089–1095 (2010)
5. Li, Z., Duan, Z., Chen, G., Huang, L.: Consensus of multiagent systems and synchronization
of complex networks: a unified viewpoint. IEEE Trans. Circuits Syst. I: Regul. Pap. 57(1),
213–224 (2010)
6. Xiong, W., Ho, D.W.C., Wang, Z.: Consensus analysis of multiagent networks via aggregated
and pinning approaches. IEEE Trans. Neural Netw. Learn. Syst. 22(8), 1231–1240 (2011)
7. Nedi, A., Ozdaglar, A., Parrilo, P.A.: Constraint consensus and optimization in multi-agent
networks. IEEE Trans. Autom. Control 55(4), 922–938 (2010)
8. Ram, S.S., Nedi, A., Veeravalli, V.V.: Distributed stochastic subgradient projection algorithms
for convex optimization. J. Optim. Theory Appl. 147(3), 516–545 (2010)
References 17

9. Johansson, B., Rabi, M., Johansson, M.: A randomized incremental subgradient method for
distributed optimization in networked systems. SIAM J. Optim. 20(3), 1157–1170 (2009)
10. Zhu, M., Martnez, S.: On distributed convex optimization under inequality and equality con-
straints. IEEE Trans. Autom. Control 57(1), 151–164 (2012)
11. Yuan, D., Xu, S., Zhao, H.: Distributed primal-dual subgradient method for multiagent opti-
mization via consensus algorithms. IEEE Trans. Syst. Man Cybern.: Syst. 41(6), 1715–1724
(2011)
12. Duchi, J.C., Agarwal, A., Wainwright, M.J.: Dual averaging for distributed optimization: con-
vergence analysis and network scaling. IEEE Trans. Autom. Control 57(3), 592–606 (2012)
13. Tsianos, K.I., Rabbat, M.G.: Distributed consensus and optimization under communication
delays. In: 49th Annual Allerton Conference, Illinois, USA, pp. 974–982 (2011)
14. Tsianos, K.I., Rabbat, M.G.: Distributed dual averaging for convex optimization under commu-
nication delays. In: American Control Conference, Montreal, Canada, pp. 1067–1072 (2012)
15. Nedi, A., Ozdaglar, A.: Cooperative distributed multi-agent optimization. In: Eldar, Y., Palo-
mar, D. (eds.) Convex Optimization in Signal Processing and Communications, pp. 340–386.
Cambridge University Press, Cambridge (2010)
16. Wang, J.: Primal and dual assignment networks. IEEE Trans. Neural Netw. Learn. Syst. 8(3),
784–790 (1997)
17. Hu, X., Wang, J.: Solving the assignment problem using continuous-time and discrete-time
improved dual networks. IEEE Trans. Neural Netw. Learn. Syst. 23(5), 821–827 (2012)
18. Nesterov, Y.: Primal-dual subgradient methods for convex problems. Math. Program. 120(1),
261–283 (2009)
19. Yuan, D., Xu, S., Zhao, H., Rong, L.: Distributed dual averaging method for multiagent opti-
mization with quantized communication. Syst. Control Lett. 61(11), 1053–1061 (2012)
20. Agarwal, A., Duchi, J.: Distributed delayed stochastic optimization (2011).
arXiv:abs/1104.5525
21. Cheng, L., Hou, Z.G., Tan, M.: A delayed projection neural network for solving linear varia-
tional inequalities. IEEE Trans. Neural Netw. Learn. Syst. 20(6), 915–925 (2009)
22. Bregman, L.: The relaxation method of finding the common point of sets and its application
to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 7(3),
200–217 (1967)
23. Nemirovski, A., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. Wiley,
New York (1983)
24. Tseng, P.: On accelerated proximal gradient methods for convex-concave optimization. Tech-
nical Reports, Department of Mathematics, University of Washington (2008)
Chapter 2
Constrained Consensus of Multi-agent
Systems with Time-Varying Topology

Abstract In this chapter, the constrained consensus problem is studied for the asyn-
chronous discrete-time multi-agent system, where each agent needs to lie in a closed
convex constraint set. The communication graphs of this asynchronous system are
assumed to be directed, unbalanced, dynamically changing. In addition, their union
graph is assumed to be strongly connected within a certain interval of finite length. To
deal with the asynchronous communication issue among agents, the original asyn-
chronous system is equivalently transformed to a synchronous one by adding some
new agents. By employing the properties of the projection on the convex sets, the
distance between the states of the agent in the newly constructed system and the
intersection set of all constraint sets for agents is estimated. Based on this estima-
tion, the original system is proven to reach consensus by showing that the linear
parts of the newly constructed system converge and the nonlinear parts vanish over
time. Finally, two numerical examples are provided to show the effectiveness of the
method and the correctness of theoretical analysis.

Keywords Constrained consensus · Multi-agent systems · Asynchronous

communication · Distributed control

2.1 Introduction

In recent years, the collective behaviors in the network of autonomous agents have
received an increasing interest due to their promising potential for widely application
areas, such as formation control [1], flocking control [2], filtering in sensor networks
[3], swarming [4], unmanned autonomous vehicles (UAV) formations [5, 6], etc.
Most of the existing works are concerned with continuous-time dynamics [7–
17]. In real world applications, the information transmission among agents may
not be continuous due to the unreliability of communications channels or the limited
sensing ability of agents. Thus, discrete-time multi-agent systems have found widely
applications and their dynamics have attracted a lot of research interests [18–27],
in which each agent synchronously receives its neighbors’ information at discrete
time instants, where the synchrony means that all agents update their states using
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 19
H. Wang et al., Distributed Optimization, Game and Learning Algorithms,
https://doi.org/10.1007/978-981-33-4528-7_2
20 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

the information of its neighboring agents at the same time. However, considering a
central synchronizing clock may not be available and the communication topology
is dynamically changing [28]. In [28], the authors studied the stationary consensus
of the asynchronous second-order multi-agent system under switching topologies,
where the asynchrony means that each agent’s update action is independent of the
others’.
In some practical applications, the states of agents may need to reach a consensus
but the state must lie in certain constraint sets. A related application of the constrained
consensus is the dynamic coalitional TU games [29]. The distributed bargaining
protocol of the coalitional TU games can be formulated as follows:
⎡ ⎤

N
xi (t + 1) = PXi (t) ⎣ ai j (t)x j (t)⎦ ,
j=1

where {1, 2, . . . , N } is the set of the players. PXi [·] is the projection operator onto
the local constraint set Xi . Xi (t) is the bounding set of player i at time t. The authors
in [29] proved the convergence of the bargaining process to a random allocation that
lines in the core of the dynamical TU game, which means that the players finally
reach an agreement via the distributed bargaining protocol with probability 1. To
the best of our knowledge, few work has considered the constrained consensus of
discrete-time multi-agent systems except in [30], where the communication delays
was considered. It should be mentioned that the communication topology consid-
ered in [30] is unbalanced and jointly connected which is less conservative than the
communication topology considered in [29], where the communication topologies
are assumed to be balanced.
Motivated by the above discussions, we study the constrained consensus of the
asynchronous multi-agent systems, where each agent is required to lie in a closed con-
vex constraint set while reaching consensus. The communication graphs are assumed
to be directed, dynamically changing, and not necessarily balanced. In addition, their
union graph is assumed to be strongly connected. Symmetry would be lost in the
presence of unbalanced graphs and asynchronous communications, rendering the
approaches in [29] are not yet applicable. The existence of the constraint sets would
introduce nonlinearity, that is, the projection error, rendering the approaches in [28]
not applicable either. To deal with the projection errors under the unbalanced com-
munication topology, we divide the proof into the following two procedures. First,
we present a transformation method, built on the seminal work [31], which equiv-
alently transforms the original asynchronous network to the synchronous one by
adding new agents. Few existing work has presented such an idea. The analysis is
then performed for the newly constructed synchronous system. Second, we employ
the properties of the projection on the convex sets, the distance from the states of the
agents to the intersection set of all agents’ constraint sets is estimated. Based on the
estimation, the original system is then proven to reach consensus by showing that the
linear parts of the newly constructed systems converge and the nonlinear parts vanish
2.1 Introduction 21

over time. Finally, two numerical examples are provided to show the effectiveness
of the theoretical results.
Notations: A vector is viewed as a column. For a vector x, x denotes the
Euclidean norm. For a vector x and a closed convex set X , P X [x] denotes projection
of x onto X , i.e., P X [x] = argmin y∈X x − y. R denotes the real number set, N
denotes the nature number set, and Rn = R × ·· · × R. For matrix A, [A]i j denotes
n
the (i, j)th component of matrix A. The superscript “T” denotes the transpose of the
vector and a matrix.

2.2 Preliminaries

The communication topology of a network of N agents is represented using a directed

graph G = (V, E, A) with the vertex set V = {1, 2, . . . , N }, the edge set E ⊂ V × V,
and the weighted adjacency matrix A. If the edge set E and the weighted adjacency
matrix A change over time, then such a time-varying graph is called as the time-
varying communication topology for networks. Consider a discrete-time multi-agent
system consisting of N agents. Each agent is regarded as a vertex in a time-varying
directed graph G(k), whose edge set is E(k) and adjacency matrix is A(k) with
respect to k. ai j (k) is the (i, j)th entry of the adjacency matrix A(k). Each edge
( j, i) ∈ E(k) is an available communication channel, through which, each agent can
receive information from its neighbors. The neighboring set of agent i at time k is
denoted by Ni (k).
The state of each agent, denoted by xi (k) ∈ Rn , is constrained to lie in a nonempty
N
closed convex set Xi ⊆ Rn , which is private for agent i. Define X = i=1 Xi , which
is unknown for all agents. Due to the privacy of the constraint sets and the asyn-
chronous clocks of each agent, we adopt the distributed asynchronous projection
algorithm, in which, at the updating time, each agent first computes the weighted
average among his own state and its neighbors’ states, then projects the averaged
value to its constraint set Xi . For all i = 1, 2, . . . , N , the algorithm is stated as
follows:
⎡ ⎤

xi (tk+1
i
) = PXi ⎣ ai j (tki )x j (tki )⎦ , (2.1)
j∈Ni (tki )

with initial value xi (0) ∈ Xi , where PXi [·] is the projection operator onto Xi . Time
t0i , t1i , . . . , tki , . . . with t0i = 0 for all i = 1, 2, . . . , N , are positive integers, at which,
agent i receives its neighbors’ states. For simplicity, we denote the updating time
t0i , t1i , . . . , tki , . . . for agent i by {tki }. We further assume that for all i = 1, 2, . . . , N
and k ∈ N, {tki } satisfies the following condition:

T ≤ tk+1
i
− tki ≤ T , (2.2)
22 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

where T and T are positive numbers that satisfy T ≤ T . Hereafter, we denote τki =
i
tk+1 − tki > 0.
Definition 2.1 ([28]) Discrete-time multi-agent system is said to be asynchronous
if for all i, j = 1, 2, . . . , N , the time series {tki } are mutually independent, i.e., {tki }
j
is independent of {tk }, i = j.
We set ai j (tki ) = 0 for all j ∈
/ Ni (tki ). Then, the asynchronous discrete-time multi-
agent system (2.1) can be rewritten as follows
⎡ ⎤

N
xi (tk+1
i
) = PXi ⎣ ai j (tki )x j (tki )⎦ . (2.3)
j=1

Proposition 2.2 ([29]) For the projection operator P on a closed convex set X ⊆
Rn , for any ω ∈ Rn and any x ∈ X ,

PX [ω] − x2 ≤ ω − x2 − PX [ω] − ω2

Proposition 2.2 establishes a relation between the projection error vector and the
feasible directions of the convex set X at the projection vector.
Assumption 2.1 (Weight Rule) There exists a constant 0 < η < 1, such that for all
i = 1, 2, . . . , N
(a) aii (tki ) > η for all k ≥ 0.
(b) ai j (tki ) > η for all k ≥ 0 and all agents j communicate directly with agent i at
time tki .
(c) ai j (tki ) > 0, j∈Ni (k) ai j (tki ) = 1 for all i, j = 1, 2, . . . , N and k ∈ N.
Assumption 2.1 says that every agent assigns a substantial weight to the infor-
mation received from its neighbors. This guarantees that the information from each
agent influences the information of each other agent persistently in time. In other
words, this assumption guarantees that the agent information is mixing at a nondi-
minishing rate in time. Without this assumption, information from some of agents
may become less influential in time, and in the limit, resulting in loss of information
from these agents.
Remark 2.3 In [30], the authors studied the constrained consensus in multi-agent
networks under the assumption that the positive edge weights are chosen from a finite
set (Assumption 2 in [30]). In this chapter, the weights can be chosen infinitely from
a bounded set, i.e., ai j (tki ) ∈ [0, 1] whenever j ∈ Ni (k) for all i = 1, 2, . . . , N and
k ∈ N.
Remark 2.4 In [29], the constrained consensus is studied under the assumption that
N
the adjacency matrix A(k) is doubly stochastic for all k ∈ N, i.e, i=1 ai j (k) = 1,
N
j=1 ai j (k) = 1 for all i, j = 1, 2, . . . , N . In this chapter, this constrained assump-
tion is removed.
2.2 Preliminaries 23

Definition 2.5 The constrained consensus problem of the multi-agent system (2.3)
is said to be reached if for some z̃ ∈ X and all i = 1, 2, . . . , N ,

lim xi (tki ) − z̃ = 0.

k→∞

Before presenting the main results, we need to give the following assumption on
the connectivity of the communication graph.
+∞

Assumption 2.2 There exists an infinite sequence of time indices ki |i=0 , where
k0 = 0,0 < km+1
 − km < M, m ∈ Z, M ∈ Z+ , such that the union of the directed
km+1 −1
graphs G(ti ) i=k m
is strongly connected.

Remark 2.6 The assumption on the strong connectivity of communication networks

is crucial for the constrained consensus of multi-agent systems [29]. It ensures that
the agents’ states are sufficiently fused over the communication network, and avoids
that certain agents remain stationary states in themselves constraint set, if so, the
network will not reach consensus due to the different local constraint sets. This
assumption is quite different from the unconstrained consensus, i.e., Xi = Rn for all
i = 1, 2, . . . , N , where the communication network only needs to contain a directed
spanning tree [22, 25, 28].

2.3 Main Results

In this section, we will investigate the constrained consensus problem of the multi-
agent system (2.3). The motivation is mainly from [28, 30]. We should first merge the
time sequence that each agent receives its neighbors’ states into a single ordered time
sequence T and then the original asynchronous discrete-time system is casted into
an equivalent augmented synchronous discrete-time multi-agent which evolves over
time sequence T . Then, we employ the mixed tools from graph theory, nonnegative
matrix theory and the infinite product of stochastic matrices to obtain the main results.

2.3.1 Model Transformation

Here, we reduce the original system with asynchronous communication to a new

system with synchronous communication under bounded communication interval.
We introduce an enlarged system that is obtained by adding new agents into the
original system in order to deal with asynchronous communication. It suffices to
add (m̂ − 1)N agents into the original systems,  where m̂ be the upper bound for the
number of elements in set t j : t j ∈ tki , tk+1
i
, j ∈ N for all i = 1, 2, . . . , N and
 
k ∈ N. By Lemma 1 in [28], we have m̂ = T /T + 1 (N − 1) + 1 where T and
T are defined in (2.2) and T /T is the largest integer no less than T /T .
24 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

We label the newly added agent corresponding to agent j by j + N , . . . , j + (m̂ −

1)N . For t with 1 ≤ t ≤ τki , agent j + (t − 1)N transmits its state to agent j + t N
and agent j + t N transmits the same state to agent j + (t + 1)N if t + 1 < τki or to
agent i with a product of weight ai j (tki ) if t + 1 = τki . Then, the weight ãi h (t) of the
enlarged system between agent i and agent h is defined as follows:
⎧
⎨ ai j (tki ), h = j + t N , t = τki ,
ãi h (t) = 1, h = j + t N , 1 ≤ t ≤ τki − 1, (2.4)
⎩
0, otherwise,
 
for k ≥ 0. Hereafter, we denote for all t ≥ 0, Ã(t) = ãi j (t) m̂ N ×m̂ N be the adja-
cency matrix and G̃(t) for all t ≥ 0 be the underlying graph of the enlarged system,
respectively. By (2.4), it is easy to see that Ã(t) is a row-stochastic matrix, but not a
column-stochastic one. On the other hand, we let R n be the constrained set of agent
i = N + 1, N + 2, . . . , m̂ N .
Using the weights rule (2.4), the discrete-time asynchronous multi-agent system
(2.3) can be written as follows:
⎡ ⎤

m̂ N
x̃i (t + 1) = PXi ⎣ ãi j (t)x̃ j (t)⎦ , (2.5)
j=1

for all i = 1, 2, . . . , m̂ N , whose initial values are given by

x̃i (0) = xi (0) for all i = 1, 2, . . . , N ,

x̃i (0) = 0 for all i = N + 1, N + 2, . . . , m̂ N .

In the following, we provide a useful lemma without proof, please refer to Lemma
3 in [28] for a similar proof.
+∞

Lemma 2.7 Let ki |i=0 be an infinite sequence of time indices, where k0 =
0, 0 < km+1 − km < M, m ∈ Z, M ∈ Z+ , such that the union of the directed
km+1 −1
graphs {G(ti )}i=k is strongly connected. Then, the union of enlarged graphs
  m
km+1 −1
G̃(ti ) i=km is also strongly connected.

Lemma 2.7 says that if the union of the original graphs is strongly connected in
certain finite time interval, then the union of the enlarged graphs is also strongly con-
nected in the same time interval. This means that the model transformation method
does not change the connectivity of the original graphs, which guarantees that infor-
mation is mixed sufficiently in the enlarged multi-agent system. In the following, we
separate the linear and nonlinear parts of the constrained multi-agent system (2.4) to
reveal the effect of projection errors for constrained consensus.
2.3 Main Results 25

x̃i (t + 1) = ωi (t) + ei (t), for all i = 1, 2, . . . , m̂ N , (2.6)

m̂ N
where ωi (t) = j=1 ãi j (t)x̃ j (t), ei (t) = PXi [ωi (t)] − ωi (t).

2.3.2 State Transition Matrix

Define the state transition matrix of (2.6) be

Φ(t, s) = Ã(t)Ã(t − 1) · · · Ã(s). (2.7)

By simple calculation, we have


m̂ N
x̃i (t + 1) = [Φ(t, s)]i j x̃ j (s)
j=1
⎛ ⎞

t m̂ N
+ ⎝ [Φ(t, r )]i j e j (r − 1)⎠ + ei (k). (2.8)
r =s+1 j=1

Clearly, the state transition matrix plays a key role in the evolution of the agents’
states. In the following, we give some properties of the state transition matrix, which
include the row-stochasticity of Φ(t, s) and its limiting matrix, the convex properties
of each column of Φ(t, s), and the estimate of the convergence rate of the transition
matrix based on the norm properties. Their proofs are omitted, because the proofs
can be found in Lemma 3 in [22] and Lemma 5 in [30].
Lemma 2.8 Under Assumptions 2.1 and 2.2.
(a) Φ(t, s) is a row-stochastic matrix for all t ≥ s ≥ 0.
(b) The limit Φ(s) = limt→∞ Φ(t, s) exists for all s ≥ 0.
(c) Φ(s) has identical rows and the rows are stochastic, i.e.,

Φ(s) = 1φ T (s),

where φ(s) ∈ Rm̂ N is a stochastic vector for each s and 1 is a vector whose
elements are 1.
(d) For all s ≥ 0, i, j = 1, 2, . . . , N ,
 
Φ(s + (m̂ N − 1)M − 1, s) i j ≥ η(m̂ N −1)M ,

where M is defined in Assumption 2.2.

(e) For all j = 1, 2, . . . , m̂ N , the entries [Φ(t, s)]i j , i = 1, 2, . . . , m̂ N , converge
to the same limit φ j (s) as t → ∞ with a geometric rate, i.e., for all j =
1, 2, . . . , m̂ N and s ≥ 0,
26 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

  −B   t−s
[Φ(t, s)]i j − φ j (s) ≤ 2 1 + η 1 − η B B
1 − ηB

for all t ≥ s and i = 1, 2, . . . , N , where B = (N − 1)M + m̂ N .

2.3.3 Constrained Consensus

In the following, we derive the main result in this chapter. First, we prove that
under Assumptions 2.1 and 2.2, the projection errors ei (t) → 0 as t → ∞ for all
i = 1, 2, . . . , m̂ N . Second, we build an auxiliary variable y(t), then prove that x̃i (t)
and ωi (t) converge to y(t) as t → ∞ for all i = 1, 2, . . . , N . Finally, we prove the
constrained consensus of the multi-agent system (2.3) can be reached.
We start by proving that the projection errors of agents vanish over time under
Assumptions 2.1 and 2.2.
Proposition 2.9 Under Assumptions 2.1 and 2.2, limt→∞ ei (t) = 0 for all i =
1, 2, . . . , m̂ N .

Proof Let z̃ be any element in X . By the strict non-expansion property of the

Euclidean projection on a closed convex set Xi , we have for all i = 1, 2, . . . , N
and t ≥ 0,

x̃i (t + 1) − z̃2 ≤ ωi (t) − z̃2 − ei (t)2 . (2.9)

Actually, by (2.6), one has x̃i (t + 1) = PXi [ωi (t)]. Thus, it follows by Proposi-
tion 2.2 that
 2
x̃i (t + 1) − z̃2 = PXi [ωi (t)] − z̃ 
 2
≤ ωi (t) − z̃2 − PXi [ωi (t)] − ωi (t)
≤ ωi (t) − z̃2 − ei (t)2 .

By (2.9), we have

x̃i (t + 1) − z̃2 ≤ ωi (t) − z̃2 . (2.10)

Thus,

max {x̃i (t + 1) − z̃} ≤ max {ωi (t) − z̃} (2.11)

i=1,2,...,N i=1,2,...,N

On the other hand, it is easy to have that

2.3 Main Results 27
 2
 m̂ N 
  
ωi (t) − z̃2 = 
 ãij (t) x̃ j (t) − z̃ 

 j=1 

m̂ N
 2
≤ ãi j (t) x̃ j (t) − z̃  , (2.12)
j=1

where the last inequality is obtained by the convexity of the operator  · 2 and the
row-stochasticity of Ã(t), i.e., m̂j=1
N
ãi j (t) = 1. Then, by (2.12), we get

  2
m̂ N
 
ωi (t) − z̃2 ≤ ãi j (t) max x̃ j (t) − z̃ 
j=1,2,...,N
j=1
 2
= max x̃i (t) − z̃ ,
i=1,2,...,N

which implies that

max ωi (t) − z̃ ≤ max x̃i (t) − z̃ . (2.13)

i=1,2,...,N i=1,2,...,N

Let D1 (t) = maxi=1,2,...,N x̃i (t) − z̃ and D2 (t) = maxi=1,2,...,N ωi (t) − z̃. Then,
by (2.10) and (2.12), we have D1 (t) and D2 (t) are monotone decreasing. Since D1 (t)
and D2 (t) are bounded, they converge to the same non-negative value d1 at t → ∞.
Again from (2.9), (2.11) and (2.13), in order to prove limt→∞ ei (t) = 0, we
only need to prove limt→∞ x̃i (t) − z̃ = d1. By contradiction, if it does not hold,
there must exist an i 0 such that the limit of x̃i0 (t)−  z̃  does not equal to d1 . We
 
suppose there exists a subsequence of x̃i0 (t) − z̃  , denoted by x̃i0 (ts ) − z̃  ,
has the limit d2 < d1 . Then, there existpositive numbers 1 , 2 and s(1 , 2 ) with
d2 + 2 < d1 − 1 , such that d2 − 2 < x̃i0 (ts ) − z̃  < d2 + 2 for all ts > s(1 , 2 )
and d1 − 1 < D1 (t) < d1 + 1 for all t > s(1 , 2 ).
Since D1 (t) < d1 + 1 for all t > s(1 , 2 ), x̃i (t) − z̃ < d1 + 1 for all t >
s(1 , 2 ) and i = 1, 2, . . . , N . Let s0 be the s0 th point defined in Assumption 2.2
and satisfy ts0 ≥ s(1 , 2 ). Then, we have
 
x̃i (s(1 , 2 ) + 1) − z̃ 
 0

≤ ωi0 (s(1 , 2 )) − z̃ 
  
m̂ N
 
≤ ãi0 i0 x̃i0 (s(1 , 2 )) − z̃  + ãi0 j x̃ j (s(1 , 2 )) − z̃ 
j=1, j=i 0
≤ η(d2 + 2 ) + (1 − η)(d1 + 1 ), (2.14)
m̂ N
where the last inequality is obtained by Assumption 2.1 and j=1 ãi j = 1 for all
i = 1, 2, . . . , N .
28 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

Similar to (2.14), it is easy to derive that

 
x̃i (s(1 , 2 ) + 2) − z̃ 
0

  
m̂ N
 
≤ ãi0 i0 x̃i0 (s(1 , 2 ) + 1) − z̃  + ãi0 j x̃ j (s(1 , 2 ) + 1) − z̃ 
j=1, j=i 0
≤ η (η(d2 + 2 ) + (1 − η)(d1 + 1 )) + (1 − η)(d1 + 1 )
≤ η2 (d2 + 2 ) + (1 − η2 )(d1 + 1 ).

Recursively, for all s(1 , 2 ) ≤ t ≤ ts0 , we have

 
x̃i (t) − z̃  ≤ η M (d2 + 2 ) + (1 − η M )(d1 + 1 ), (2.15)
0

where the last inequality is obtained by 0 < η < 1 and ts0 − s(1 , 2 ) ≤ M.
By Lemma 2.8(a), Φ(t, s) defined in (2.7) is a row-stochastic matrix
 for all t ≥ s ≥
0. By Lemma 2.8(d), we obtain that Φ(s1 + (m̂ N −1)M −1, s1 ) ji0 ≥ η(m̂ N −1)M for
 
all j = 1, 2, . . . , N and s1 ≥ 0. Also, it follows from (2.15) that x̃i0 (ts0 ) − z̃  ≤
η M (d2 + 2 ) + (1 − η M )(d1 + 1 ). Thus, we have for all i = 1, 2, . . . , N
 
x̃i (ts + (m̂ N − 1)M) − z̃ 
 0
 
 
≤  Φ(ts0 + (m̂ N − 1)M − 1, ts0 ) ii0 x̃i0 (ts0 ) − z̃ 
 
 m̂ N 
    
+  Φ(ts0 + (m̂ N − 1)M − 1, ts0 ) i j x̃ j (ts0 ) − z̃ 

 j=1, j=i0 
   
≤ Φ(ts0 + (m̂ N − 1)M − 1, ts0 ) ii0 x̃i0 (ts0 ) − z̃ 
  !
+ 1 − Φ(ts0 + (m̂ N − 1)M − 1, ts0 ) ii0 (d1 + 1 )
 
≤ η(m̂ N −1)M η M (d2 + 2 ) + (1 − η M )(d1 + 1 )
!
+ 1 − η(m̂ N −1)M (d1 + 1 ), (2.16)

where the last inequality is obtained by Lemma 2.8(d).

Let 0 < 1 < η2(2−η (d1 −d2 ) ηm̂ N (d1 −d2 )
m̂ N
m̂ N ) and 0 < 2 < 2
. Then, we can check that d2 +
2 < d1 + 1 . Then, by simple calculation, we obtain from (2.16) that
 
x̃i (ts + (m̂ N − 1)M) − z̃  < d1 − 1 , (2.17)
0

which contradicts to D1 (t) > d1 − 1 for all t > s(1 , 2 ). Hence, we have
limt→∞ x̃i (k) − z̃ = d1 holds for all i = 1, 2, . . . , N , i.e., limt→∞ ei (k) = 0
for all i = 1, 2, . . . , N .

Define an auxiliary variable y(t) as

2.3 Main Results 29


m̂ N 
t 
m̂ N
y(t) = φi (s)x̃i (s) + φ j (r )e j (r − 1) + ei (t), (2.18)
i=1 r =s+1 j=1

where φi (s) is the ith element of φ(s) defined in Lemma 2.8. By (2.8), we have


m̂ N
x̃i (t + 1) = [Φ(t, s)]i j x̃ j (s)
j=1
⎛ ⎞

t 
m̂ N
+ ⎝ [Φ(t, r )]i j e j (r − 1)⎠ + ei (k). (2.19)
r =s+1 j=1

The following property shows that ωi (t) and x̃i (t) converge to y(t) as t → ∞.
Proposition 2.10 Under Assumptions 2.1 and 2.2, limt→∞ ωi (t) − y(t) = 0 and
limt→∞ x̃i (t) − y(t) = 0.

Proof By Proposition 2.9, we have limt→∞ ei (t) = 0 for all i = 1, 2, . . . , N .

Then, for any 3 > 0 and all i = 1, 2, . . . , N , there exists a positive integer s2 , such
that whenever t ≥ s > s2 , ei (t) < 3 .
By Lemma 2.8, Eqs. (2.18) and (2.19), we get

x̃i (t + 1) − y(t + 1)


m̂ N
  
≤ [Φ(t, s)]i j − φ j (s) x̃ j (s)
j=1


t 
m̂ N
  
+ [Φ(t, r )]i j − φ j (r ) e j (r − 1)
r =s+1 j=1

1 + η−B   t−s 
m̂ N
 
≤2 1 − η B B x̃ j (s)
1−η B
j=1

1 + η−B    t−r 
t m̂ N
+ 1 − η B B
e j (r − 1)
1 − η B r =s+1 j=1

1 + η−B   t−s 
m̂ N
 
≤2 1 − η B B x̃ j (s)
1 − ηB j=1
  t−s−1  − 1
1 + η−B 1 − ηB B − 1 − ηB B
+ m̂ N 3  − 1 . (2.20)
1 − ηB 1 − ηB B

In view of arbitrary choice of 3 , taking the limits of both sides of (2.20) as t → ∞,

we have for all i = 1, 2, . . . , N ,
30 2 Constrained Consensus of Multi-agent Systems with Time-Varying Topology

lim x̃i (t) − y(t) = 0.

t→∞

 
Then, it follows ωi (t) − y(t) ≤ m̂j=1 N
[Φ(k, k)]i j x̃ j (t) − y(t) ≤ m̂ N
  j=1
x̃ j (t) − y(t). Then, we have for all i = 1, 2, . . . , N ,

lim ωi (t) − y(t) = 0,

t→∞

which completes the proof.

Now, we are in the position to get the main results of this chapter, which guarantees
the constrained consensus of the multi-agent system (2.3) can be finally reached.
Theorem 2.11 Under Assumptions 2.1 and 2.2. Constrained consensus of multi-
agent system (2.3) can be reached.
Proof Proposition 2.9 has proven that for any z̃ ∈ X sequence {x̃i (t) − z̃} is con-
vergent for all i = 1, 2, . . . , N . By Proposition 2.10, we have that limt→∞ x̃i (t) −
y(t) = 0 for all i = 1, 2, . . . , N . Thus, from y(t)−z̃ ≤ x̃i (t) − y(t) + x̃i (t) −
z̃, we have the sequence {y(t) − z̃} is convergent for any z̃ ∈ X . Thus, y(t) must
converge to a certain point x̃ ∈ X . That is, for x̃ ∈ X , there exists a subsequence
{y(tm )} of {y(t)}, such that limm→∞ y(tm ) − x̃ = 0, i.e., for any 4 > 0, there
exists a constant t0 , whenever tm > t0 , we have y(tm ) − z̃ < 21 1 . Note from Propo-
sition 2.10 that limt→∞ x̃i (t) − y(t) = 0. That is, for 4 > 0, there exists a con-
stant t1 , whenever t > t1 , we have xi (t) − y(t) < 21 4 . Set t2 = max{t0 , t1 }, we
have for all tm > t2 , xi (tm ) − x̃ ≤ xi (tm ) − y(tm ) + y(tm ) − x̃ ≤ 4 . Thus,
for all t > tm > t2 and i = 1, 2, . . . , N , we have xi (t) − x̃ ≤ 4 , which means
that limt→∞ xi (t) − x̃ = 0 for all i = 1, 2, . . . , N , implying that the constrained
consensus for the multi-agent system (2.3) is reached.

2.4 Numerical Examples

In this section, two numerical examples are given to illustrate the effectiveness of
the method and the correctness of theoretical analysis.

2.4.1 Constrained Consensus of a Simple Directed

Multi-agent System

We first illustrate the constrained consensus of a simple multi-agent system with 4

agents, whose states are two-dimensional vectors, and the communication graphs
are shown in Fig. 2.1. The agents in the multi-agent system evolve in the following
asynchronous way:
2.4 Numerical Examples 31

Fig. 2.1 Asynchronous communication topologies of a simple multi-agent system

(1) Agent 2 receives the state information from agent 1 at update times 4k + 1,
k ∈ N, whose communication graph G1 is shown in Fig. 2.1a;
(2) Agent 3 receives the state information from agent 2 at update times 4k + 2,
k ∈ N, whose communication graph G2 is shown in Fig. 2.1b;
(3) Agent 4 receives the state information from agent 3 at update times 4k + 3,
k ∈ N, whose communication graph G3 is shown in Fig. 2.1c;
(4) Agent 1 receives the state information from agent 4 at update times 4k + 4,
k ∈ N, whose communication graph G4 is shown in Fig. 2.1d.

Apparently, none of these graphs is strongly connected, but their union is. Thus,
Assumption 2.2 is satisfied. The edge weights are set to be a21 = 0.4, a22 = 0.6 in
G1 ; a32 = 0.5, a33 = 0.5 in G2 ; a43 = 0.3, a33 = 0.7 in G3 ; a14 = 0.5, a11 = 0.5 in
G4 . The agents’ initial values are randomly chosen from their local constraint sets.
The simulation results are shown in Figs. 2.2 and 2.3. Figures 2.2 and 2.3 provide
plots of the component-wise evolutions of the agents’ states xi1 , xi2 , i = 1, 2, 3, 4,
respectively. It can be seen from Figs. 2.2 and 2.3 that the agents’ states converge to
(1, 1)T ∈ X , which means that the constrained consensus has been finally reached.
The numerical results are consistent with that of theoretic analysis, it therefore con-
cludes that the correctness of the theoretical analysis for a simple multi-agent system.
Another Random Scribd Document
with Unrelated Content
streets and lanes and piazzas obstructed with broken furniture of every sort;
vilely smelling currents of black filth, and pools and lakelets of the same;
and—mercy on us!—corpses everywhere in the quieter squares—corpses of
wretches who had crawled there to die; corpses reeking in the sunlight;
corpses that even the clouds of horrid vultures refused to put a talon in.
“Such was Santiago. I had come for copy, and I soon had enough of it.
“ ‘Let’s get out of this, bo’s’n. Can’t we spend the night up yonder
among the hills and palm trees?’
“ ‘Yes,’ the good fellow answered, cheerily. ‘And luckily the wind’s
about a N.N.E.’
“We didn’t leave the city empty-handed, though. One hotel was doing a
roaring trade, and when we found ourselves, an hour before sunset, high up
among the woods, we had enough of the good things of this life to have
stood a five days’ siege.
“Perhaps we didn’t make a hearty supper! Oh no, sailor-men never eat
and drink!
“We had some wine anyhow, for our stomachs’ sake, let me say, and to
eliminate the perfume of sweet Santiago, which seemed still to hang around
us.
“The sunset was ineffably beautiful, the clouds and the bay were
streaked with the colours of tropical birds; of those very birds that sang
their evening songs above us, while the breeze sighed through the foliage.
“Twilight does not last long here, however, but a big round moon rose
slowly over the hills, and there would be neither darkness nor danger to-
night.
“ ‘I say, bo’s’n,’ I cried, ‘you were in the Merrimac with gallant Hobson.
Tell us your version. Have another cigar, and another glass of wine. Keeps
away infection, you know.’
“The bo’s’n needed no second bidding. He had a bo’s’n’s nip—four
fingers high—and the wine was brandy too.
“ ‘Ahem! Yes, I was in the Merrimac, and so was Jack Hardy, here.’
“ ‘Well,’ I cried, ‘I am in luck. Wait, bo’s’n, till I light up. Now, then,
heave round, my friend. Sure you’re not thirsty?’
“ ‘No, sirree. I feel that last little tot in my eye like. Ever seen Hobson?
Well, you’ll like ’im when you does. You’ve seen a yacht, spick and span,
new, that can rip through a stormy sea, hang or move like a Mother Carey’s
chicken, and do ’most anything. That’s him. That’s Hobson. Bless you, sir,
the old men didn’t like the youngster’s brave proposal at first. They pooh-
poohed it, as ye might say. Even Schley himself laughed a little, as, in his
fatherly way, he put a hand on young Hobson’s shoulder. I was as close to
’em, sir, as I am to Jack here. “Admiral Cervera,” he says, “is in yonder
right enough. Only wish the beggar would come out. He’s bottled.”
“ ‘“Ay, admiral,” says Hobbie, as we calls him for fond like, “and I want
to cork the bottle. Give me that old collier the Merrimac, and, with a few
volunteers, I’ll take her in and sink her right across the narrow neck, ’twixt
Canores and Estrella Points, and——”
“ ‘“And where will you and your men be then?” says Schley.
“ ‘“I’ll give you my word of honour, sir, I’ll go to heaven, almost
cheerfully, as soon’s we bottle up the dirty Don! Besides, sir,” he says,
“why smash that fine fleet up, when it would make so grand an addition to
the American Navy?”
“ ‘Yes; and it were that very argerment, I guess, that carried the pint, wi’
the captains in council assembled. Volunteers! Ay, in course; half the navy
would have volunteered to steam to certain death with young Hobson. It
was the forlornest o’ hopes ever led.
“ ‘Look you, see, sir.’ The bo’s’n paused a minute to draw with his knife
a rough sketch of Santiago bay and city on the ground.
“ ‘That’s my map, like, o’ the place lying down yonder beneath us in the
moonlight. Them things there at sea is the fleet—our fleet. You’ll have to
take Cervera’s for granted, but one of his ships lay here, you see, to guard
the entrance. The crosses is the batteries, and they did blaze and batter us
that awful night!’
“The bo’s’n paused a moment, and laid his hand affectionately on Jack
Hardy’s shoulder.
“ ‘Me and my young pal here,’ he continued, ‘had known one another
for months afore then. There was something about the lad that made me like
him. See’d him throw his extra garments one day and go like thunder for
big Nat Dowlais, ’cause he’d kicked the ship’s cat. Ay, and welted him well,
too. I took to talkin’ more to Jack after that. But I couldn’t get down deep
enough to the boy’s heart. There was something under the surface; I could
tell that. Jack was no ordinary bit o’ ship’s junk. Bless you, sir, there’s
hundreds o’ gentlemen’s sons before the mast—but they’re not all like Jack
Hardy. Jack was more like a stage sailor than anything else. Everything he
put on was so darned natty—his hands so white and soft, though his face
and neck was brown. Then he talked American like a book. Played the
piano, too, like a freak, and was often in the ward-room in consequence.
And blowed if I didn’t hear the master-at-arms—bloomin’ old brass-bound
Jimmy Legs—more’n once call him “sir.”
“ ‘Well, the Merrimac was ’long-side and ready. Incloodin’ Lieutenant
Hobson himself, eight of us were chosen for this deed o’ danger. Torpedoes
were arranged in the hold. Hobson would stand by the helmsman, Hobson
would touch the button and sink her, and, at a word, we should leap into the
sea and swim for the dinghy towin’ astern, for this was our only hope o’
salvation.
“ ‘Jack, here, had stood by my side among the volunteers, but the poor
lad was passed over. Don’t nudge me, Jackie lad; I’m goin’ to tell the truth,
the whole bloomin’ truth, and nothin’ but—so there! I’ll never forget, sir,
the look o’ disappointment on the lad’s face just then. Some time after, I
found him for’ard with his back to the ship and his face to the sea. He
looked smartly up, but I could see by the starlight there were tears on his
face.
“ ‘He said nothing, but walked away impatient like, and I saw him no
more for a time.’
“The bo’s’n leaned towards me now, and his eyes sparkled in the
moonlight. He touched my knee with his horny palm.
“ ‘We steamed away,’ he said, in a hoarse half-whisper—steamed into
the darkness and away from the flag-ship. Not a sound for a time save the
hollow dump o’ the screw and the swirl o’ the seethin’ seas!
“ ‘In silence we steamed—it might have been for half an hour, but it
seemed like an age—an age of blackness and terror. Nothing was nateral
like. The ship was a death-ship, the figures agin the bulwarks yonder were
spectres. I would have given worlds to have heard but a word, a laugh, a
cough even!
“ ‘I said there were eight of us! By the sky above us yonder, sir, there
were nine!
“ ‘I guessed at once who the ninth was, and I shuddered a bit when I
thought of brave, foolish Hardy here. For never a stroke could he swim, and
his coming with us to-night was sheer madness—nay, more, it looked like
suicide.
“ ‘Soon after Jack slid slowly up towards me, and his left arm clutched
my right as I clutch yours now. Every one of us, sir, was stripped to the
waist. Every one wore a lifebelt save Jack Hardy. He was a stowaway, and
not in it.
“ ‘“Oh, boy,” I said, speaking in a whisper, “why have you done this?”
“ ‘“Hush!” he answered. “My time is mebbe short, mate, and you’ve
always been my friend. So listen. Something tells me you’ll be saved, but I
am here to die. I want you to bear a message to my parents—to my mother
especially. Her address you’ll find in my ditty-box. But go to see her, Sam,
when the war is over. Far away west my people live in opulence, and I’m an
only son. Father taunted me with cowardice, and I ran away and came to
sea. Tell father I forgave him. Tell mother——” Ah, sir, just here the lad
broke down. He’s only a boy. “Tell mother,” he sobbed, “how her Jack died
for his country. Tell her I felt she’d forgiven me—that will please her—that
my every dream was of home and her, that——”
“ ‘“A boat on the weather-bow,” cried a man to Hobson. “Shall we fire?”
“ ‘“No,” cried Hobson; “never a shot.”
“ ‘It had been a picket. We heard her officer shout in Spanish to give
way with a will, and she disappeared up into the darkness of the channel we
were now entering.
“ ‘The end was coming; the end was very near, and we all knew it.’
* * * * *
“While the bo’s’n had been telling his story, young Hardy sat silent, but
he spoke now almost for the first time.
“ ‘A moment, sir. The bo’s’n won’t tell you, but I must. He tore off his
lifebelt, and fastened it around me. He swore I must wear it or he would
fling it into the sea. That’s all!’
“ ‘Well, sir,’ continued the bo’s’n, ‘the awful silence was speedily
broken. They had seen us only as a dark mass, black as the rocks that
towered above us. Then their fire opened. We’ll never be under such a fire
again as that, sir, and live. Shells burst above us, around us, shells riddled
our hull, and raked our spar-deck, and crushed into our deck-house.
Fragments and splinters flew about in all directions. I think most of us were
flat on our faces just then, and I lay beside Jackie here holding his hand. No
tremor there, though! No signs of fear! And the fire poured into us from
three sides, sir, from the batteries of Socappa on the left, from Morro on the
right, and from a warship ahead.
“ ‘Speak of thunder. Pah! thunder isn’t in it with such a devil’s din as
this, and lightning ’gainst those gun-gleams would have been like the glint
of a farthing candle!
“ ‘Then we saw brave Hobson’s figure—unearthly tall it looked. No
voice could be heard, only his arms waved us to the bulwarks.
“ ‘Next second it seemed we were all in the water, as a roar louder than
the artillery shook the sky, shook the hills, and silenced even the batteries.
“ ‘The ship was sinking beside us! We were all but drawn into the
whirlpool, but I held Jack’s hand and toughly towed him off.
“ ‘But the dinghy was gone, and the rudder too, and the Merrimac sank,
not across, but along the channel. So our forlorn hope had been led in vain.
The Spanish fleet was bottled still, but not corked, sir.’
“He paused for a moment.
“ ‘Ah, sir, no one there would ever forget that night, nor the hours we
passed under a tilted grating that God in His mercy had put it into some
one’s head to attach by a rope to the ship. We could just get under this
catamaran and hold on to the spars above.
“ ‘Hour after hour of darkness went by. Boats passed and repassed, and
we could hear the men talking. Had they known there were nine heads
under that grating, short would have been our shrift, sir.
“ ‘And all these hours we hardly spoke. We almost feared to breathe
aloud.
“ ‘More than once I thought that Jackie here was dead or dying, but I
whispered cheering words to him. More than once I trembled as my feet
were touched by slimy sharks. How they did not tear me down I cannot tell
you. Seems to me, sir, ’twere a ’tarposition o’ Providence like.
“ ‘But daylight came at last, and Cervera’s own boat and Cervera
himself.
“ ‘Hobson’s voice was feeble enough now, but he managed to hail her.
“ ‘“Por Dios!” we heard the white-haired admiral cry. “Do the dead talk
to us?”
 “ ‘But we were saved, and taken to the Spanish ship. Yes, sir, treated
with every kindness, made prisoners, but released at long, long last, even
before sweet Santeehager fell.
“ ‘Well, that’s my yarn, sir, and it’s all as true as the stars above us.’
“ ‘And Jack Hardy here,’ I ventured to ask, ‘was he reprimanded?’
“ ‘Tried by drum-head he was, sir. Condemned to death for desertion,
and pardoned all in one sentence.’ ”
“ ‘Ah, sir,’ the brave bo’s’n added, ‘I’ll bet my boots that Jack Hardy is a
midshipman before this cruel war is over. Thank ye, sir, I don’t mind if I do;
and I’ll give ye a toast, too—

“ ‘“May the Stars and Stripes we love so well,

With Britain’s flag entwine,
And we’re goin’ to give the world—fits,
When the two brave fleets combine.“ ‘“

* * * * *
The Walrus sailed on and on around the great Antarctic continent, but
never saw her consort till once more the two ships met safe and sound at
Kerguelen Isle.

END OF BOOK II
 BOOK III

ON THE GREAT ANTARCTIC

CONTINENT

CHAPTER I

A STRANGE DISCOVERY—SHEELAH AND TAFFY

“She is bound to be,” said Captain Mayne Brace, a day or two before the
good ship Walrus reached Kerguelen. “Bound to be, Mr. Armstrong. She is
the better craft of the two, you know.”
He was talking to Ingomar and Walter, one evening in October, while
they all sat together in the cosy saloon, not a mile away from the stove.
Ingomar and Brace were smoking the pipe of peace, and sipping their
coffee (which they placed, to keep warm, on top of the stove), between each
longdrawn sip. Walter was reading one of Scott’s novels, or trying to, for he
was listening to the conversation all the same. Charlie was missing to-night.
I rather think he would have been found, if any one had cared to look for
him, forward in the galley, listening to the men’s yarns, or playing a
hornpipe to please them.
“Well, yes, she is bound to be, in the natural course of events, because,
as you say, she has faster sailing qualities, and all that; but——”
“Ah!” interrupted Mayne Brace, with a smile, and another hearty pull at
his coffee; “we must not think of the ‘might be,’ or the ‘may be.’ Else we’d
go on thinking and get nervous, and end in believing, that because we did
not meet the Sea Elephant somewhere to the east of Dougherty Islands, she
has been taken aback in a squall, and gone down stern foremost, with all
hands. Or that she had, at the very least, broken her screw.”
“Steward!”
“Ay, ay, sir!”
“Put more coals on the fire.”
“Yes, sir.”
“And just replenish our cups of coffee. Fresh ground, isn’t it?”
“That it be, sir.
“Dumpty always roasts it himself, and I grinds it. A main good hand
Dumpty is, sir, at roasting coffee. A little morsel of lard in the bottom of the
pan to keep the beans from burning, a good clear fire, and keep them
moving and moving; and there you be, sir.”
“Steward!”
“Sir to you again, sir.”
“Ever anybody ask you for a recipe for roasting coffee?”
“Milk and sugar, sir?”
The milk was another invention of the steward. It was a fresh gull’s egg,
beaten and mixed with hot water, and sweetened with pure preserved milk.
On the whole, everybody did his best on board the old Walrus.
The men forward to-night were very jolly, for, being so near to the end
of their exceedingly long voyage, the captain had spliced the main brace,
that is, he had added one modest glass of rum to their nightly allowance. I
don’t believe in rum myself, but when one is writing a sea story, one must
adhere to the truth. The man who does not face realities and the naked truth,
is like the fabled ostrich that hid its head in the sand when danger
approached.
The men drank “sweethearts and wives,” or “wives and sweethearts,” in
the real good old British fashion. The married men, you know, drank “wives
and sweethearts.” The bachelors, and they were nearly all of that
persuasion, put the “sweethearts” to the front.
They had mixed the grog with a good deal of hot water and sugar to
make it last. But they toasted each other also; and it was, “Here’s to you,
Jack;” or, “Here’s to you, Bill,” or Tom or Joe, as the case might be. And
“We’ve been shipmates now more’n a year, and never a word atween us,
bar a sea-boot now and then.”
And they toasted “The Captain.” “And he is a good fellow,” was the
remark of one sailor, “though a stickler for duty.”
“Ah! Well, Sconce, dooty is dooty all the world. Stick by that, and we’ll
all do well.”
“Dooty,” said another, “is the needle wot points to the Pole, and the Pole
is Heaven itself.”
“Very good sentiment for you, Jack. Here’s to dooty!”
“Now, sir”—this to Charlie—“touch her up, sir. Give us ‘Homeward
Bound,’ and we’ll all chime in, from Dumpty downwards, to the nipper wot
tends the dogs.”
“Homeward Bound” was given with glee; but, of course, it was only a
make-believe, because there wasn’t much home life about Kerguelen.
They sighted the island after passing McDonald and Heard Isles.
Charlie again. He had been determined to be first to see land.
Before the entrance to the creek or natural harbour, where the men and
animals were, is a spit of rocky land, a rugged kind of breakwater, and had
the Sea Elephant been the first inside, her top-masts would have shown
over this.
But here was never a ship’s mast to be seen.
On the shore, high up on a braeside, was an outlook, and the Walrus’s
people saw both American and British ensigns dipped to welcome the
Walrus.
The Walrus returned the salute.
Then flags of all kinds were set in motion, and the signalmen on board
and on shore were very busy indeed, for a time.
“Yes, all was well, now,” said the signalman on shore, “but two dogs
dead, and one Innuit. Sea Elephant had never been seen.”
The anchor was hardly let go when the officer’s boat was alongside, and
he was heartily welcomed down below to exchange experiences.
He and his men had been very busy all the time, and they were ably
assisted and supported by the kindly Yak-Yaks. He spoke in the very
highest terms of Slap-dash, the chief. In the dreary days of winter, when the
island was deep in snow, snow-shoe expeditions were got up; but sleighing,
especially with the bears, who were better suited to the rough work, was
preferred. The Yak-Yak died of inflammation. One dog fell over a cliff and
was killed at once. The other was found dead. Both were buried side by
side, and cairns mark their resting-place. “There is a cairn also,” said Slator,
“on the poor Yak-Yak. I think we nearly all dropped some tears at his
grave.”
I suppose they did, reader, for in the loneliness of such a place as this the
heart is sometimes very near the throat. Sunshine brings mirth and
happiness, gloom depresses, and there is always a certain amount of sadness
in even the songs of northern nations, such as Iceland, Scotland, and
Norway.
Both Charlie and Walt had some doubt as to how the Yak-Yak dogs
would receive them again. But, accompanied by Ingomar, they boldly
marched some distance into the interior, to the kennels. It was the afternoon
of what had been a glorious day, and they had doffed their fur caps and
coats.
The bears were not at home just then. Both bears and dogs, indeed, had
gone away to roam the wilds nearly every day, but the Bruins, with the
dogs, always came shambling or trotting back at eventide, to sleep and to
eat.
They were away then at this moment, and Slap-dash proposed that, with
the Newfoundlands and pet collie, they should all march forth to meet them.
Strangely enough, they had a rendezvous on a hill-top, where most of
them met every night, and from this a beaten track to the camp.
To-day several of the dogs were already at the place of meeting, several
were straggling up from seawards, and in front (for no dog was permitted to
walk behind him) was Gruff, with his well-beloved wife Growley.
When within about seventy yards of the place, where Ingomar and the
boys were standing, both stopped short and sniffed the air. Then Growley
gave vent to a half-choked roar of rage, that shook the hills—well, if it
didn’t shake the hills, it shook the hearts of Charlie and Walt.
“Strangers!” Growley seemed to shout. “I’ll tear ’em limb from limb!”
Gruff rounded on her at once, and promptly knocked her down.
 Then Gruff came trotting on, and Nora and Nick and the collie ran off to
meet them, our heroes following.
That was a pas de joie, a joy-dance, if ever there was a joy-dance in this
world; and those sceptical creatures, who would class dogs and our other
dumb friends as mere automata, would have been converted on the spot to
the dear old doctrine, that animals have souls, had they but seen that dance.
It was too absurdly intrinsically droll for description. The other two
bears, Grumpey and Meg, came up and joined, and presently all the rest of
the bonnie dogs.
They went round and round our heroes in a hairy hurricane; they
pretended to worry each other, they barked and roared, and grumbled and
growled, till the boys’ sides were sore with laughing.
Surely such a scene of merriment was never before witnessed, and when
all had quietened down somewhat, they went amicably back to the kennels.
This is not one of Grimms’ fairy tales, mind, rather is it a fairy tale of
science and natural history, and these, readers mine, are all true.
* * * * *
A whole week passed away, but still no Sea Elephant.
Captain Mayne Brace had taken in more coals, and his arrangements
were all complete, so he was becoming impatient; but at long last the ship
hove in sight over the horizon, and the union was complete.
On comparing logs, it was found that they must have passed each other
at night, and had been probably within ten nautical miles of each other.
The bigger ship had taken many observations, and done a much quicker
voyage. But, knowing that he could be at Kerguelen much sooner than the
Walrus, a happy thought had occurred to Captain Bell. He would run up to
the Cape of Good Hope and endeavour to get a cargo of coals.
Although the war was raging, he succeeded, and now these were landed
in case of emergency, each ship just taking enough for the grand new cruise.
I need hardly say that the meeting between Curtis and Ingomar was most
cordial.
A grand ball was given on shore on the night of re-union.
Sailors are not sailors unless they can have a bit of fun.
 It was a ball of a somewhat heterogeneous description, for men waltzed
with men, though Slap-dash did some really graceful movements with Gruff
and the other bears as partners. There were no ladies, you see, but all the
more freedom and merriment.
Yet, stay; I must qualify this statement. The Eskimos, Yaks, Innuits,
Teelies, or any other name you choose to give them, are droll creatures.
They all dress alike in skins, and their faces are all about the same shape.
Now the very day before the Walrus and Sea Elephant sailed, all being
then on board, except a change of men who were to remain at Kerguelen for
observation duty, Slap-dash came up and saluted Captain Bell.
“Four of my rascals,” he said, “want to speak to you directly.”
Then the four “rascals” were led up and threw themselves on their faces
before Captain Bell as if they had been worshipping the sun.
“Get up, get up,” said Bell, “and speak like men.”
They arose at once and stood before him, and two took a step in advance
of the other two.
“We not all men-people, sir,” said one.
“We not all men-people,” said the other.
Captain Bell began to frown.
“Dis ees my ole woman-people,” said the first speaker.
“Dis ees my ole mudder-people,” said the other.
“Slap-dash,” cried Bell, “did you know this?”
“Not befo’ dis morning, sah; no, no.”
Captain Bell was puzzled and silent. He addressed Ross, the officer who
had been left in charge at Kerguelen.
“No, sir,” said this gentleman; “I don’t see how we can send them on
shore. We can’t want the whole four. They will pine and die if separated.
That would be a dead certainty.”
“Very dead,” said Bell, smiling.
“Besides, though no one suspected their sex, that one called Sheelah is
an excellent cook, and both are capital nurses. We were sick sometimes. We
had green fever in winter, and certain I am that they nursed us back to life.”
The carpenter was next called for.
 “Carpenter,” said Bell, “a small screen berth will be wanted below in
some corner, a kind of l-l-ladies’ cabin. Do ye hear?”
“Well, sir, I do hear, because I’m not deaf; but I don’t understand.”
“Then just do as you are told, Mr. Inglis.”
“Certainly, sir, certainly.”
So a little privacy was obtained for Sheelah and Taffy, and, as it turned
out afterwards, no one was the loser for the “women-people” being on
board.
Do coming events throw their shadows before?
Perhaps they do. Anyhow, when the two ships looked their last on
Kerguelen—the last for a long time, at all events—there was more silence
on board than is usual with sailors going off to sea.
They knew the dangers they were going to encounter, but they were all
quite acclimatized to the rigorous Antarctic climate by this time, and there
was not a man on board, British or American, who was not prepared to do
his best. Which of us can do more?

CHAPTER II

A FIGHT BETWEEN MEN AND ICE

The Sea Elephant’s cruise around the great Antarctic continent, and all her
captain and bold men did, and said and saw, would make a book in itself.
That may one day see the light, as well as the adventures of the men left
behind at Kerguelen.
We must now follow our heroes into a country as widely different in
every way as Scotland or England is from the moon.
Now, having been a boy myself, not so very long ago—apparently—and
being still a boy at heart, I know that boys do not as a rule care for
geography. That is because it is taught in a stupidly, awkward way at
schools, a method being adopted which is devoid of all interest. But never
mind, I do wish you for once in a way to take a look at the map here
presented to you. The ships were off south and east from Kerguelen Isle,
and the first port to be struck was Termination Land. It was not to be the
termination of their cruise, however, by a very long way.
 Would you be surprised to learn that there are two poles in the south, and
two in the north, the magnetic and real poles.
The real axis, the hub of our “terral” wheel, is the one we have to deal
with.
Here all meridians may be supposed to meet at a point.
There would in reality be no more south for a man standing at this pole.
Let him look in which ever way he liked, to Africa, to South America, or
New Zealand; it would all be north, north, north. No east, no west, just
north.
The Sea Elephant and her sister, the Walrus, were not to be run into any
danger along the coast of Wilkes’ Land, which marvellous line of shore
may be said to stretch from Termination Land and Island, right away to
Ringgold’s Knoll, far, far east. It is, or is supposed to be, the longest stretch
of coast land in, or any way around, the Antarctic. There is no mistake
about this being land, nor that it is indented with bays and gulphs, just as
the west coast of Scotland or Norway is; and these indentations may really
divide the continent in places.
I only want to give you some rough idea of this land coast. Had you then
been able to sail along it many thousands of years ago—and you would
have had to be up very early indeed to do so—before there was any ice here
at all, when the shores were green and forest-clad, the sight you would have
witnessed would have been a very beautiful one indeed! Hills and vales and
mountain land, and probably in the farther interior, vast sierras, the woods
teeming with strange animals; and strange birds would have been there, too,
sailing over the forests, or floating on blue seas, alive with myriads of fish
of various species, many now lost and gone, others still extant because they
have migrated.
But now, though the same formation of surface and contour of hills may
remain, they are all, all snow-clad, and protected seawards by a barrier, or
barriers of ice, of every description, which few mariners would care to
negotiate.
* * * * *
The weather continued favourable, but there were many days of darkness
and gloom; and after Termination Land had been reached, it was not
considered advisable—strong and well fortified though the ships were—to
be among the ice when the shadows of great clouds enveloped the land, or
when storms were threatened. But when the sun shone, and the ice was
open, then they boldly ventured to push their way through, either under
steam, or under sail.
Ice like this closes very suddenly, and if the captain of an exploring ship
is not very clever, he may get caught, and a week’s imprisonment counts
against a ship when making a voyage.
Sailing in a pack like this, a vessel to a landsman would seem to be in a
very dangerous position.
She may be, though no one on board appears to think so. The ice is here,
the ice is there, the ice is all around; flat bergs, like what you meet in the
north; pancake ice, lakes of slush, and those terrible masses, or square
mountains of land-ice—a characteristic feature of this country—with caved
perpendicular sides, striated on the horizontal, or, if they have been melted
by the sun at one side, oblique, and glittering gorgeously blue, green, or
paley white, in the sun’s rays.
But all, big or small, covered with snow, so that their very whiteness
dazzles the eyes. But at this season there were birds everywhere, and seals
of many species. The penguins, I need hardly add, were a very curious
sight, as they stood or staggered about on the low flat bergs. Our heroes saw
some sea-elephants, though I believe these, as a rule, are far more common
to the south of Tierra Del Fuego.
One day, when the ships were pretty close together, and well in through
the ice, the sky cleared far too quickly to please Captain Mayne Brace. He
knew at once that John Frost would have them in his clutches, if they did
not soon beat a retreat.
So he signalled to his consort, and both vessels quickly had their heads
turned to the north.
They might have found themselves clear in a few hours had it not
suddenly come on to blow from the cold and icy south.
The ice began to pack.
Steam was got up with the greatest despatch, and nearly all sail taken in.
Luckily there was no swell, else there would have been pressure enough to
have thrown both vessels on their beam-ends on a floe.
The Sea Elephant was leading, and by-and-by the Walrus managed to
creep right into her wake. This was an advantage for a time. A south wind,
even with a clear sky, would naturally open the ice, but there was some
demon current working underneath that they could not account for; and
while they were still two miles from clear and open water, they found
themselves rapidly becoming part and parcel of the pack.
Break the ice, did you say? I should smile. You may get steam
machinery to smash bay-ice, or splinter pancake, but not your solid, heavy
pieces. Oh no! So men who have inventions of this sort should sell them to
farmers at home to break up their mill dams in winter.
Then came a battle ’twixt men and ice. Men with their cunning, ice with
its force of movement, slow but sure.
Both ships got closer together, the Sea Elephant leading, all hands that
could be spared from both ships, over the side in front of the foremost.
Armed with great poles, they moved the bergs on every side.
It was bitterly cold work, and the pieces moved but slowly.
Under all the pressure of steam she could produce without risk, aided by
the men over the side, the Sea Elephant forged her way slowly, fathom by
fathom, indeed, but after a time that to our heroes seemed interminable, her
jib-boom hung over the black water.[D]
Then came the scramble to get inboard, and though their fingers were
about as hard as boards, and some had frozen faces, in less than ten minutes
all hands were once more on their respective decks.
Sail was once more set, fires were banked—save the coals they must—
and away they went, right merrily, to the east again, the wind well on the
starboard beam.
Although the men had raised a cheer when the ships were quite out of
that ugly pack, there was no fear in any breast.
“Would there have been much danger if we had been beset in there,
uncle?” Charlie ventured to ask the captain, at supper.
“A fig for the danger, boy. We’ll never be out of that, but we came to
find the South Pole, or get somewhere near it.”
Ingomar smiled.
“Well, then, Hans, we have come to make a big record.”
“That will beat all creation, captain.”
 “Yes, beat all creation, and it would have been misfortune, to say the
least of it, to have got beset. That’s all. Yes, thanks, steward, I’ll have
another slice.”
* * * * *
The two ships stood steadily onwards now, day after day, sailing
whenever they could, steaming only when obliged to, for the economy of
coal had to be studied, and that, too, most carefully.
Captain Bell, of the Sea Elephant, came now to be recognized as head of
the expedition, though on every occasion that was deemed important a
council was called and the opinions of all officers taken.
He was now always called The Admiral, but not to his face. He was
none too fond of fine titles.
And the Sea Elephant was called the Flag Ship, for short.
One day, when in the neighbourhood of the Knoll, the Admiral signalled
to the Walrus, that as they would soon round Wilkes’ Land and stand down
south, it would be best for all hands to bend their cold-weather gear.
In shore English that would signify, give out the supplies of winter
clothing.
As it turned out, this was very excellent advice indeed.
The Eskimos had their supply first and foremost, and this they had made
themselves, under the supervision of Slap-dash, and from seal-skins with
the hair on.
Slap-dash assured Captain Bell that there was nothing so good for
keeping out the cold, and his words turned out to be true. Most, however, of
the sailors and their officers still stuck to flannel and fur.
Both Charlie and Walter had a very great desire to see the inside of a real
ice-cave. These caves look like archways, or the openings into tunnels, and
are formed by the dash of the waves on huge bergs of land-ice, or even in
the sides of the ice-barrier itself.
They had their desire fulfilled one day, while the ships lay almost
motionless on the dark water.
There wasn’t a breath of wind, nor was there any fog. And the surveyors
were engaged very busily indeed, in taking soundings, and bringing up
specimens of the mud or clay at the bottom for examination.
 Fires were banked, but the ships were at no great distance from a lofty
ice-wall, at the foot of which were several caves.
They rowed on shore at sunset.
And the appearance of that sunset was in itself a sight to behold!
The sun was sinking slowly down to the north of west, and in a cloudless
sky. It seemed a larger sun than our young heroes had ever yet beheld, and
cast its reflection on the heaving waves ’twixt boat and horizon, in a very
remarkable way; for although the sheen was bright, it was not dazzling. Nor
was the sun itself. But nearer to the spot where our heroes stood, on the
field of level ice betwixt them and the ice-caves, were many shades of opal
and pearl.
“We must be moving,” said Ingomar, “at last, boys, or we will not get
home to-night.”
“Oh!” cried Walter, “I wouldn’t mind staying here all night to look at the
sky.”
“Nor I,” said Charlie. “I’d like to sleep in the snow. Nothing could harm
us except the frost, and we should be in our sleeping-bags, so that couldn’t
hurt much.”
“There are no snakes here, anyhow.” This from “wise Walter,” as Charlie
sometimes called him chaffingly.
“No, Walt; and no burglars, either.”
There was one thing to be said for the dogs, Nick and Nora and Wallace.
They had long ago fully made up their minds to enjoy themselves to the
fullest extent, whenever they had the chance.
They were tearing round and round on the ice-floe at this moment,
wriggling and jumping and playing at leap-frog, while Nick would pause
every moment to fill his mouth with snow and fling it over his neighbour’s
shoulders.
The boys must have just one more look at that sky before they entered
the ice-cave.
Lo! what a change. The sun was all but down, and sea and sky had
changed to orange, deep and charming. The very snow was orange.
But judge of their disappointment when they entered the first cave and
found that all was pitch dark.
 CHAPTER III

THE BEAUTY AND MARVELS OF AN ICE-CAVE

“Oh, what a shame!” cried Walt, impatiently. “We did expect to see
something real splendid.”
Ingomar laughed.
“You are snow-blind, boys, just for the moment. If you’d come when I
told you, when the sun was still above the horizon, you would have had a
daylight view.
“The sun can’t be expected to stay for you. He has to rise and shine on
other seas, if not on other lands.”
But when their eyes became more accustomed to the twilight, they could
see that they were in a vast vaulted cave, solid ice and snow beneath them,
and strange uncanny shapes sparkling in the semi-darkness beyond.
Three men had accompanied Ingomar and the boys, and one was
carrying a bag.
“Be cautious how you move, lads, else one of you may go through into
the sea, and never be seen again.”
“But the ice feels very strong.”
“Yes; it is perhaps a foot thick, and that is strong enough for anything.
But there are ‘pussy-holes’ here and there, up through which seals crawl to
sleep, and on these the ice is very thin.”
Just as he spoke, there was a sudden and angry roar heard ahead of them,
where something black and big reared itself, and two fierce eyes glared at
the intruders.
The boys clutched each other in superstitious fear, and stepped quickly
back.
It was only a large seal, however, but so quickly did it retreat that
Ingomar had not the slightest idea what species it was.
I may say for the seals here in the Antarctic, which number four or five
species, that though in the breeding season they have certain habitats, after
that happy time is over they are free to wander where they please, and often
turn up in strange places. It is the same with Arctic seals.
 An eared seal, whose fur has been much sought after, is now, I think,
almost extinct, owing to the murderous greed of the sealer. I think it would
be well if there were a close season for all species. But this is a digression.
Let us return to the cave.
The somewhat mysterious bag carried into the cave was now opened,
and Ingomar, bending down, extracted some of what he termed theatrical
properties therefrom.
Next moment, on the touch of a button, the whole of this cave was filled
with dazzling light.
What a sight!
“Oh—h—h!”
That was all our boys could say for a moment or two.
No stalactite cave probably ever rivalled the beauty of this.
And here were stalactites, too, in the form of depending icicles, dozens,
scores, hundreds of them, and, seen by the electric light, they emitted all the
colours of the rainbow.
They walked cautiously on and on a long way into the bowels of this
mighty cavern, watching the floor for pussy-holes.
No one could even guess where the seal had gone.
“Well,” said Charlie, as they came at last to the end of the ocean-
hollowed cave, “I should really have expected to find mermaids here.”
“Now,” said Ingomar, “one more transformation scene, or perhaps two,
and then the pantomime is over.”
As he spoke he touched a spring, and, wonderful to say, the cave was
illuminated with brightest crimson, then with orange and red again. So on to
the pure white light, and in this they found their way to the mouth of the
cave, and made their exit and presently their way to the boats.
“We’ve seen a sight,” said Ingomar, “that is surely worth coming to the
Antarctic to look upon.”
“Yes,” said Charlie, thoughtfully.
“Oh,” cried Walt, “will you do it again some time?”
Ingomar laughed.
“It all depends,” he said.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebooknice.com