Information Technology and Control 2021/2/50 357

Cloud-Based Multi-Robot Path Planning in Complex and Crowded

ITC 2/50
Environment Using Fuzzy Logic and Online Learnings
Information Technology
and Control
Vol. 50 / No. 2 / 2021
Received 2020/12/29 Accepted after revision 2021/05/18
pp. 357-374
DOI 10.5755/j01.itc.50.2.28234 http://dx.doi.org/10.5755/j01.itc.50.2.28234

HOW TO CITE: Zagradjanin, N., Rodic, A., Pamurar, D., Pavkovic, B. (2021). Cloud-Based Multi-Robot Path Planning in Complex
and Crowded Environment Using Fuzzy Logic and Online Learning. Information Technology and Control, 50(2), 357-374. https://doi.
org/10.5755/j01.itc.50.2.28234

Cloud-Based Multi-Robot
Path Planning in Complex and
Crowded Environment Using
Fuzzy Logic and Online Learning
Novak Zagradjanin
University of Belgrade, School of Electrical Engineering, Bulevar kralja Aleksandra 73, 11120 Belgrade, Serbia

Aleksandar Rodic
University of Belgrade, School of Electrical Engineering, Bulevar kralja Aleksandra 73, 11120 Belgrade, Serbia
Mihailo Pupin Institute, Volgina 15, 11060 Belgrade, Serbia

Dragan Pamucar
Department of Logistics, Military academy, University of Defence, Pavla Jurisica Sturma 33, 11000 Belgrade, Serbia

Bojan Pavkovic
Military technical institute, Ratka Resanovica 1, 11030 Belgrade, Serbia

Corresponding author: [email protected]

This paper considers an autonomous cloud-based multi-robot system designed to execute highly repetitive tasks
in a dynamic environment such as a modern megastore. Cloud level is intended for performing the most demand-
ing operations in order to unload the robots that are users of cloud services in this architecture. For path planning
on global level D* Lite algorithm is applied, bearing in mind its high efficiency in dynamic environments. In order
to introduce smart cost map for further improvement of path planning in complex and crowded environment, im-
plementation of fuzzy inference system and learning algorithm is proposed. The results indicate the possibility of
applying a similar concept in different real-world robotics applications, in order to reduce the total paths length,
as well as to minimize the risk in path planning related to the human-robot interactions.
KEYWORDS: multi-robot system, path planning, cloud technology, fuzzy logic, learning algorithm.
358 Information Technology and Control 2021/2/50

1. Introduction
The basic purpose of an algorithm used in the robot constraints. Therefore, a human-aware algorithm for
path planning process is to determine a valid path planning takes into account the crowds (as well as
from the start to the goal position of the robot in its other limitations not covered by this paper) on local
configuration space. Robot configuration space im- level and defines a valid local path. Such solutions are
plies an approach that fully defines the location of a common, but if the algorithm for planning the path on
robot, that is, all its degrees of freedom [24], in the global level does not fully take into account crowds,
environment in which it moves and performs tasks. this can reduce the efficiency of the entire system and
In the case of multi-robot path planning, the situa- increase the risk of the task [2]. In addition, in some
tion is much more complicated since the growth of situations a local planner cannot solve the problem, so
the joint configuration space is exponentially pro- it is necessary to replan the path on global level during
portional to the number of robots [19]. Therefore, the robot motion. These situations are specially con-
single-robot path planning techniques cannot be sidered in this paper because they can significantly
applied in the case of a multi-robot system without slow the robot and cause at the same time increasing
some adaptation. the total paths length.
Generally speaking, there are two approaches that In order to construct a crowd-sensitive global plan-
can be applied to algorithms designed for planning the ner, knowledge about the crowds global changes or
path of a multi-robot system [36]. The first approach behavior is required. This is not always feasible, be-
involves a coupled technique that treats the multi-ro- cause over time the crowds can behave differently. In
bot system as a single entity so that the paths of all ro- some cases, crowd density has a relatively repeatable
bots of that system are calculated simultaneously and dynamic of change on daily, weekly or some other
coordinated in the joint configuration space. There- time basis. For instance, in some part of the store the
fore, this approach can guarantee the completeness of crowd is likely to be larger every day in the morning,
the solution (all paths will be calculated if a solution in the second one every weekend, in the third one for
is possible), but its disadvantages are that it is diffi- holidays or special events. Therefore, it is desirable
cult to apply to a large number of robots and usually that robots should be capable to predict crowd densi-
does not provide a solution in real time. The second ty change, whenever and wherever possible, in order
approach involves a decoupled technique that, within to improve the efficiency of task and motion planning.
a multi-robot system, first calculates the path of each To this end, the implementation of fuzzy inference
individual robot independently of the others, and then system (FIS) is proposed, which should ensure that in
applies some method of motion coordination to avoid the initial phase of planning the global planner takes
conflict situations. This approach is fast and usually into consideration the crowds whose density change
provides a real-time solution, but its disadvantages is predictable. The goal is to penalize the path passing
are that it cannot guarantee the completeness of the through the high-cluttered parts of the store propor-
solution, as well as that movement of multiple robots tionately to the crowd density.
along independently calculated paths can cause con- In addition to using FIS, robots have the ability to
flicting situations that cannot be solved. learn online in order to reuse detection of unexpect-
This paper discusses an autonomous cloud-based ed long-lasting changes (that do not move/change for
multi-robot system designed to perform highly repet- a certain period of time) in environment structures
itive tasks in a dynamic environment such as a mod- from previous iterations for more efficient task and
ern megastore. In robotics, the method often used to motion planning in the next iteration. The changes
calculate the path involves the use of an algorithm for that are referred to here are, for example, accidentally
planning rough path on global level integrated with spilled larger quantities of goods that occupy certain
an algorithm for planning precise path on local level parts of the store for some time, temporarily blocked
[11]. In complex environments such as a megastore, or opened passages, etc. To this end, it is proposed to
algorithm for planning the path on global level defines use the learning algorithm based on the online statis-
path that do not take into account crowds and other tical estimation of changes in the environment.
 Information Technology and Control 2021/2/50 359

D* Lite is applied as graph-based algorithm for plan- of the system. The same idea can be implemented in
ning the path on global level. In this paper, it is adapt- industrial and other similar environments.
ed so that it can be used to plan the path of a multi-ro-
bot system. A decoupled path planning technique
is applied with robots motion coordination to avoid
conflict situations. 2. Related Work
As it is known, graph based algorithms require a cost Graph-based techniques are used with both algo-
map of the robot environment to operate on. Conse- rithms - algorithms for planning the path for sin-
quently, one of the approaches to improve path plan- gle-robot and algorithms for planning the path for
ning is to generate a smart cost map. The cost map is multi-robot system. M* algorithm [35] is based on
used to store various information about the environ- the search for path in joint configuration space with
ment that affects the robots motion. This area has dimensionality gradation depending on the need for
not been sufficiently studied, especially in the case motion coordination. In [4] A* algorithm for multi-ro-
of crowded environment, and provides many op- bot path planning is proposed using decoupled ap-
portunities for further research. In that context, the proach and priorities. A lattice variant of A* algorithm
contribution of this paper is reflected in the original for planning the path for multi-robot system is pro-
application of FIS to exploit predictable changes in posed in [8]. D* is an incremental algorithm widely
crowd density (in spatio-temporal domain), in order used for path planning in a completely unknown or
to generate smart cost map to improve autonomous partially known environment, both for single robot
robot path planning in crowded environment. In ad- and for multi-robot systems [6]. A multi-agent ver-
dition, it is also proposed a learning algorithm that sion of D* Lite, which is a variant of D* [18], is pro-
has the same purpose as the FIS, but exploits data posed in [1]. Another approach to apply D* Lite for
of other characteristics. This learning algorithm is planning the path for multi-robot system with imple-
based on the M-out-of-N detector, which is most mented concept of parallel processing is proposed in
commonly used in radar technology. In this case, the [30]. Here, during the planning, other robots, except
original application of this technique is presented to the origin, are considered as obstacles.
detect the presence or absence of an obstacle in a cell In addition to the above mentioned, in recent years one
of smart cost map. of the main directions of research in robotics is appli-
In summary, the common goal of applying fuzzy log- cation of different learning techniques to robot path
ic and online learning is to generate a smart cost map planning, in order to improve its efficiency. In [29] a
that is used for defining path in the initial planning mission coordination architecture is proposed, which
phase so that they are similar to the real ones to a executes planning the path through check-points and
feasible extent. This will be an attempt to reduce the assigning priorities, based on statistical estimation of
number of paths corrections during robots motion changes in the environment with purpose to improve
and thus to improve system efficiency. Replanning the replanning process. The robot motion planner pro-
the paths increases the execution time of the task to posed in [20, 25, 43] uses a formed database that stores
a lesser or larger extent, often implies robots delays, segments of the local paths depending on the geomet-
degrades path quality as well as overall performance ric shape of the obstacles, in order to form a global path
of the system. As a result, it can be expected that the based on them depending on the specific situation. An-
total length of the paths will be reduced (the sum of other learning based method to reuse information from
the path length of all robots in all iterations in one cy- previous motion plans, the environment, and types of
cle), making them on average “smoother” in terms of the obstacles is presented in [15]. Path planning using
a smaller number of sharp changes in direction (eas- experience is also proposed in [14, 32, 33], where so-
ier to follow by robots). The aim is also to reduce the called experience graphs are introduced, which in fact
risk in path planning, which refers to the movement represent a path network from previous iterations.
of humans and robots in the same environment, that This approach is very useful in the environment which
is, to ensure an amount of safe space between robots possesses a significant amount of underlying structure
and crowds. That improves the overall performance that is under-utilized in a classic path planning and
360 Information Technology and Control 2021/2/50

mobile manipulation. If there is no experience plan- approach to path planning, in terms of optimizing the
ning is done from the beginning, of course. In [5] and global cost map taking into account all factors affect-
[9], the concept of learning based on the experience of ing the robots motion in the environment and having
trajectories in high-dimensional space is described, in mind a mission specificity that requires the man-
and this knowledge is used by robots for planning in agement of risks arising from different sources. In [2]
the next iterations. An interesting approach of using two algorithms (CUSUM-A* and Risk-A*) that learn
learned maneuvers from previous situations that re- a cost map in crowded environments and plan from
quired avoiding collisions with obstacles in order to it are described. These algorithms continuously up-
improve planning in subsequent iterations is consid- grade crowd models based on current observations.
ered in [34]. In [28] the concept of robot path planning The CUSUM-A* algorithm uses in the path planning
using external preferences and suggestions by the user process prior knowledge of changes in the crowds in
is addressed. the spatio-temporal domain, while Risk-A* focuses
When using graph based algorithms, researchers of- on optimizing path costs in terms of human-robot in-
ten focus on smart cost map to improve path planning teractions.
process. In [12] it is described how the clever design Based on the analysis of the literature, it can be con-
of a grid based cost map contributes to the robustness cluded that the smart cost map is a good approach
of the system in a real application. This approach to improve robots’ path planning in general, even in
used so-called combined cost map. It incorporates a environments they share with humans. Despite the
cost map derived from known static obstacles, as well unpredictability of human movement, it can often be
as a cost map based on the structure of roads in the stated that in public objects, especially in stores and
environment and other available information about similar environments, crowd density has a relatively
terrain that can affect the robots motion. Dynamic repeatable dynamic of change (in spatio-temporal do-
obstacles in this cost map are presented as a high cost main), that can be used to generate smart cost maps
region around the obstacle to ensure a safe distance. to improve robot path planning. The existing liter-
In [26] a cost map based on probability is presented, ature does not sufficiently consider this issue. The
which takes into account the uncertainty of terrain same can be said for the application of learning al-
characteristics for each grid cell in the map. The fi- gorithms to improve robot path planning in crowded
nal cost map allows the calculation of the optimal environments. Keeping that in mind, the tendency of
path between the two points in the environment, as this paper is to contribute to overcoming mentioned
well as the distribution of likely paths between the research gap.
points. To construct a crowd-sensitive path planner
based on smart cost map is a special challenge. In [40]
it is presented approach based on learning the cost
map that best defines previously observed motion 3. System Overview
and trajectories of pedestrians. This approach uses a Multi-robot systems have many applications and the
cost map at the beginning of the process that ignores field of application is constantly expanding. For the
predictions of future pedestrian locations. Then, in purpose of performing mission successfully, the allo-
such a map, robot paths are planned, at the same time cation of tasks among the robots in multi-robot sys-
pedestrian trajectories are simulated (on the basis of tem, path planning and motion coordination must be
previous observations), and based on that, new costs primarily defined. With the aim of a wider application
are defined in a cost map proportional to the hin- of a group of robots, they need to be as simple as pos-
drance probability at each cell. The paper [37] pres- sible, energy efficient, low cost, and at the same time
ents similar cloud-based path planning concept using to perform the assigned tasks reliably. Clearly, some
D* Lite algorithm, but there the implementation of of these requirements are conflicting. The solution
Multi-criteria decision making (MCDM) using Full can be based on cloud technology. With the applica-
consistency method (FUCOM) for improvement the tion of cloud technology, it can be achieved that the
efficiency of path planning is proposed. The applica- multi-robot system uses the capacities and resources
tion of MCDM using FUCOM provides an adaptive of the cloud (information, memory, communication
 Information Technology and Control 2021/2/50 361

and other) [3, 13]. In practice, this usually means that architecture has been designed, which consists of
the cloud processes a large amount of data, while ro- the following modules: Data base, Store, Goods flow,
bots, as cloud service users, receive the necessary Warehouse, System manager and Robots (Figure 1).
data and information that enables them to perform
tasks [7, 16]. Let’s take the modern megastore as a typ- Figure 1
ical example of a complex dynamic system, where the Block diagram of the system architecture
flow of goods and money takes place and where peo-
ple are present as customers and employees. In such
a system, a group of robots may first be intended for requests f
smart logistics i.e. to perform tasks such as: goods de- store, mak
livery (delivery robots), autonomous goods handling warehous
(manipulation robots), autonomous scouting (scout requests
robots), cleaning (cleaning service robots), info ser- number o
vice (info robots), shopping assistance (shopping ro- making th
bot assistant), etc. Planning and executing the above the start a
tasks at the same time involves great resources for replannin
data processing. For these reasons, the environment inference
of modern megastore can be considered as one of the improve t
possible scenarios where a multi-robot system based nation, fi
on cloud technology can be deployed. In order for this map upda
complex system to perform tasks with sufficient reli-
ability in the cloud environment, the megastore must
have an appropriate information infrastructure, that
it is fully covered with a distributed network of differ- TheFigure
module1. Block diagram of the system architecture.
“Data base” is intended to provide a store
ent sensors, as well as with wireless communication and warehouse map (free fields, dynamic and static
between the participants of all involved processes. The module “Data base” is intended to provide a
obstacles, shelves with goods), as well as information
Any change in the environment is identified on the store and warehouse map (free fields, dynamic and
on articles specification and distribution on shelves
cloud level, processed and necessary information re- static obstacles, shelves with goods), as well as in-
in the store and in the warehouse.
lated to the changes is exchanged between networked formation on articles specification and distribution on
The module
shelves “Store”
in the store simulates
and in a dynamic environ-
the warehouse.
clients. This approach makes possible to organize the
ment in the store
The module “Store”in sense
simulatesof appearance
a dynamicand move-
environ-
performing of different tasks in megastore with syn-
ments
ment in theof the buyers.
store Theof
in sense number of buyers
appearance and ismove-
given
chronized accomplishment and to manage collabora-
as anofinitial
ments parameter
the buyers. The innumber
the phase of initialization,
of buyers is given
tive strategies of multi-robot system. One of the basic
tasks in given model of high automated megastore is as while theirparameter
an initial moving is in of athe
random
phase nature. This mod-
of initialization, Figure 2. M
ule also simulates the changes in environment in the black – w
planning the path of robots in presence of static and while their moving is of a random nature. This mod- st
sense of blocking certain initially free
ule also simulates the changes in environment in the cells or releas-
moving obstacles. For example, the typical mission
ing of
sense theblocking
initially certain
blockedinitially
transits.free
Thecells
initial map of
or releas- The loa
that can be put to robot in charge is to transport goods
the store and warehouse is shown in Figure
ing the initially blocked transits. The initial map of 2. domly sel
from the warehouse to the store in advance specified
place, for the required time, moving along the path theThe
storemodule
and warehouse
“Goods flow”is shown in Figure
monitors 2. of the
picking Based
that is most favourable in terms of multiple criteria. The module
goods from the“Goods
shelvesflow”
in themonitors
store andpicking of theof
purchase system in
This is just one of the complex tasks in the described goods from (with
the same the shelves
the cash in the storeverification).
register and purchaseThis of that “Syst
scenario relocated from the robot level to the cloud themodule
same also(withprocesses
the cashthe register verification).
loss of the goods due to This
the operation
level and as such it represents the focus of this work. module also processes
expiration date and/or thedamaging
loss of the goods
etc., due the
updates to the
list formation
For the needs of Matlab simulation of an autonomous and schedule
expiration of goods
date and/or in the store
damaging etc.,and generates
updates the
the list tween rob
multi-robot system designed to perform highly repet- andgoods amendments.
schedule of goods in the store and generates the tem and
itive tasks of goods transport from the warehouse to goods amendments.
The module “Warehouse” monitors the situation in planning,
the store and other tasks in complex dynamic envi- Thewarehouse
the module “Warehouse”
and delivery monitors
dynamics the(updates
situation the
in
ronment, such as a modern megastore, a cloud-based thelist
warehouse
and schedule andofdelivery
warehoused dynamics
goods).(updates the
list and schedule of warehoused goods).
The module “Robots” in a simulated environment
represents robots that move along defined paths, ob-
serve the environment with their sensors, and perform
assigned tasks. In this case, the system consists of
three robots for the transport of goods (homogeneous
system).
 362 Information Technology and Control 2021/2/50

The module “Robots” in a simulated environment operations related to gathering and grouping of infor-
represents robots that move along defined paths, ob- mation, their processing, allocation of tasks between
serve the environment with their sensors, and per- robots, application of the fuzzy inference system and
form assigned tasks. In this case, the system consists learning algorithm, path planning and replanning,
of three robots for the transport of goods (homoge- motion coordination, etc.
neous system).
The module “System manager” is a cloud level. It Figure 3
Gantt chart of operations allocation between the modules
is designed to perform the most complicated tasks,
primarily related to data processing and activity co-
ordination, which ensure the successful functioning
9

Data base
is usually do
of multi-robot system, for example: receipt of the re- 8 Store ment that ar
quests for adding the goods to the shelves in the store,
Goods flow
System manager
pass, while t
7
ing its motio
making the plan of transport the goods from the ware-
Robot1

6
Robot2
this data. In
house to the store (in accordance to the received re- Robot3
Warehouse may occur d
quests for adding the goods to the shelves, the number 5
environment
requests for adding the goods to the shelves in the

Moduls
of available robots and optimization criteria), mak- dynamic and
store, making the plan of transport the goods from the 4

ing the plan of engaging the available robots with the ning also cha
warehouse to the store (in accordance to the received 3
start and goal positions of each robot, planning or re- the solution
requests for adding the goods to the shelves, the
planning
number the pathrobots
of available of robots,
and application
optimizationofcriteria),
the fuzzy 2
optimal or e
inference system and learning algorithm in order the mentione
making the plan of engaging the available robots withto 1

process (exp
the start and goal positions of each robot, planningcoor-
improve the path planning efficiency, motion or 0
path from th
dination,the
replanning finding a solution
path of to conflicting
robots, application situations,
of the fuzzy
0 0.2 0.4 0.6 0.8 1 1.2

map update with gathered data. Time [s] tion of the ro

inference system and learning algorithm in order to search result
improve the path planning efficiency, motion coordi- Figure 3. Gantt chart of operations allocation between the modules. lems are mo
Figurefinding
nation, 2 a solution to conflicting situations, mental algor
A direct consequence of this is the unloading of robots
mapMap of thewith
update storegathered
and warehouse
data. A direct consequence of this is the unloading of bearing in m
as they are able to use cloud services.
robots as they are able to use cloud services. ment map ar
Since thethe
Since mentioned
mentionedarchitecture
architecture isis of
of aamodular
modular previous sea
type, it can be adapted for various other applications.
type, it can be adapted for various other applications. correct an ex
This
Thismay
mayinclude
includeincreasing
increasing the
the number
number of of robots in a
robots in to these algo
system,
a system,of of
heterogeneous
heterogeneousororhomogenous
homogenous type. D* Lite is
rithm. It is a
vide a from it by a
ic and 4. Improvement of Multi-Robot Path Planning respect. D*
as in-
ion on
4. Improvement of Multi-Robot Path tion basicall
4.1. Some Properties of D* Lite Algorithm Then, if chan
Planning Lite perform
viron- rent solution
4.1. In orderProperties
Some to plan the robot motion,
of D* Lite the environment
Algorithm
move- map can be presented as a 2D uniform resolution late the new
given In order to grid.
occupancy plan Inthethis
robot
grid,motion,
each celltheisenvironment
assigned a search result
blue
Figure 2. –Map
entrance/exit, redwarehouse
of the store and – shelves with
(bluegoods in store,
– entrance/exit,
zation, pink – shelves with goods in warehouse map can be
traversal presented
cost as a zero,
greater than 2D uniform resolutionthe
which represents oc- if a graph ch
black – wall,
black – wall, green – cash register, red – shelves with goods in
mod- green – cash register, difficultygrid.
cupancy of entering that each
In this grid, cell. cell
Thisisgrid is usually
assigned a tra- search (does
store, pink – shelves with goods in warehouse).
in the approximated
versal as a than
cost greater discrete graph
zero, and represents
which some graph- the This makes
the basic A*
releas- The load of each module in one iteration of a ran- difficulty of entering that cell. This grid is path
based search technique can be applied for plan-
usually ap-
ning. One ofasthe techniques by which this graph-based
can be per- D* Lite u
map of The selected
domly load of each module
scenario in oneiniteration
is given of a random-
Figure 3. proximated a discrete graph and some
formed is to define the centre of each cell as a single from sgoal to
Based on Figure
ly selected scenario3,isitgiven
can in
beFigure
concluded
3. that the search technique can be applied for path planning. and updates
node in the corresponding graph. Adjacent nodes are
of the system
Basedintensively
on Figure uses cloud
3, it can be resources
concludedinthat
thethesense
sys- One of the techniques
connected by which
with the edges. this is
Edge cost can be performed
usually adopt- characterize
ase of thattem
“System manager”
intensively usesexecutes the most in
cloud resources demanding
the sense isedtoasdefine
some combination of the traversal cost of node
the centre of each cell as a single nodesin • g(s), th
. This operations related
that “System to gathering
manager” executes andthegrouping of in-
most demanding the corresponding
it connects graph.
and the edge Adjacent
length nodes are
[11]. Generally con-
speak- found so far;
to the formation, their processing, allocation of tasks be- ing, the purpose of robot path planning algorithms, • h(s) or
he list tween robots, application of the fuzzy inference sys- including those from the A* family, is to find a path cost of movi
es the tem and learning algorithm, path planning and re- from start to goal position with minimal total cost. ensures that
planning, motion coordination, etc. The A* algorithm calculates a path assuming that most optimi
tion in the graph in which the search is performed is static, optimal path
es the i.e. not to change. However, the real situation is usu- search);
ally often such that the environment in which the • f(s) = g
 Information Technology and Control 2021/2/50 363

nected with the edges. Edge cost is usually adopted D* Lite unlike basic version of A* searches path from
as some combination of the traversal cost of nodes it sgoal to sstart. During operation, D* Lite generates and
connects and the edge length [11]. Generally speaking, updates the value of the following functions that
the purpose of robot path planning algorithms, in- characterize cell s:
cluding those from the A* family, is to find a path from _ g(s), the minimum cost of moving from s
goal to s
start to goal position with minimal total cost. found so far;
The A* algorithm calculates a path assuming that the _ h(s) or heuristic value, estimates the minimum
graph in which the search is performed is static, i.e. cost of moving from s to sstart. Using heuristic value
not to change. However, the real situation is usually ensures that the search tree is directed toward the
often such that the environment in which the robot most optimistic cells in terms of belonging to the
moves is only partially known or even completely un- optimal path from start to goal cell (this speeds up
known. In this case, path planning process is usually the search);
done assuming that parts of the environment that are _ f(s) = g(s) + h(s), estimates the minimum cost of
unknown at that moment are free to pass, while the moving from sstart via s to sgoal;
robot observes the environment during its motion _ rhs(s) = min
s’∈Succ(s)(c(s, s’) + g(s’)) or rhs(s) = 0
and the map is updated on the basis of this data. In ad-
if s = sgoal, one step lookahead estimate of s and
dition, changes in the environment may occur during
represents path cost derived from looking at the
the robot motion. In both cases, the environment map
g values of its neighbors. In implementation, each
available for mission planning is dynamic and there-
cell maintains a pointer to the cell from which it
fore the graph used for path planning also changes.
derives its rhs value, so the robot should follow the
A consequence of this may be that the solution gen-
pointers from its current cell to pursue an optimal
erated by the A* algorithm is not optimal or even not
path to the goal.
valid. In order to avoid this, in the mentioned cases A*
algorithm resets the search process (expansion of the The pseudocode of of D* Lite is shown in Algorithm 1
cells) and calculates the path from the beginning (rel- [22] and its main steps will be explained below. During
ative to the current position of the robot), without the the path searching, D* Lite classifies cells as consis-
possibility of using the search results from the previ- tent if g(s) = rhs(s), or inconsistent in other case. Cells
ous steps. These problems are more effectively solved that are inconsistent can be overconsistent (if g(s) >
by so-called incremental algorithms (an extension rhs(s)) or underconsistent (if g(s) < rhs(s)). Similar to
of A* algorithm), bearing in mind that when changes other algorithms from the A* and D* family, D* Lite
in the environment map are detected, they use the generates and updates OPEN list (set of inconsistent
results from the previous search steps to calculate a cells) to perform efficient expansion of these cells
new one or to correct an existing path if necessary. D* and develop a search tree. Expansion of cells (the cell
Lite belongs to these algorithms [17]. becomes expanded when it is removed from OPEN
D* Lite is a variant of the more famous D* algorithm. list) is based on the key function as in line 8, while
It is at least as efficient as the D*, and differs from it its value is defined as in line 1. Initializing the D* Lite
by algorithmic steps and is simpler in this respect. D* algorithm involves defining the goal cell parameters
Lite works by generating an initial solution basically so that it becomes inconsistent and inserted into the
in a similar way as the A* algorithm. Then, if chang- OPEN list (lines 15-17). The planner then calculates
es in the environment are detected, D* Lite performs the minimum cost path by calling the ComputePath()
a replanning process to check the current solution function (line 19). The robot moves along the calcu-
and, if necessary, to correct it or calculate the new lated path (sstart is also updated). If some changes in
one with the maximum possible use of search results the environment occur during the motion of the ro-
from previous iterations. In other words, if a graph bot within the range of its sensor, the algorithm up-
change occurs D* Lite does not reset the search (does dates the rhs parameters of all cells directly affected
not return that process to the beginning). This makes by the change and inserts those cells that become in-
D* Lite significantly more efficient than the basic A* consistent in the OPEN list (lines 20-23). After that,
algorithm. it performs checking/replanning the path by recall-
364 Information Technology and Control 2021/2/50

ing the ComputePath() function. Line 18 essentially function for checking the validity of the current path
means that in a real implementation the planner ends and to correct it if necessary. Within any particular
its work when it becomes sstart = sgoal (when the robot execution of this function no one cell is expanded
reaches the goal cell). more than twice [23]. Multiplying the calling of this
function generally causes a cumulative increase in
Algorithm 1: D* Lite algorithm (basic version) the total number of expanded cells in planning/re-
key(s) planning process.
return [min(g(s), rhs(s)) + h(sstart, s); min(g(s), rhs(s))];
4.2. Fuzzy Inference System
UpdateCell(s)
02. if s was not visited before The megastore environment usually involves static
structures, such as walls and shelves, whose location
03. g(s) = ∞;
is apriori known and they are not considered as par-
04. if (s ≠ sgoal) rhs(s) = mins’∈Succ(s)(c(s, s’) + g(s’));
ticular challenge for generating the path of robots. On
05. if (s ∈ OPEN) remove s from OPEN; other side, people (workers and/or buyers) that move
06. if (g(s) ≠ rhs(s)) insert s into OPEN with key(s); relatively unpredictably can slow or stop the robots,
ComputePath() increasing their travel time and total paths length.
07. while (mins∈OPEN(key(s)) < key(sstart) OR rhs(sstart) ≠ g(sstart)) Modern technologies provide a relatively simple
08. remove cell s with the minimum key from OPEN; monitoring of statistics on the average popularity in
09. if (g(s) > rhs(s)) some location depending on the time. One of the ex-
amples of statistical data processing in this regard is
10. g(s) = rhs(s);
Google’s Popular times graph. This graph shows how
11. for all s’ ∈ Pred(s) UpdateCell(s’); busy some location typically is during different times
12. else of the day and it is based on the average number of vis-
13. g(s) = ∞; itors over the last several weeks.
14. for all s’ ∈ Pred(s) ∪ {s} UpdateCell(s’); From the aspect of robots path planning in crowded
Main() and complex environment, such as megastore, chang-
15. g(sstart) = rhs(sstart) = ∞; g(sgoal) = ∞; es in crowd density or the average number of buyers
16. rhs(sgoal) = 0; OPEN = ∅; per unit of the time are important information, be-
17. insert sgoal into OPEN with key(sgoal); cause they can affect navigation costs, as it was men-
tioned. If dynamic of change of crowd density in store
18. forever
or its parts is relatively repeatable over time (for ex-
19. ComputePath(); ample at a daily or weekly level), this can be used to
20. wait for changes in edge costs; improve the efficiency of task and motion planning
21. for all directed edges (u, v) with changed edge costs for robots that operate in this environment. Having
22. Update the edge cost c(u, v); that in mind, in this paper the application of fuzzy
23. UpdateCell(u); inference system is proposed in order to regulate the
change in the traversal cost of the cells in the map to
If the edge costs are equal to the lengths of the cor- follow the patterns of change in the crowd density.
responding transitions, then D* Lite generates the In our approach, traversal cost of the cells depends
path of the least cost which in this case is also the on the time (i.e. iteration number of the robots en-
shortest path in the graph-representation of the en- gagement) and the distance of the cells from the pre-
vironment. defined part of the store. In this way, D* Lite will gen-
A measure of the efficiency of algorithms that per- erate initial path through the areas that are expected
form path planning based on graph search is the num- to be high-cluttered only when open space or other
ber of expanded nodes [21, 22]. By expanding a fewer more favorable options are not available.
cells, the algorithm comes to the solution (calculates Suppose that the map of store (Figure 2) is presented as
the path) faster. Analyzing from the aspect of D* Lite a discrete grid of 16x16 cells, intended for planning the
algorithm, any detected change in the environment path on global level. In order to present the proposed
during the motion of the robot leads to calling the concept for improvement of path planning in this me-
 - probability of occupancy of the cells nearest to
the third row of shelves depending on the itera-
tion number (part of the day) is as shown
Information Technology and Control 2021/2/50
in Fig- 365

ure 4,
gastore scenario,-ourmoving
approach makes away fromthatthe_ third
assumption moving row of the
away from shelves for
third row of shelves for one
in one part of store - around the third row of shelves cell in all directions the probability of occupancy
(Figure 2), the dynamic oneof change
cell in all directions
of crowd density is the probability of occu-
decreases by 33%.
relatively repeatablepancyon a dailydecreases
level. From theby 33%.
aspect
of the search algorithm, a larger number of buyers in Also, it is supposed that the time between two con-
Also,
some part of the store mean ithigher
is supposed that the
probability of occu- timeiterations
secutive between two con-
of engagement of a multi-robot
secutive
pancy of appropriate cells initerations of repet-
map. Therefore, the engagement
system for
of a
transport
multi-robot
of goods is approximately
stant - about 10 minutes. Working time of mega-
con-
b) Input varia
itive statistics is supposed to be as follows:
_ probability system
of occupancy forof the
transport
cells nearestof goods
to the storeis approximately
is from con-
6h to 23h, so the total number of itera-
third rowstant
of shelves- about 10 minutes. Working time of megastore system is
depending on the iteration tions of the engagement of the multi-robot
number (part of the day) is as shown in Figure 4, about 100.
is from 6h to 23h, so the total number of iterations of
the engagement of the multi-robot system is about
Figure 4
Probability of100.
occupancy of the cells nearest to the third row of shelves depending on the time

f crowd density in Block diagram of the proposed fuzzy inference c) O

able over time (for system is shown in Figure 5.
), this can beFigure
used5 Figure 4. Probability of occupancy of the cells nearest to the third F
d motion planning
Block row of
diagram of fuzzy inference system for shelves
calculatingdepending onofthe
the traversal cost time.
the cells
vironment. Having
plication of fuzzy
der to regulate the
cells in the map to
e crowd density. In
e cells depends on
he robots engage-
s from the prede-
D* Lite will gen- Figure 5. Block diagram of fuzzy inference system for calculating
Block diagram of the proposed fuzzy the
that are expected traversal
inference cost of Fuzzy
system the cells.
logic techniques are efficient in solving com-
is shown in Figure 5. plex, ill-defined problems that are characterized by
pen space or other
The fuzzy membership Thefunctions
fuzzy membership
for the input lin- functions for the
uncertainty input lin- and fuzziness of infor-
of environment
lable. mation [36, 37]. Taking into account that disturbanc-
guistic variables,guistic
as well asvariables, as well vari-
the output linguistic as the output linguistic vari-
(Figure 2) isablepre-
are given in able
Figureare
6. given in Figure 6.
es and noises are common sources of uncertainties, it
can be concluded that from the aspect of fuzzy imple-
cells, intended for
The fuzzy rules for determining the traversal cost of mentation this system is highly resistant to noise and
In order to present
cells in the map are constructed as in Table 1. disturbance [38, 39, 40].
ment of path plan-
ur approach makes
- around the third
amic of change of
le on a daily level.
 ill gen-
r prede-
xpectedother the traversal cost of the cells.
ill gen- The 5.fuzzy
Figure Blockmembership
diagram of fuzzy functions for the
inference system forinput lin-
calculating
r other the traversal cost of the cells.
guistic variables, as well as the output linguistic vari-
xpected
is pre- 366 The fuzzy membership functions
Information Technology and Control for the input lin- 2021/2/50

or other able arevariables,

guistic given in Figure
as well6.as the output linguistic vari-
dedis pre-for The fuzzy membership functions for the input lin-
present able are given in Figure 6.
ded for Figureguistic
6 variables, as well as the output linguistic vari-
is
hpresent pre-
plan- Fuzzyable are functions
membership given in Figure 6.
ded for
hmakesplan-
hepresent
third
makes morning noon afternoon evening

angeh plan-
of
Degree of membership

he third
hy makes
level.
ange of
helarger
third
y level.
eange mean
larger
of
a) Input variable 1 “iteration number” (i.e. part of the day)
y level.
ete meancells a) Input variable 1 “iteration number” (i.e. part of the day)
apposedlarger iteration number
te cells
e mean a) Input variable
variable11 “iteration
“iteration number” (i.e. part
day)of the day)
pposed (a) Input number” (i.e. part of the
te cells
arest to
upposed
e itera-
arest to small medium large
in Fig-
e itera-
arest to
Degree of membership

in Fig-
hevesitera-
for
in
fves Fig-
occu-for
f occu-
lves
wo con- for
ofi-robot
occu- distance
wo con- b) Input variable 2 “distance of the cell to the third row of shelves
ly con-
i-robot (in2 “distance
(b) Input variable
b) Input variable 2 “distance number ofof
of the thecells)”
cell to the third row of shelves (in number of cells)”
cell to the third row of shelves
wo
gastore con-
ly con- (in number of cells)”
ti-robot
ions of b) Input variable 2 “distance of the cell to the third row of shelves
gastore
lyabout
con- small medium
(in number of large
cells)”
ions of
egastore
s about
Degree of membership

tions of
s about

traversal cost

(c) Output variable “traversal cost of the cell”

c) Output variable “traversal cost of the cell”
c) Output variable “traversal cost of the cell”
the third Figure 6. Fuzzy membership functions.
c) Figure
Output6.variable “traversal cost
Fuzzy membership of the cell”
functions.
the third

the third Figure 6. Fuzzy membership functions.

 Information Technology and Control 2021/2/50 367

Table 1
Fuzzy rules table for calculating the traversal cost of the cells in the map

Rule number Fuzzy input 1 Fuzzy input 2 Fuzzy output

1 morning small distance medium cost

2 morning not small distance small cost
3 noon small distance large cost
4 noon medium distance medium cost
5 noon large distance small cost
6 afternoon small distance medium cost
7 afternoon not small distance small cost
8 evening - small cost
Connection type “AND”
The fuzzy rules for determining the traversal cost regard to the actual bit value, where N is the window
of cells in the map are constructed as in Table 1. length (Figure 7). Applied to specific case, this means
4.3. Online Learning thatisifdeclared
cell in the Nassuccessive
blocked. Initerations an same
general, the obstacle is
logic
Table 1 detected M times in a cell that should be free accord-
is applied to cells that should be blocked according to
In addition
Fuzzy to for
rules table fuzzy inference
calculating system,
the traversal costitofis
theproposed
cells
ingbasic
to themap.basic
in the map the Ofmap, then
course, during
there willgeneration
be iterationsprocess
that
to integrate in considered architecture, under the in-
some cells are outside of the sensors range of all this
of the initial path in the first subsequent iteration ro-
troducedFuzzy
assumptions, an online learning Fuzzyalgorithm cell is
Rule Fuzzy bots, sodeclared
at the endasofblocked. In general,
these iterations the same
statuses as inlogic
the
based oninput
binary
1 moving-window or M-out-of-N detec- is applied to map
cellsare
thatassigned
should be toblocked according to
number input 2 output
start (basic) them (with the aim
tor1[10]. With
morningthis learning-based
small distance approach,
medium we cost
will try the basic map. Of course, there will be iterations that
of statistical estimation in the learning algorithm).
to 2obtainmorning
reusing datanotcollected
small distance small cost
with robots’ sensors some cells are outside of the sensors range of all ro-
3 noon small distance
from previous iterations whenever possible, large incost
order to bots, so
Figure 7 at the end of these iterations statuses as in the
4 noon improve the
additionally medium distance
performance medium
of cost
multi-robot start (basic)
Block diagram of map are assigned
learning system to them (with the aim
5 noon large distance
system with every execution of repetitive tasks small cost of statistical estimation in the learning algorithm).
6 afternoon small distance medium cost
As the robots travel and observe the environment,
7 afternoon not small distance small cost
they update the map. The updating of the map in es-
8 evening - small cost
sence depends on the number of robots, their paths,
Connection type “AND”
as well as the range of sensors. In order to apply pro-
posed
Fuzzylearning algorithm, are
logic techniques status of eachincell
efficient of map
solving
observed by iterations of robots engagement
complex, ill-defined problems that are characterized is pre-
by uncertainty of environment and fuzziness of in-“1”
sented as binary time series of “0” (free cell) and Figure 7. Block diagram of learning system.
(blocked[36,
formation cell).37].
The moving-window
Taking into accountslides on these
that disturb-
bits and
ances and noises
the change of thesources
are common start cell status (status
of uncertainties, ThisThis
detector
detector provides
providesthetheforgetting
forgettingofofpast
past itera-
itera-
it according to the basic
can be concluded map)the
that from foraspect
the initial pathim-
of fuzzy gen- tions,
tions, which
which isis regulated
regulated byby window
window length
length N. Pa-Pa-
eration process
plementation in the first
this system subsequent
is highly resistantiteration
to noise is rameter
rameter M M isis adjusted
adjusted according
according to to the
the specific
specific envi-
envi-
and disturbance
declared when[38, test39, 40].
statistics satisfies the following: ronment,
ronment, depending
depending on on the
the period
period ofof engagement
engagement of
there exist at least M of N bits of the opposite value in robots
robotsandandexpected
expected changes
changes in in the
the environment.
environment.
4.3. Online
regard Learning
to the actual bit value, where N is the window Suggested
Suggested learning
learning system
system has ahas a simple
simple structure
structure and
length (Figure 7). Applied to specific case, this means therefore it is very easy to be implemented. After After
and therefore it is very easy to be implemented. each
In addition each iteration, “System Manager” performs the anal-
that if in the Ntosuccessive
fuzzy inference system,
iterations it is pro-
an obstacle is de- iteration, “System Manager” performs the analysis of
posed
tectedtoMintegrate
times inin considered
a cell architecture,
that should under
be free according detected changes in the environment, as wellas
ysis of detected changes in the environment, aswell
the
the as the analysis of the effects of applying the learning
to introduced
the basic map, assumptions,
then duringan online learning
generation algo- of
process analysis of the effects of applying the learning algo-
rithm based on binary moving-window or M-out-of-N algorithm in previous iterations. The results of these
the initial path in the first subsequent iteration this rithm in previous iterations. The results of these anal-
detector [10]. With this learning-based approach, we analysis can be used in further upgrading of the pro-
will try to obtain reusing data collected with robots’ posed concept to correct the application of the learn-
sensors from previous iterations whenever possible, ing algorithm (in terms of correction the values of
in order to additionally improve the performance of parameters M and N, etc).
multi-robot system with every execution of repetitive The resistance of this segment of the system to the
tasks influence of disturbances and noises mostly depends
As the robots travel and observe the environment, on the probability of detection PD and false alarm PF
they update the map. The updating of the map in es- of the applied M-out-of-N detector, which will be
sence depends on the number of robots, their paths, defined below [10].
368 Information Technology and Control 2021/2/50

ysis can be used in further upgrading of the proposed with the specification of the start and the goal po-
concept to correct the application of the learning al- sitions of each robot,
gorithm (in terms of correction the values of parame- _ planning/replanning of robots paths,
ters M and N, etc). _ implementation of fuzzy inference system and
The resistance of this segment of the system to the learning algorithm with a purpose of improvement
influence of disturbances and noises mostly depends the path planning efficiency,
on the probability of detection PD and false alarm PF _ motion coordination,
of the applied M-out-of-N detector, which will be de- _ solving a conflict situations and updating of envi-
fined below [10].
ronment map based on information collected with
If the probability of detection of single cell status
robots sensors.
changing in map by robot’s sensors, i.e. bit status
changing in learning algorithm in each of N single bits These are the most complicated and time consuming
is pd, then the probability of declaring a initial bit sta- tasks, so they are performed on cloud level in simula-
tus changing in at list M out of N bits, is: tion (Figure 1).
In order to test the characteristics of the proposed ap-
N N
N k proach for improvement the efficiency of path plan-
∑ P(k ) ∑   pd (1 − pd )
( N −k )
=PD = . ning on global level, we applied it to a large number of
=k M=k k M 
various data sets in simulated megastore scenario. The
Similarly, if the probability of false alarm per single map of store in accordance with Section 4.2. is repre-
cell status changing in map by robot’s sensors is pfa, sented as a discrete 16x16 grid. To apply graph-based
then the probability of declaring a false alarm in at search technique, it is adopted that the robot can move
least M out of N bits, resulting in the declaration of a from an unblocked cell to one of the eight neighboring
false initial bit status changing, is: cells, if it is also unblocked (eight-connected graph).
The costs of edges are obtained by multiplying of the
traversal cost of the cell in which the robot makes the
N
N k
 p fa (1 − p fa )
( N −k )
=PF ∑ k
k =M 
. move and the length of the edge (one for orthogonal
 movements or √2 for diagonal movements), while the
costs of edges into obstacles are infinite.
In real systems, PD and PF depend on the characteristics
The information about environment changing is not
of the robot’s sensors, but also on the characteristics of
apriori available to robots and system manager. As
the multi-robot system and the environment in which
the robots move from start to goal locations, they re-
the robots move, and it is usually valid that pd ≫ pfa.
ceive new information concerning the traversability
of surrounding cells within sensor range (one cell in
all directions).
5. Simulation, Results and Discussion Fuzzy system and learning algorithm were tested sepa-
The simulation was organized completely in accor- rately to simplify the analysis of the effect of their appli-
dance with Figure 1 and the description of the system cation. This approach is applied because these two sys-
architecture given in Section 3. tems work independently and are designed to improve
All modules are included, but some processes can be path planning by exploiting different information from
underlined as specific from the aspect of the topic of the environment and its dynamics. With this in mind,
this paper, as follows: in a real situation they would not interfere with each
_ checking the shelves and determining the percent- other, but the effects of their application would collec-
age of their fullness with goods after the specified tively contribute to the improvement of path planning.
intervals, The cloud approach has always been applied, both in
_ generating a notification if the shelf fullness is low- the case of the fuzzy system application and in the case
er than the preset level, of the learning algorithm application.
_ planning of engagement the available robots for In order to test fuzzy inference system presented in
goods transport from the warehouse to the store Section 4.2., the probability of occupancy of the cells
 Figu

Th
Information Technology and Control 2021/2/50 369 impl
of th
defin
nearest to the third row of shelves depending on the match in general case, having in mind that during the cells
iteration number was exactly as on the graph in the robot motion along the initial path the changes in an erate
Figure 4, but with the possibility of deviation in each environment are detected which can require the cor- or no
iteration of ± 20% (to test at the same time the robust- rection of the motion plan. glob
ness of the system and the proposed method from the The situation shown in Figure 8 a) and Figure 8 b) Figu
aspect of fuzzy implementation). Also, moving away a) iteration №=5 (without FIS)
implies the time when the crowd in the observed part crow
from the third row of shelves for one cell in all direc- of the store is not large, so the fuzzy inference system
tions the probability of occupancy decreases by 33%. Figure 8. Path planning results without using FIS and with using 1 Figu
wa
defines relatively low traversal cost FIS. of the appropriate glob
The traversal costs of the cells are defined in accor-
cells in the map. In this case, D* Lite algorithm gen- infer
dance with the Figure 6.
erates theThesamesituation shownregardless
initial solution in Figureof 8whether a) and Figure 8plann b)T
The results of robots path planning in the phase of or not fuzzy logic is applied. There is no correction of impl
implies the time when the crowd in the observed part path
goods transport from the warehouse to the store, in global path, of th
iterations that present the effects of using fuzzy infer-
of thealso.store Onis thenotother side,sointhe
large, Figure
fuzzy 8 c)inference
and system ation
Figure defines
8 d), path planning low
relatively in situation
traversal of cost
the largest
of the appropriate defin
ence system, are shown in Figure 8. crowd density is presented. It can be seen that Robot tem
cells
Dashed line represents initially computed path, while
cells in the map. In this case, D* Lite algorithm gen- be co
1 was forced to make two corrections of the path (on erate
continuous line is final path. These two lines do not erates the same initial solution
global level) in the situation without using the fuzzy
regardless of whether fuzzy
or not fuzzy logic is applied. There is no correctionof or
ofthno
Figure 8 global path, also. On the other side, in Figure 8 c) andW glob
Figurea) iteration
8 d), path Figu
Path planning results without using FIS and with using FIS
a) iteration №=5 (without FIS) №=5planning
(without FIS) in situation of the largest itera
crowdb)density iterationis №=5presented.
(with FIS) It can be seen that Robot
crow
paths
1 wa
1 Figure
was forced to makeresults
8. Path planning two without
correctionsusing FISof and
thewith
pathusing
(on
paths
global level) in the situation FIS. without using the fuzzy glob
plem
infer
inference system. The assumption is that a local 2% c
The situation
planner cannot solve shown thisin problem
Figure 8in a) aand
safeFigure
way so 8plan
b)
a
apath
fuz
implies
path the timeon
replanning whenglobalthelevel
crowd is in the observed
needed. In the situ-part
ationA
of thewith
ation storethe is not large, soof
application thethefuzzy
fuzzy inference
inference system
sys-
ing
tem a
defines
tem there relatively
was no low traversal
correction of cost of theAlso,
the path. appropriate
it can
discr
be c
cells
be in the map.
concluded thatInpath thisincase, D* Lite
situation algorithm
without usinggen- the
In
fuzz sc
erates inference
fuzzy the same system initial solution
is quite regardless
risky from of thewhether
aspecteral
or the
of not human-robot applied. There is no correctionofofth
fuzzy logic isinteractions.
time
global
We have path, run also.several
On the cycles
other side, within100 Figure 8 c) andW
consecutive enga
itera
Figure 8 d),
iterations in path
everyplanning
cycle and in situation
calculatedof mean the largest
total
a)b)iteration
(a) iteration№=5
iteration №=5(without
№=5 (without FIS)
FIS) b) iteration
(b) iteration №=5
№=5 (with FIS)
(with FIS) with
path
(with FIS) crowd
paths densityper
c) length
iteration is presented.
№=36 cycle.
(withoutWe FIS)Itgot
canthat be seen that Robot
the mean total
(T21
path
1 was length
paths forced per to make
cycletwo in thecorrections
situationofwith the path
the im- (on
The
plem
global level) of
plementation in the fuzzy situation systemwithout using for
is shorter the about
fuzzy
are
2% s
inference
2% compared system.
to theThe option assumption
without theisapplication
that a local of c
aplanner
fuzzy system.cannot solve this problem in a safe way soa fuz a
pathAlso, the simulation experiments for testing learn- A
replanning on global level is needed. In the situ-
ationapproach
ing with the were application
conducted of the in fuzzy
the sameinference
map sys- ing
and
tem there
discrete grid.wasTwo no correction
scenarios are of presented
the path. Also, in Figure discr
it can 9.
be scenario
In concluded that path
I there is one in obstacle
situationthat without usingsev-
occupied thesc
In
fuzzycells
eral inference system is quite T1=30T,
with occupancy-time risky fromwhere Teral
the aspect is
of the human-robot interactions.
time between two consecutive iterations of robots time
We have run
engagement. several cycles
In scenario II three with 100 consecutive
obstacles appeared enga
iterations
with in every (without
occupancy-times cycle FIS) and
T21,calculatedT22 and mean totalT23with
c) iteration №=36 c)d)iteration
iteration№=36№=36 (with FIS)
(c)b) iteration
iteration №=5(without
№=36 (without
FIS)
(with FIS)
FIS) paths(d)length
(T21≠T22≠T23) iteration per cycle.
with
№=36 We
FIS) got
duration
(with that the
between 10Tmeanand 35T. (T21
total
Figur
pathsparameters
The length per ofcycle binaryin the situation with detector
moving-window the im- The
plementation
are set to M=3ofand theN=5.fuzzy system is shorter for about are s
2% compared to the option without the application of
a fuzzy system.
Also, the simulation experiments for testing learn-
ing approach were conducted in the same map and
 plementation of the fuzzy system is shorter for about Fo
2% compared to the option without the application of 100 c
a fuzzy system. lated
Also, the simulation experiments for testing learn- length
370 Information Technology and Control 2021/2/50
ing approach were conducted in the same map and
Th
discrete grid. Two scenarios are presented in Figure 9.
In scenario I there is one obstacle that occupied sev- transp
eralthe
cells with occupancy-time for sc
inference system. The assumption is that a local plan- Also, simulation experimentsT1=30T, where
for testing T is
learning
ner cannot solve this problem in a safe way so a path
time between two consecutive iterations of robots
approach were conducted in the same map and dis-
effect
replanning on global level is needed. In the situation
engagement. In scenario II three obstacles appeared
crete grid. Two scenarios are presented in Figure 9. ure 10
with occupancy-times T21, T22 and T23 Th
with the application
c) iterationof№=36
the (without
fuzzy FIS)
inference system (T21≠T22≠T23) with duration between 10T and 35T. Figure
there was no correction of the path. Also, it can be Figure 9 Table
Figure 10. Path planning results without using learning algorithm
The parameters of and
binary moving-window detector
concluded that path in situation without using the Scenarios I and
a)IIScenario
overview
are set to M=3 and N=5. I (new
withobstacles
using
(without are shown
learning
learning) in grey)
algorithm.
fuzzy inference system is quite risky from the aspect Fo
of the human-robot interactions. For every scenario we have run several cycles with Mean
100 c
100 consecutive iterations in every cycle and calcu-
We have run several cycles with 100 consecutive iter- lated
ations in every cycle and calculated mean total paths
lated mean expanded cells and mean total paths Scen
length
length per cycle. We got that the mean total paths length per cycle.
Th
length per cycle in the situation with the implementa- The results of robots path planning in the phase of
transp I
tion of the fuzzy system is shorter for about 2% com- transport of goods from the warehouse to the store,
for sc
pared to the option without the application of a fuzzy for scenarios I and II in iterations that present the II
effect
system. effects of using learning algorithm, are shown in Fig-
d) iteration №=36 (with FIS) Figureure
9. Scenarios ureBa 10
10. I and II overview (new obstacles are shown in
Figure 10 grey). Th
total
The10.statistics
Figure Path planning for described
results without scenarios is shown
using learning algorithmin
Path planning results without using learning algorithm and with using learning algorithm Table
ing le
Table b)2.Scenario andI with(withusing learning algorithm.
learning)
a) Scenario I (without learning) comp
a) Scenario I (without learning)
For every scenario weTable have2 run several cycles with (Figu
Mean
Mean total paths length reduction with
100 consecutive iterations in every cycle and calcu- using learning algorithmapplic
lated mean Mean expanded total paths length per cycle of
cells and mean total paths infere
100 iterations ReductionScen
length
Scenarioper cycle. (%)
path c
without learning with learning
The results algorithm
of robots path planning algorithmin the phasethe of e
transport
I of goods 4342from the warehouse 4289 to the store, crI
1,22% the
for scenarios I 4558and II in iterations initialII
II 4391 that present3,66%the
effects of using learning algorithm, are shown in Fig- ed ch
ure 10. Th
Ba
Based on Table 2, it can be concluded that mean metho
The statistics for described scenarios is showntotal in
total paths lengths are significantly reduced with us- menta
Table 2. ing le
ing learning algorithm
b) Scenario I (with learning)(Figure 10 b and Figure 10 tion d), a
a)b)Scenario
(a) ScenarioI I(without
Scenario I(without learning)
(with learning)
learning) (b) Scenario
c) Scenario I (with learning)
II (without learning) comp
compared to situations when learning is not applied and th
Table 2 (Figu
(Figure 10 a and Figure
Mean total paths length reduction 10with
c). using
Thislearning
is because
algorithmthe
applic
application of Meanlearning
total paths algorithm,
length per as well
cycle of as the fuzzy
infere
inference
Scenario system, provides an adaptive approachpath
100 iterations Reductionto
path calculating withoutby learning
D* Lite, with withlearning (%) 6. Coc
taking into account
algorithm algorithm the e
the experience in early planning phase. In this way
I 4342 4289 1,22% theIn cr
the creation of smart cost map is achieved, so that the
II paths on global 4558 level are searched 4391 initial
robot
initial with 3,66%
predict-
ed
tasksch
ed changes in the environment.
megaTh
Based on Table of
The robustness 2, itthecansystem
be concluded
and the that mean
proposed
metho
perfo
total
method pathsfromlengths
the aspect are significantly
of learning reduced
algorithmwith us-
imple-
menta
ing learning
mentation algorithm
depends (Figureon
in general 10probability
b and Figure 10 data
d),
of detec-
b) Scenario I (with learning) tion
applic a
compared
tion and false
c) to situations
Scenario II alarm of when
(without applied
learning) learning is not detector
M-out-of-N applied
c) Scenario II (without learning)
(c) Scenario II (without learning) d) Scenario
(d) Scenario IIII(with
(with learning)
learning) and
ing a th
(Figure
and this 10 a and
problem isFigure
considered 10 c).in This
Sectionis 4.3.
because the
application of learning algorithm, as well as the fuzzy coord
inference system, provides an adaptive approach to
path calculating by D* Lite, with taking into account 6. Co
6. Conclusions
the experience in early planning phase. In this way
theIncreation of smart
this paper cost map is achieved,
an autonomous so that
cloud-based theIn
multi-
initial robot
robot paths
systemondesigned
global level
to are searched
execute withrepetitive
highly predict-
ed changes tasks
tasks in a in the environment.
dynamic environment such as a modern
mega
 Information Technology and Control 2021/2/50 371

In scenario I there is one obstacle that occupied

several cells with occupancy-time T1=30T, where 6. Conclusions
T is time between two consecutive iterations of In this paper an autonomous cloud-based multi-robot
robots engagement. In scenario II three obstacles system designed to execute highly repetitive tasks in a
appeared with occupancy-times T21, T22 and T23 dynamic environment such as a modern megastore is
(T21≠T22≠T23) with duration between 10T and considered. Cloud level is intended for performing the
35T. The parameters of binary moving-window de- most demanding operations related to data process-
tector are set to M=3 and N=5. ing, allocation of tasks between robots, application of
For every scenario we have run several cycles with the fuzzy inference system and learning algorithm,
100 consecutive iterations in every cycle and cal- path planning and replanning, motion coordination,
culated mean expanded cells and mean total paths etc. A direct consequence of this is the unloading of
length per cycle. robots because they are users of cloud services in this
The results of robots path planning in the phase of architecture.
transport of goods from the warehouse to the store, for D* Lite is applied as algorithm for path planning on
scenarios I and II in iterations that present the effects global level, taking into account its high efficiency in
of using learning algorithm, are shown in Figure 10. environments that are partially known or completely
The statistics for described scenarios is shown in unknown. In order to further improve path planning
Table 2. process in complex and crowded environment, imple-
mentation of smart cost map based on fuzzy inference
Table 2 system and learning algorithm is proposed.
Mean total paths length reduction with using learning
Proposed concept substantially reduces the total
algorithm
paths length. This makes preconditions for energy
Mean total paths length per cycle of saving, as well as for increasing the average speed of
100 iterations Reduction
Scenario the robots, so that they reach the destination for a
without learning with learning (%)
shorter time and perform tasks more efficiently. Also,
algorithm algorithm
the risk in path planning related to the human-robot
I 4342 4289 1,22% interactions can be reduced.
II 4558 4391 3,66% The important advantage of the system is simplicity
of implementation. Its main disadvantage is sensitiv-
ity of the communication loss.
Based on Table 2, it can be concluded that mean to-
tal paths lengths are significantly reduced with us- A perspective for future works would be to implement
ing learning algorithm (Figure 10 b and Figure 10 d), some advanced machine learning techniques in com-
compared to situations when learning is not applied bination with multi-criteria decision making meth-
(Figure 10 a and Figure 10 c). This is because the ap- ods, in order to reinforce smart cost map intended
plication of learning algorithm, as well as the fuzzy for path planning. The idea is based on the fact that a
inference system, provides an adaptive approach to dynamic environment may have, besides crowds and
path calculating by D* Lite, with taking into account obstacles, other specifics that affect the robots mo-
the experience in early planning phase. In this way tion [41, 42] and that are desirable to consider in the
the creation of smart cost map is achieved, so that the initial planning process.
initial paths on global level are searched with predict-
Acknowledgement
ed changes in the environment.
The robustness of the system and the proposed meth- The authors of this paper would like to express their
od from the aspect of learning algorithm implemen- gratitude to the Ministry of Education, Science and
tation depends in general on probability of detection Technological Development of Republic of Serbia,
and false alarm of applied M-out-of-N detector and for their support of Contract No 451-03-68/2020-
this problem is considered in Section 4.3. 14/200325.
372 Information Technology and Control 2021/2/50

References
1. Al-Mutib, K., AlSulaiman, M., Emaduddin, M., Ram- botics and Au-tomation, Seattle, WA, USA, 2015, 900-
dane, H., Mattar, E. D* Lite Based Real-Time Multi- 905. https://doi.org/10.1109/ICRA.2015.7139284
Agent Path Planning in Dynamic Environments. In 2011 10. Dillard, G. M. A moving window detector for binary integ-
Third Interna-tional Conference on Computational In- ra-tion. IEEE Transactions on Information Theory, 1967,
telligence, Modelling & Simulation, Langkawi, Malay- 13(1), 2-6. https://doi.org/10.1109/TIT.1967.1053967
sia, 2011. https://doi.org/10.1109/CIMSim.2011.38
11. Ferguson, D., Stentz, A. Field D*: An Interpola-
2. Aroor, A., Epstein, S. L., Korpan, R. Online Learning tion-Based Path Planner and Replanner. Robotics Re-
for Crowd-Sensitive Path Planning. Proceedings of the search - Results of the 12th International Symposium of
17th In-ternational Conference on Autonomous Agents Robotics Research, San Francisco, CA, USA, 2005, 239-
and MultiA-gent Systems, 2018, 1702-1710. 253. https://doi.org/10.1007/978-3-540-48113-3_22
3. Benavidez, P., Muppidi, M., Rad, P., Prevost, J. J., Jamshi- 12. Ferguson, D., Likhachev, M. Efficiently Using Cost
di, M., Brown, L. Cloud-Based Realtime Robotic Visual Maps For Planning Complex Maneuvers. Proceedings
SLAM. Proceedings of the 2015 Annual IEEE Systems of International Conference on Robotics and Automa-
Conference, Vancouver, BC, Canada, 2015, 773-777. tion - Workshop on Planning with Cost Maps, 2008.
https://doi.org/10.1109/SYSCON.2015.7116844
13. Hunziker, D., Gajamohan, M., Waibel, M., D’Andrea,
4. Bennewitz, M., Burgard, W., Thrun, S. Optimizing
R. Rapyuta: The RoboEarth Cloud Engine. 2013 IEEE
Schedules for Prioritized Path Planning of Multi-Robot
Interna-tional Conference on Robotics and Automa-
Systems. Pro-ceedings of the 2001 IEEE International
tion, Karlsruhe, Germany, 2013, 438-444. https://doi.
Conference on Ro-botics and Automation, Seoul, South
org/10.1109/ICRA.2013.6630612
Korea, 2001, 271-276. https://doi.org/10.1109/RO-
BOT.2001.932565 14. Hwang, V., Phillips, M., Srinivasa, S., Likhachev, M. Lazy
Validation of Experience Graphs. 2015 IEEE Interna-
5. Berenson, D., Abbeel, P., Goldberg, K. A Robot Path
tional Conference on Robotics and Automation, Seat-
Planning Framework that Learns from Experience.
tle, WA, USA, 2015, 912-919. https://doi.org/10.1109/
2012 IEEE Inter-national Conference on Robotics and
ICRA.2015.7139286
Automation, Saint Paul, MN, USA, 2012, 3671-3678.
https://doi.org/10.1109/ICRA.2012.6224742 15. Jetchev, N., Toussaint, M. Trajectory Prediction: Learn-
ing to Map Situations to Robot Trajectories. Proceed-
6. Brumitt, B., Stentz, A. GRAMMPS: A Generalized Mis-
ings of the 26th Annual International Conference on
sion Planner for Multiple Mobile Robots in Unstruc-
Machine Learning, Montreal, Quebec, Canada, 2009,
tured Envi-ronments. Proceedings of the 1998 IEEE
449-456. https://doi.org/10.1145/1553374.1553433
International Con-ference on Robotics and Automa-
tion, Leuven, Belgium, 1998, 1564-1571. https://doi. 16. Kamburugamuve, S., He, H., Fox, G. C., Zhao, W. Cloud
org/10.1109/ROBOT.1998.677360 based Real-Time Multi-Robot Collision Avoidance for
7. Choudhary, S., Carlone, L., Nieto, C., Rogers, J., Liu, Z., Swarm Robotics. International Journal of Grid and Dis-
Christensen, H. I., Dellaert, F. Multi Robot Object-Based tribut-ed Computing, 2016, 9(6), 339-358. https://doi.
SLAM. Proceedings of the International Symposium on org/10.14257/ijgdc.2016.9.6.30
Ex-perimental Robotics, Tokyo, Japan, 2016, 729-741. 17. Koenig, S., Likhachev, M. D* Lite. Proceedings of the
https://doi.org/10.1007/978-3-319-50115-4_63 Eight-eenth National Conference on Artificial Intelli-
8. Cirillo, M., Uras, T., Koenig, S. A Lattice-Based Ap- gence and Fourteenth Conference on Innovative Appli-
proach to Multi-Robot Motion Planning for Non-Ho- cations of Artifi-cial Intelligence, Edmonton, Alberta,
lonomic Vehicles. 2014 IEEE/RSJ International Canada, 2002, 476-483
Conference on Intelligent Ro-bots and Systems, Chi- 18. Koenig, S., Likhachev, M. Fast Replanning for Naviga-
cago, IL, USA, 2014, 232-239. https://doi.org/10.1109/ tion in Unknown Terrain. IEEE Transactions on Ro-
IROS.2014.6942566 botics, 2005, 21(3), 354-363. https://doi.org/10.1109/
9. Coleman, D., Sucan, I. A., Moll, M., Okada, K., Correll, TRO.2004.838026
N. Experience-Based Planning with Sparse Roadmap 19. LaValle, S. M. Planning Algorithms. Cambridge University
Spanners. 2015 IEEE International Conference on Ro- Press, 2006. https://doi.org/10.1017/CBO9780511546877
 Information Technology and Control 2021/2/50 373

20. Lien, J.-M., Lu, Y. Planning Motion in Environments ference on Net-working, Sensing and Control, Taipei,
with Similar Obstacles. In 2009 Conference Robotics: Taiwan, 2015, 353-353, https://doi.org/10.1109/ICN-
Science and Systems V, 2009. https://doi.org/10.15607/ SC.2015.7116061
RSS.2009.V.012 31. Precup, R.-E., Preitl, S., Petriu, E., Bojan-Dragos, C.-
21. Likhachev, M., Ferguson, D., Gordon, G., Stentz, A., A., Szed-lak-Stinean, A.-I., Roman, R.-C., Hedrea,
Thrun, S. Anytime Search in Dynamic Graphs. Jour- E.-L. Model-Based Fuzzy Control Results for Net-
nal Artificial Intel-ligence, 2008, 172(14), 1613-1643. worked Control Systems. Re-ports in Mechanical En-
https://doi.org/10.1016/j.artint.2007.11.009 gineering, 2020, 1(1), 10-25. https://doi.org/10.31181/
22. Likhachev, M., Ferguson, D., Gordon, G., Stentz, A., rme200101010p
Thrun, S. Anytime Dynamic A*: An Anytime, Replan- 32. Phillips, M., Likhachev, M. Speeding up Heuristic Com-
ning Algorithm. Proceedings of the 15th International puta-tion in Planning with Experience Graphs. 2015
Conference on Auto-mated Planning and Scheduling, IEEE Interna-tional Conference on Robotics and Au-
Monterey, CA, USA, 2005, 262-271. tomation, Seattle, WA, USA, 2015, 893-899. https://doi.
23. Likhachev, M., Ferguson, D., Gordon, G., Stentz, A., org/10.1109/ICRA.2015.7139283
Thrun, S. Anytime Dynamic A*: The proofs. Technical 33. Phillips, M., Cohen, B., Chitta, S., Likhachev, M.
Report, CMU-RI-TR-05-12, Robotics Institute, Carne- E-graphs: Bootstrapping Planning with Experience
gie Mellon University, 2005. Graphs. In 2012 Conference Robotics: Science and Sys-
24. Lozano-Perez, T. Spatial Planning: A Configuration tems VIII, 2012. https://doi.org/10.15607/RSS.2012.
Space Approach. IEEE Transactions on Comput- VIII.043
ing, 1983, C-32(2), 108-120. https://doi.org/10.1109/ 34. Saha, O., Dasgupta, P. Fast Path Planning Using Expe-
TC.1983.1676196 rience Learning from Obstacle Patterns. In 2016 Con-
25. Martin, S. R., Wright, S. E., Sheppard, J. W. Offline and ference AAAI Spring Symposium Series, Palo Alto, Cal-
Online Evolutionary Bi-directional RRT Algorithms for ifornia, 2016, 60-67.
Effi-cient Replanning in Environments with Moving 35. Wagner, G., Choset, H. M*: A Complete Multirobot Path
Obstacles. 2007 IEEE International Conference on Au- Planning Algorithm with Performance Bounds. 2011
tomation Science and Engineering, Scottsdale, AZ, USA, IEEE/RSJ International Conference on Intelligent Ro-
2007. https://doi.org/10.1109/COASE.2007.4341761 bots and Systems, San Francisco, CA, USA, 2011, 3260-
26. Murphy, E. Planning and Exploring Under Uncertainty. 3267. https://doi.org/10.1109/IROS.2011.6095022
PhD thesis, University of Oxford, 2010. 36. Wiktor, A., Scobee, D., Messengery, S., Clark, C. Decen-
27. Messinis, S., & Vosniakos, G. C. (2020). An Agent-Based tral-ized and Complete Multi-Robot Motion Planning
Flexible Manufacturing System Controller with Pe- in Con-fined Spaces. 2014 IEEE/RSJ International
tri-Net En-abled Algebraic Deadlock Avoidance. Re- Conference on Intelligent Robots and Systems, Chi-
ports in Mechanical Engineering, 1(1), 77-92. https:// cago, USA, 2014, 1168-1175. https://doi.org/10.1109/
doi.org/10.31181/rme200101077m IROS.2014.6942705

28. Nardi, L., Stachniss, C. Experience-Based Path Plan- 37. Vilela, M., Oluyemi, G., Petrovski, A. A Fuzzy Inference
ning for Mobile Robots Exploiting User Preferences. System Applied to Value of Information Assessment for
2016 IEEE/RSJ International Conference on Intelli- Oil and Gas Industry. Decision Making: Applications in
gent Robots and Systems, 2016, 1170-1176. https://doi. Man-agement and Engineering, 2019, 2(2), 1-18. https://
org/10.1109/IROS.2016.7759197 doi.org/10.31181/dmame1902001v
29. Oliver, C. S., Saptharishi, M., Dolan, J. M., Trebi-Ol- 38. Milosevic, T., Pamucar, D., Chatterjee, P. Model for Se-
lennu, A., Khosla, P. K. Multi-Robot Path Planning lecting a Route for the Transport of Hazardous Mate-
by Predicting Struc-ture in a Dynamic Environment. rials Using a Fuzzy Logic System. Military Technical
Proceedings of the Confer-ence on Mechatronic Sys- Courier, 2021, 69(2), 355-390. https://doi.org/10.5937/
tems, 2000, 555-560. https://doi.org/10.1016/S1474- vojtehg69-29629
6670(17)39203-0 39. Dimic, S., Ljubojevic, S. Decision Making Model in For-
30. Peng, J.-H., Li, I.-H., Chien, Y.-H., Hsu, C.-C., Wang, W.- est Road Network Management. Military Technical
Y. Multi-Robot Path Planning Based on Improved D* Courier, 2019, 67(1), 93-115. https://doi.org/10.5937/
Lite Al-gorithm. 2015 IEEE 12th International Con- vojtehg67-18446
374 Information Technology and Control 2021/2/50

40. Vilela, M., Oluyemi, G., & Petrovski, A. A Holistic Ap- 42. Ziebart, B. D., Ratliff, N., Gallagher, G., Mertz, C., Peter-
proach to Assessment of Value of Information (VOI) son, K., Bagnell, J. A., Hebert, M., Dey, A. K., Srinivasa,
with Fuzzy Data and Decision Criteria. Decision S. Plan-ning-Based Prediction for Pedestrians. Pro-
Making: Applications in Management and Engineer- ceedings of the 2009 IEEE/RSJ International Con-
ing, 2020, 3(2), 97-118. https://doi.org/10.31181/dma- ference on Intelligent ro-bots and systems, St. Louis,
MO, USA, 2009, 3931-3936. https://doi.org/10.1109/
me2003097v
IROS.2009.5354147
41. Zagradjanin, N., Pamucar, D., Jovanovic, K. Cloud-
43. Zucker, M., Kuffner, J., Branicky, M. Multipartite RRTs
Based Multi-Robot Path Planning in Complex and
for Rapid Replanning in Dynamic Environments. Pro-
Crowded Envi-ronment with Multi-criteria Decision ceedings of the 2007 IEEE International Conference
Making Using Full Con-sistency Method. Symmetry, on Robotics and Automation, Rome, Italy, 2007, 1603-
2019, 11(10), 1-15. https://doi.org/10.3390/sym11101241 1609. https://doi.org/10.1109/ROBOT.2007.363553

This article is an Open Access article distributed under the terms and conditions of the Creative
Commons Attribution 4.0 (CC BY 4.0) License (http://creativecommons.org/licenses/by/4.0/).