Multi-Agent Systems

1. Introduction to Agent-Based Systems

Florin Leon
“Gheorghe Asachi” Technical University of Iași, Romania
Faculty of Automatic Control and Computer Engineering

http://florinleon.byethost24.com/lect_mas.html

2024
Introduction to Agent-Based Systems
1. Introduction
2. Agents
3. Intelligent Agents
4. Mobile Agents
5. Agent Applications
6. Conclusions

2
Introduction to Agent-Based Systems
1. Introduction
2. Agents
3. Intelligent Agents
4. Mobile Agents
5. Agent Applications
6. Conclusions

3
Trends in Computing
 Ubiquity
 Interconnection
 Intelligence
 Delegation
 Human-orientation

4
Ubiquity
 The cost of computing capability has been continuously reduced
 Processing elements can be introduced almost everywhere
(e.g., Internet of Things)

5
Ubiquity
 The cost of computing capability has been continuously reduced
 Processing elements can be introduced almost everywhere

6
Interconnection
 First computers: isolated entities that communicated only
with the human operator
 Nowadays, computer systems are connected to a network or
the Internet, there are large distributed systems

7
Interconnection
 Rethinking the bases of computer science?
 Computations seen as interaction processes, where the
interactions are more important than the algorithm
 E.g., centralized vs. decentralized sorting
 QuickSort
 Compare to right and exchange numbers if right is lower

8
Intelligence
 Determined by:
 The increasing complexity of the problems
 The need to automate the finding of their solutions

 Automated stock ordering systems for business, maintaining

inventory
 Numerical calculations for accountants

9
Intelligence
 Urban planning  Healthcare

10
Delegation
 The transfer of control to computerized systems
 For example, the automatic piloting systems of aircraft

11
Delegation
 Autonomous driving

12
Human-Orientation
 Progressing away from the direct
programming of machines
 The use of metaphors similar to the way in
which people understand reality
 Generations of programming languages
 Machine code, assembly language, high level
languages, LLMs?
 The evolution of user interfaces
 Switches / lights, CUI, GUI
 The evolution of paradigms
 Structured programming, abstract data types,
object-oriented programming, functional
programming

13
Global Computing
 What kind of computational models are needed to
exploit very large systems?
 E.g., systems with 1010 processors distributed everywhere

14
Autonomy / Independence
 Entities that operate without human intervention
 They must represent the interests of the user in the
interactions with other systems or with other people
 The problems are different from traditional distributed
systems; they include a social dimension
 Such entities are the agents

15
The Main Ideas
 An agent is capable of autonomous action
 Its user or owner does not control it step by step
 A multi-agent system is composed of interacting
agents
 Cooperation
 Coordination
 Negotiation
 Communication

16
Agents and Multi-Agent Systems
 Design
 Agent design vs. society design
 Micro-macro distinction
 Issues
 Selfishness vs. cooperation
 Coordination to achieve common goals
 Recognizing conflicts and negotiation
 Communication languages expressive enough

17
Vision and Motivation
 Autonomy
 Control of spacecraft: Remote Agent, NASA Deep Space 1 (1998-2001)
 Cooperation
 Air traffic control systems: OASIS, Sydney airport (prototype, 1995)
 Road traffic control systems: Distributed Vehicle Monitoring Testbed
 Search and negotiation
 Searching for a complete holiday package
 Negotiating public procurement offers

18
Perspectives on the Field
 Agents as a paradigm for software engineering
 Solution for the increasing complexity
 Architectures with components that dynamically interact
 Agent-related concepts such as autonomy are relevant, e.g.,
for edge computing and Internet of Things
 Local, decentralized decisions
 Scalability
 Communication and collaboration, information and resources sharing
to achieve common goals effectively
 Performance adjustments to use resources efficiently
 Fault tolerance

19
MAS and Social Sciences
 Social sciences aim to understand the behavior of societies
 They can use MAS as a tool
 In reality, probably not prediction, but simulation to understand social
processes
 Typical headlines:
 An AI Can Simulate an Economy Millions of Times to Create Fairer Tax
Policy
 Multi-Agent Systems Can Model the Complex Processes of a City
 AI Can Predict Your Future Behavior with Powerful New Simulations
 Agents as a tool for understanding human society
 Asimov, Foundation, psychohistory

20
MAS and Game Theory (GT)
 Turing and von Neumann were interested in both computer
science and game theory
 Afterwards, the two areas separated (GT  economy)
 Lately, results from GT have been successfully applied in MAS
(“rational agent”)
 GT provides descriptive concepts
 MAS uses these results in a computational environment
 Is the concept of a “rational agent” useful for understanding
human society?

21
MAS and Distributed or Concurrent Systems
 Agents are autonomous; synchronization and coordination
structures are not static but dynamic (determined at runtime)
 Agent middleware frameworks ensure concurrent and
distributed execution
 Agents have economic interactions, each has its own interest
(i.e., of its owner)
 Negotiations for a compromise
 Actors: receive messages and react (change their state, decide,
send messages, create other actors)

22
Introduction to Agent-Based Systems
1. Introduction
2. Agents
3. Intelligent Agents
4. Mobile Agents
5. Agent Applications
6. Conclusions

23
Definitions
 An agent is a software or hardware entity that is situated in
some environment, and that is capable of autonomous action in
order to meet its design objectives

 A multi-agent system is a loosely coupled collection of agents

that can work together to solve problems that are beyond the
capabilities or knowledge of individual agents

24
The Fundamental Structure

25
Details
 An agent has effectoric capability: the ability to modify the
environment
 Usually an agent does not have full control over the environment,
it can only influence the environment (partial control)
 The environment can be non-deterministic: the same actions have
different effects
 Actions may fail
 Actions have preconditions, e.g., “lift the table” (strength), “buy a
Ferrari” (money)
 The agent has to decide which actions to perform from a large set of
possibilities

26
Examples of Agents
 Apple Siri (2011)
 Controlling music, messages, and calls
 Setting reminders and timers based on voice commands
 Offering suggestions for calendar events and reminders based on usage patterns
 Amazon Alexa (2014)
 Automatically adjusting smart home devices based on user routines
 Reminding users of tasks
 Suggesting new skills (functionalities) based on usage patterns
 Google Assistant (2016)
 Providing commute updates proactively
 Booking reservations at user-specified times
 Suggesting calendar events based on email content
 Email notification agents
27
Examples of Agents
 Control systems
 Thermostats
 if temperature is too cold then turn heating on
 if temperature is ok then turn heating off
 The effect is not guaranteed, e.g., if the door is open
 Auto-pilots for aviation
 Space probes
 Controllers for nuclear reactors

28
The Environment
 Accessible vs. inaccessible
 Accessible: complete, accurate, up-to-date information about the
state of the environment
 Deterministic vs. non-deterministic
 Deterministic: every action has a single guaranteed effect
 Static vs. dynamic
 Static: the environment remains unchanged except for the effects
of the actions of the agent
 Discrete vs. continuous
 Discrete: finite number of states

29
Accessibility
 We do not know the current temperature at the North Pole
(but with additional effort, we could)
 We are forbidden to know an opponent’s cards in a poker game
 Accessible environments are simpler
 Decisions depend on the quality of information

30
Determinism
 Usually we imagine or want software environments to be
governed by clear rules
 In a deterministic environment, the agents do not need to test
the results of their actions
 In a non-deterministic environment, agents have partial control
and their actions may fail
 If necessary, they have to rebuild their plans
 A very complex deterministic environment can be treated as
non-deterministic

31
Dynamism
 Classic planning methods in AI implicitly assume a static
environment
 In a static environment, the agent can predict the state of the
environment after the execution of an action
 No synchronization or coordination is needed among agents
 In a dynamic environment, there are concurrent processes
 If the agent does not take any action at some point, other agents may
still change the environment
 The state of the environment can even change during the execution
of an action by an agent
 The agent has to repeatedly gather information in order to determine
the state of the environment
32
Continuity
 Examples:
 Discrete environment: chess
 Continuous environment: driving a car

 Computers are discrete, continuity is approximated

 Information may be lost by discretization

 In a discrete environment it is possible (in principle) to

enumerate all the states for finding the optimal action

33
Open Environments
 The most general category:
 Inaccessible
 Non-deterministic
 Dynamic
 Continuous

34
Episodic Environments
 The performance of an agent depends on several
discrete episodes
 There is no connection between its performance in different
episodes
 For example, an email sorting agent

35
Examples

36
Real-Time Systems
 For real-time systems, time plays a role in the evaluation of agent
performance
 Types of interactions
 Making quick decisions to act
 Quick attainment of a state
 Repeating an action as often as possible
 The search for optimal actions depends on time
 Exhaustive search is usually not possible
 n possible actions, n! possible sequences of actions
 Every realistic system is real-time, in a way
 Examples: decisions in milliseconds
 PRS, space probe
 Remote Agent, Deep Space 1

37
Introduction to Agent-Based Systems
1. Introduction
2. Agents
3. Intelligent Agents
4. Mobile Agents
5. Agent Applications
6. Conclusions

38
Definition
 An intelligent agent is an agent with a flexible
behavior:
 Proactiveness: it has a goal-oriented behavior, and can take
the initiative
 Reactivity: it responds in a timely fashion to changes in the
environment
 Social ability: it can interact with other agents, and possibly
humans

39
Proactiveness
 Typical programs (C/Java/C#) are proactive
 It is assumed that the preconditions and the postconditions are
satisfied
 Completely correct program with respect to its
preconditions and postconditions
 Termination is guaranteed when executed in a state where its
preconditions are satisfied
 Upon termination, the postconditions are guaranteed
 Formal specification
 Divide and conquer, top-down refinements, etc.

40
Proactiveness
 In MAS, the environment may change during execution
 Preconditions may become false
 If the goal is no longer valid, the action must not be continued

41
Reactivity
 For example, a server is reactive, it responds to requests
 An agent must make local decisions that have global
consequences
 Fairness
 For example, an agent that gives access to a printer
 First process p1 , then process p2
 It is a fair rule in isolation
 But what if this rule is always applied?
 The long-term effects are difficult to understand

42
Reactivity
 Agents must respond to changes
 A purely reactive agent may never focus on a goal long
enough to actually achieve it

 Finding a balance between proactiveness and reactivity is

a key issue in agent design

43
Social Ability
 The usual data exchange among computers does not have
a social nature
 Social ability refers to the possibilities of cooperation with
other agents with common goals
 The goals of the others must be understood
 Negotiation

44
Agents and Objects
 In object-oriented programming (OOP) we also talk about actors
and messages
 The main difference: the degree of autonomy
 In objects, methods are called directly
 The control flow moves directly into the method
 The decision to execute is at the source (the object that calls the method)
 Agents receive requests to perform actions
 An agent can refuse a request
 The decision to execute is at the destination (the agent that receives the
request)
 In OOP an object can also have a single public method and
conditions in the called method, but this is not the nature of OOP

45
Agents and Objects
 “Objects do it for free; agents do it for money”

46
Introduction to Agent-Based Systems
1. Introduction
2. Agents
3. Intelligent Agents
4. Mobile Agents
5. Agent Applications
6. Conclusions

47
Motivation
 People increasingly use mobile devices
 They may not be permanently connected
 Bandwidth is sometimes insufficient, there may be network
errors
 Every time the device reconnects, its address may change

48
Mobility
 Mobility is the change of the perspective an agent has on its
environment
 It can be a change of physical location (hardware agent, robot) or the
migration to another machine (software agent)
 In a more restrictive sense, a mobile agent is an autonomous
program that can move within a heterogeneous network
 The agent decides when it moves
 It stops its execution on a machine, it moves and resumes its execution
from the same point on another machine
 Agent mobility differs from process mobility in distributed
systems
 In distributed systems, the operating system decides (e.g., for load
balancing); an agent decides by itself
49
Examples
 An agent leaves from a mobile device to the Internet, collects
information and brings an answer
 The network may fall in the meantime, the agent does not
require a permanent connection with the mobile device
 Conversely, a network application can send agents to mobile
devices

50
Advantages of Mobile Agents
 Reducing network traffic (moving the computation to the data)
 Overcoming network latency (agents can execute locally)
 Asynchronous, autonomous execution (agents can be
dispatched into a network and operate independently)
 Integration of heterogeneous systems (the same agent
framework installed on various environments)
 Robustness and fault tolerance (graceful degradation)
 Encapsulation of communication protocols (an agent can be a
kind of adapter)

51
Implementation Issues
 Security
 Protecting hosts from malicious agents
 Protecting agents from malicious hosts
 Portability and standardization
 Agents must run on compatible platforms
 Code representation should be independent of the platform:
Java bytecode, DotNet MSIL / CIL
 Performance and scalability
 Interpreted languages ​are slower
 Just-in-time compilation

52
Introduction to Agent-Based Systems
1. Introduction
2. Agents
3. Intelligent Agents
4. Mobile Agents
5. Agent Applications
6. Conclusions

53
Main Applications
 Multi-agent simulations of real systems of phenomena, in order
to understand them better
 Road traffic simulations
 Social simulations

 Autonomous entities
 Self-driving, autonomous cars
 Different types of auto-pilots
 Space probes, rovers for exploring other planets

54
 Other Applications
 Commercial applications  Entertainment
 Electronic commerce  Games
 Information management:  Films
gathering, filtering  Medical applications
 Business process management  Patient monitoring
 Industrial applications  Healthcare
 Manufacturing
 Process control
 Air traffic control
 Telecommunications
 Transportation systems

For details, read the course support

55
Conclusions
 Agents answer recent trends in computing: ubiquity,
interconnection, intelligence, delegation and human-orientation
 An agent is a software or hardware entity that is situated in some
environment, and that is capable of autonomous action in order to
meet its design objectives
 An intelligent agent is an agent with a flexible behavior:
proactiveness, reactivity and social ability
 A multi-agent system is a loosely coupled collection of agents that can
work together to solve problems that are beyond the capabilities or
knowledge of individual agents

56