Unit 5: Decision Making

1. Basics of Utility Theory

• Utility as a Measure of Preference
Utility theory is used in AI to quantify the preferences of an agent. It
assigns a numerical value (called utility) to each possible outcome,
indicating how desirable it is. Higher utility values represent more
preferred outcomes.
• Expected Utility Principle
When facing uncertainty, agents don't just pick the most likely outcome—
they calculate the expected utility, which is the weighted average of all
possible outcomes, based on their probabilities and utilities. This helps
agents make optimal decisions under uncertainty.
• Utility Functions
A utility function maps states or outcomes to real numbers. This allows
agents to rank different outcomes, not just on whether they succeed or
fail, but how good or bad each outcome is. For example, an autonomous
car may have a utility function considering safety, time, and fuel
efficiency.
• Risk Attitude and Utility Curves
Utility theory also accounts for different risk preferences: risk-averse,
risk-neutral, and risk-seeking. A risk-averse agent prefers a certain
outcome over a gamble with higher or equal expected utility, while a risk-
seeking one may choose gambles for potentially higher rewards.

2. Basics of Decision Theory

• Decision Theory Framework
Decision theory combines probability theory and utility theory to guide
rational decision-making. It provides a formal framework for choosing
the best action by evaluating the expected utility of each possible choice.
• Components of Decision Problems
A typical decision problem consists of: a set of actions, a set of possible
states of the world, a probability distribution over states, and a utility
function that evaluates outcomes. The goal is to select the action with the
highest expected utility.
 • Bayesian Decision Theory
This is a fundamental model where agents update their beliefs using
Bayes’ theorem and then make decisions based on these updated beliefs.
It's especially important when new evidence becomes available.
• Real-World Application
In AI, decision theory is applied in various fields, including diagnostic
systems, recommendation systems, and strategic planning agents, where
agents must make rational choices even under uncertainty.

3. Sequential Decision Theory

• Decision-Making Over Time
Sequential decision theory deals with scenarios where decisions are made
in a sequence, and each decision affects future decisions and outcomes.
It's not about one-time choices but long-term planning.
• Markov Decision Processes (MDPs)
A common model for sequential decision-making. It includes states,
actions, transition probabilities, rewards, and a discount factor. The goal
is to find a policy that maximizes the expected cumulative reward.
• Value and Policy Iteration
These are two dynamic programming methods used to solve MDPs.
Value iteration updates utility values iteratively, while policy iteration
updates the action policy directly. Both methods aim to reach optimal
decision policies.
• Partially Observable MDPs (POMDPs)
In real-world AI, agents may not have full observability. POMDPs model
such situations by including belief states and observation probabilities.
They are used in robotics, natural language processing, and planning.

4. Decision Problems
• Single-Agent vs. Multi-Agent
Decision problems can involve a single decision-maker (like a robot
navigating a room) or multiple agents (like cars on a road). In multi-agent
settings, agents must also predict and respond to the actions of others.
 • Deterministic vs. Stochastic Problems
In deterministic problems, outcomes are fully determined by actions. In
stochastic (probabilistic) problems, outcomes are uncertain and described
by probability distributions.
• Static vs. Dynamic Problems
Static problems involve a one-time decision with no future implications,
while dynamic problems require a sequence of decisions that affect the
agent's future environment and options.
• Constraints and Objectives
Many decision problems have constraints (e.g., time, resources, budget)
and multiple objectives (e.g., maximize profit while minimizing risk),
which complicate the decision-making process and require multi-
objective optimization.

5. Elementary Game Theory

• What is Game Theory?
Game theory is the study of strategic interaction between agents
(players), where the outcome for each depends on the actions of all. It's
crucial in multi-agent AI environments.
• Types of Games
o Cooperative vs. Non-Cooperative: In cooperative games, agents
form alliances and share rewards; in non-cooperative games, each
agent acts in their own interest.
o Zero-Sum vs. Non-Zero-Sum: In zero-sum games, one player’s
gain is another’s loss. In non-zero-sum games, all players can
benefit or suffer.
o Simultaneous vs. Sequential: In simultaneous games, players act
at the same time. In sequential games, players act one after another,
considering previous actions.
• Nash Equilibrium
A stable state where no player can benefit by changing their strategy
unilaterally. It is a key solution concept in game theory and is used in
designing AI for auctions, negotiations, and autonomous systems.
 • Application in AI
Game theory underlies negotiation bots, auction algorithms, multi-agent
robotics, and strategies in competitive environments like online games or
automated bidding systems.

6. Sample Applications of Decision-Making in AI

• Medical Diagnosis Systems
AI systems use decision theory to recommend treatment options by
evaluating the expected outcomes and patient risks based on probabilistic
models.
• Autonomous Vehicles
Self-driving cars use sequential decision-making to decide actions like
lane changes, speed adjustments, and obstacle avoidance over time.
• Recommendation Engines
Systems like Netflix or Amazon use utility theory to recommend movies
or products by maximizing the user's satisfaction (utility) based on past
behavior.
• Robotic Path Planning
Robots use decision algorithms to navigate through complex
environments while minimizing energy and maximizing task efficiency.
• Game AI
Game-playing agents (like AlphaGo) use a combination of decision
theory, reinforcement learning, and game theory to choose optimal moves
against human or computer opponents.
• Financial Decision Systems
AI in finance evaluates risk and return for investments using decision
models, helping in portfolio optimization and automated trading.