Foundation of AI

1) Philosophy (the study of the fundamental nature of

knowledge)
Can formal rules be used to draw valid conclusions?
How does the mind arise from a physical brain?
Where does knowledge come from?
How does knowledge lead to action?

Aristotle (384–322 B.C.), was the first to formulate a precise set of laws
governing the rational part of the mind. He developed an informal system of
syllogisms for proper reasoning, which in principle allowed one to generate
conclusions mechanically, given initial premises.

Thomas Hobbes (1588–1679) proposed that reasoning was like numerical

computation that "we add and subtract in our silent thoughts."

2) Mathematics
What are the formal rules to draw valid conclusions?
What can be computed?

Formal science required a level of mathematical formalization in three

fundamental areas: logic, computation, and probability.

Logic:

George Boole (1815–1864), who worked out the details of propositional,

or Boolean, logic.
In 1879, Gottlob Frege (1848–1925) extended Boole’s logic to include
objects and relations, creating the first-order logic that is used today.

First-order logic – Contains predicates, quantifiers, and variables

E.g. Philosopher(a) ⇒ Scholar(a)

∀x, effect_carona(x) ⇒ quarantine(x)

1 / 29
 ∀x, King(x) ^ Greedy (x) ⇒ Evil (x)

3) Economics
How should we make decisions so as to maximize payoff?
How should we do this when the payoff may be far in the future?

The science of economics got its start in 1776, when Scottish philosopher
Adam Smith treated it as a science, using the idea that economies can be
thought of as consisting of individual agents maximizing their own economic
well-being.

4) Neuroscience (1861-present)
How do brains process information?

Neuroscience is the study of the nervous system, particularly the brain. The
exact way in which the brain enables thought is one of the great mysteries of
science. It has been appreciated for thousands of years that the brain is
somehow involved in thought, because of the evidence that strong blows to
the head can lead to mental incapacitation. The brain is recognized as the
seat of consciousness. Before then, candidate locations included the heart,
the spleen, and the pineal gland. The truly amazing conclusion is that a
collection of simple cells can lead to thought, action, and consciousness or,
brains cause minds. It has also long been known that human brains are
somehow different; in about 335 B.C. Aristotle wrote, “Of all the animals, man
has the largest brain in proportion to his size.”

5) Psychology
How do humans and animals think and act?

The principal characteristic of cognitive psychology is that the brain

processes and processes information. The claim is that beliefs, goals, and
reasoning steps can be useful components of a theory of human behavior.

The knowledge-based agent has three key steps:

2 / 29
 1. Stimulus is translated into an internal representation
2. The representation is manipulated by cognitive processes to derive new
internal representations
3. These are translated into actions

6) Computer Engineering
How can we build an efficient computer?

For artificial intelligence to succeed, we need two things: intelligence and an

artifact. The computer has been the artifact of choice. The modern digital
electronic computer was invented independently and almost simultaneously
by scientists in three countries embattled in World War II.

History of AI
Maturation of Artificial Intelligence (1943-1952)
Year 1943: The first work which is now recognized as AI was done by
Warren McCulloch and Walter Pitts in 1943. They proposed a model of
artificial neurons.
Year 1949: Donald Hebb demonstrated an updating rule for modifying
the connection strength between neurons. His rule is now called
Hebbian learning.
Year 1950: Alan Turing, an English mathematician, pioneered Machine
learning in 1950. Alan Turing published "Computing Machinery and
Intelligence" in which he proposed a test. The test can check the
machine's ability to exhibit intelligent behavior equivalent to human
intelligence, called a Turing test.

The Birth of Artificial Intelligence (1952-1956)

Year 1955: Allen Newell and Herbert A. Simon created the "first
artificial intelligence program" which was named "Logic Theorist". This
program proved 38 of 52 Mathematics theorems and found new and
more elegant proofs for some theorems.

3 / 29
 Year 1956: The word "Artificial Intelligence" was first adopted by
American Computer scientist John McCarthy at the Dartmouth
Conference. For the first time, AI was coined as an academic field. At
that time high-level computer languages such as FORTRAN, LISP, or
COBOL were invented. The enthusiasm for AI was very high at that time.

The Golden Years - Early Enthusiasm (1956-1974)

Year 1966: The researchers emphasized developing algorithms that can
solve mathematical problems. Joseph Weizenbaum created the first
chatbot in 1966, which was named ELIZA.
Year 1972: The first intelligent humanoid robot was built in Japan,
named WABOT-1.

The First AI Winter (1974-1980)

The duration between 1974 to 1980 was the first AI winter duration. AI winter
refers to the time period where computer scientists dealt with a severe
shortage of funding from the government for AI research. During AI winters,
the interest of publicity on artificial intelligence decreased.

A Boom of AI (1980-1987)
Year 1980: After the AI winter duration, AI came back with "Expert
System". Expert systems were programmed to emulate the decision-
making ability of a human expert. In 1980, the first national conference
of the American Association of Artificial Intelligence was held at
Stanford University.

The Second AI Winter (1987-1993)

The duration between the years 1987 to 1993 was the second AI Winter
duration. Again, investors and the government stopped funding for AI
research due to high costs but not efficient results. The expert system such as
XCON was very cost-effective.

4 / 29
The Emergence of Intelligent Agents (1993-2011)
Year 1997: In 1997, IBM Deep Blue beat world chess champion Gary
Kasparov and became the first computer to beat a world chess
champion.
Year 2002: For the first time, AI entered the home in the form of
Roomba, a vacuum cleaner.
Year 2006: AI came into the business world. Companies like Facebook,
Twitter, and Netflix started using AI.

Deep Learning, Big Data, and Artificial General Intelligence

(2011-present)
Year 2011: IBM's Watson won Jeopardy, a quiz show, where it had to
solve complex questions as well as riddles. Watson proved that it could
understand natural language and solve tricky questions quickly.
Year 2012: Google launched an Android app feature "Google Now",
which was able to provide information to the user as a prediction.
Year 2014: Chatbot "Eugene Goostman" won a competition in the
infamous "Turing test."
Year 2018: The "Project Debater" from IBM debated on complex topics
with two master debaters and performed extremely well. Google
demonstrated an AI program "Duplex" which was a virtual assistant that
took hairdresser appointments on call, and the lady on the other side
didn't notice that she was talking with a machine.

The State of the Art

What can AI do today?

A concise answer is difficult because there are so many activities in so many

subfields. Following are a few applications that show the state of the art:

1. Speech Recognition: A traveler calling United Airlines to book a flight

can have the entire conversation guided by an automated speech
recognition and dialog management system.

5 / 29
 2. Robotic Vehicles: Today's robots are far more sophisticated than their
predecessors. Many are semi-autonomous, with machine vision systems
to interact within a changing environment. Some can even work side-by-
side with humans. All signs suggest we are in the middle of a new
industrial robot boom.
3. Autonomous Planning and Scheduling: A hundred million miles
from Earth, NASA's Remote Agent program became the first on-board
autonomous planning program to control the scheduling of operations
for a spacecraft.
4. Game Playing (GP): It is the design of artificial intelligence programs
to be able to play more than one game successfully. For many games like
chess, computers are programmed to play these games using a specially
designed algorithm, which cannot be transferred to another context. For
example, a chess-playing computer program cannot play checkers.
5. Spam Fighting: The spam filter now uses an artificial neural network
to detect and block especially sneaky or dishonest spam mails.
6. Machine Translation: A computer program automatically translates
from Arabic to English, allowing an English speaker to understand the
content.

Above are just a few examples of artificial intelligence systems that exist
today. Not magic or science fiction, but rather science, engineering, and
mathematics.

What is an Agent?
An agent can be anything that perceives its environment through
sensors and acts upon that environment through actuators.
An Agent runs in the cycle of perceiving, thinking, and acting.
An agent gets percepts one at a time and maps this percept sequence to
action.
Percepts are the electrical signals from sensors after processing objects
in the visual field (like location, colors, loudness, direction, etc.)

6 / 29
Types of Agents:
Human-Agent: A human agent has eyes, ears, and other organs which
work for sensors and hand, legs, vocal tract work for actuators.
Robotic Agent: A robotic agent can have cameras, infrared range
finder, NLP for sensors and various motors for actuators.
Software Agent: Software agent can have keystrokes, file contents as
sensory input and act on those inputs and display output on the screen.

Hence the world around us is full of agents such as thermostats, cellphones,

cameras, and even we are also agents. Before moving forward, we should first
know about sensors, effectors, and actuators.

Sensors:

Sensor is a device that detects the change in the environment and sends the
information to other electronic devices. An agent observes its environment
through sensors.

Actuators:

Actuators are the components of machines that convert energy into motion.
The actuators are only responsible for moving and controlling a system. An
actuator is a device that causes machines or other devices to operate. An
actuator can be an electric motor, gears, rails, etc.

Effectors:
Effectors are the devices that affect the environment. Effectors can be legs,
wheels, arms, fingers, wings, fins, and display screens.

Intelligent Agents:

An intelligent agent is an autonomous entity that acts upon an environment

using sensors and actuators to achieve goals. An intelligent agent may learn

7 / 29
from the environment to achieve its goals. A thermostat is an example of an
intelligent agent.

Following are the main four rules for an AI agent:

1. An AI agent must have the ability to perceive the environment.

2. The observation must be used to make decisions.
3. Decision should result in an action.
4. The action taken by an AI agent must be a rational action.

Structure of Intelligent Agent

An intelligent agent is anything that perceives its environment, takes actions
autonomously to achieve goals, and may improve its performance with
learning or may use knowledge. It interacts with surroundings via perception
through sensors and actions through effectors or actuators.

Agent = Architecture + Agent Function + Agent Program

The IA structure consists of three main parts: architecture, agent function,
and agent program.

Architecture:

This refers to machinery or devices that consist of actuators and sensors. The
intelligent agent executes on this machinery. Examples include a personal
computer, a car, or a camera.

Agent Function:

This is a function in which actions are mapped from a certain percept

sequence.
[ f: P* → A ]
Percept sequence refers to a history of what the intelligent agent has
perceived.

8 / 29
Agent Program:

This is an implementation or execution of the agent function. The agent

function is produced through the agent program’s execution on the physical
architecture.

Interaction of Agents with Environment

Interaction of the Agent with the environment uses Sensors and Effectors.
Sensors perceive the environment and the actuators or effectors act upon that
environment. This interaction can occur in two different ways:

Perception:

Perception is a passive interaction between the agent and the environment

where the environment remains unchanged when the agent takes up
information from the environment. This involves gaining information using
'Sensors' from the surroundings without any change to the surroundings.

Action:
Action is an active interaction between the agent and the environment where
the environment changes when the action is performed. This involves the
utilization of an 'Effector' or an 'Actuator' which completes an action but leads
to changes in the surroundings while doing so. For example, in the case of a
virtual agent, when the virtual agent reads and interprets the information
provided by the user, it is known as 'Perception' while when it replies to the
user based on the interpretation it is known as 'Action'.

Characteristics of Intelligent Agents

Intelligent agents have the following distinguishing characteristics:

They have some level of autonomy that allows them to perform certain
tasks on their own.

9 / 29
 They have a learning ability that enables them to learn even as tasks are
carried out.
They can interact with other entities such as agents, humans, and
systems.
New rules can be accommodated by intelligent agents incrementally.
They exhibit goal-oriented habits.
They are knowledge-based. They use knowledge regarding
communications, processes, and entities.

Rational Agent:
A rational agent is an agent that has clear preferences, models uncertainty,
and acts in a way to maximize its performance measure with all possible
actions. A rational agent is said to perform the right things. AI is about
creating rational agents to use for game theory and decision theory for
various real-world scenarios. For an AI agent, the rational action is most
important because in the AI reinforcement learning algorithm, for each best
possible action, the agent gets a positive reward and for each wrong action,
an agent gets a negative reward.

Rationality:

The rationality of an agent is measured by its performance measure.

Rationality can be judged on the basis of the following points:

Performance measure which defines the success criterion.

Agent prior knowledge of its environment.
Best possible actions that an agent can perform.
The sequence of percepts.

PEAS Representation
PEAS is a type of model on which an AI agent works upon. When we define
an AI agent or rational agent, then we can group its properties under the
PEAS representation model. It is made up of four words:

10 / 29
 P: Performance measure
E: Environment
A: Actuators
S: Sensors

Solve exercise of PEAS for the following:

Taxi driver
Medical diagnosis system
Satellite Image System
Refinery Controller
Part Picking Robot
Interactive English Tutor

Deterministic vs. Stochastic

If the next state of the environment is completely determined by the
current state and the action executed by the agent, then we say the
environment is deterministic;
Otherwise, it is stochastic.

Episodic vs. Sequential

In an episodic task environment, the agent's experience is divided into
atomic episodes.
Each episode consists of the agent perceiving and then performing a
single action.
Crucially, the next episode does not depend on the actions taken in
previous episodes.
For example, an agent that has to spot defective parts on an assembly
line bases each decision on the current part, regardless of previous
decisions;
In sequential environments, on the other hand, the current decision
could affect all future decisions.
Chess and taxi driving are sequential.

11 / 29
Discrete vs. Continuous
The discrete/continuous distinction can be applied to the state of the
environment, to the way time is handled, and to the percepts and
actions of the agent.
For example, a discrete-state environment such as a chess game has a
finite number of distinct states.
Chess also has a discrete set of percepts and actions.
Taxi driving is a continuous-state and continuous-time problem:
The speed and location of the taxi and of the other vehicles sweep
through a range of continuous values and do so smoothly over
time.
Taxi-driving actions are also continuous (steering angles, etc.)

Single Agent vs. Multiagent

An agent solving a crossword puzzle by itself is clearly in a single-agent
environment
Whereas an agent playing chess is in a two-agent environment.
Multiagent is further classified into two ways:
Competitive multiagent environment
Cooperative multiagent environment

Types of Agents in Artificial Intelligence With Examples

Based on their degree of perceived intelligence and capability, types of agents
in artificial intelligence can be divided into:

Simple Reflex Agents

Model-Based Agents
Goal-Based Agents
Utility-Based Agents
Learning Agents

Performance can be improved and better action can be generated for each of
these types of agents in AI.

12 / 29
Simple Reflex Agents

This is a simple type of agent that works on the basis of the current percept
and not based on the rest of the percepts history. The agent function, in this
case, is based on condition-action rule where the condition or the state is
mapped to the action such that action is taken only when the condition is true
or else it is not. If the environment associated with this agent is fully
observable, only then is the agent function successful, if it is partially
observable, in that case, the agent function enters into infinite loops that can
be escaped only on randomization of its actions. The problems associated
with this type include very limited intelligence, no knowledge of non-
perceptual parts of the state, huge size for generation and storage, and
inability to adapt to changes in the environment.
Example: A thermostat in a heating system.

Model-Based Agents
Model-based agents utilize the condition-action rule, where it works by
finding a rule that will allow the condition, which is based on the current
situation, to be satisfied. Irrespective of the first type, it can handle partially
observable environments by tracking the situation and using a particular
model related to the world. It consists of two important factors, which are
Model and Internal State. The model provides knowledge and understanding
of the process of occurrence of different things in the surroundings such that
the current situation can be studied and a condition can be created. Actions
are performed by the agent based on this model. Internal State uses the
perceptual history to represent a current percept. The agent keeps track of
this internal state and is adjusted by each of the percepts. The current
internal state is stored by the agent inside it to maintain a kind of structure
that can describe the unseen world. The state of the agent can be updated by
gaining information about how the world evolves and how the agent's action
affects the world.
Example: A vacuum cleaner that uses sensors to detect dirt and obstacles and
moves and cleans based on a model.

13 / 29
Goal-Based Agents
This type takes decisions on the basis of its goal or desirable situations so that
it can choose such an action that can achieve the goal required. It is an
improvement over model-based agents where information about the goal is
also included. This is because it is not always sufficient to know just about the
current state, knowledge of the goal is a more beneficial approach.

The aim is to reduce the distance between action and the goal so that the best
possible way can be chosen from multiple possibilities. Once the best way is
found, the decision is represented explicitly which makes the agent more
flexible.

It carries out considerations of different situations called searching and

planning by considering long sequences of possible actions for confirming its
ability to achieve the goal. This makes the agent proactive. It can easily
change its behavior if required.

Example: A chess-playing AI whose goal is winning the game.

Utility-Based Agents
The Utility-Based Agents are similar to the Goal-Based Agents. There are
multiple sequences in which a goal state can be achieved. Some sequences
may be better than the others. The success or failure of this agent depends on
the choice of the appropriate route taken to reach the goal.

The difference in Goal-Based and Utility-Based Agents lies in the fact that
Goal-Based Agents only measure if a state can be a possible goal state or not.
But the Utility-Based Agents keep a measure of the desirability
(preference/utility) of these multiple routes to reach the goal state. This
measure is kept by a utility function. The utility function maps a state with a
real number which describes the happiness degree of the agent.

A Utility-Based Agent has to keep a regular track of its environment. This

enables it to reach a goal state in the quickest, cheapest, and safest way. A
rational agent selects the best possible action based on the utility of the

14 / 29
output. Hence these agents involve a great deal of perception, reasoning,
learning, and representation.

Learning Agents
As the name suggests, a learning agent has the capability to learn from its
past experiences. It starts its working in unknown environments by using its
basic knowledge and then it automatically adapts to the environment through
its learning process to reach its goal.

These agents comprise the following components:

Learning Element: It is through the learning element that the agent

learns about the changes in the environment.
The Critic: The critic gives feedback to the learning agent. The critic
evaluates the feedback based on the performance standards fixed in
advance. The agent comes to know about its performance based on the
feedback given to it by the critic.
Performance Element: This is responsible for selecting the best
external action for the agent.
Problem Generator: This component suggests actions for the agent
which are based on new informative experiences. It also keeps track of
the past.

These agents are hence capable of learning, analyzing their performance,

and finding new and innovative ways to improve their performance.

Algorithm under Informed Search

1. Greedy Best-First Search
2. A*

Greedy Best-First Search

Greedy best-first search is an informed search algorithm where the

evaluation function is strictly equal to the heuristic function, disregarding the
edge weights in a weighted graph because only the heuristic value is
15 / 29
considered. In order to search for a goal node, it expands the node that is
closest to the goal as determined by the heuristic function. This approach
assumes that it is likely to lead to a solution quickly. However, the solution
from a greedy best-first search may not be optimal since a shorter path may
exist.

In this algorithm, search cost is at a minimum since the solution is found

without expanding a node that is not on the solution path. This algorithm is
minimal, but not complete, since it can lead to a dead end. It’s called
“Greedy” because at each step it tries to get as close to the goal as it can.

Evaluation Function

The evaluation function, ( f(x) ), for the greedy best-first search algorithm is
the following:
[ f(x) = h(x) ]

Here, the evaluation function is equal to the heuristic function. Since this
search disregards edge weights, finding the lowest-cost path is not
guaranteed.

Heuristic Function

A heuristic function, ( h(x) ), evaluates the successive node based on how

close it is to the target node. In other words, it chooses the immediate low-
cost option. As this is the case, however, it does not necessarily find the
shortest path to the goal.

Suppose a bot is trying to move from point A to point B. In greedy best-first

search, the bot will choose to move to the position that brings it closest to the
goal, disregarding if another position ultimately yields a shorter distance. In
the case that there is an obstruction, it will evaluate the previous nodes with
the shortest distance to the goal, and continuously choose the node that is
closest to the goal.

Steps of the Algorithm

1. Initialize a tree with the root node being the start node in the open list.

16 / 29
 2. If the open list is empty, return a failure, otherwise, add the current
node to the closed list.
3. Remove the node with the lowest ( h(x) ) value from the open list for
exploration.
4. If a child node is the target, return a success. Otherwise, if the node has
not been in either the open or closed list, add it to the open list for
exploration.

A* Search
A* Search is an informed best-first search algorithm that efficiently
determines the lowest cost path between any two nodes in a directed
weighted graph with non-negative edge weights. This algorithm is a variant of
Dijkstra’s algorithm. A slight difference arises from the fact that an evaluation
function is used to determine which node to explore next.

Evaluation Function

The evaluation function, ( f(x) ), for the A* search algorithm is the following:
[ f(n) = g(n) + h(n) ]

Where ( g(n) ) represents the cost to get to node ( n ) and ( h(n) ) represents
the estimated cost to arrive at the goal node from node ( n ). For the
algorithm to generate the correct result, the evaluation function must be
admissible, meaning that it never overestimates the cost to arrive at the goal
node.

Steps

The A* algorithm is implemented in a similar way to Dijkstra’s algorithm.

Given a weighted graph with non-negative edge weights, to find the lowest-
cost path from a start node ( S ) to a goal node ( G ), two lists are used:

An open list, implemented as a priority queue, which stores the next

nodes to be explored. Because this is a priority queue, the most
promising candidate node (the one with the lowest value from the
evaluation function) is always at the top. Initially, the only node in this
list is the start node ( S ).

17 / 29
 A closed list which stores the nodes that have already been evaluated.
When a node is in the closed list, it means that the lowest-cost path to
that node has been found.

To find the lowest cost path, a search tree is constructed in the following way:

1. Initialize a tree with the root node being the start node ( S ).
2. Remove the top node from the open list for exploration.
3. Add the current node to the closed list.
4. Add all nodes that have an incoming edge from the current node as
child nodes in the tree.
5. Update the lowest cost to reach the child node.
6. Compute the evaluation function for every child node and add them to
the open list.

Just like in Dijkstra’s algorithm, the lowest cost is updated as the algorithm
progresses in the following way:
[ \text{current_lowest_cost} = \min(\text{current_lowest_cost},
\text{parent_node_cost} + \text{edge_weight}) ]

All nodes except for the start node start with a lowest cost of infinity. The
start node has an initial lowest cost of 0. The algorithm concludes when the
goal node ( G ) is removed from the open list and explored, indicating that a
shortest path has been found. The shortest path is the path from the start
node ( S ) to the goal node ( G ) in the tree.

Note: The algorithm ends when the goal node ( G ) has been explored, NOT
when it is added to the open list.

Decision Trees
A decision tree is a type of supervised machine learning used to categorize or
make predictions based on how a previous set of questions were answered.
The model is a form of supervised learning, meaning that the model is
trained and tested on a set of data that contains the desired categorization.

Some key terms of a decision tree

18 / 29
 Root node: The base of the decision tree.
Splitting: The process of dividing a node into multiple sub-nodes.
Decision node: When a sub-node is further split into additional sub-
nodes.
Leaf node: When a sub-node does not further split into additional sub-
nodes; represents possible outcomes.
Pruning: The process of removing sub-nodes of a decision tree.
Branch: A subsection of the decision tree consisting of multiple nodes.

Advantages of Decision Trees

Works for numerical or categorical data and variables.

Models problems with multiple outputs.
Tests the reliability of the tree.
Requires less data cleaning than other data modeling techniques.

Disadvantages

Affected by noise in the data.

Not ideal for large datasets.
Can disproportionately value, or weigh, attributes.
The decisions at nodes are limited to binary outcomes, reducing the
complexity that the tree can handle.
Trees can become very complex when dealing with uncertainty and
numerous linked outcomes.

Factor for Decision Trees:

1. Entropy

Entropy basically tells us how impure a collection of data is. The term impure
here defines non-homogeneity. In other words, we can say, “Entropy is the
measurement of homogeneity. It returns us the information about an
arbitrary dataset that how impure/non-homogeneous the dataset is". Entropy
controls how a Decision Tree decides to split the data.

19 / 29
Learning from Examples
An agent is learning if it improves its performance on future tasks after
making observations about the world.

Forms of Learning

Any component of an agent can be improved by learning from data. The

improvements, and the techniques used to make them, depend on four major
factors:

Which component is to be improved.

What Prior knowledge the agent already has.
What Representation is used for the data and the component.
What Feedback is available to learn from.

There are three types of learning:

Supervised Learning
Supervised learning is a learning method in which a model learns from a
labeled dataset containing input-output pairs. Each input in the dataset has a
corresponding correct output (the label), and the model's task is to learn the
relationship between the inputs and outputs. This enables the model to make
predictions on new, unseen data by applying the learned mapping.

Example of Supervised Learning

Predicting house prices: The input might be house features such as size,
location, and number of bedrooms, and the output would be the house price.
The supervised learning model would learn the relationship between these
features and house prices from historical data, and then it could predict
prices for new houses entering the market.

Categories of Supervised Learning

Regression: When dealing with real-valued output variables like "price"

or "temperature," several popular Regression algorithms come into play,

20 / 29
 such as the Simple Linear Regression Algorithm, Multivariate
Regression Algorithm, Decision Tree Algorithm, and Lasso Regression.
Classification: In instances where the output variable is a category,
like distinguishing between 'spam' and 'not spam' in email filtering,
several widely-used classification algorithms come into play. These
encompass the following algorithms: Random Forest, Decision Tree,
Logistic Regression, and Support Vector Machine.

Advantages of Supervised Learning

Effectiveness: Supervised learning can predict outcomes based on

past data.
Simplicity: It's relatively easy to understand and implement.
Performance Evaluation: It is easy to measure the performance of a
supervised learning model since the ground truth (labels) is known.
Applications: Can be used in various fields like finance, healthcare,
marketing, etc.
Feature Importance: It allows an understanding of which features are
most important in making predictions.

Disadvantages of Supervised Learning

Dependency on Labeled Data: Supervised learning requires a large

amount of labeled data, which can be expensive and time-consuming.
Overfitting: Models can become too complex and fit the noise in the
training data rather than the actual signal, which degrades their
performance on new data.
Generalization: Sometimes, these models do not generalize well to
unseen data if the data they were trained on does not represent the
broader context.

Applications of Supervised Learning

Healthcare: Used to predict patient diagnoses based on symptoms and

past medical history.
Finance: For credit scoring and predicting stock prices.
Retail: To forecast sales, recommend products, and personalize
marketing.

21 / 29
 Autonomous Vehicles: These are used to recognize traffic signs and
pedestrians.
Speech Recognition: In virtual assistants and transcription services.

Unsupervised Learning

In unsupervised learning, the agent learns patterns in the input even though
no explicit feedback is supplied. Unlike supervised learning, where the
training data includes both input vectors and corresponding target labels,
unsupervised learning algorithms try to learn patterns and relationships
directly from the input data.

Example of Unsupervised Learning

Clustering: A common unsupervised learning technique is clustering, where

data is grouped into subsets (clusters) such that data in each cluster are more
similar than those in others. For instance, a company could use clustering to
segment its customers based on purchasing behavior without prior
knowledge of the customer groups' characteristics.

Categories of Unsupervised Learning

Clustering: Grouping similar instances into clusters (e.g., k-means,

hierarchical clustering). Some popular clustering algorithms are the K-
Means Clustering algorithm, Mean-shift algorithm, DBSCAN Algorithm,
Principal Component Analysis, and Independent Component Analysis.
Association: Discovering rules that capture interesting relationships
between variables in large databases (e.g., market basket analysis).
Some popular algorithms of Association are the Apriori Algorithm,
Eclat, and FP-growth algorithm.
Dimensionality Reduction: Reducing the number of random variables
under consideration (e.g., PCA, t-SNE), which helps to simplify the data
without losing important information.

Advantages of Unsupervised Learning

Discovering Hidden Patterns: It can identify patterns and

relationships in data that are not initially evident.

22 / 29
 No Need for Labeled Data: Works with unlabeled data, making it
useful where obtaining labels is expensive or impractical.
Reduction of Complexity in Data: Helps reduce the dimensionality of
data, making complex data more comprehensible.
Feature Discovery: This can be used to find useful features that can
improve the performance of supervised learning algorithms.
Flexibility: Can handle changes in input data or the environment since
it doesn’t rely on predefined labels.

Disadvantages of Unsupervised Learning

Interpretation of Results: The results can be ambiguous and harder to

interpret than those from supervised learning models.
Dependency on Input Data: The output quality heavily depends on the
quality of the input data.
Lack of Precise Objectives: Without specific tasks like prediction or
classification, the direction of learning is less focused, leading to less
actionable insights.

Applications of Unsupervised Learning

Customer Segmentation: Businesses use clustering to segment

customers based on behaviors and preferences for targeted marketing.
Anomaly Detection: Identifying unusual data points can be critical in
fraud detection or network security.
Recommendation Systems: Associative models help build
recommendation systems that suggest products based on user behavior.
Feature Elicitation: Used in preprocessing steps to extract new
features from raw data which can improve the accuracy of predictive
models.
Image Segmentation: Applied in computer vision to divide an image
into meaningful segments and analyze each segment individually.

Reinforcement Learning
In reinforcement learning, the agent learns from a series of reinforcements—
rewards or punishments. For example, the lack of a tip at the end of the
journey gives the taxi agent an indication that it did something wrong.
23 / 29
Example of Reinforcement Learning

Chess game: A classic example of reinforcement learning is the game of

chess. In this scenario, the RL agent learns to play chess by playing games
against opponents. Each move the agent makes results in a new board state
and possibly a reward (such as capturing an opponent's piece) or a penalty
(such as losing a piece). The agent learns effective strategies over time by
maximizing its cumulative rewards (ultimately aiming to win games).

Categories of Reinforcement Learning

Model-based RL: In this category, the agent builds a model of the

environment and uses it to predict future rewards and states. This allows
the agent to plan by considering potential future situations before taking
action.
Model-free RL: Here, the agent learns to act without explicitly
constructing a model of the environment. It directly learns the value of
actions or action policies from experience.
Partially Observable RL: This type involves situations where the agent
doesn't have access to the full state of the environment. The agent must
learn to make decisions based on incomplete information, often using
strategies that involve maintaining internal state estimates.

Advantages of Reinforcement Learning

Adaptability: RL agents can adapt to new environments or changes

within their environment, making them suitable for dynamic and
uncertain situations.
Decision-Making Autonomy: RL agents make decisions based on
learned experiences rather than pre-defined rules, which can be
advantageous in complex environments where manual behavior
specification is impractical.
Continuous Learning: Since the learning process is continuous, RL
agents can improve their performance over time as they gain more
experience.
Handling Complexity: RL can handle problems with high complexity
and numerous possible states and actions, which might be infeasible for
traditional algorithms.

24 / 29
 Optimization: RL is geared towards optimization of the decision-
making process, aiming to find the best sequence of actions for any
given situation.

Disadvantages of Reinforcement Learning

Dependency on Reward Design: The effectiveness of an RL agent is

heavily dependent on the design of the reward system. Poorly designed
rewards can lead to unwanted behaviors.
High Computational Cost: Training RL models often requires
significant computational resources and time, especially as the
complexity of the environment increases.
Sample Inefficiency: RL algorithms typically require many
interactions with the environment to learn effective policies, which can
be impractical in real-world scenarios where each interaction could be
costly or time-consuming.

Applications of Reinforcement Learning

Autonomous Vehicles: RL is used to develop autonomous driving

systems, helping vehicles learn to navigate complex traffic
environments safely.
Robotics: RL enables robots to learn complex tasks like walking,
picking up and manipulating objects, and interacting with humans and
other robots in a dynamic environment.
Gaming: In the gaming industry, RL is used to develop AI that can
challenge human players, adapt to their strategies, and provide
engaging gameplay.
Finance: RL can be applied to trading and investment strategies where
the algorithm learns to make buying and selling decisions to maximize
financial return.
Healthcare: RL algorithms are being explored for various applications
in healthcare, including personalized treatment recommendation
systems and management of healthcare logistics.

25 / 29
Ensemble Learning
Ensemble learning is a combination of several learning models in one
problem. These models are known as weak learners. The intuition is that
when you combine several weak learners, they can become strong learners.
Each weak learner is fitted on the training set and provides predictions
obtained. The final prediction result is computed by combining the results
from all the weak learners.

Basic Ensemble Learning Techniques

Max Voting

In classification, the prediction from each model is a vote. In max voting, the
final prediction comes from the prediction with the most votes.

Let’s take an example where you have three classifiers with the following
predictions:

Classifier 1 – class A
Classifier 2 – class B
Classifier 3 – class B

The final prediction here would be class B since it has the most votes.

Averaging

In averaging, the final output is an average of all predictions. This goes for
regression problems. For example, in random forest regression, the final
result is the average of the predictions from individual decision trees.

Let’s take an example of three regression models that predict the price of a
commodity as follows:

Regressor 1 – 200
Regressor 2 – 300
Regressor 3 – 400

The final prediction would be the average of 200, 300, and 400.
26 / 29
Weighted Average

In weighted averaging, the base model with higher predictive power is more
important. In the price prediction example, each of the regressors would be
assigned a weight. The sum of the weights would equal one. Let’s say that the
regressors are given weights of 0.35, 0.45, and 0.2 respectively. The final
model prediction can be computed as follows:

[ 0.35 \times 200 + 0.45 \times 300 + 0.2 \times 400 = 285 ]

Key Difference of Bagging and Boosting

1. Approach: Bagging builds multiple models independently in parallel,
while boosting builds models sequentially, with each iteration
correcting the errors of the previous ones.
2. Weights: Bagging treats all data points equally, whereas boosting
assigns different weights to data points, prioritizing those that were
previously misclassified.
3. Model Complexity: Bagging often uses the same type of base model
across all iterations, whereas boosting typically employs a series of weak
models, gradually increasing their complexity.
4. Final Prediction: Bagging combines predictions by averaging or voting,
while boosting combines predictions with weighted averaging.

27 / 29
Bagging (Bootstrap Aggregating)

1. Step 1: Bagging involves creating multiple subsets of the original

dataset through a process called bootstrapping.
2. Step 2: Bootstrapping randomly samples the data with replacement,
creating diverse training sets for each model.
3. Step 3: Then, individual models (often decision trees) are trained on
these subsets in parallel.
4. Step 4: The final prediction is obtained by averaging (for regression) or
voting (for classification) the predictions of all individual models.

The primary benefits of bagging include reducing overfitting, improving

stability, and increasing accuracy. The diversity of training sets prevents
models from relying on the same patterns, leading to better generalization.

28 / 29
Boosting

Boosting, on the other hand, aims to correct the errors made by previous
models sequentially. Unlike bagging, boosting assigns different weights to
each data point, emphasizing the misclassified ones. It trains weak models in
an iterative manner, where each model focuses on the samples that the
previous models struggled with. The final prediction is a weighted sum of the
individual model predictions.

Boosting algorithms, such as AdaBoost and Gradient Boosting, tend to have a

strong focus on improving accuracy, especially when dealing with complex
datasets. They can adapt well to noise and outliers, capturing intricate
relationships within the data.

29 / 29