UNIVERSITY INSTITUTE OF COMPUTING

MASTER OF COMPUTER APPLICATIONS

MACHINE LEARNING
22CAT-701/22CAT-731

DISCOVER . LEARN . EMPOWER

1
 Syllabus
Unit-3
• Natural Language Processing: Feature Engineering on Text Data, Accessing Text
corpora, Regular Expression for detecting word pattern, useful Applications of
Regular Expression.
• Genetic Algorithms – Motivation, Genetic Algorithms, An illustrative Example,
Hypothesis, Space Search, Genetic Programming, Models of Evolution and
Learning, Parallelizing, Genetic Algorithms.
• Reinforcement Learning – Introduction, The Learning Task, Q Learning, Non-
Deterministic, Rewards and Actions, Temporal Difference Learning, Generalizing
from Examples.
• Self-Study: Conditional Frequency Distribution, Accessing Text from the web and
from disk, Text Processing with Unicode.
2
 Natural Language Processing (NLP)
What is NLP?
• NLP stands for Natural Language Processing, which is a part of Computer
Science, Human language, and Artificial Intelligence. It is the technology that is
used by machines to understand, analyse, manipulate, and interpret human's
languages. It helps developers to organize knowledge for performing tasks such
as translation, automatic summarization, Named Entity Recognition (NER),
speech recognition, relationship extraction, and topic segmentation.

3
 Natural Language Processing (NLP)
History of NLP
• (1940-1960) - Focused on Machine Translation (MT)
• The Natural Languages Processing started in the year 1940s.
• 1948 - In the Year 1948, the first recognisable NLP application was introduced in
Birkbeck College, London.
• 1950s - In the Year 1950s, there was a conflicting view between linguistics and
computer science. Now, Chomsky developed his first book syntactic structures and
claimed that language is generative in nature.
• In 1957, Chomsky also introduced the idea of Generative Grammar, which is rule
based descriptions of syntactic structures.
• (1960-1980) - Flavored with Artificial Intelligence (AI)

4
 Natural Language Processing (NLP)
History of NLP
• In the year 1960 to 1980, the key developments were:
• Augmented Transition Networks (ATN)
• Augmented Transition Networks is a finite state machine that is capable of
recognizing regular languages.
• Case Grammar
• Case Grammar was developed by Linguist Charles J. Fillmore in the year 1968.
Case Grammar uses languages such as English to express the relationship between
nouns and verbs by using the preposition.
• In Case Grammar, case roles can be defined to link certain kinds of verbs and
objects.
• For example: "Neha broke the mirror with the hammer". In this example case
grammar identify Neha as an agent, mirror as a theme, and hammer as an
instrument. 5
 Natural Language Processing (NLP)
Advantages of NLP
• NLP helps users to ask questions about any subject and get a direct response within
seconds.
• NLP offers exact answers to the question means it does not offer unnecessary and
unwanted information.
• NLP helps computers to communicate with humans in their languages.
• It is very time efficient.
• Most of the companies use NLP to improve the efficiency of documentation
processes, accuracy of documentation, and identify the information from large
databases.

6
 Natural Language Processing (NLP)
Disadvantages of NLP
• A list of disadvantages of NLP is given below:
• NLP may not show context.
• NLP is unpredictable
• NLP may require more keystrokes.
• NLP is unable to adapt to the new domain, and it has a limited function that's why
NLP is built for a single and specific task only.

7
 Natural Language Processing (NLP)
Components of NLP
• There are the following two components of NLP -
• 1. Natural Language Understanding (NLU)
• Natural Language Understanding (NLU) helps the machine to understand and analyse human
language by extracting the metadata from content such as concepts, entities, keywords, emotion,
relations, and semantic roles.
• NLU mainly used in Business applications to understand the customer's problem in both spoken and
written language.
• NLU involves the following tasks -
• It is used to map the given input into useful representation.
• It is used to analyze different aspects of the language.
• 2. Natural Language Generation (NLG)
• Natural Language Generation (NLG) acts as a translator that converts the computerized data into
natural language representation. It mainly involves Text planning, Sentence planning, and Text
Realization.
8
 Natural Language Processing (NLP)
Phases of NLP
• There are the following five phases of NLP:

9
 Natural Language Processing (NLP)
1. Lexical Analysis and Morphological
• The first phase of NLP is the Lexical Analysis. This phase scans the source code as
a stream of characters and converts it into meaningful lexemes. It divides the whole
text into paragraphs, sentences, and words.
2. Syntactic Analysis (Parsing)
• Syntactic Analysis is used to check grammar, word arrangements, and shows the
relationship among the words.
Example: Agra goes to the Poonam
• In the real world, Agra goes to the Poonam, does not make any sense, so this
sentence is rejected by the Syntactic analyzer.
3. Semantic Analysis
• Semantic analysis is concerned with the meaning representation. It mainly focuses
on the literal meaning of words, phrases, and sentences.
10
 Natural Language Processing (NLP)
4. Discourse Integration
• Discourse Integration depends upon the sentences that proceeds it and also invokes
the meaning of the sentences that follow it.
5. Pragmatic Analysis
• Pragmatic is the fifth and last phase of NLP. It helps you to discover the intended
effect by applying a set of rules that characterize cooperative dialogues.
For Example: "Open the door" is interpreted as a request instead of an order.

11
 Natural Language Processing (NLP)
Why NLP is difficult?
• NLP is difficult because Ambiguity and Uncertainty exist in the language.
Ambiguity
• There are the following three ambiguity -
Lexical Ambiguity
• Lexical Ambiguity exists in the presence of two or more possible meanings of the
sentence within a single word.
Example:
Manya is looking for a match.
• In the above example, the word match refers to that either Manya is looking for a
partner or Manya is looking for a match. (Cricket or other match)

12
 Natural Language Processing (NLP)
Syntactic Ambiguity
• Syntactic Ambiguity exists in the presence of two or more possible meanings
within the sentence.
Example:
• I saw the girl with the binocular.
• In the above example, did I have the binoculars? Or did the girl have the
binoculars?
Referential Ambiguity
Referential Ambiguity exists when you are referring to something using the pronoun.
• Example: Kiran went to Sunita. She said, "I am hungry."
• In the above sentence, you do not know that who is hungry, either Kiran or Sunita.

13
 Natural Language Processing (NLP)
NLP Libraries
• Scikit-learn: It provides a wide range of algorithms for building machine learning
models in Python.
• Natural language Toolkit (NLTK): NLTK is a complete toolkit for all NLP
techniques.
• Pattern: It is a web mining module for NLP and machine learning.
• TextBlob: It provides an easy interface to learn basic NLP tasks like sentiment
analysis, noun phrase extraction, or pos-tagging.
• Quepy: Quepy is used to transform natural language questions into queries in a
database query language.
• SpaCy: SpaCy is an open-source NLP library which is used for Data Extraction,
Data Analysis, Sentiment Analysis, and Text Summarization.
• Gensim: Gensim works with large datasets and processes data streams.
14
 Feature Engineering
What is a feature?
• Generally, all machine learning algorithms take input data to generate the output.
The input data remains in a tabular form consisting of rows (instances or
observations) and columns (variable or attributes), and these attributes are often
known as features. For example, an image is an instance in computer vision, but a
line in the image could be the feature. Similarly, in NLP, a document can be an
observation, and the word count could be the feature. So, we can say a feature is an
attribute that impacts a problem or is useful for the problem.
What is Feature Engineering?
• Feature engineering is the pre-processing step of machine learning, which extracts
features from raw data. It helps to represent an underlying problem to predictive
models in a better way, which as a result, improve the accuracy of the model for
unseen data. The predictive model contains predictor variables and an outcome
variable, and while the feature engineering process selects the most useful predictor
variables for the model.
15
Feature Engineering

16
 Feature Engineering
These processes are described as below:
1.Feature Creation: Feature creation is finding the most useful variables to be used
in a predictive model. The process is subjective, and it requires human creativity
and intervention. The new features are created by mixing existing features using
addition, subtraction, and ration, and these new features have great flexibility.
2.Transformations: The transformation step of feature engineering involves
adjusting the predictor variable to improve the accuracy and performance of the
model. For example, it ensures that the model is flexible to take input of the variety
of data; it ensures that all the variables are on the same scale, making the model
easier to understand. It improves the model's accuracy and ensures that all the
features are within the acceptable range to avoid any computational error.

17
 Feature Engineering
1.Feature Extraction: Feature extraction is an automated feature engineering
process that generates new variables by extracting them from the raw data. The
main aim of this step is to reduce the volume of data so that it can be easily used
and managed for data modelling. Feature extraction methods include cluster
analysis, text analytics, edge detection algorithms, and principal components
analysis (PCA).
2.Feature Selection: While developing the machine learning model, only a few
variables in the dataset are useful for building the model, and the rest features are
either redundant or irrelevant. If we input the dataset with all these redundant and
irrelevant features, it may negatively impact and reduce the overall performance
and accuracy of the model. Hence it is very important to identify and select the
most appropriate features from the data and remove the irrelevant or less important
features, which is done with the help of feature selection in machine
learning. "Feature selection is a way of selecting the subset of the most relevant
features from the original features set by removing the redundant, irrelevant, or
noisy features." 18
 Feature Engineering
Below are some benefits of using feature selection in machine learning:
• It helps in avoiding the curse of dimensionality.
• It helps in the simplification of the model so that the researchers can easily
interpret it.
• It reduces the training time.
• It reduces overfitting hence enhancing the generalization.

19
 Feature Engineering
Need for Feature Engineering in Machine Learning
• In machine learning, the performance of the model depends on data pre-processing
and data handling. But if we create a model without pre-processing or data
handling, then it may not give good accuracy. Whereas, if we apply feature
engineering on the same model, then the accuracy of the model is enhanced. Hence,
feature engineering in machine learning improves the model's performance. Below
are some points that explain the need for feature engineering:
• Better features mean flexibility.
In machine learning, we always try to choose the optimal model to get good results.
However, sometimes after choosing the wrong model, still, we can get better
predictions, and this is because of better features. The flexibility in features will
enable you to select the less complex models. Because less complex models are
faster to run, easier to understand and maintain, which is always desirable.

20
 Feature Engineering
• Better features mean simpler models.
If we input the well-engineered features to our model, then even after selecting the
wrong parameters (Not much optimal), we can have good outcomes. After feature
engineering, it is not necessary to do hard for picking the right model with the most
optimized parameters. If we have good features, we can better represent the
complete data and use it to best characterize the given problem.
• Better features mean better results.
As already discussed, in machine learning, as data we will provide will get the
same output. So, to obtain better results, we must need to use better features.

21
 Feature Engineering
Steps in Feature Engineering
• The steps of feature engineering may vary as per different data scientists and ML
engineers. However, there are some common steps that are involved in most
machine learning algorithms, and these steps are as follows:
• Data Preparation: The first step is data preparation. In this step, raw data acquired
from different resources are prepared to make it in a suitable format so that it can
be used in the ML model. The data preparation may contain cleaning of data,
delivery, data augmentation, fusion, ingestion, or loading.
• Exploratory Analysis: Exploratory analysis or Exploratory data analysis (EDA) is
an important step of features engineering, which is mainly used by data scientists.
This step involves analysis, investing data set, and summarization of the main
characteristics of data. Different data visualization techniques are used to better
understand the manipulation of data sources, to find the most appropriate statistical
technique for data analysis, and to select the best features for the data.
22
 Feature Engineering
Feature Engineering Techniques
Some of the popular feature engineering techniques include:
1. Imputation
• Feature engineering deals with inappropriate data, missing values, human
interruption, general errors, insufficient data sources, etc. Missing values within the
dataset highly affect the performance of the algorithm, and to deal with them
"Imputation" technique is used. Imputation is responsible for handling
irregularities within the dataset.
• For example, removing the missing values from the complete row or complete
column by a huge percentage of missing values. But at the same time, to maintain
the data size, it is required to impute the missing data, which can be done as:
• For numerical data imputation, a default value can be imputed in a column, and
missing values can be filled with means or medians of the columns.
• For categorical data imputation, missing values can be interchanged with the 23
maximum occurred value in a column.
 Feature Engineering
2. Handling Outliers
• Outliers are the deviated values or data points that are observed too away from other
data points in such a way that they badly affect the performance of the model.
Outliers can be handled with this feature engineering technique. This technique first
identifies the outliers and then remove them out.
• Standard deviation can be used to identify the outliers. For example, each value
within a space has a definite to an average distance, but if a value is greater distant
than a certain value, it can be considered as an outlier. Z-score can also be used to
detect outliers.

24
 Feature Engineering
3. Log transform
• Logarithm transformation or log transform is one of the commonly used
mathematical techniques in machine learning. Log transform helps in handling the
skewed data, and it makes the distribution more approximate to normal after
transformation. It also reduces the effects of outliers on the data, as because of the
normalization of magnitude differences, a model becomes much robust.
Note: Log transformation is only applicable for the positive values; else, it will give an
error. To avoid this, we can add 1 to the data before transformation, which ensures
transformation to be positive.
4. Binning
• In machine learning, overfitting is one of the main issues that degrade the
performance of the model and which occurs due to a greater number of parameters
and noisy data. However, one of the popular techniques of feature engineering,
"binning", can be used to normalize the noisy data. This process involves
segmenting different features into bins. 25
 Feature Engineering
5. Feature Split
• As the name suggests, feature split is the process of splitting features intimately into
two or more parts and performing to make new features. This technique helps the
algorithms to better understand and learn the patterns in the dataset.
• The feature splitting process enables the new features to be clustered and binned,
which results in extracting useful information and improving the performance of the
data models.
6. One hot encoding
• One hot encoding is the popular encoding technique in machine learning. It is a
technique that converts the categorical data in a form so that they can be easily
understood by machine learning algorithms and hence can make a good prediction. It
enables group the of categorical data without losing any information.

26
 Regular Expression & Its Applications
A regular expression (regex) is a sequence of characters that define a search pattern.
Here’s how to write regular expressions:

1.Start by understanding the special characters used in regex, such as “.”, “*”, “+”,
“?”, and more.
2.Choose a programming language or tool that supports regex, such as Python, Perl, or
grep.
3.Write your pattern using the special characters and literal characters.
4.Use the appropriate function or method to search for the pattern in a string.

27
 Regular Expression & Its Applications
Examples:
1.To match a sequence of literal characters, simply write those characters in the
pattern.
2.To match a single character from a set of possibilities, use square brackets, e.g.
[0123456789] matches any digit.
3.To match zero or more occurrences of the preceding expression, use the star (*)
symbol.
4.To match one or more occurrences of the preceding expression, use the plus (+)
symbol.
5.It is important to note that regex can be complex and difficult to read, so it is
recommended to use tools like regex testers to debug and optimize your patterns.
• A regular expression (sometimes called a rational expression) is a sequence of
characters that define a search pattern, mainly for use in pattern matching with
strings, or string matching, i.e. “find and replace” like operations. 28
 Genetic Algorithm in Machine Learning
• Genetic algorithms provide an approach to learning that is based loosely on
simulated evolution.
• Hypotheses are often described by bit strings whose interpretation depends
on the application.
• The search for an appropriate hypothesis begins with a population of initial
hypotheses.
• Members of the current population give rise to the next generation population by
means of operations such as random mutation and crossover, which are
patterned after processes in biological evolution.
• The hypotheses in the current population are evaluated relative to a given
measure of fitness, with the most fit hypotheses selected probabilistically as
seeds for producing the next generation.

29
 Genetic Algorithm in Machine Learning
• Genetic algorithms (GAS) provide a learning method motivated by an analogy to
biological evolution.
• GAs generate successor hypotheses by repeatedly mutating and
recombining parts of the best currently known hypotheses.
• At each step, the current population is updated by replacing some fraction of
the population by offspring of the most fit current hypotheses.
• The process forms a generate-and-test beam-search of hypotheses, in which
variants of the best current hypotheses are most likely to be considered next.

30
 Genetic Algorithm in Machine Learning
• GAs search a space of candidate hypotheses to identify the best
hypothesis.
• In GAs the "best hypothesis" is defined as the one that optimizes a predefined
numerical measure for the problem at hand, called the hypothesis fitness.
• For example, if the learning task is the problem of approximating an unknown
function given training examples of its input and output, then fitness could be
defined as the accuracy of the hypothesis over this training data.

31
Genetic Algorithm in Machine Learning

32
Genetic Algorithm in Machine Learning

33
Genetic Algorithm in Machine Learning

34
 Genetic Algorithm in Machine Learning
• A certain number of hypotheses from the current population are selected for
inclusion in the next generation.
• These are selected probabilistically, where the probability of selecting hypothesis
hi is given by

• The probability that a hypothesis will be selected is proportional to its own

fitness and is inversely proportional to the fitness of the other competing
hypotheses in the current population.

35
 Genetic Algorithm in Machine Learning
• Hypotheses in GAs are often represented by bit strings, so that they can be
easily manipulated by genetic operators such as mutation and crossover.
• How if-then rules can be encoded by bit strings
– Consider the attribute Outlook, which can take on any of the three
values Sunny, Overcast, or Rain.
– the string 010 represents the constraint that Outlook must take on the
second of these values, , or Outlook = Overcast.
– The string 011 represents the more general constraint that allows two
possible values, or (Outlook = Overcast or Rain).
– The string 111 represents the most general possible constraint, indicating
that we don't care which of its possible values the attribute takes on.

36
 Genetic Algorithm in Machine Learning
• The crossover operator produces two new offspring from two parent
strings, by copying selected bits from each parent.
• The bit at position i in each offspring is copied from the bit at position i in
one of the two parents.
• The choice of which parent contributes the bit for position i is
determined by an additional string called the crossover mask.
• There are different crossover operators.
– Single-point crossover
– Two-point crossover
– Uniform crossover

37
 Genetic Algorithm in Machine Learning
•In single-point crossover, the crossover mask is always constructed so that it
begins with a string containing n contiguous 1s, followed by the necessary
number of 0s to complete the string.
•This results in offspring in which the first n bits are contributed by one
parent and the remaining bits by the second parent.
• Each time the single-point crossover operator is applied, the
crossover point n is chosen at random, and the crossover mask is then
created and applied.

38
 Genetic Algorithm in Machine Learning
• The fitness function defines the criterion for ranking potential hypotheses
and for probabilistically selecting them for inclusion in the next generation
population.
• If the task is to learn classification rules, then the fitness function typically
has a component that scores the classification accuracy of the rule over a set
of provided training examples.
• Often other criteria may be included as well, such as the complexity or
generality of the rule.
• More generally, when the bit-string hypothesis is interpreted as a complex
procedure (e.g., when the bit string represents a collection of if-then rules),
the fitness function may measure the overall performance of the resulting
procedure rather than performance of individual rules.
39
 Genetic Algorithm in Machine Learning
• The probability that a hypothesis will be selected is given by the ratio of its
fitness to the fitness of other members of the current population.
• This method is called fitness proportionate selection, or roulette wheel selection

Tournament selection
• two hypotheses are first chosen at random from the current population.
• With some predefined probability p the more fit of these two is then selected,
and with probability (1 - p) the less fit hypothesis is selected.
Rank selection,
• the hypotheses in the current population are first sorted by fitness.
• The probability that a hypothesis will be selected is then proportional to its rank
in this sorted list, rather than its fitness.
40
 Reinforcement Learning in Machine Learning
What is Reinforcement Learning?
• Reinforcement Learning is a feedback-based Machine learning technique in which
an agent learns to behave in an environment by performing the actions and seeing
the results of actions. For each good action, the agent gets positive feedback, and for
each bad action, the agent gets negative feedback or penalty.
• In Reinforcement Learning, the agent learns automatically using feedbacks without
any labeled data, unlike supervised learning.
• Since there is no labeled data, so the agent is bound to learn by its experience only.
• RL solves a specific type of problem where decision making is sequential, and the
goal is long-term, such as game-playing, robotics, etc.

41
 Reinforcement Learning in Machine Learning
• The agent interacts with the environment and explores it by itself. The primary goal
of an agent in reinforcement learning is to improve the performance by getting the
maximum positive rewards.
• The agent learns with the process of hit and trial, and based on the experience, it
learns to perform the task in a better way. Hence, we can say that "Reinforcement
learning is a type of machine learning method where an intelligent agent (computer
program) interacts with the environment and learns to act within that." How a
Robotic dog learns the movement of his arms is an example of Reinforcement
learning.
• It is a core part of Artificial intelligence, and all AI agent works on the concept of
reinforcement learning. Here we do not need to pre-program the agent, as it learns
from its own experience without any human intervention.

42
Reinforcement Learning in Machine Learning

43
 Reinforcement Learning in Machine Learning
Terms used in Reinforcement Learning
• Agent(): An entity that can perceive/explore the environment and act upon it.
• Environment()bolo: A situation in which an agent is present or surrounded by. In RL,
we assume the stochastic environment, which means it is random in nature.
• Action(): Actions are the moves taken by an agent within the environment.
• State(): State is a situation returned by the environment after each action taken by
the agent.
• Reward(): A feedback returned to the agent from the environment to evaluate the
action of the agent.
• Policy(): Policy is a strategy applied by the agent for the next action based on the
current state.
• Value(): It is expected long-term retuned with the discount factor and opposite to the
short-term reward.
44
 Reinforcement Learning in Machine Learning
Key Features of Reinforcement Learning
• In RL, the agent is not instructed about the environment and what actions need to be
taken.
• It is based on the hit and trial process.
• The agent takes the next action and changes states according to the feedback of the
previous action.
• The agent may get a delayed reward.
• The environment is stochastic, and the agent needs to explore it to reach to get the
maximum positive rewards.

45
 Reinforcement Learning in Machine Learning
Approaches to implement Reinforcement Learning
There are mainly three ways to implement reinforcement-learning in ML, which are:
1.Value-based:
The value-based approach is about to find the optimal value function, which is the
maximum value at a state under any policy. Therefore, the agent expects the long-
term return at any state(s) under policy π.
2.Policy-based:
Policy-based approach is to find the optimal policy for the maximum future rewards
without using the value function. In this approach, the agent tries to apply such a
policy that the action performed in each step helps to maximize the future reward.
The policy-based approach has mainly two types of policy:
1. Deterministic: The same action is produced by the policy (π) at any state.
2. Stochastic: In this policy, probability determines the produced action.

46
 Reinforcement Learning in Machine Learning
What is Q-learning?
• Q-learning is a machine learning approach that enables a model to iteratively learn
and improve over time by taking the correct action. Q-learning is a type of
reinforcement learning.
• With reinforcement learning, a machine learning model is trained to mimic the way
animals or children learn. Good actions are rewarded or reinforced, while bad
actions are discouraged and penalized.
• With the state-action-reward-state-action form of reinforcement learning, the
training regimen follows a model to take the right actions. Q-learning provides a
model-free approach to reinforcement learning. There is no model of the
environment to guide the reinforcement learning process. The agent -- which is the
AI component that acts in the environment -- iteratively learns and makes
predictions about the environment on its own.

47
 Reinforcement Learning in Machine Learning
How does Q-learning work?
• Q-learning models operate in an iterative process that involves multiple components
working together to help train a model. The iterative process involves the agent
learning by exploring the environment and updating the model as the exploration
continues. The multiple components of Q-learning include the following:
• Agents. The agent is the entity that acts and operates within an environment.
• States. The state is a variable that identifies the current position in an environment
of an agent.
• Actions. The action is the agent's operation when it is in a specific state.
• Rewards. A foundational concept within reinforcement learning is the concept of
providing either a positive or a negative response for the agent's actions.
• Episodes. An episode is when an agent can no longer take a new action and ends up
terminating.
• Q-values. The Q-value is the metric used to measure an action at a particular state. 48
 Reinforcement Learning in Machine Learning
How does Q-learning work?
• Q-learning models operate in an iterative process that involves multiple components
working together to help train a model. The iterative process involves the agent
learning by exploring the environment and updating the model as the exploration
continues. The multiple components of Q-learning include the following:
• Agents. The agent is the entity that acts and operates within an environment.
• States. The state is a variable that identifies the current position in an environment
of an agent.
• Actions. The action is the agent's operation when it is in a specific state.
• Rewards. A foundational concept within reinforcement learning is the concept of
providing either a positive or a negative response for the agent's actions.
• Episodes. An episode is when an agent can no longer take a new action and ends up
terminating.
• Q-values. The Q-value is the metric used to measure an action at a particular state. 49
 Reinforcement Learning in Machine Learning
What is a Q-table?
• The Q-table includes columns and rows with lists of rewards for the best actions of
each state in a specific environment. A Q-table helps an agent understand what
actions are likely to lead to positive outcomes in different situations.
• The table rows represent different situations the agent might encounter, and the
columns represent the actions it can take. As the agent interacts with the
environment and receives feedback in the form of rewards or penalties, the values in
the Q-table are updated to reflect what the model has learned.
• The purpose of reinforcement learning is to gradually improve performance through
the Q-table to help choose actions. With more feedback, the Q-table becomes more
accurate so the agent can make better decisions and achieve optimal results.
• The Q-table is directly related to the concept of the Q-function. The Q-function is a
mathematical equation that looks at the current state of the environment and the
action under consideration as inputs. The Q-function then generates outputs along
with expected future rewards for that action in the specific state 50
 Reinforcement Learning in Machine Learning
What are the advantages of Q-learning?
• The Q-learning approach to reinforcement learning can potentially be advantageous
for several reasons, including the following:
• Model-free. The model-free approach is the foundation of Q-learning and one of the
biggest potential advantages for some uses. Rather than requiring prior knowledge
about an environment, the Q-learning agent can learn about the environment as it
trains. The model-free approach is particularly beneficial for scenarios where the
underlying dynamics of an environment are difficult to model or completely
unknown.
• Off-policy optimization. The model can optimize to get the best possible result
without being strictly tethered to a policy that might not enable the same degree of
optimization.
• Flexibility. The model-free, off-policy approach enables Q-learning flexibility to
work across a variety of problems and environments.
51
 Reinforcement Learning in Machine Learning
What are the disadvantages of Q-learning?
• The Q-learning approach to reinforcement model machine learning also has some
disadvantages, such as the following:
• Exploration vs. exploitation tradeoff. It can be hard for a Q-learning model to find
the right balance between trying new actions and sticking with what's already
known. It's a dilemma that is commonly referred to as the exploration vs.
exploitation tradeoff for reinforcement learning.
• Curse of dimensionality. Q-learning can potentially face a machine learning risk
known as the curse of dimensionality. The curse of dimensionality is a problem with
high-dimensional data where the amount of data required to represent the
distribution increases exponentially. This can lead to computational challenges and
decreased accuracy.
• Overestimation. A Q-learning model can sometimes be too optimistic and
overestimate how good a particular action or strategy is.
52
 Reinforcement Learning in Machine Learning
What are the disadvantages of Q-learning?
• The Q-learning approach to reinforcement model machine learning also has some
disadvantages, such as the following:
• Exploration vs. exploitation tradeoff. It can be hard for a Q-learning model to find
the right balance between trying new actions and sticking with what's already
known. It's a dilemma that is commonly referred to as the exploration vs.
exploitation tradeoff for reinforcement learning.
• Curse of dimensionality. Q-learning can potentially face a machine learning risk
known as the curse of dimensionality. The curse of dimensionality is a problem with
high-dimensional data where the amount of data required to represent the
distribution increases exponentially. This can lead to computational challenges and
decreased accuracy.
• Overestimation. A Q-learning model can sometimes be too optimistic and
overestimate how good a particular action or strategy is.
53
THANK YOU

54