13.

ARTIFICIAL
INTELLIGENCE
MSc PHAM NGOC HA

Software Engineering Department

Content

▪ 13.1 Introduction
▪ 13.2 Knowledge Presentation
▪ 13.3 Expert Systems
▪ 13.4 Perception
▪ 13.5 Neuron Network
▪ 13.6 Deep Learning

2
Objectives

After studying this chapter, the student should be able to:

▪ Define and give a brief history of artificial intelligence.

▪ Describe how knowledge is represented in an intelligent agent.

▪ Show how expert systems can be used when a human expert is not available.

▪ Show how an artificial agent can be used to simulate mundane tasks performed by human beings.

▪ Show how expert systems and mundane systems can use different search techniques to solve
problems.

▪ Show how the learning process in humans can be simulated, to some extent, using neural networks
that create the electronic version of a neuron called a perceptron.

▪ Show how the deep learning working and the advantages of deep learning.

3
1 - INTRODUCTION

4
1. What is artificial intelligence?

▪ Although there is no universally agreed

definition of artificial intelligence, we
accept the following definition that
matches the topics covered in this
section.

5
2. History of artificial intelligence

▪ Artificial intelligence is an
independent field of study is
relatively new but it has some roots
in the past. We can say that it
started 2400 years ago when the
Greek philosopher Aristotle invented
the concept of logical reasoning.
▪ However, the main idea of a thinking
machine came from Alan Turing,
who proposed the Turing test. The
term ‘artificial intelligence’ was first
coined by John McCarthy in 1956.

Figure 13.1 History of artificial intelligence 6

3. The Turing test

▪ In 1950, Alan Turing proposed the Turing Test, which

provides a definition of intelligence in a machine. The
test simply compares the intelligent behaviour of a
human being with that of a computer:
▪ An interrogator asks a set of questions that are
forwarded to both a computer and a human
being.
▪ The interrogator receives two sets of
responses, but does not know which set comes
from the human and which set from the
computer.
Figure 13.2 Simple Turing Test diagram
▪ After careful examination of the two sets, if the
interrogator cannot definitely tell which set has
come from the computer and which from the
human, the computer has passed
7
4. Intelligent agents

▪ An intelligent agent is a system that perceives

its environment, learns from it, and interacts with
it intelligently. Intelligent agents can be divided
into two broad categories:

▪ Software agents is a set of programs that

are designed to do particular tasks. For
example, some intelligent systems can be
used to organize electronic mail (email).

▪ Physical agents (robot) is a programmable

system that can be used to perform a variety
of tasks. Simple robots can be used in
manufacturing to do routine jobs such as
assembling, welding, or painting.

▪ Some organizations use mobile robots that Figure 13.3 Simple reflex agent diagram
do delivery jobs such as distributing mail or
correspondence to different rooms.
8
5. Programming languages

▪ Data scientists choose different programming languages based on ease of use, simplicity in
programming syntax, number of machine learning libraries available, integration with other
programs like cloud infrastructure or visualization software, and computational speed and
efficiency. There are two most popular programing languages in AI
1. Python 2. Prolog

This language is widely used by programmers AI developers appreciate it for its high level of
because of its pure syntax and the logical, abstraction, built-in search engine, non-
strictly grammatical construction of the determinism, etc.
program. This is one of the few languages that
● relatively fast development speed; represents the paradigm of declarative
● rich and diverse set of libraries and tools; programming. The learning curve here is quite
● balance of low-level and high-level high.
programming; ● powerful and flexible programming
● allows you to test algorithms without structure;
having to implement them; ● data structuring on the basis of a tree;
● is still being actively developed. ● automatic rollback option..
9
6. Programming languages (cont)

3. LISP 4. JAVA

Since the 1960s, Lisp has been widely Java is, without a doubt, a significant AI
utilized in scientific research in the disciplines programming language. One reason for
of natural languages, theorem proofs, and the this is the language’s widespread use in
solution of artificial intelligence problems. mobile app development. It’s a fantastic
Lisp was intended as a practical match, given how many mobile apps make
mathematical notation for programming, but it use of AI.
quickly became a popular choice among AI Java also makes use of simplified
developers. More importantly, the man who debugging, and its easy-to-use syntax
created Lisp was very influential in the field of offers graphical data presentation and
AI, so much of his work had been incorporates both WORA and Object-
implemented for a long time. Oriented patterns.
10
7. Programming languages (cont)

5. C++ 6. R

C++ is another language that has been R may not be the ideal language for AI,
around for a long time yet is still a viable but it excels at crunching massive
option for AI. One of the reasons for this is numbers, making it superior to Python
the language’s broad flexibility, which makes at scale.
it ideal for resource-intensive applications.
C++ is a low-level language that helps the AI R is a suitable language for Artificial
model in production run more smoothly. And, Intelligence because of its built-in
while C++ isn’t everyone’s first pick for AI functional programming, vectorial
engineers, it’s worth noting that many deep computation, and Object-Oriented
and machine learning libraries are created in nature.
the language. 11
2- Knowledge Presentation

12
1. Introduction
▪ If an artificial agent is supposed to solve some problems related to the real world, it
needs to be able to represent knowledge somehow. Facts are represented as data
structures that can be manipulated by programs stored inside the computer.
▪ There are four common methods for representing knowledge: semantic networks,
frames, predicate logic, and rule-based systems.

Figure 13.4 Four common methods for representing knowledge 13

2. Semantic networks

▪ Semantic networks were developed in the early 1960s by Richard H. Richens. A semantic
network uses directed graphs to represent knowledge.
▪ Semantic networks use vertices to represent concepts, and edges (denoted by arrows) to
represent the relation between two concepts (Figure 13.3).

Figure 13.5 A simple semantic network 14

3. Semantic networks (cont)

Concepts Relations

To develop an exact definition of a In a semantic network, relations are

concept, experts have related the shown by edges. An edge can define a
definition of concepts to the theory of subclass relation—
sets. A concept, therefore, can be the edge is directed from the subclass
thought of as a set or a subset. For to its superclass. An edge can also
example, define an instance
Animal defines the set of all animals, relation—the edge is directed from the
horse defines the set of all horses and is instance to the set to which it belongs.
a subset of the set animal. An object is a An edge can
member (instance) of a set. Concepts also define an attribute of an object
are shown by vertices. (color, size, …).

15
4. Frames presentation
▪ Frames are closely related to semantic
networks. In semantic networks, a graph is
used to represent knowledge: in frames, data
structures (records) are used to represent the
same knowledge.

▪ One advantage of frames over semantic

networks is that programs can handle frames
more easily than semantic networks.

▪ Objects A node in a semantic network

becomes an object in a set of frames, so an
object can define a class, a subclass, or an
instance of a class
Figure 13.6 A set of frames representing
▪ Slots Edges in semantic networks are semantic network
translated into slots—fields in the data
structure. The name of the slot defines the
type of the relationship and the value of the
slot completes the relationship.

16
5. Predicate logic
▪ The most common knowledge representation is predicate logic. Predicate logic can be
used to represent complex facts. It is a well-defined language developed via a long
history of theoretical logic.
▪ Propositional logic is a language made up from a set of sentences that can be used
to carry out logical reasoning about the world.
▪ Examples,
▪ P(x) = x is a student.
▪ P(classroom) = Classroom is empty.
▪ P(classroom) = Classroom has 25 students
▪ And we can combine quantifiers and predicates,
▪ (Ɐx)P(x) = x passed the exam.
▪ Meaning: All the x passed the exam.
▪ (Ǝx)P(x) = x passed the exam.
▪ Meaning: Some of the x passed the exam. (At least one of the x has passed the
exam.)

17
6. Predicate logic (cont)

Operators
Propositional logic uses five operators, as
shown below:

Sentence
A sentence in this language is defined recursively as shown below:
1. An uppercase letter, such as A, B, S, or T, that represents a statement in a natural languages,
is a sentence.
2. Any of the two constant values (true and false) is a sentence.
3. If P is a sentence, then ¬P is a sentence.
4. If P and Q are sentences, then P ∨ Q, P ∧ Q, P → Q, and P ↔ Q are sentences

18
7. Rule-based systems

▪ A rule-based system represents knowledge using a set of rules that can be used to
deduce new facts from known facts. The rules express what is true if specific
conditions are met.
▪ A rule-based database is a set of if… then… statements in the form in which A is
called the antecedent and B is called the consequent. Note that in a rule-based
system, each rule is handled independently without any connection to other rules.

19
8. Rule-based systems (cont)
Components
▪ A rule-based system is made up of three components: an interpreter (or inference
engine), a knowledge base, and a fact database, as shown in Figure 18.4.

Figure 13.7 The components of a rule-based system

20
3 - Expert Systems

21
1. Introduction

▪ Expert systems use the knowledge representation languages discussed in the

previous section to perform tasks that normally need human expertise. They can be
used in situations in which that expertise is in short supply, expensive, or unavailable
when required.
▪ An expert system is divided into two subsystems: the inference engine and the
knowledge base. The knowledge base represents facts and rules. The inference
engine applies the rules to the known facts to deduce new facts. Inference engines
can also include

Figure 13.8 A diagram Expert Systems 22

2. Example Expert systems

▪ For example, in medicine, an expert system can narrow down a set of symptoms to a
likely subset of causes, a task normally carried out by a doctor.

Figure 13.9 A diagram of Medical Expert Systems 23

3. Extracting knowledge

▪ An expert system is built on predefined knowledge about its field of expertise.

▪ Extracting knowledge from an expert is normally a difficult task, for several reasons:
1. The knowledge possessed by the expert is normally heuristic: it is based on
probability rather than certainty.
2. The expert often finds it hard to express their knowledge in such a way that it can
be stored in a knowledge base as exact rules. For example, it is hard for an
electrical engineer to show how, step by step, a faulty electric motor can be
diagnosed. The knowledge is normally intuitive.
3. Knowledge acquisition can only be done via personal interview with the expert,
which can be a tiring and boring task if the interviewer is not an expert in this type of
interview.

24
4. Extracting knowledge (cont)

▪ An expert system in medicine, for example, is built on the knowledge of a doctor

specialized in the field for which the system is built: an expert system is supposed to
do the same job as the human expert.

Figure 13.10: System Architecture for Medical Knowledge Extraction 25

5. Extracting facts

▪ To be able to infer new facts or perform

actions, a fact database is needed in
addition to the knowledge base for a
knowledge representation language.
▪ The fact database in an expert system is
case-based, in which facts collected or
measured are entered into the system to be
used by the inference engine.

Figure 13.11: Flow Chart for Developing Case Based Reasoning Expert Systems
26
6. Architecture

▪ An expert system can have up to seven components: user, user interface, inference
engine, knowledge base, fact database, explanation system, and knowledge base editor.
▪ The inference engine is the heart of an expert system: it communicates with the
knowledge base, fact database, and the user interface.

Figure 13.12 Medical Expert Systems 27

7. Architecture (cont)

▪ User The user is the entity that uses the system to benefit from the expertise offered.
▪ User interface The user interface allows the user to interact with the system. The user
interface can accept natural language from the user and interpret it for the system.
Most user interfaces also offer a user-friendly menu system
▪ Inference engine The inference engine is the heart of the system that uses the
knowledge base and the fact database to infer the action to be taken.
▪ Knowledge base The knowledge base is a collection of knowledge based on
interviews with experts in the relevant field of expertise.
▪ Fact database The fact database in an expert system is case-based. For each case,
the user enters the available or measured data into the fact database to be used by
the inference engine for that particular case.
▪ Explanation system The explanation system, which may not be included in all
systems, is used to explain the rationale behind the decision made by the inference
engine.
▪ Knowledge base editor The knowledge base editor, which may not be included in all
systems, is used to update the knowledge base if new experience has been obtained 28
from experts in the field.
4- Perception

29
1. Introduction

• One of the goals in artificial intelligence

is to create a machine that behaves like
an expert—an expert system.
• Another goal is to create a machine that
behaves like an ordinary human. One of
the meanings of the word ‘perception’ is
understanding what is received through
the senses—sight, hearing, touch, smell,
taste.
• A human being sees a scene through the
eyes, and the brain interprets it to extract
the type of the objects in the scene. A
human being hears a set of voice signals
through the ears, and the brain interprets
it as a meaningful sentence, and so on.

Figure 13.13 Perception diagram 30

2. Image processing
▪ Image processing or computer vision is an area of AI that deals with the perception
of objects through the artificial eyes of an agent, such as a camera. An image
processor takes a two-dimensional image from the outside world and tries to create
a description of the three-dimensional objects present in the scene
▪ The input presented to an image processor is one or more images from the scene,
while the output is a description of the objects in the scene. The processor uses a
database containing the characteristics of objects for comparison (Figure 13.12).

Figure 13.14 Components of an image processor 31

3. Image processing - Edge detection
▪ Image processing or computer vision is an area of AI that deals with the perception
of objects through the artificial eyes of an agent, such as a camera. An image
processor takes a two-dimensional image from the outside world and tries to create
a description of the three-dimensional objects present in the scene
▪ The input presented to an image processor is one or more images from the scene,
while the output is a description of the objects in the scene. The processor uses a
database containing the characteristics of objects for comparison (Figure 13.14).

Figure 13.14 Edge detection process 32

4. Image processing - Segmentation
▪ Image processing or computer vision is an area of AI that deals with the perception
of objects through the artificial eyes of an agent, such as a camera. An image
processor takes a two-dimensional image from the outside world and tries to create
a description of the three-dimensional objects present in the scene
▪ The input presented to an image processor is one or more images from the scene,
while the output is a description of the objects in the scene. The processor uses a
database containing the characteristics of objects for comparison (Figure 13.15).

Figure 13.15 Components of an image processor 33

5. Image processing - Finding depth
▪ Image processing or computer vision is an area of AI that deals with the perception
of objects through the artificial eyes of an agent, such as a camera. An image
processor takes a two-dimensional image from the outside world and tries to create
a description of the three-dimensional objects present in the scene
▪ The input presented to an image processor is one or more images from the scene,
while the output is a description of the objects in the scene. The processor uses a
database containing the characteristics of objects for comparison (Figure 13.16).

Figure 13.16 Components of an image processor 34

6. Language understanding

▪ One of the inherent capabilities of a human being is to understand—that is, interpret—

the audio signals that they perceive. A machine that can understand natural language
can be very useful in daily life.
▪ For example, it can replace—most of the time—a telephone operator. It can also be
used on occasions when a system needs a predefined format of queries.

▪ We can divide the task of a

machine that understands natural
language into four consecutive
steps:
 speech recognition,
 syntactic analysis,
 semantic analysis, and
 Pragmatic analysis.

Figure 13.17 Steps of understands natural language

35
7. Language understanding - Speech recognition

▪ The first step in natural language processing is speech recognition. In this step, a
speech signal is analyzed and the sequence of words it contains are extracted. The
input to the speech recognition subsystem is a continuous (analog) signal: the output is
a sequence of words. The signal needs to be divided into different sounds, sometimes
called phonemes.
▪ The sounds then need to be combined into words. The detailed process, however, is
beyond the scope of this book: we leave the task to specialized books in speech
recognition.

36
Figure 13.18 overview of Speech recognition
8. Language understanding - Syntactic analysis

▪ The syntactic analysis step is used to define how words are to be grouped in a
sentence.
▪ This is a difficult task in a language like English, in which the function of a word in a
sentence is not determined by its position in the sentence.
▪ For example, in the following two sentences:

it is always John who is rewarded, but in the first sentence John is in the last position
and Mary is in the first position.
A machine that hears any of the above sentences needs to interpret them correctly
and come to the same conclusion no matter which sentence is heard.
37
5 - Neuron Network

38
1. Introduction

▪ Enabling an artificial intelligence agent to learn is definitely not an easy task.

However, several methods have been used in the past that create hope for the future.
Most of the methods use inductive learning or learning by example.
▪ A large set of problems and their solutions are given to the machine from which to
learn. In this section we discuss only one of these methods, which can be described
without complex mathematical concepts: neural networks.
▪ Neural networks try to simulate the learning process of the human brain using a
network of neurons.

39
Figure 13.19 A simplified view of an Artificial Neural Network
2. History of Neural Networks

▪ The first neural network was conceived of by Warren McCulloch and Walter Pitts in
1943. They wrote a seminal paper on how neurons may work and modelled their ideas
by creating a simple neural network using electrical circuits.

▪ This breakthrough model paved the way for neural network research in two areas

Figure 13.20 Biological processes in the brain Figure 13.21 The application of neural networks in AI

40
3. Biological neurons

▪ The human brain has billions of processing units, called neurons. Each
neuron, on average, is connected to several thousand other neurons. A
neuron is made of three parts: soma, axon, and dendrites, as shown in Figure
13.22.

Figure 13.22 A simplified diagram of a neuron

41
4. Perceptron

▪ A perceptron is an artificial neuron similar to a single biological neuron. It takes a set

of weighted inputs, sums the inputs, and compares the result with a threshold value. If
the result is above the threshold value, the perceptron fires, otherwise, it does not.
When a perceptron fires, the output is 1: when it does not fire, the output is zero.
▪ Figure 13.23 shows a perceptron with five inputs (x1 to x5), and five weights (w1 to
w5). In this perceptron, if T is the value of the threshold, the value of output is
determined

42
Figure 13.23 A perceptron
5. Example Perceptron

▪ Assume a case study with three inputs and

one output. There are already four
examples with known inputs and outputs, as
shown in the following table:

▪ This set of inputs is used to train a perceptron with all equal weights (w1 5 w2 5 w3).
▪ The threshold is set to 0.8. The original weight for all inputs is 50 per cent. The weights
remain the same if the output produced is correct—that is, matches the actual output.
▪ The weights are increased by 10 per cent if the output produced is less than the output
data: the weights are decreased by 10 per cent if the output produced is greater than the
output data.

43
6. Multiple layer

▪ Several layers of perceptions can be combined to create multilayer neural networks.

▪ The output from each layer becomes the input to the next layer. The first layer is
called the input layer, the middle layers are called the hidden layers, and the last layer
is called the output layer. The nodes in the input layer are not neurons, they are only
distributors. The hidden nodes are normally used to impose the weight on the output
from the previous layer. Figure 13.24 shows an example of a neural network with three
layers

44
Figure 13.24 Multiple layer ANN
7. Applications

▪ Handwriting Recognition – The

idea of Handwriting recognition
has become very important. This
is because handheld devices like
the Palm Pilot are becoming very
popular. Hence, we can use
Neural networks to recognize
handwritten characters.

▪ Traveling Salesman Problem –

Neural networks can also solve
the traveling salesman problem.
But this is to a certain degree of
approximation only.

45
8. Applications

▪ Image Compression– Vast amounts

of information is received and
processed at once by neural networks.
This makes them useful in image
compression. With the Internet
explosion and more sites using more
images on their sites, using neural
networks for image compression is
worth a look.
▪ Stock Exchange Prediction – The
day-to-day business of the stock
market is very complicated. Many
factors weigh in whether a given stock
will go up or down on any given day.
Thus, Neural networks can examine a
lot of information in a fast manner and
sort it all out. So we can use them to
predict stock prices. 46
6- DEEP LEARNING

47
1. Introduction

▪ Deep learning is a branch of

machine learning that uses data,
loads and loads of data, to teach
computers how to do things only
humans were capable of before.
▪ Deep learning is based on the
concept of artificial neural
networks, or computational
systems that mimic the way the
human brain functions.

Figure 13.25 Deep Learning vs Human Brain

48
2. TECHNOLOGY

Deep learning is a fast-growing field, and new architectures, variants appear

every few weeks. We'll see discuss the major three:

1. Convolution Neural Network

(CNN)
CNNs exploit spatially-local
correlation by enforcing a local
connectivity pattern between
neurons of adjacent layers.

Figure 13.26 A sample Convolution Neural Network

(CNN)

49
3. TECHNOLOGY (cont)

2. Recurrent Neural Network (RNN)

▪ RNNs are called recurrent because they perform the same task for every element of
a sequence, with the output being depended on the previous computations. Or RNNs
have a “memory” which captures information about what has been calculated so far.

Figure 13.27 A simplified diagram of a Recurrent Neural Network

50
4. TECHNOLOGY (cont)

3. Long-Short Term Memory LSTM

▪ can learn "Very Deep Learning" tasks that require memories of events that happened
thousands or even millions of discrete time steps ago.LSTM works even when there
are long delays, and it can handle signals that have a mix of low and high frequency
components.

Figure 13.28 A simplified diagram of a Long-Short Term Memory

51
5. ADVANTAGES

1. It does feature extraction, no

need for engineering features
2. Moving towards raw features
3. Better optimization
4. A new level of noise
robustness
5. Multi-task and transfer
learning
6. Better Architectures
Figure 13.29 Machine Learning Vs Deep Learning

52
6. CHALLENGES

1. Need a large dataset

2. Because you need a large dataset, training time is usually significant.
3. The scale of a net's weights is important for performance. When the features are of
the same type this is not a problem. However, when the features are heterogeneous,
it is.
4. Parameters are hard to interpret--although there is progress being made.
5. Hyperparamter tuning is non-trivial.

53
7. REAL TIME APPLICATIONS
1. Automatic Colorization of Black and White Images

2. Automatically Adding Sounds To Silent Movies

3. Automatic Machine Translation

4. Object Classification and Detection in Photographs

5. Automatic Handwriting Generation

54
Q&A

55