Artificial Intelligence

EXPERT SYSTEMS
Probability Based Expert Systems
 Probability Based Expert Systems
• Expert takes decisions based on institutions
• Expert system is a intuitive decision making
process
• Decisions are based on statements using
probability
• Such systems are called probability based
expert systems
 Sources of uncertainty
• Data
– missing data, unreliable, ambiguous, imprecise representation,
inconsistent, subjective, derived from defaults, …

• Expert knowledge
– inconsistency between different experts
– plausibility
• “best guess” of experts
– quality
• causal knowledge
– deep understanding
• statistical associations
– observations
– scope
• only current domain?
 Sources of uncertainty
• Knowledge representation
– restricted model of the real system
– limited expressiveness of the representation mechanism
• Inference process
– deductive
• the derived result is formally correct, but wrong in the real
system
– inductive
• new conclusions are not well-founded
– unsound reasoning methods
 Sources of uncertainty
• individual rules
– errors
• domain errors
• representation errors
• inappropriate application of the rules
– likelihood of evidence
• for each premise
• for the conclusion
• combination of evidence from multiple premises
 Sources of uncertainty
• conflict resolution
• if multiple rules are applicable, which one is selected
– explicit priorities, provided by domain experts
– implicit priorities derived from rule properties
» specificity of patterns, ordering of patterns creation time of rules, most recent
usage, …

• compatibility
– contradictions between rules
– subsumption
• one rule is a more general version of another one
– redundancy
– missing rules
– data fusion
• integration of data from multiple sources
 Types of uncertainty
• Type 1
• Uncertainty about the truth of the inputs
• User is best qualified to make judgments
• System may provide advice/ assistance
• Type 2
• Uncertainty about the validity of the rules
• System expert should make risk estimates
/ assign probabilistic values
 Uncertainty - Example
Person A walks on Campus towards the bus stop. A few hundred
yards away A sees someone and is quite sure that it's his next-
door neighbor B who usually goes by car to the University. A
screams B's name.

default - A wants to take a bus

belief, (un)certainty - it's the neighbor B
probability, default, uncertainty - the neighbor goes home by car
default - A wants to get a lift
default - A wants to go home
 Notations
• Probability if event A occurring
– P(A)
• Probability of events A1, A2 & A3 occurring
– P(A1 and A2 and A3)
– P(A1A2A3)
• Probability of A1 or A2 or A3 occurring
– P(A1 or A2 or A3)
– P(A1+A2+A3)
 Probability Definitions
• Probability is
• a measure of the uncertainty about the likelihood of
an event occurring
• Expected frequency with which it will occur in future
• Limit of the ratio of the number of observed
occurrences of an event to the total number of trials,
as the number of trials increase without limits
• Measure of degree of belief that an event will occur
 Probability
• Probability can be expressed mathematically
as a numerical index with a range between
zero (an absolute impossibility) to unity (an
absolute certainty).
• Most events have a probability index strictly
between 0 and 1, which means that each
event has at least two possible outcomes:
favorable outcome or success,
and unfavorable outcome or failure.
 Probability

the number of successes

P(success) 
the number of possible outcomes

the number of failures

P( failure) 
the number of possible outcomes
 Probability
• If s is the number of times success can occur,
and f is the number of times failure can
occur, then
s
P success  p 
s f
f
P failure  q 
s f
pq1
 Probability
• joint probability
• The number of times A and B can occur, or the
probability that both A and B will occur, is
called the joint probability of A and B.
• Represented as p(AÇB).
 Probability
• Conditional probability
– Let A be an event B be another event.
– A and B are not mutually exclusive
– The probability that event A will occur if event B occurs is
called the conditional probability.
– Conditional probability is denoted as p(A|B)
– vertical bar represents GIVEN expression
– interpreted as “Conditional probability of event A occurring
given that event B has occurred”.
the number of times A and B can occur
p AB 
–. the number of times B can occur
 Probability
• Conditional probability
– Probability of A occurring given B has occurred

p A B
p AB 
pB
– Probability of B occurring given A has occurred

p B A
p BA 
pA
 Probability

p B A  p B A  p A
p A B  p B A  p A

p A B
p AB  Bayesian rule
pB
 Probability - Bayesian rule
• We have

p BAp A
p AB 
pB
• where:
• p(A|B) is the conditional probability that event A occurs given that event
B has occurred;
• p(B|A) is the conditional probability of event B occurring given that
event A has occurred;
• p(A) is the probability of event A occurring;
• p(B) is the probability of event B occurring.
 Probability - Bayesian rule
• The joint probability of A if A is dependent on
multiple events
n n
p A Bi  p A Bi  p Bi
i 1 i 1

A B1
B4

B3 B2
 Probability - Bayesian rule
• If the occurrence of event A depends on only
two mutually exclusive events, B and NOT B ,
we obtain:

• Similarly

• Substituting in Bayesian rule, we get

 Notations
• C is event / conclusion
• P(C) is probability rule C is fired
• Ei is lower nodes
• E is event that all lower rules are true
• P(E) is probability all lower rules are true
• P(C/E) is probability that a rule will fire provided
all lower rules are true
• P(C/⌐E) is probability that a rule will fire provided
all lower rules are false
 Implementing Bayesian Rules
• P(C)= P(C/E) + P(C/⌐E)
• Applying Bayesian Rule
• P(C)= P(C/E) * P(E)+ P(C/⌐E) * P(⌐E)
• P(C/E) and P(C/⌐E) is estimated by system
expert for each node
• Calculating probability of each node find
probability of a rule is more a exercise than a
judgment
 Probability models
Task : Assigning probability value for each node
Probability models used for assigning probability
values for each node
• Monte Carlo simulation
• Dependence models
– General dependence model
– Relevant symptoms dependence model
– Independence model
– General dependence model with assumptions
 Monte Carlo Simulation
Algorithm/process
• Define a domain of possible inputs.
• Generate inputs randomly for each input
from a probability distribution over the
domain.
• Perform a deterministic computation on
the inputs to generate probability value.
• Repeat numerous times
• Aggregate the results.
 Monte Carlo Simulation
• Assign P(Ei) = random value between(0,1)
• (assign random values for all inputs)
• For an AND node P(E)= P(E1,E2,E3,E4,E5,…,En)
• For an OR node P(E)= p(E1+E2+E3+E4+….+En)
• Calculate P(C/E)
• Calculate P(C/⌐E)
• Iteration ends
• Repeat 1,000 times
• Aggregate results of P(C/E) & P(C/⌐E)
 General Dependence Model
• Requires all the probabilities
• P(Ei/S1∩S2 ∩ …. ∩Sk)
• Where,
• P(Ei) is probability of a node (disease being
diagnosed)
• S1, S2,…, Sk, are probabilities of symptoms
∴ P(Ei/S1∩S2 ∩ …. ∩Sk) =
(│Ei/S1∩S2 ∩ …. ∩Sk│/N)/ │S1∩S2 ∩ …. ∩Sk│/N
 General Dependence Model
∴ P(Ei/S1∩S2 ∩ …. ∩Sk) =
(│Ei/S1∩S2 ∩ …. ∩Sk│/N)/ │S1∩S2 ∩ …. ∩Sk│/N
P(Ei/S1∩S2 ∩ …. ∩Sk) =
P(Ei/S1∩S2 ∩ …. ∩Sk)/ P(S1∩S2 ∩ …. ∩Sk)
P(Ei/S1∩S2 ∩ …. ∩Sk) =
(P(Ei)P(S1∩S2 ∩ …. ∩Sk)/Ei)/P(S1∩S2 ∩ …. ∩Sk)
 General Dependence Model

Problems
• Requires all the probabilities
– (Probabilities of all the symptoms)
• Extremely large number of parameters
• For n diseases & m symptoms
– (n+1)2m parameters
 Relevant Symptoms Dependence
Model
• Use only relevant symptoms for calculating
the probability model
• Q = {S1, S2,…, Sj} relevant symptoms
• ⌐Q = {S1, S2,…, Sj} non-relevant/independent
• (Ei∩S1∩S2 ∩ …. ∩Sk)is replaced by
(Ei∩(∩Sj))j∈Q* ∏Pjj∈ ⌐Q
• Number of parameters For n diseases & m
symptoms is reduced to (2r+1n+m)
 Independence Model
• Assumes symptoms are independent
• ∴
• = (P(Ei)/P(S1∩S2 ∩ …. ∩Sk))
*(P(s1/Ei)*P(s2/Ei)*….*P(sk/Ei))

• Number of parameters For n diseases & m

symptoms is reduced to (nr+n+m+1)
General Dependence Model With
Assumptions
For each disease E
• Select non relevant symptoms (S1,s2,..,Sj)
• Mark P(Sj/Ei) for these symptoms as ‘NULL’
• For n diseases & m symptoms parameters
(n+1)2m will be much less than (nr+n+m+1)
• Problems
– Model is not realistic
– Not easy to update
 Uncertainty Measures
• Most common uncertainty measures are
• Certainty factors
• Evidence theory
• Fuzzy logic
 Certainty Factors
• MB Belief Measure
– Measure of relative decrement
in disbelief in hypothesis h due
to some information e
(≅relative increment of belief in h
due to e)
• To remove negative values the
equation can be written as
• MB takes a value between
(0.1)
•
 Certainty Factors
• MD Disbelief Measure
– Measure of relative decrement
in belief in hypothesis h due to
some information e
(≅relative increment of disbelief in
h due to e)
• To remove negative values the
equation can be written as
• MB takes a value between
(0.1)
•
 Certainty Factors
CF Certainty factor
CF(h,e) = MB(h,e) - MD(h,e)
• Properties
• CF assumes value between -1, 1
• If h is true ⇒ MB(h,e)=1,MD(h,e)=0,CF(h,e)=1
– 1⇒ definitely true
• If h is false ⇒ MB(h,e)=0,MD(h,e)=1,CF(h,e)=-1
– -1 ⇒ definitely false
• If h is neither true or false CF(h,e)=0
– 0⇒ no opinion
• CF(h,e)+CF(⌐h,e)=0
 Dempster-Shafer Theory of Evidence
• Frame of discernment U – a space/ set of
possible events/answers/options
• Probability m on P(U) - unit of evidence resulting
from a given information unit,
• Belief function

• Plausibility
 Dempster-Shafer Theory of Evidence
Properties
• Pl(A)≥Cr(A)
• Pl(φ)=Cr(φ)=0
• Pl(U)=Cr(U)=0
• Pl(A)+ Pl(⌐A)≥ 1
• Cr(A)+ Cr(⌐A)≤ 1
• Cr(⌐A)+ Pl(A) = 1
• ⇒ Pl(A) = 1 - Cr(⌐A)
 Fuzzy Logic
Probability
• Given P(A) & P(B)
• It is still impossible to determine
• P(A∩B)
• P(A∪B)
 Fuzzy Logic
Fuzzy Logic
• Probability P() is replaced by possibility ∏
• Given ∏(A) & ∏(B)
• 0≤ ∏(A) ≤1
• ∏(⌐A)=1 - ∏(A)
• ∏(A∩B) = min(∏(A), ∏(B))
• ∏(A∪B) = max(∏(A), ∏(B))
• ∏(A→B) = min(1,1 - ∏(A)+ ∏(B))
 Propagation of Uncertainty

When evidence form several single have to be

mixed to obtain mixed evidence
 Propagation of Uncertainty
• Certainty factor

• MB(h1∩h2,e) = min(MB(h1,e),MB(h2,e))
• MD(h1∩h2,e) = max(MD(h1,e),MD(h2,e))
• MB(h1∪h2,e) = max(MB(h1,e),MB(h2,e))
• MD(h1∪h2,e) = min(MD(h1,e),MD(h2,e))
 Propagation of Uncertainty
• Theory of evidence

Where,
• K is a normalizing constant that makes m a
probability
 Propagation of Uncertainty
• Fuzzy logic
• We use
• ∏(A∩B) = min(∏(A), ∏(B))
• ∏(A∪B) = max(∏(A), ∏(B))
• To derive
• ∏(p∧q(A))= min(∏p(A), ∏q(A))
• ∏(p∨q(A))= max(∏p(A), ∏q(A))
• ∏(⌐p(A))= 1-(∏p(A))
 Control Structures With Uncertainty
• expert system exhibits uncertainty ⇒
• exhaustive search through all rules
• select rule with highest probability
• ⇒lot of computer time
• can not rely upon basis Data driven/ Goal
driven control structures
 Control Structures With Uncertainty
Control structure for uncertainty base expert system
• Define rule set (set of rules)
• Expert assigns all inputs probabilities (0,1)
• Perform exhaustive search
• Each conclusion will have highest possible probability
– (if conclusions not providing acceptable probability values)
– Expert ⇒ check assigned probability values or
– Expert ⇒ re design expert system
decision threshold value = acceptable possibility value for each conclusion
• For each conclusion check with decision threshold value
– if > threshold value accept as true and proceed
– if < threshold value accept as false and proceed
• Will result in final conclusions with maximum probability
 Control Structures With Uncertainty

Control structure for uncertainty base expert

system requiring sequential analysis
• Define rule set (set of rules)
• For each rule/ item assign cost
• At each stage select least cost evidence
 Control Structures With Uncertainty

Control structure for uncertainty base expert

system using Baye’s theroem
• Define rule set (set of rules)
• Define probability values between (0,1) for
each input
• At each intermediate node that rank highest
among available nodes at each intermediate
conclusion
 Expert System Tools - Nature
System tools

Programming tools

Knowledge Engg tools

System building Tools

Support Facilities

Expert system tools :- Programming systems that

simplifies the job of constructing an expert system
 Programming tools
Problem Oriented
Programming tools
Symbol manipulation

• Problem oriented languages designed for specific problems

• Symbol manipulation Languages designed AI applications
• Useful for representing complex concepts
• Flexibility for expert builder
• Eg. Lisp
• Cannot help on
– How to represent knowledge
– How to access knowledge
 Knowledge Engineering tools
Skeletal
Knowledge Engg tools
General purpose

• Skeletal system (expert system without

knowledge) Used for specific domain only
• General purpose can handle different domains
& problems
• Expert system building language with support
environment
 System building Tools
Knowledge accq Tools
System building Tools
Designing Tools

• Knowledge acquisition Tools

• Support’s
– acquisition of knowledge
– representing knowledge
• Designing Tools
• Aids in designing expert system
 Support Facilities
Programming Tools
Support Facilities
Explanation Tools
• Programming Tools
• Debugging tools: tracking, break Package
• KB editors: support editing KB
• Explanation Tools
• Built in I/O facilities: interactively communicate with Expert
system
• Explanation facilities: respective reasoning, description of
rules that led to current conclusion
Expert System Tools - Categories
Expert System tools

Inductive

Simple Rule Based

Structured Rule Based

Hybrid

Domain Specific
 Inductive Tools
• Based on machine learning
• Derived from experiments conducted in
machine learning
• Usually large – run on main frames
• Smaller tools can run on PC’s
• Developer feeds examples
• Tools convert examples into rules
• Tools determine the order for the rules
 Simple Rule Based Tools
• Uses simple if-then else rules
• Lacks editing features
• Not structured or organized
• Used only if rules< 500
• Simple effective & easy
 Structured Rule Based Tools
• Uses simple if-then rules
• Provides editing features
• Rules are organized/ structured
• Offers
– context trees
– Multiple instantiations
– Rule sets (rules can be sub divided into separate sets)
• Used if rules >500
• Can run on PC’s
 Hybrid Tools
• Object Oriented
• Problem/facts are represented as objects
• Objects contain both rules and facts
• Support large rules>500 to thousands
• Facilitates complex and graphical interfaces
• Powerful should be Used only if it is worth the
cost
• Usually viewed as research tool
 Domain Specific Tools
• Designed to be used for a specific domain only
• Can use any technique
• Can be categorized in any of the above
categories based on the technique used
 Expert system shells
• Shell: A special purpose tool designed for
certain types of application in which the user
must only supply the knowledge base
• shell is a ready-made expert system
• with the knowledgebase missing
• instructions for building a knowledgebase
• Used for
– Building Expert system
– Training
 Expert system shells
• Shell should support
• Inference engine
• Knowledge base
• user system interface
• Explanation facility
• Knowledge acquisition facility
 Expert system shells
• Shell components (categories)
• Knowledge Engineer development aids
• Knowledge acquisition aids
• Data management aids
• Inference engines
• User system interfaces
• Explanation facility
• Language interfaces
 Commercial packages
• Commercial tools for designing expert system
shells
– Automated Reasoning Tool (ART)
– Knowledge Engineering Environment (KEE)
– RuleMaster
assignment
Selecting Right Tool For Job
assignment
 Applications Of Expert System
• Product specific
– Commodity based
– Embedded
– Eg. Vehicles, home appliances
• Service specific
– Generic, Task based
– Computer help services
• Organization specific
– Corporate goal based
– Corporate use
– Intelligent customer assistant, intell’ proposal generator
• Tack specific
– Personal work stations
– Assist single user
– Intelligent programmer assistant, intelligent sales assist