Lecture 17: Logic II

CS221 / Spring 2018 / Sadigh

 Review: ingredients of a logic
Syntax: defines a set of valid formulas (Formulas)
Example: Rain ∧ Wet
Semantics: for each formula, specify a set of models (assignments /
configurations of the world)
Wet
0 1
Example: 0
Rain

Inference rules: given f , what new formulas g can be added that are
guaranteed to follow ( fg )?

Example: from Rain ∧ Wet, derive Rain

CS221 / Spring 2018 / Sadigh 1

 Review: schema
Syntax Semantics

formula

models

Inference
rules

CS221 / Spring 2018 / Sadigh 2

• Logic provides a formal language to talk about the world.
• The valid sentences in the language are the logical formulas, which live in syntax-land.
• In semantics-land, a model represents a possible configuration of the world. An interpretation function
connects syntax and semantics. Specifically, it defines, for each formula f , a set of models M(f ).
 Review: inference task
Inference task
Input:
Knowledge base KB (e.g., {Rain, Rain → Wet})
Query formula f (e.g., Wet)
Output:
Whether KB entails f (KB |= f )? (e.g., yes)
(KB |= f ⇐⇒ M(f ) ⊇ M(KB))

CS221 / Spring 2018 / Sadigh 4

• A knowledge base is a set of formulas we know to be true. Semantically the KB represents the conjunction
of the formulas.
• The central goal of logic is inference: to figure out whether a query formula is entailed by, contradictory
with, or contingent on the KB (these are semantic notions defined by the interpretation function).
 Review: inference algorithm
Inference algorithm:
(repeatedly apply inference rules)
KB f

Definition: modus ponens inference rule

p1 , ··· , pk , (p1 ∧ · · · ∧ pk ) → q
q

Desiderata: soundness and completeness

CS221 / Spring 2018 / Sadigh 6

• The unique thing about having a logical language is that we can also perform inference directly on syntax
by applying inference rules, rather than always appealing to semantics (and performing model checking
there).
• We would like the inference algorithm to be both sound (not derive any false formulas) and complete
(derive all true formulas). Soundness is easy to check, completeness is harder.
 Review: formulas
Propositional logic: any legal combination of symbols

(Rain ∧ Snow) → (Traffic ∨ Peaceful) ∧ Wet

Propositional logic with only Horn clauses: restricted

(Rain ∧ Snow) → Traffic

CS221 / Spring 2018 / Sadigh 8

• Whether a set of inference rules is complete depends on what the formulas are. Last time, we looked at
two logical languages: propositional logic and propositional logic restricted to Horn clauses (essentially
formulas that look like p1 ∧ · · · ∧ pk → q), which intuitively can only derive positive information.
 Approaches
Formulas allowed Inference rule Complete?

Propositional logic (only Horn clauses) modus ponens yes

Propositional logic modus ponens no

Propositional logic resolution yes

CS221 / Spring 2018 / Sadigh 10

• We saw that if our logical language was restricted to Horn clauses, then modus ponens alone was sufficient
for completeness. For general propositional logic, modus ponens is insufficient.
• In this lecture, we’ll see that a more powerful inference rule, resolution, is complete for all of propositional
logic.
 Roadmap

Resolution in propositional logic

First-order logic

CS221 / Spring 2018 / Sadigh 12

 Horn clauses and disjunction
Written with implication Written with disjunction
A→C ¬A ∨ C
A∧B →C ¬A ∨ ¬B ∨ C
• Literal: either p or ¬p, where p is a propositional symbol
• Clause: disjunction of literals
• Horn clauses: at most one positive literal
Modus ponens (rewritten):
A, ¬A∨C
C

• Intuition: cancel out A and ¬A

CS221 / Spring 2018 / Sadigh 13

• Modus ponens can only deal with Horn clauses, so let’s see why Horn clauses are limiting. We can
equivalently write implication using negation and disjunction. Then it’s clear that Horn clauses are just
disjunctions of literals where there is at most one positive literal and zero or more negative literals. The
negative literals correspond to the propositional symbols on the left side of the implication, and the positive
literal corresponds to the propositional symbol on the right side of the implication.
• If we rewrite modus ponens, we can see a ”canceling out” intuition emerging. To make the intuition a bit
more explicit, remember that, to respect soundness, we require {A, ¬A ∨ C} |= C; this is equivalent to:
if A ∧ (¬A ∨ C) is true, then C is also true. This is clearly the case.
• But modus ponens cannot operate on general clauses.
 Resolution [Robinson, 1965]
General clauses have any number of literals:

¬A ∨ B ∨ ¬C ∨ D ∨ ¬E ∨ F

Example: resolution inference rule

Rain ∨ Snow, ¬Snow ∨ Traffic
Rain ∨ Traffic

Definition: resolution inference rule

f1 ∨ · · · ∨ fn ∨ p, ¬p ∨ g1 ∨ · · · ∨ gm
f1 ∨ · · · ∨ fn ∨ g1 ∨ · · · ∨ gm

CS221 / Spring 2018 / Sadigh 15

• Let’s try to generalize modus ponens by allowing it to work on general clauses. This generalized inference
rule is called resolution, which was invented in 1965 by John Alan Robinson.
• The idea behind resolution is that it takes two general clauses, where one of them has some propositional
symbol p and the other clause has its negation ¬p, and simply takes the disjunction of the two clauses
with p and ¬p removed. Here, f1 , . . . , fn , g1 , . . . , gm are arbitrary literals.
 Soundness of resolution
Rain ∨ Snow, ¬Snow ∨ Traffic
(resolution rule)
Rain ∨ Traffic
M(Rain ∨ Snow)∩M(¬Snow ∨ Traffic)⊆?M(Rain ∨ Traffic)

Snow Snow Snow

0 1 0 1 0 1
0,0 0,0 0,0
Rain, Traffic

Rain, Traffic

Rain, Traffic
0,1 0,1 0,1
1,0 1,0 1,0
1,1 1,1 1,1

Sound!

CS221 / Spring 2018 / Sadigh 17

• Why is resolution logically sound? We can verify the soundness of resolution by checking its semantic
interpretation. Indeed, the intersection of the models of f and g are a superset of f ∨ g.
 Conjunctive normal form
So far: resolution only works on clauses...but that’s enough!

Definition: conjunctive normal form (CNF)

A CNF formula is a conjunction of clauses.

Example: (A ∨ B ∨ ¬C) ∧ (¬B ∨ D)

Equivalent: knowledge base where each formula is a clause

Proposition: conversion to CNF

Every formula f in propositional logic can be converted into an

equivalent CNF formula f 0 :
M(f ) = M(f 0 )

CS221 / Spring 2018 / Sadigh 19

• But so far, we’ve only considered clauses, which are disjunctions of literals. Surely this can’t be all of
propositional logic... But it turns out it actually is in the following sense.
• A conjunction of clauses is called a CNF formula, and every formula in propositional logic can be converted
into an equivalent CNF. Given a CNF formula, we can toss each of its clauses into the knowledge base.
• But why can every formula be put in CNF?
 Conversion to CNF: example
Initial formula:
(Summer → Snow) → Bizzare
Remove implication (→):
¬(¬Summer ∨ Snow) ∨ Bizzare
Push negation (¬ ) inwards (de Morgan):
(¬¬Summer ∧ ¬Snow) ∨ Bizzare
Remove double negation:
(Summer ∧ ¬Snow) ∨ Bizzare
Distribute ∨ over ∧:
(Summer ∨ Bizzare) ∧ (¬Snow ∨ Bizzare)
CS221 / Spring 2018 / Sadigh 21
• The answer is by construction. There is a six-step procedure that takes any propositional formula and
turns it into CNF. Here is an example of how it works (only four of the six steps apply here).
 Conversion to CNF: general
Conversion rules:
f ↔g
• Eliminate ↔: (f →g)∧(g→f )
f →g
• Eliminate →: ¬f ∨g
¬(f ∧g)
• Move ¬ inwards: ¬f ∨¬g
¬(f ∨g)
• Move ¬ inwards: ¬f ∧¬g
¬¬f
• Eliminate double negation: f
f ∨(g∧h)
• Distribute ∨ over ∧: (f ∨g)∧(f ∨h)

CS221 / Spring 2018 / Sadigh 23

• Here are the general rules that convert any formula to CNF. First, we try to reduce everything to negation,
conjunction, and disjunction.
• Next, we try to push negation inwards so that they sit on the propositional symbols (forming literals).
Note that when negation gets pushed inside, it flips conjunction to disjunction, and vice-versa.
• Finally, we distribute so that the conjunctions are on the outside, and the disjunctions are on the inside.
• Note that each of these operations preserves the semantics of the logical form (remember there are many
formula that map to the same set of models). This is in contrast with most inference rules, where the
conclusion is more general than the conjunction of the premises.
• Also, when we apply a CNF rewrite rule, we replace the old formula with the new one, so there is no
blow-up in the number of formulas. This is in contrast to applying general inference rules. An analogy:
conversion to CNF does simplification in the context of full inference, just like AC-3 does simplification in
the context of backtracking search.
 Resolution algorithm

Recall: entailment and contradiction ⇔ satisfiability

KB |= f KB ∪ {¬f } is unsatisfiable

KB |= ¬f KB ∪ {f } is unsatisfiable

Algorithm: resolution-based inference

• Convert all formulas into CNF.

• Repeatedly apply resolution rule.
• Return unsatisfiable iff derive false.

CS221 / Spring 2018 / Sadigh 25

• After we have converted all the formulas to CNF, we can repeatedly apply the resolution rule. But what
is the final target?
• Recall that both testing for entailment and contradiction boil down to checking satisfiability. Resolution
can be used to do this very thing. If we ever apply a resolution rule (e.g., to premises A and ¬A) and we
derive false, which is clearly a contradiction, then the set of formulas in the knowledge base is unsatisfiable.
• If we are unable to derive false, that means the knowledge base is satisfiable because resolution is complete.
However, unlike in model checking, we don’t actually produce a concrete model that satisfies the KB.
 Resolution: example
Knowledge base (is it satisfiable?):
KB = {A → (B ∨ C), A, ¬B, ¬C}
Convert to CNF:
KB = {¬A ∨ B ∨ C, A, ¬B, ¬C}
Repeatedly apply resolution rule:
false

C ¬C

B∨C ¬B

¬A ∨ B ∨ C A

CS221 / Spring 2018 / Sadigh

Unsatisfiable! 27
• Here’s an example of taking a knowledge base, converting it into CNF, and applying resolution. In this
case, we derive false, which means that the original knowledge base was unsatisfiable.
 Time complexity
Definition: modus ponens inference rule
p1 , ··· , pk , (p1 ∧ · · · ∧ pk ) → q
q

• Each rule application adds clause with one propositional symbol

⇒ linear time

Definition: resolution inference rule

f1 ∨ · · · ∨ fn ∨ p, ¬p ∨ g1 ∨ · · · ∨ gm
f1 ∨ · · · ∨ fn ∨ g1 ∨ · · · ∨ gm

• Each rule application adds clause with many propositional symbols

⇒ exponential time

CS221 / Spring 2018 / Sadigh 29

• There we have it — a sound and complete inference procedure for all of propositional logic (although we
didn’t prove completeness). But what do we have to pay computationally for this increase?
• If we only have to apply modus ponens, each propositional symbol can only get added once, so with the
appropriate algorithm (forward chaining), we can apply all necessary modus ponens rules in linear time.
• But with resolution, we can end up adding clauses with many propositional symbols, and possibly any
subset of them! Therefore, this can take exponential time.
 Summary

Horn clauses any clauses

modus ponens resolution

linear time exponential time

less expressive more expressive

CS221 / Spring 2018 / Sadigh 31

• To summarize, we can either content ourselves with the limited expressivity of Horn clauses and obtain an
efficient inference procedure (via modus ponens).
• If we wanted the expressivity of full propositional logic, then we need to use resolution and thus pay more.
 Roadmap

Resolution in propositional logic

First-order logic

CS221 / Spring 2018 / Sadigh 33

 Limitations of propositional logic
Alice and Bob both know arithmetic.

AliceKnowsArithmetic ∧ BobKnowsArithmetic

All students know arithmetic.

AliceIsStudent → AliceKnowsArithmetic
BobIsStudent → BobKnowsArithmetic
...

Every even integer greater than 2 is the sum of two primes.

???

CS221 / Spring 2018 / Sadigh 34

• If the goal of logic is to be able to express facts in the world in a compact way, let us ask ourselves if
propositional logic is enough.
• Some facts can be expressed in propositional logic, but it is very clunky, having to instantiate many different
formulas. Others simply can’t be expressed at all, because we would need to use an infinite number of
formulas.
 Limitations of propositional logic
All students know arithmetic.

AliceIsStudent → AliceKnowsArithmetic
BobIsStudent → BobKnowsArithmetic
...

Propositional logic is very clunky. What’s missing?

• Objects and relations: propositions (e.g., AliceKnowsArithmetic)

have more internal structure (alice, Knows, arithmetic)
• Quantifiers and variables: all is a quantifier which applies to each
person, don’t want to enumerate them all...

CS221 / Spring 2018 / Sadigh 36

• What’s missing? The key conceptual observation is that the world is not just a bunch of atomic facts, but
that each fact is actually made out of objects and relations between those objects.
• Once facts are decomposed in this way, we can use quantifiers and variables to implicitly define a huge
(and possibly infinite) number of facts with one compact formula. Again, where logic excels is the ability
to represent complex things via simple means.
 First-order logic

Syntax Semantics

formula

models

Inference
rules

CS221 / Spring 2018 / Sadigh 38

• We will now introduce first-order logic, which will address the representational limitations of propositional
logic.
• Remember to define a logic, we need to talk about its syntax, its semantics (interpretation function), and
finally inference rules that we can use to operate on the syntax.
 First-order logic: examples
Alice and Bob both know arithmetic.

Knows(alice, arithmetic) ∧ Knows(bob, arithmetic)

All students know arithmetic.

∀x Student(x) → Knows(x, arithmetic)

CS221 / Spring 2018 / Sadigh 40

• Before formally defining things, let’s look at two examples. First-order logic is basically propositional logic
with a few more symbols.
 Syntax of first-order logic
Terms (refer to objects):
• Constant symbol (e.g., arithmetic)
• Variable (e.g., x)
• Function of terms (e.g., Sum(3, x))
Formulas (refer to truth values):
• Atomic formulas (atoms): predicate applied to terms (e.g.,
Knows(x, arithmetic))
• Connectives applied to formulas (e.g., Student(x) →
Knows(x, arithmetic))
• Quantifiers applied to formulas (e.g., ∀x Student(x) →
Knows(x, arithmetic))

CS221 / Spring 2018 / Sadigh 42

• In propositional logic, everything was a formula (or a connective). In first-order logic, there are two types
of beasts: terms and formulas. There are three types of terms: constant symbols (which refer to specific
objects), variables (which refer to some unspecified object to be determined by quantifiers), and functions
(which is a function applied to a set of arguments which are themselves terms).
• Given the terms, we can form atomic formulas, which are the analogue of propositional symbols, but with
internal structure (e.g., terms).
• From this point, we can apply the same connectives on these atomic formulas, as we applied to propositional
symbols in propositional logic. At this level, first-order logic looks very much like propositional logic.
• Finally, to make use of the fact that atomic formulas have internal structure, we have quantifiers, which
are really the whole point of first-order logic!
 Quantifiers
Universal quantification (∀):

Think conjunction: ∀x P (x) is like P (A) ∧ P (B) ∧ · · ·

Existential quantification (∃):

Think disjunction: ∃x P (x) is like P (A) ∨ P (B) ∨ · · ·

Some properties:
• ¬∀x P (x) equivalent to ∃x ¬P (x)
• ∀x ∃y Knows(x, y) different from ∃y ∀x Knows(x, y)

CS221 / Spring 2018 / Sadigh 44

• There are two types of quantifiers: universal and existential. These are basically glorified ways of doing
conjunction and disjunction, respectively.
• For crude intuition, we can think of conjunction and disjunction as very nice syntactic sugar, which can
be rolled out into something that looks more like propositional logic. But quantifiers aren’t just sugar,
and it is important that they be compact, for sometimes the variable being quantified over can take on an
infinite number of objects.
• That being said, the conjunction and disjunction intuition suffices for day-to-day guidance. For example,
it should be intuitive that pushing the negation inside a universal quantifier (conjunction) turns it into a
existential (disjunction), which was the case for propositional logic (by de Morgan’s laws). Also, one cannot
interchange universal and existential quantifiers any more than one can swap conjunction and disjunction
in propositional logic.
 Natural language quantifiers
Universal quantification (∀):

Every student knows arithmetic.

∀x Student(x)→Knows(x, arithmetic)

Existential quantification (∃):

Some student knows arithmetic.

∃x Student(x)∧Knows(x, arithmetic)

Note the different connectives!

CS221 / Spring 2018 / Sadigh 46

• Universal and existential quantifiers naturally correspond to the words every and some, respectively. But
when converting English to formal logic, one must exercise caution.
• Every can be thought of as taking two arguments P and Q (e.g., student and knows arithmetic). The
connective between P and Q is an implication (not conjunction, which is a common mistake). This makes
sense because when we talk about every P , we are only restricting our attention to objects x for which
P (x) is true. Implication does exactly that.
• On the other hand, the connective for existential quantification is conjunction, because we’re asking for
an object x such that P (x) and Q(x) both hold.
 Some examples of first-order logic
There is some course that every student has taken.

∃y Course(y) ∧ [∀x Student(x) → Takes(x, y)]

Every even integer greater than 2 is the sum of two primes.

∀x EvenInt(x) ∧ Greater(x, 2) → ∃y ∃z Equals(x, Sum(y, z)) ∧ Prime(y) ∧ Prime(z)

If a student takes a course and the course covers a concept, then the
student knows that concept.

∀x ∀y ∀z (Student(x) ∧ Takes(x, y) ∧ Course(y) ∧ Covers(y, z)) → Knows(x, z)

CS221 / Spring 2018 / Sadigh 48

• Let’s do some more examples of converting natural language to first-order logic. Remember the connectives
associated with existential and universal quantification!
• Note that some English words such as a can trigger both universal or existential quantification, depending
on context. In A student took CS221, we have existential quantification, but in if a student takes CS221,
..., we have universal quantification.
• Formal logic clears up the ambiguities associated with natural language.
 First-order logic

Syntax Semantics

formula

models

Inference
rules

CS221 / Spring 2018 / Sadigh 50

• So far, we’ve only presented the syntax of first-order logic, although we’ve actually given quite a bit of
intuition about what the formulas mean. After all, it’s hard to talk about the syntax without at least a
hint of semantics for motivation.
• Now let’s talk about the formal semantics of first-order logic.
 Models in first-order logic
Recall a model represents a possible situation in the world.

Propositional logic: Model w maps propositional symbols to truth values.

w = {AliceKnowsArithmetic : 1, BobKnowsArithmetic : 0}

First-order logic: ?

CS221 / Spring 2018 / Sadigh 52

• Recall that a model in propositional logic was just an assignment of truth values to propositional symbols.
• A natural candidate for a model in first-order logic would then be an assignment of truth values to grounded
atomic formula (those formulas whose terms are constants as opposed to variables). This is almost right,
but doesn’t talk about the relationship between constant symbols.
 Graph representation of a model
If only have unary and binary relations, a model w as be represented a
directed graph:
w
Student bob

alice o1 o2 robert
Knows Knows

o3

arithmetic

• Nodes are objects, labeled with constant symbols

• Directed edges are binary relations, labeled with predicate symbols;
unary relations are additional node labels
CS221 / Spring 2018 / Sadigh 54
• A better way to think about a first-order model is that there are a number of objects in the world
(o1 , o2 , . . . ); think of these as nodes in a graph. Then we have relations between these objects. Predicates
that take two arguments can be visualized as labeled edges between objects. Predicates that take one
argument can be visualized as node labels (but these are not so important).
• So far, the objects are unnamed. We can access individual objects directly using constant symbols, which
are labels on the nodes.
 Models in first-order logic

Definition: model in first-order logic

A model w in first-order logic maps:

• constant symbols to objects

w(alice) = o1 , w(bob) = o2 , w(arithmetic) = o3

• predicate symbols to tuples of objects

w(Knows) = {(o1 , o3 ), (o2 , o3 ), . . . }

CS221 / Spring 2018 / Sadigh 56

• Formally, a first-order model w maps constant symbols to objects and predicate symbols to tuples of
objects (2 for binary predicates).
 A restriction on models
John and Bob are students.
Student(john) ∧ Student(bob)
w1 w2 w3
Student Student Student Student Student

john o1 o2 bob john o1 bob john o1 o2 o3 bob

• Unique names assumption: Each object has at most one constant

symbol. This rules out w2 .
• Domain closure: Each object has at least one constant symbol.
This rules out w3 .
Point:
constant symbol object

CS221 / Spring 2018 / Sadigh 58

• Note that by default, two constant symbols can refer to the same object, and there can be objects which no
constant symbols refer to. This can make life somewhat confusing. Fortunately, there are two assumptions
that people sometimes make to simplify things.
• The unique names assumption says that there’s at most one way to refer to an object via a constant
symbol. Domain closure says there’s at least one. Together, they imply that there is a one-to-one
relationship between constant symbols in syntax-land and objects in semantics-land.
 Propositionalization
If one-to-one mapping between constant symbols and objects (unique
names and domain closure),
first-order logic is syntactic sugar for propositional logic:
Knowledge base in first-order logic
Student(alice) ∧ Student(bob)
∀x Student(x) → Person(x)
∃x Student(x) ∧ Creative(x)

Knowledge base in propositional logic

Student(alice) ∧ Student(bob)
(Student(alice) → Person(alice)) ∧ (Student(bob) → Person(bob))
(Student(alice) ∧ Creative(alice)) ∨ (Student(bob) ∧ Creative(bob))

Point: use any inference algorithm for propositional logic!

CS221 / Spring 2018 / Sadigh 60
• If a one-to-one mapping really exists, then we can propositionalize all our formulas, which basically unrolls
all the quantifiers into explicit conjunctions and disjunctions.
• The upshot of this conversion, is that we’re back to propositional logic, and we know how to do in-
ference in propositional logic (either using model checking or by applying inference rules). Of course,
propositionalization could be quite expensive and not the most efficient thing to do.
 First-order logic
Syntax Semantics

formula

models

Inference

rules

CS221 / Spring 2018 / Sadigh 62

• Now we look at inference rules which can make first-order inference much more efficient. The key is
to do everything implicitly and avoid propositionalization; again the whole spirit of logic is to do things
compactly and implicitly.
 Definite clauses
∀x ∀y ∀z (Takes(x, y) ∧ Covers(y, z)) → Knows(x, z)

Note: if propositionalize, get one formula for each value to (x, y, z), e.g.,
(alice, cs221, mdp)

Definition: definite clause (first-order logic)

A definite clause has the following form:

∀x1 · · · ∀xn (a1 ∧ · · · ∧ ak ) → b
for variables x1 , . . . , xn and atomic formulas a1 , . . . , ak , b (which
contain those variables).

CS221 / Spring 2018 / Sadigh 64

• Like our development of inference in propositional logic, we will first talk about first-order logic restricted
to Horn clauses, in which case a first-order version of modus ponens will be sound and complete. After
that, we’ll see how resolution allows to handle all of first-order logic.
• We start by generalizing definite clauses from propositional logic to first-order logic. The only difference
is that we now have universal quantifiers sitting at the beginning of the definite clause. This makes sense
since universal quantification is associated with implication, and one can check that if one propositionalizes
a first-order definite clause, one obtains a set (conjunction) of multiple propositional definite clauses.
 Modus ponens (first attempt)

Definition: modus ponens (first-order logic)

a1 , . . . , ak ∀x1 · · · ∀xn (a1 ∧ · · · ∧ ak ) → b
,
b

Setup:

Given P (alice) and ∀x P (x) → Q(x).

Problem:

Can’t infer Q(alice) because P (x) and P (alice) don’t match!

Solution: substitution and unification

CS221 / Spring 2018 / Sadigh 66

• If we try to write down the modus ponens rule, we would fail.
• As a simple example, suppose we are given P (alice) and ∀x P (x) → Q(x). We would naturally want to
derive Q(alice). But notice that we can’t apply modus ponens because P (alice) and P (x) don’t match!
• Recall that we’re in syntax-land, which means that these formulas are just symbols. Inference rules don’t
have access to the semantics of the constants and variables — it is just a pattern matcher. So we have to
be very methodical.
• To develop a mechanism to match variables and constants, we will introduce two concepts, substitution
and unification for this purpose.
 Substitution
Subst[{x/alice}, P (x)] = P (alice)

Subst[{x/alice, y/z}, P (x) ∧ K(x, y)] = P (alice) ∧ K(alice, z)

Definition: Substitution

A substitution θ is a mapping from variables to terms.

Subst[θ, f ] returns the result of performing substitution θ on f .

CS221 / Spring 2018 / Sadigh 68

• The first step is substitution, which applies a search-and-replace operation on a formula or term.
• We won’t define Subst[θ, f ] formally, but from the examples, it should be clear what Subst does.
• Technical note: if θ contains variable substitutions x/alice we only apply the substitution to the free
variables in f , which are the variables not bound by quantification (e.g., x in ∃y, P (x, y)). Later, we’ll see
how CNF formulas allow us to remove all the quantifiers.
 Unification
Unify[Knows(alice, arithmetic), Knows(x, arithmetic)] = {x/alice}
Unify[Knows(alice, y), Knows(x, z)] = {x/alice, y/z}
Unify[Knows(alice, y), Knows(bob, z)] = fail
Unify[Knows(alice, y), Knows(x, F (x))] = {x/alice, y/F (alice)}

Definition: Unification

Unification takes two formulas f and g and returns a substitution

θ which is the most general unifier:
Unify[f, g] = θ such that Subst[θ, f ] = Subst[θ, g]
or ”fail” if no such θ exists.

CS221 / Spring 2018 / Sadigh 70

• Substitution can be used to make two formulas identical, and unification is the way to find the least
committal substitution we can find to achieve this.
• Unification, like substitution, can be implemented recursively. The implementation details are not the most
exciting, but it’s useful to get some intuition from the examples.
 Modus ponens

Definition: modus ponens (first-order logic)

a01 , . . . , a0k
∀x1 · · · ∀xn (a1 ∧ · · · ∧ ak ) → b
0
,
b
Get most general unifier θ on premises:
• θ = Unify[a01 ∧ · · · ∧ a0k , a1 ∧ · · · ∧ ak ]
Apply θ to conclusion:
• Subst[θ, b] = b0

CS221 / Spring 2018 / Sadigh 72

• Having defined substitution and unification, we are in position to finally define the modus ponens rule for
first-order logic. Instead of performing a exact match, we instead perform a unification, which generates
a substitution θ. Using θ, we can generate the conclusion b0 on the fly.
• Note the significance here: the rule a1 ∧ · · · ∧ ak → b can be used in a myriad ways, but Unify identifies
the appropriate substitution, so that it can be applied to the conclusion.
 Modus ponens example

Example: modus ponens in first-order logic

Premises:
Takes(alice, cs221)
Covers(cs221, mdp)

∀x ∀y ∀z Takes(x, y) ∧ Covers(y, z) → Knows(x, z)

Conclusion:
θ = {x/alice, y/cs221, z/mdp}
Derive Knows(alice, mdp)

CS221 / Spring 2018 / Sadigh 74

• Here’s a simple example of modus ponens in action. We bind x, y, z to appropriate objects (constant
symbols), which is used to generate the conclusion Knows(alice, mdp).
 Complexity
∀x ∀y ∀z P (x, y, z)

• Each application of Modus ponens produces an atomic formula.

• If no function symbols, number of atomic formulas is at most

(num-constant-symbols)(maximum-predicate-arity)

• If there are function symbols (e.g., F ), then infinite...

Q(a) Q(F (a)) Q(F (F (a))) Q(F (F (F (a)))) ···

CS221 / Spring 2018 / Sadigh 76

• In propositional logic, modus ponens was considered efficient, since in the worst case, we generate each
propositional symbol.
• In first-order logic, though, we typically have many more atomic formulas in place of propositional symbols,
which leads to a potentially exponentially number of atomic formulas, or worse, with function symbols,
there might be an infinite set of atomic formulas.
 Complexity

Theorem: completeness

Modus ponens is complete for first-order logic with only Horn

clauses.

Theorem: semi-decidability

First-order logic (even restricted to only Horn clauses) is semi-

decidable.
• If KB |= f , forward inference on complete inference rules will
prove f in finite time.
• If KB 6|= f , no algorithm can show this in finite time.

CS221 / Spring 2018 / Sadigh 78

• We can show that modus ponens is complete with respect to Horn clauses, which means that every true
formula has an actual finite derivation.
• However, this doesn’t mean that we can just run modus ponens and be done with it, for first-order logic
even restricted to Horn clauses is semi-decidable, which means that if we a formula is entailed, then we
will be able to derive it, but if it is not entailed, then we don’t even know when to stop the algorithm —
quite troubling!
• With propositional logic, there were a finite number of propositional symbols, but now the number of
atomic formulas can be infinite (the culprit is function symbols).
• Though we have hit a theoretical barrier, life goes on and we can still run modus ponens inference to get
a one-sided answer. Next, we will move to working with full first-order logic.
 Resolution
Recall: First-order logic includes non-Horn clauses

∀x Student(x) → ∃y Knows(x, y)

High-level strategy (same as in propositional logic):

• Convert all formulas to CNF
• Repeatedly apply resolution rule

CS221 / Spring 2018 / Sadigh 80

• To go beyond Horn clauses, we will develop a single resolution rule which is sound and complete.
• The high-level strategy is the same as propositional logic: convert to CNF and apply resolution.
 Conversion to CNF
Input:

∀x (∀y Animal(y) → Loves(x, y)) → ∃y Loves(y, x)

Output:

(Animal(Y (x)) ∨ Loves(Z(x), x)) ∧ (¬Loves(x, Y (x)) ∨ Loves(Z(x), x))

New to first-order logic:

• All variables (e.g., x) have universal quantifiers by default
• Introduce Skolem functions (e.g., Y (x)) to represent existential
quantified variables

CS221 / Spring 2018 / Sadigh 82

• Consider the logical formula corresponding to Everyone who loves all animals is loved by someone. The
slide shows the desired output, which looks like a CNF formula in propositional logic, but there are two
differences: there are variables (e.g., x) and functions of variables (e.g., Y (x)). The variables are assumed
to be universally quantified over, and the functions are called Skolem functions and stand for a property
of the variable.
 Conversion to CNF (part 1)
Anyone who likes all animals is liked by someone.
Input:
∀x (∀y Animal(y) → Loves(x, y)) → ∃y Loves(y, x)
Eliminate implications (old):
∀x ¬(∀y ¬Animal(y) ∨ Loves(x, y)) ∨ ∃y Loves(y, x)
Push ¬ inwards, eliminate double negation (old):
∀x (∃y Animal(y) ∧ ¬Loves(x, y)) ∨ ∃y Loves(y, x)
Standardize variables (new):
∀x (∃y Animal(y) ∧ ¬Loves(x, y)) ∨ ∃z Loves(z, x)

CS221 / Spring 2018 / Sadigh 84

• We start by eliminating implications, pushing negation inside, and eliminating double negation, which is
all old.
• The first thing new to first-order logic is standardization of variables. Note that in ∃x P (x) ∧ ∃x Q(x),
there are two instances of x whose scopes don’t overlap. To make this clearer, we will convert this into
∃x P (x) ∧ ∃y Q(y). This sets the stage for when we will drop the quantifiers on the variables.
 Conversion to CNF (part 2)
∀x (∃y Animal(y) ∧ ¬Loves(x, y)) ∨ ∃z Loves(z, x)

Replace existentially quantified variables with Skolem functions (new):

∀x [Animal(Y (x)) ∧ ¬Loves(x, Y (x))] ∨ Loves(Z(x), x)

Distribute ∨ over ∧ (old):

∀x [Animal(Y (x)) ∨ Loves(Z(x), x)] ∧ [¬Loves(x, Y (x)) ∨ Loves(Z(x), x)]

Remove universal quantifiers (new):

[Animal(Y (x)) ∨ Loves(Z(x), x)] ∧ [¬Loves(x, Y (x)) ∨ Loves(Z(x), x)]

CS221 / Spring 2018 / Sadigh 86

• The next step is to remove existential variables by replacing them with Skolem functions. This is perhaps
the most non-trivial part of the process. Consider the formula: ∀x ∃y P (x, y). Here, y is existentially
quantified and depends on x. So we can mark this dependence explicitly by setting y = Y (x). Then the
formula becomes ∀x P (x, Y (x)). You can even think of the function Y as being existentially quantified
over outside the ∀x.
• Next, we distribute disjunction over conjunction as before.
• Finally, we simply drop all universal quantifiers. Because those are the only quantifiers left, there is no
ambiguity.
• The final CNF formula can be difficult to interpret, but we can be assured that the final formula captures
exactly the same information as the original formula.
 Resolution

Definition: resolution rule (first-order logic)

f1 ∨ · · · ∨ fn ∨ p, ¬q ∨ g1 ∨ · · · ∨ gm
Subst[θ, f1 ∨ · · · ∨ fn ∨ g1 ∨ · · · ∨ gm ]
where θ = Unify[p, q].

Example: resolution
Animal(Y (x)) ∨ Loves(Z(x), x), ¬Loves(u, v) ∨ Feeds(u, v)
Animal(Y (x)) ∨ Feeds(Z(x), x)
Substitution: θ = {u/Z(x), v/x}.

CS221 / Spring 2018 / Sadigh 88

• After convering all formulas to CNF, then we can apply the resolution rule, which is generalized to first-
order logic. This means that instead of doing exact matching of a literal p, we unify atomic formulas p
and q, and then apply the resulting substitution θ on the conclusion.
 Summary
Propositional logic First-order logic
model checking n/a
⇐ propositionalization
modus ponens modus ponens++
(Horn clauses) (Horn clauses)
resolution resolution++
(general) (general)
++: unification and substitution

Key idea: variables in first-order logic

Variables yield compact knowledge representations.

CS221 / Spring 2018 / Sadigh 90
• To summarize, we have presented propositional logic and first-order logic. When there is a one-to-one
mapping between constant symbols and objects, we can propositionalize, thereby converting first-order
logic into propositional logic. This is needed if we want to use model checking to do inference.
• For inference based on syntactic derivations, there is a neat parallel between using modus ponens for Horn
clauses and resolution for general formulas (after conversion to CNF). In the first-order logic case, things
are more complex because we have to use unification and substitution to do matching of formulas.
• The main idea in first-order logic is the use of variables (not to be confused with the variables in variable-
based models, which are mere propositional symbols from the point of view of logic), coupled with quan-
tifiers.
• Propositional formulas allow us to express large complex sets of models compactly using a small piece of
propositional syntax. Variables in first-order logic in essence takes this idea one more step forward, allowing
us to effectively express large complex propositional formulas compactly using a small piece of first-order
syntax.
• Note that variables in first-order logic are not same as the variables in variable-based models (CSPs). CSPs
variables correspond to atomic formula and denote truth values. First-order logic variables denote objects.