Knowledge Engineering

Quang Nhat NGUYEN

(Academic year 2010-2011) Hanoi University of Science and Technology School of Information and Communication Technology

Content
Introduction First-order logic Knowledge representation Semantic Web Logic L i programming i
Resolution Forward reasoning g Backward reasoning

Uncertain reasoning Expert systems Knowledge discovery by Machine learning Knowledge discovery by Data mining
Knowledge Engineering
2

Logical inference in First-order logic (1)

Logical inference problem
Given a knowledge base KB (a set of sentences) and a sentence , does the KB semantically entail ?

KB ?
In all interpretations in which sentences in the KB are true, is also true?

Question: Is there a procedure that can decide the entailment for all possible input sentences in a finite number of steps For the propositional logic: Yes However, the logical inference problem in the first-order logic is undecidable
Knowledge Engineering
3

Logical inference in First-order logic (2)

Is the Truth-table approach a viable approach for the first order first-order logic? No! Why not? It would require us to enumerate and list all possible interpretations I I = (assignments of symbols to objects, predicates to relations and functions to relational mappings) There are t many possible i t Th too ibl interpretations! t ti !

Knowledge Engineering

Logical inference in First-order logic (3)

Is the Inference-rule approach a viable approach for the first order logic? first-order-logic?
Yes

The inference rules represent sound inference patterns that one can apply to sentences in the KB Reuse inference rules from the propositional logic
Modus ponens, Resolution, And-introduction And-elimination ponens Resolution And-introduction, And-elimination, Or-introduction, Negation elimination

Additional inference rules are needed for sentences with quantifiers ( ) tifi (, We need to handle variables
Must involve variable substitutions
Knowledge Engineering
5

Variable substitutions
Variables in FOL sentences can be substituted with ground terms Ground terms = {constants, variables, functions} A variable substitution is a mapping from variables to ground terms {x1 /t1, x2/t2,} Application of the substitution to FOL sentences
SUBST({x/Tuan, y/Hai}, Likes(x, y)) = Likes(Tuan,Hai) SUBST ({x/z, y/fatherOf(Vu)}, Likes(x, y)) = Likes (z, fatherOf(Vu))

Knowledge Engineering

Inference rules for quantifiers (1)

Universal instantiation (elimination)
To b tit t T substitute a variable with a ground term i bl ith dt x: (x) ( ) (a) ; a is a ground term

Example
x: Likes(x, IceCream) Likes(Tuan, IceCream) By the substitution of the variable x with the constant Tuan Denoted as: SUBST({x/a}, x: (x)) = (a)

Universal Instantiation can be applied many times to produce many different consequences
Knowledge Engineering
7

Inference rules for quantifiers (2)

Existential instantiation (elimination)
To substitute a variable with a ground term that does not appear elsewhere in the KB x: (x) (a) ; a does not appear in KB

Example
x: Kill(x Victim) Kill(Murderer Victim) Kill(x, Kill(Murderer, By the substitution of the variable x with the constant Murderer Denoted as: SUBST({x/a} x: (x)) = (a) SUBST({x/a},

Existential Instantiation can be applied once

Then the existentially quantified sentence can be discarded
Knowledge Engineering
8

Unification (1)
Problem in inference: Universal instantiation gives many opportunities for substituting variables with ground terms
x: (x) (a) ; a is a ground term

Solution: Try substitutions of similar sentences in KB Example

Lets assume we have KB containing the 3 following sentences x: King(x) Greedy(x) Evil(x) King(John) y: Greedy(y) If we use a substitution = {x/John,y/John}, then we can infer Evil(John)
Knowledge Engineering
9

Unification (2)
A unification takes two similar sentences and computes the substitution that makes them look the same if it same, exists
UNIFY(p,q) = such that SUBST(,p) = SUBST(,q)

Examples
UNIFY(Knows(John, x), Knows(John, Jane)) = {x/Jane} UNIFY(Knows(John, x), Knows(y, Bill)) = {x/Bill, y/John} UNIFY(Knows(John, x), Knows(y, MotherOf(y))) = {x/MotherOf(John), /J h } { /M th Of(J h ) y/John} UNIFY(Knows(John, x), Knows(x, Elizabeth )) = fail Because no
substitution can make the two sentence are the same (i.e., x cannot take on the values John and Elizabeth at the same time)
Knowledge Engineering
10

Most general unifier

UNIFY should return a substitution that makes the two arguments look the same But there could be more But, than one such unifier For example, to unify Knows(John,x) and Knows(y,z) p y ( ) (y )
1 = {y/John, x/z } 2 = {y/John, x/John, z/John}

In the above example, the f first unifier is more general f than the second For every unifiable pair of expressions, there is a single expressions most general unifier (MGU) that is unique up to renaming of variables
In the above example: MGU = {y/John, x/z}
Knowledge Engineering
11

Resolution inference rule (1)

Basic propositional version , Equivalent to ,

Knowledge Engineering

12

Resolution inference rule (2)

FOL version:
p1 pj pm, q1 qk qn SUBST (, (p1 pj-1 pj+1 pm q1 qk-1 qk+1 qn))

where qk = qk and UNIFY(pj, qk) = q q

is the most general unifier (MGU) of pj and qk

The literals pj and qk are called complementary literals
Because each one unifies with the negation of the other

The resulting disjunction is called a resolvent

Knowledge Engineering

13

Resolution inference rule Examples (1)

Example 1
Rich(x) Unhappy(x) Rich(Me) Unhappy(x), Unhappy(Me) by the most general unifier = {x/Me}

Example 2 Lets assume that KB contains the following sentences

PhD(x) HighlyQualified(x) PhD(x) EarlyEarnings(x) HighlyQualified(x) Rich(x) EarlyEarnings(x) Rich(x)

Try resolution to infer Rich(Me) y ( )

Knowledge Engineering
14

Resolution inference rule Examples (2)

Convert KB to the equivalent sentences PhD(x) HighlyQualified(x) ( ) g yQ ( ) PhD(x) EarlyEarnings(x) HighlyQualified(x) Rich(x) EarlyEarnings(x) Rich(x) EarlyEarnings(x) From PhD(x) HighlyQualified(x) and PhD(x) EarlyEarnings(x) and using the MGU = {x/Me}, we infer HighlyQualified(Me) EarlyEarnings(Me) From HighlyQualified(x) Rich(x) and HighlyQualified(Me) EarlyEarnings(Me) and using the MGU = {x/Me}, we infer y g ( ) g { }, Rich(Me) EarlyEarnings(Me) From Rich(Me) EarlyEarnings(Me) and EarlyEarnings(x) Rich(x) and using the MGU = {x/Me}, we infer Rich(Me) {x/Me}
Knowledge Engineering
15

Resolution-based proving
Proof by refutation: To prove KB , we need to prove that { , {KB, } is unsatisfiable } Main steps of the resolution-based proving procedure
1. Convert {KB, } to CNF with ground terms and universal

variables only i bl l 2. Apply repeatedly the resolution rule while keeping track and consistency of substitutions 3. Stop when either an empty set (contradiction) is derived or no more new resolvents (conclusions) follow

At a resolution step a contradiction occurs if an empty step, clause (equivalent to false) is derived

p, p {}
Knowledge Engineering
16

Conjunctive normal form

To be able to do resolution, the given formulas have to be in conjunctive normal form (CNF) A literal is an atomic formula or the negation of an atomic formula
An atomic formula is also called a positive literal The negation of an atomic formula is called a negative literal

A clause is a disjunction of literals f

A special clause called the empty clause equivalent to false

A FOL formula is in conjunctive normal form (CNF) if it is a conjunction of disjunctions of literals

Equivalently, if it is a set of clauses
Knowledge Engineering

17

Conversion to CNF (1)

1. Eliminate implications and equivalences using the laws

(p q) (p q) (p q) ((p q) (q p)) ((p q) (q p))

2. 2 Move negations inwards using the laws

(p q) (p q) (p q) (p q) ( x: p(x)) ( x: p(x)) ( x: p(x)) ( x: p(x)) p p p

3. Rename variables so that each quantifier has a unique variable

( x: p(x))

( x: q(x)) ( x: p(x))
Knowledge Engineering

( y: q(y))
18

Conversion to CNF (2)

4. Eliminate existential quantifiers ()

If no universal quantifier () occurs before an existential quantifier (x), drop the existential quantifier and replace all occurrences of the existential quantifier variable x by a new constant (called a Skolem constant) E.g., ( x: p(A) q(x)) (p(A) q(B)) If universal quantifiers y1,, yn precede an existential q y, , y p quantifier (x), drop the existential quantifier and replace all occurrences of the existential quantifier variable x by the term f(y1, , yn) where f is a new function symbol (called a Skolem function) E.g., ( x, y: p(x) q(y)) ( x: p(x) q(f(x)) where f(x) is a Skolem function
Knowledge Engineering

19

Conversion to CNF (3)

5. Drop all universal quantifiers ()
E.g., E g ( x: p(x) q(f(x))) (p(x) q(f(x)))

6. Convert to CNF by using the distributive law of over

The result is a CNF with variables constants functions variables, constants, E.g., (p (q r)) (p q) (p r)

7. Flatten nested conjunctions or disjunctions. Then write

each disjunction on a separate line

Knowledge Engineering

20

Conversion to CNF Example (1)

Problem: Convert to CNF the following sentence: x: (y: p(x y) (y: q(x y) r(x y))) p(x, (y: q(x, r(x,
1. Eliminate implications

x: ((y: ( ( ( p(x, y)) (y: ( ( y) r(x, y)))) )) ( (q(x, ) ( ))))

2. Move inwards

x: ((y: p(x, y)) (y: (q(x, y) r(x, y))))

3. Rename variables

x: ((y: p(x, y)) (z: (q(x, z) r(x, z))))

Knowledge Engineering

21

Conversion to CNF Example (2)

4. Skolemize (i.e., eliminate existential quantifiers)

x: (p(x, f1(x)) (q(x f2(x)) r(x, f2(x)))) ( p(x (q(x, r(x

5. Drop universal quantifiers

p(x, f1( )) ( ( f2( )) r(x, f2( ))) ( (x)) (q(x, (x)) ( (x)))
6. Distribute over

(p(x, f1(x)) q(x, f2(x))) (p(x, f1(x)) r(x, f2(x)))

7. Write each disjunction on a separate line

p(x, f1(x)) q(x, f2(x)) p(x, f1(x)) r(x, f2(x))

Knowledge Engineering
22

Resolution-based proving Example

The crime example is described as follows: The law says that it is a crime for an American to sell weapons to hostile nations The country Nono, an enemy of America, has some y y missiles All of the country Nonos missiles were sold to it by Colonel West who is an American Use resolution to prove that West is a criminal!

Knowledge Engineering

23

Example: Formalization in FOL (1)

... it is a crime for an American to sell weapons to hostile nations nations
x,y,z: American(x) Weapon(y) Sells(x, y, z) Hostile(z) Criminal(x)

Nono ... has some missiles

x: Owns(Nono, x) Missile(x)

All of the country Nonos missiles were sold to it by Colonel West

x: Owns(Nono, x) Missile(x) Sells(West, x, Nono)

Knowledge Engineering

24

Example: Formalization in FOL (2)

Missiles are weapons
x: Missile(x) Weapon(x)

An enemy of America is a hostile nation

x: Enemy(x, America) Hostile(x) y( ) ( )

West, who is an American

American(West)

Nono, an enemy of America ...

Enemy(Nono, America)

To be proved that West is a criminal

Criminal(West)
Knowledge Engineering

25

Example: Convert to CNF form

American(x) Weapon(y) Sells(x, y, z) Hostile(z) Criminal(x) Owns(Nono, M1) Missile(M1) Missile(x) Owns(Nono, x) Sells(West, x, Nono) Missile(x) Weapon(x) Enemy(x, America) Hostile(x) American(West) ( ) Enemy(Nono, America) Criminal(West) ( )
Knowledge Engineering
26

Example: Proof

Knowledge Engineering

27

Horn normal form

A formula is in the Horn norm form if:
The formula is a conjunction () of clauses Each clause is a disjunction () of literals, in which at most 1 (possibly no!) positive literal E.g., (p q) (p r s)

The k Th knowledge base (KB) can b represented i th H l d b be t d in the Horn norm form
Rules (p ( p1 p2 pn q) p p Equivalent to the rule: (p1 p2 pn q) Facts p1, q Integrity constraints (p1 p2 pn) Equivalent to the rule: (p1 p2 pn false) ( )
Knowledge Engineering
28

Generalized Modus Ponens inference rule

(p1 p2 pn q), p1, p2, , pn q The application of the Generalized Modus Ponens inference rule requires th t all th sentences i KB must i f l i that ll the t in t be in the Horn norm form The Generalized Modus Ponens inference rule can be applied in both reasoning strategies Forward reasoning o a d easo g Backward reasoning

Knowledge Engineering

29

Forward reasoning
Problem: Given a set of premises (i.e., rules and facts) stored in the knowledge base KB, prove the conclusion Q Idea: Iterate the two following steps until the conclusion is derived Apply rules whose condition clause match KB For each matched rule, add its conclusion clause to KB ,

Knowledge Engineering

30

Forward reasoning Example (1)

Knowledge Engineering

31

Forward reasoning Example (2)

Knowledge Engineering

32

Forward reasoning Example (3)

Knowledge Engineering

33

Forward reasoning Example (4)

Knowledge Engineering

34

Forward reasoning Example (5)

Knowledge Engineering

35

Forward reasoning Example (6)

Knowledge Engineering

36

Forward reasoning Example (7)

Knowledge Engineering

37

Backward reasoning
Idea: the backward reasoning process starts from the conclusion Q To prove Q by using the set of premises (i.e., rules and facts) in KB
Check if Q has been proved (in KB) If not yet, prove all the conditions of a rule (in KB) whose conclusion clause is Q

Avoid loops
Check if the new derived clause has already been in the list of tobe-proved ones? If already, dont add it again!

Avoid to repeat the proof of a proved clause

The clause has already been proved true The clause has already been proved unsatisfiable (false) in KB
Knowledge Engineering
38

Backward reasoning Example (1)

Knowledge Engineering

39

Backward reasoning Example (2)

Knowledge Engineering

40

Backward reasoning Example (3)

Knowledge Engineering

41

Backward reasoning Example (4)

Knowledge Engineering

42

Backward reasoning Example (5)

Knowledge Engineering

43

Forward vs. Backward reasoning

Forward reasoning is a data-driven inference process
E.g., E g Object recognition routine decision recognition,

Forward reasoning may do some redundant work inferences that are irrelevant to the goal Backward reasoning is a goal-driven inference process, appropriate for problem-solving problem solving
E.g., How do I get into a PhD program?

Knowledge Engineering

44