Summary: Artificial Intelligence 1

Ramon Meffert, 2015

Introduction
This summary is based on Artificial Intelligence: A Modern Approach (3rd edition), as taught by Arnold Meijster
in the course Artificial Intelligence 1. The chapters included in this summary (as per Meijsters assignment)
are:
Chapters 1, 2 and 3 in their entirety;
Chapter 4, excluding 4.2 and 4.5;
Chapter 5, up to 5.4.3;
Chapter 6, excluding pages 225 and 226;
Chapters 7, 8, 9 and 13 in their entirety.

Contents
Contents
1

Introduction
1
What is AI? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1
Acting humanly: The Turing Test approach . . . . . . . . .
1.2
Thinking humanly: The cognitive modeling approach . . .
1.3
Thinking rationally: The laws of thought approach . . . .
1.4
Acting rationally: The rational agent approach . . . . . . . .
2
The Foundations of Artificial Intelligence . . . . . . . . . . . . . . .
2.1
Philosophy . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2
Mathematics . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3
Economics . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4
Neuroscience . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5
Psychology . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6
Computer engineering . . . . . . . . . . . . . . . . . . . . . .
2.7
Control theory and cybernetics . . . . . . . . . . . . . . . . .
2.8
Linguistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
The History of Artificial Intelligence . . . . . . . . . . . . . . . . . .
3.1
The gestation of artificial intelligence (1943-1955) . . . . . .
3.2
The birth of artificial intelligence (1956) . . . . . . . . . . . .
3.3
Early enthusiasm, great expectations (1952-1969) . . . . . .
3.4
A dose of reality (1966-1973) . . . . . . . . . . . . . . . . . .
3.5
Knowledge-based systems: The key to power? (1969-1979) .
3.6
AI becomes an industry (1980-present) . . . . . . . . . . . .
3.7
The return of neural networks (1986-present) . . . . . . . .
3.8
AI adopts the scientific method (1987-present) . . . . . . . .
3.9
The emergence of intelligent agents (1995-present) . . . . .
3.10 The availability of very large data sets (2001-present) . . . .
4
The State of the Art . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

3
3
3
3
3
3
3
3
4
4
4
4
4
5
5
5
5
5
5
5
6
6
6
6
6
6
6

Intelligent Agents
1
Agents and Environments . . . . . . . . . . . . . . . .
2
Good Behavior: The Concept of Rationality . . . . . .
2.1
Rationality . . . . . . . . . . . . . . . . . . . . .
2.2
Omniscience, learning, and autonomy . . . . .
3
The Nature of Environments . . . . . . . . . . . . . .
3.1
Properties of task environments . . . . . . . .
4
The Structure of Agents . . . . . . . . . . . . . . . . .
4.1
Agent Programs . . . . . . . . . . . . . . . . .
4.2
Simple reflex agents . . . . . . . . . . . . . . .
4.3
Model-based reflex agents . . . . . . . . . . . .
4.4
Goal-based agents . . . . . . . . . . . . . . . .
4.5
Utility-based agents . . . . . . . . . . . . . . .
4.6
Learning Agents . . . . . . . . . . . . . . . . .
4.7
How the components of agent programs work

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

7
7
7
7
7
7
8
8
8
8
8
9
9
9
9

Solving Problems by Searching

1
Problem-Solving Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1
Well-defined problems and solutions . . . . . . . . . . . . . . . . . . . .
1.2
Formulating problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
Example Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1
Toy problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
Searching for Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1
Infrastructure for search algorithms . . . . . . . . . . . . . . . . . . . . .
3.2
Measuring problem-solving performance . . . . . . . . . . . . . . . . . .
4
Uninformed Search Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1
Breadth-first search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2
Uniform-cost search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3
Depth-first search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4
Depth-limited search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5
Iterative deepening depth-first search . . . . . . . . . . . . . . . . . . . .
4.6
Bidirectional search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.7
Comparing uninformed search strategies . . . . . . . . . . . . . . . . . .
5
Informed (Heuristic) Search Strategies . . . . . . . . . . . . . . . . . . . . . . . .
5.1
Greedy best-first search . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2
A* search: Minimizing the total estimated solution cost . . . . . . . . .
5.3
Memory-bounded heuristic search . . . . . . . . . . . . . . . . . . . . . .
5.4
Learning to search better . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
Heuristic Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1
The effect of heuristic accuracy on performance . . . . . . . . . . . . . .
6.2
Generating admissible heuristics from relaxed problems . . . . . . . . .
6.3
Generating admissible heuristics from subproblems: Pattern databases
6.4
Learning heuristics from experience . . . . . . . . . . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

9
9
9
9
10
10
10
10
10
11
11
11
11
11
12
12
12
12
12
12
13
13
13
13
13
13
14

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

Overview of pseudocode for agent types

14

Overview of pseudocode for search algorithms

15

Glossary

18

Chapter 1: Introduction
Intelligence is trying to understand how we, as
humans, think; Artificial Intelligence is trying to
build entities that can think.

What is AI?

Intelligence can be classified based on two bases:

the way an agent acts and the way an agent thinks.
Either can be done humanly or rationally. As the
name implies, thinking or acting humanly means
that an agent acts or thinks in a way that is similar to humans. Rationality is an ideal measure of
performance, based on some objective ideal.

1.1

Acting humanly: The Turing Test approach

The Turing Test was designed to provide a satisfactory operational definition of intelligence, and is
defined as follows:
An intelligent machine must be able to:
communicate successfully (natural language
processing);
store what it knows or hears (knowledge representation);

1.3

Syllogisms provide patterns for argument structures that are always correct, i.e. Socrates is a
man; All men are mortal; Therefore, Socrates is
mortal. These laws of thought, nowadays referred
to as logic, was refined by 19th century logicians.
Through these refinements, it was possible to describe relations between objects in the world objectively. The results of these refinements is called the
logicist tradition; within artificial intelligence, it attempts to create intelligent systems based on these
premises.
1.4

adapt to new circumstances and to detect and

extrapolate patters (machine learning).
An extended version of the Turing Test, known as
the total Turing Test, includes a video signal and
the possibility for the interrogator to pass physical
objects the the machine. Therefore, to pass the total
Turing Test, a machine must also include computer
vision and robotics.

1.2

Thinking humanly: The cognitive modeling

approach

Another approach to creating intelligence is by

studying the human mind. Basically, there are three
ways to do this: introspection (observing ones own
thoughts); psychological experiments (observing a
person in action); and brain imaging (observing the
brain in action). These methods are part of cognitive science, an interdisciplinary field combining experimental psychological data and computer
models.

Acting rationally:
proach

The rational agent ap-

An agent is something that acts. A rational agent

acts to achieve the best outcome, or when there is
uncertainty, the best expected outcome. However,
since perfect rationality in complex systems is at
best infeasible, perfect rationality is a good starting
point for analysis. Limited Rationality is the implementation of rationality while keeping in mind
the computational demands, i.e. making sure the
computations take less time at the cost of some
quality of performance.

2
use this information to answer questions and
to draw new conclusions (automated reasoning);

Thinking rationally: The laws of thought

approach

2.1

The Foundations of Artificial Intelligence

Philosophy

Philosophy has supplied some valuable concepts

that are used in artificial intelligence. Many influential philosophers, such as Descartes, Aristotle
and Leibniz coined terms and theories that are still
used today. Rationalism, for example; the movement which advocates reasoning as the method for
understanding the world. Dualism is the theory
that part of the human mind/soul/spirit is outside
of (the laws of) nature. Materialism is the opposite,
arguing that the brains operation according to the
laws of physics constitutes the mind. The empiricism movement argues that everything can be understood by experiencing it. This is extended in the
idea of induction, i.e. general rules are acquired by
exposure to repeated associations between their elements. The doctrine of logical positivism holds
that all knowledge can be characterized by logical theories connected, ultimately, to observation
sentences that correspond to sensory inputs. The
Confirmation Theory attempts to analyze the acquisition of knowledge from experience by defining
an explicit computational procedure for extracting
knowledge from elementary experiences.

2.2

Mathematics

In order for Artificial Intelligence to become a formal science, the theories founded by the logicians
required a level of mathematical formalization in
thee fundamental areas: logic, computation, and
probability. At the basis of this lies the concept
of the algorithm; a series of operations based on
logic and mathematics to find a specific sort of output based on a specific sort of input. As Gdel
has shown, limits on deduction exist: his incompleteness theorem showed that in any formal theory as strong as the elementary theory of natural
numbers (Peano arithmetics), there are true statements that are undecidable in the sense that they
have no proof within the theory. This also means
that not everything is computable; they cannot be
represented by algorithms. Even though it is impossible to give a formal definition to this notion,
it is generally accepted that any function that is
computable by the Turing Machine is by extension
also computable in general (Church-Turing thesis).
Tractability is another important concept, roughly
stating that a problem must not take an exponential amount of time compared to the amount of instances given as input. This means that a problem
is intractable if this is the case. To recognize an intractable problem, the theory of NP-completeness
provides a method. Any problem class to which
the class of NP-complete problems can be reduced
is likely to be intractable. Probability also plays
a big role in AI, outlining the chances of a certain
outcome for an event. Bayes Rule is an instance of
a probability rule that plays a large role in AI.
2.3

Economics

Several important notions originate in the field of

Economic science. For example, utility deals with
the mathematics of preferred outcomes. Decision
theory combines probability theory and utility theory and provides a formal and complete framework
for decisions made under uncertainty. This is only
applicable in large economies where the actions
of individuals do not count significantly. If they do
count, the domain of game theory applies: individual actions have great consequences, and rational
agents should adopt policies that are (or at least
appear to be) randomized. Operations research
deals with the situations in which a payoff is not
immediately clear, but only arises after a sequence
of actions. A specific class of formalized decision
problems is called Markov decision processes. For
many applications, satisficingmaking decisions
that are good enough, rather than calculating the
optimal decisionhave a better description of actual human behavior.

2.4

Neuroscience

Neuroscience is the study of nervous systems, in

particular the brain. Although the exact workings
of the brain are still unclear, the fact that it facilitates thought has been clear for thousands of year,
as damage to the head leads to great mental incapacitation. The brain consists of neurons, simple
nerve cells which, in their large numbers, can lead
to thought, action and consciousness.
Brains cause minds. John Searle
Comparing the number of nerve cells to some
statistical numbers of processing units in modern
units leads some people, futurists, to believe that a
singularity is approaching: at this point, computers reach a superhuman level of performance.
2.5

Psychology

Psychology concerns itself with the understanding

of the workings of the human mind. As one way
of creating intelligence is through mimicking human intelligence, this field of study is an important
influence on AI. An initiating movement within
psychology was behaviorism: this movement rejected any theory involving mental processes on the
grounds that introspection could not provide reliable evidence. Instead, they focused on objective
measures of of percepts leading to responses. This
worked quite well for rats and pigeons, but not so
much for humans.
Cognitive psychology, which focuses on the
brain as an information-processing device.
It
reestablished the legitimacy of mental terms such
as beliefs and goals, arguing that they are just
as scientific as any other terms. Kenneth Craik
specified the three key steps of a knowledge-based
agent: (1) translating the stimulus to an internal
representation; (2) manipulating of the representation by cognitive processes; (3) translating these
into action.
2.6

Computer engineering

Computer engineering facilitates the very existence

of artificial intelligence. The modern computer
was invented thrice, almost independently and almost simultaneously. In World War II, many advances were made in computing technology, resulting in the firs operational, the first operational programmable and the first electronic computer in consequence. The first programmable machine was a
loom that used punch cards to store patters to be
woven. The software side of computing science has
greatly facilitated AI as well, but AI has since made
some significant contributions to software, such as
the linked list data type.

2.7

Control theory and cybernetics

Control theory entails the procedure in which an

artifact (a non-living object in the real world) controls itself to maintain a stable state. Advances in
mathematical and computational models of cognition produced a spike in interest in cybernetics,
highlighting the possibility of artificially intelligent
machines. The idea that a homeostatic device (i.e.
a device maintaining a internal balance through
appropriate feedback loops) could achieve stable
adaptive behavior was coined by W. Ross Ashby.
Modern control theory has as its goal the design of
systems that maximize an objective function over
time.
2.8

Linguistics

In 1957, B.F. Skinner published Verbal Behavior, a

book explaining language using behaviorist science. Noam Chomsky, having just published his
own theory in Syntactic Structures, wrote a review
criticizing Skinners work for not recognizing the
notion of creativity in language, which, for example, would explain how a child could understand
and produce sentences it had not heard before. Because Chomskys theories were so rigid, it provided
the necessary formality for the creation of the field
of computational linguistics or natural language
processing. This has made it possible for computers to understand language.

3
3.1

The History of Artificial Intelligence

The gestation of artificial intelligence (19431955)

Hebbian learning is a system of learning based

on work done by Warren McCulloch and Walter
Pitts, who created a computational network based
on neurons, drawing from the basic physiology of
the brain, a formal analysis of propositional logic
due to Russell and Whitehead and Turings theory of computation. It was a primitive neural network capable of computing simple formulas and
introduced weighted edges. The first real neural
network was implemented in 1950; it was called
SNARC.
3.2

The birth of artificial intelligence (1956)

A meeting of several influential people in AI occurred when a two-month workshop was organized
in the summer of 1956. In this workshop, the Logic
Theorist, created by Simon and Newell, stole the
show by being a program that was able to reason,
proving many theorems in chapter 2 of Russell and
Whiteheads Principia Mathematica. This workshop

shows why AI has become a separate field; contrary

to many fields that aim to pursue similar objectives,
AI embraces the idea of duplicating human faculties such as creativity, self-improvement and language use.
3.3

Early enthusiasm, great expectations (19521969)

Compared to the capabilities of computers at the

time, the early years of AI were quite fascinating.
Seeing as most these machines could do was some
simple arithmetic, anything remotely clever was
received as an amazing feat of technology. Even
though the programs were still very basic, Newell
and Simon were able to formulate the physical
symbol system hypothesis, stating that a physical symbol system has the necessary and sufficient
means for general intelligent action. A number
of AI programs were created, and the high-level
programming language LISP was created, which
remained the most widely-used programming language in AI for the next 30 years.
A certain set of programs worked on so-called
microworlds. These were limited domains on
which operations could be done, such as solving close-form calculus integration problems. A
famous example of a microworld program is
SHRDLU, which was able to reason about a spatial environment and perform some simple actions
on it.
Hebbs learning methods were enhanced by
Bernie Widrow, who called his networks adalines, and by Frank Rosenblatt with is perceptrons. The perceptron convergence theorem says
that the learning algorithm can adjust the connection strengths of a perceptron to match any input
data, provided such a match exists.
3.4

A dose of reality (1966-1973)

Following the enthusiasm and successes of early

AI, some very hopeful predictions were made
about what AI would be able to do in the near
future. These predictions, however, turned out to
mostly come true, but not quite within the timeframe imagined; it took, for example, 40 years instead of 10 to create a chess champion program and
have a significant mathematical theorem be solved
by a computer. Another telling example of the high
expectations of AI clearly showed when a computer
program meant to translate Russian documents to
English retranslated The spirit is willing but the
flesh is weak as The vodka is good but the meat
is rotten. It also became clear that many methods used before were actually relatively intractable,
meaning that they did not scale well. For exam-

ple, the microworlds mentioned earlier worked because they contained very few objects, so it was
feasible to try every combination until eventually
an answer was found. It became clear that The
fact that a program can find a solution in principle does
not mean that the program contains any of the mechanisms needed to find int in practice. Early experiments in machine evolution (now called genetic
algorithms) were based on the undoubtedly correct belief that by making an appropriate series of
small mutations to a machine code program, one
can generate a program with good performance for
any particular task. This proved to be unsuccessful, as after many hours of computation time, almost no progress was shown. Modern versions of
genetic algorithms show better results (i.e. they actually work.) It was also shown that, even though
perceptrons could be shown to learn anything they
were capable of representing, they could represent
very little. Modern-day multi-layer networks actually are capable of doing this, and nowadays they
work quite well. This is ironic, because this type of
neural networks was actually invented in 1969.
3.5

Knowledge-based systems:
power? (1969-1979)

The key to

The early interpretations of the workings of AI envisioned a general-purpose search mechanism that
would find any complete solutions. These were
called weak methods, because they do not scale
well. Logically, the next step in developing AI programs were the so-called expert systems, which
had a vast knowledge base, and were intended to
solve very specific problems. An example of this is
DENDRAL, which was a program used to derive
the molecular formula based on mass spectrometer
data. A derivate of DENDRAL was MYCIN, which
was used to diagnose blood infections. It used a calculus of uncertainty called certainty factors. Several programming languages were developed following the rise in popularity of AI programs. Prolog and PLANNER were popular, while others followed a more structured approach, following Minskys idea of frames, assembling facts about particular object and event types and arranging the types
into a large taxonomic hierarchy analogous to a biological taxonomy.

ten aboutuntil the mid-1980s, when at least four

groups reinvented the back-propagation learning
algorithm they found. These models were called
connectionist models were seen as direct competitors both to the symbolic models promoted by
Newell and Simon and the logicist approach.
3.8

AI adopts the scientific method (1987present)

Contrary to the early days of AI, most developments nowadays are very neatthey are based
upon earlier theorems, models and knowledge. It
relies heavily on mathematical rigor and statistical
analysis. An example of this is found in Hidden
Markov Models, which have a solid mathematical basis, and they are generated by a process of
training on a large corpus of real speech data. The
fact that speech technology as well as handwriting recognition technology are entering the market shows that these systems are not only becoming
better but also are applicable in everyday life.
Another upcoming technology is data mining,
a probabilistic approach to neural networks. The
Bayesian network formalism was inverted to allowefficient representation of, and rigorous reasoning with, uncertain knowledge.
3.9

The emergence of intelligent agents (1995present)

After the encouraging successes of some AI programs, people started to reconsider the idea of the
whole-agent problem. Due to the growing success and influence of the internet, bots became relatively common. Some people disagree with the
notion that these kind of bots are what AI should
encompass; they reason that AI should have more
human-like intelligence, i.e. creating a human-level
AI or HLAI. A related idea is the subfield of Artificial General Intelligence or AGI, which looks for a
universal algorithm for learning and acting in any
environment. Guaranteeing that what we create is
really Friendly AI is also a concern. (#RobotOverlords)
3.10

The availability of very large data sets

(2001-present)

AI became an industry and people were earning

money doing things with it. yay.

Instead of focusing on the algorithm, it was shown

that it is sometimes more important to look at the
data to be processed than to be picky about the algorithm.

3.7

3.6

AI becomes an industry (1980-present)

The return of neural networks (1986-present)

As mentioned before, the modern version of neural

networks was actually invented in 1969, but forgot-

The State of the Art

AI can do quite a few impressive things. Look in

the book for some examples; they do not seem to

For each possible percept sequence, a rational agent should select an action that is
expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.

be important for the exam.

Chapter 2: Intelligent Agents

1

Agents and Environments

An agent is anything that can be viewed as perceiving its environment through sensors and acting
upon that environment through actuators. Percepts
are the agents perceptual inputs at any given instant. An agents percept sequence is the complete
history of everything the agent has ever perceived.
The agent function is the function that maps any
given percept sequence to an action. This agent
function is achieved through the use of the agent
program; it is a concrete implementation, whereas
the agent function is the mathematical representation. It is also possible to represent the agent program using a table. However, for most programs,
this table is virtually infinite.

Good Behavior: The Concept of Rationality

A rational agent is an agent that does the right

thingconceptually speaking, every entry in the
table for the agent function is filled out correctly.
However, doing the right thing is a very vague
concept. This is answered by considering the consequences of the agents actions. The actions modify the environment states, and when the sequence
of states is desirable, the agent has performed well.
The notion of desirability is capture by a performance measure that evaluates any given sequence
of environment states. It is better to design performance measures according to what one actually
wants in the environment, rather than according to
how one thinks the agent should behave.
2.1

Rationality

The definition of a rational agent is as follows:

2.2

Omniscience, learning, and autonomy

Keep in mind that rationality does not entail omniscience. Whereas an omniscient agent could predict its environment perfectly, a rational agent can
only make reasonable assumptions. Rationality
maximizes expected performance, while perfection
maximizes actual performance. Hence, information gatheringdoing actions in order to modify
future perceptsis an important part of rationality.
Exploration is also an important form of information gathering. Besides gathering information, it
is also important that an agent learns as much as
possible from what is perceives. The prior knowledge is, generally, simply not enough to perform
well. The extent to which the agent relies on its
prior knowledge rather than on its own percepts,
we say that the agent lacks autonomy. A rational
agent should be autonomous and effectively independent.

The Nature of Environments

The task environment is the environment in which

a task is to be executed. This can be defined using
the PEAS description (Performance, Environment,
Actuators, Sensors). The performance measure describes the goals for the agent; the environment describes the kind of environment the agent will be
acting in; the actuators describes the available actuators the agent has, i.e. in what ways the agent can
modify its environment; the sensors describes the
sensors that are available to the agent, i.e. the way
it can receive input.
In contrast, some software agents or softbots
exist in rich, unlimited domains.

3.1

Properties of task environments

Although there is a vast number of task environments that might arise in AI, they can be categorized along
a fairly small number of dimensions:

fully observable
partially observable
unobservable
single agent
multi-agent
competitive
cooperative
deterministic
stochastic
uncertain
nondeterministic
episodic

sequential
static
dynamic
semi-dynamic
discrete
continuous
known
unknown

an agents sensors give it access to the complete (relevant) state of the

environment at each point in time
an agents sensors might be noisy, or part of the environment might simply not be visible to the agents sensors
the agent has no sensors
the agent is the only agent to be taken into consideration
the agent might not be the only agent in the environment, but this depends on the definition of agent
the performance measure of another agent decreases the performance
measure of the own agent
the agents share a (somewhat) common performance measure
the next state of the environment is completely determined by the current
state and the action executed by the agent
not deterministic
an environment is uncertain if it is not fully observable or not fully deterministic
an environment is nondeterministic if actions are characterized by their
possible outcomes, but no probabilities are attached to them
in an atomic task environment, the agents experience is divided into
atomic episodes, in which it receives a percept and then performs a single
action, which does not depend on the actions taken in previous episodes.
the current decision could affect all future decisions.
the environment does not change while the agent is deliberating
it does
the environment does not change, but the performance score does
finite number of distinct states and set of percepts and actions
infinite number of distinct states or set of percepts or actions
the agents knowledge of all details of the environment represent the entire environment
it does not

The Structure of Agents

The agent program requires some kind of architecture to run on; thus, the agent is comprised of the
combination of the agent program and the architecture.

4.1

Agent Programs

There are multiple types of agent programs, some

more effective than others. A relatively naive approach is the table-drive agent: it simply stores every percept with a corresponding action. As one
might expect, these are, for most tasks, rather infeasible; an automated taxi would generate a lookup
table with over 10250,000,000,000 entries.

4.2

Simple reflex agents

Simple reflex agents these agents select actions on

the basis of the current percept, ignoring the rest
of the percept history. They work using conditionaction rules, written as if X then Y. Simple reflex
agents might run into infinite loops; one solution
to this problem is to randomize (certain aspects of)
the agents actions.
4.3

Model-based reflex agents

A method of handling partial observability is the

maintenance of some internal state that depends
on the percept history and thereby reflects at least
some of the unobserved aspects of the current state.
In order for this to work, the agent needs some kind
of model of the world. This can be very simple (e.g.
Boolean circuits) or very complicated (e.g. complete scientific theories). An agent that uses such

a model is called a model-based agent.

4.4

Goal-based agents

A goal is necessary for making decisions; even

when an agent knows something about its environment, making a choice is arbitrary unless there is a
goal. The subfields search and planning focus on
these kinds of problems.
4.5

Utility-based agents

While goals provide direction and often a path to

the solution, they do not necessarily generate the
best solution. A way to solve this is by introducing
utility, which assigns values to different options
it is basically trying to optimize the utility function. This can bee seen in analogy with a taxi driver
and their client; the better the route, the happier
the client. Because agents generally cannot be omniscient, the expected utility is used.
4.6

Learning Agents

A learning agent relies on four basic principles in

order to operate: a learning element, a performance element, a critic and a problem generator.
The learning element is responsible for making improvements to the performance element by taking
input from the critic. The problem generator tries
to find new and informative experiences, as to maximize the former elements. A reward or penalty is
used as a form of direct feedback on the quality of
the agents behavior.
4.7

How the components of agent programs

work

There are roughly three ways of representing the

environment that the agent inhabits. Ordered
on terms of increasing complexity and expressive power (expressiveness): atomic, factored and
structured. In an atomic representation, each state
of the world is indivisibleit has no internal structure. A factored representation splits up each state
into a fixed set of variables or attributes, each of
which can have a value. Structured representation
allow for explicit descriptions of relationships between objects.

Chapter 3: Solving Problems by

Searching
Problem-solving agents are a kind of goal-based
agents, using an atomic representation. Goal-based

agents that use a factored or structured representation are usually called planning agents. This chapters aims to precisely define problems and their
solutions. Uninformed and informed algorithms
are consideredthe first being algorithms that are
given no information about the problem other than
its definition, and the latter require some guidance.

Problem-Solving Agents

Goals help organize behavior by limiting the objectives that the agent is trying to achieve and hence
the actions it needs to consider. Goal formulation
is based on the current situation and the agents
performance measure. Problem formulation is the
process of deciding what actions and states to consider, given a goal. An agent with several immediate options of unknown value can decide what to
do by fist examining future actions that eventually
lead to to states of known value.
For problem-solving agents, we assume that the
environment is observable, discrete, known and deterministic. The process of looking for a sequence
of actions that reaches the goal is called search. The
intended output is the solution. Once a solution is
found, the execution phase is entered. The agent,
while executing the solution, ignores the percepts
as they are known in advance. This is called an
open-loop system.
1.1

Well-defined problems and solutions

A problem can be defined by five components: the

initial state; the possible actions that are applicable in a state s; the transition model, i.e. a description of what each action does; the goal test to check
whether the goal has been reached; and the path
cost function that assigns a numeric cost to each
path. Any applicable state is a successor. Together,
the initial state, actions and transition model implicitly define the state space of the problem, which
forms a directed network or graph. A path in the
state space is a sequence of states connected by a
sequence of actions. The step cost of taking action
a in state s to reach state s is denoted by c(s, a, s0 ).
A solution to a problem is an action sequence that
leads from the initial state to a goal state. an optimal solution has the lowest path cost among all
solutions.
1.2

Formulating problems

The process of removing detail from a representation is called abstraction. The abstraction is valid if
we can expand any abstract solution into a solution
in the more detailed world.

Example Problems

A distinction is made between toy problems and

real-world problems. A toy problem is intended to
illustrate or exercise various problem-solving methods. A real-world problem is one whose solutions
people actually care about.1
2.1

Child nodes are generated using the function

child-node.
..............
The appropriate data structure for nodes is a
queue. The operations on a queue are as follows:
empty(queue): returns true only if there are no
mere elements in the queue;
pop(queue): removes the first element of the
queue and returns it;

Toy problems

Toy problems have either an incremental formulation or a complete-state formulation.

Searching for Solutions

To represent a solution to a problem, a search tree

is used. At the root is the initial state, the branches
are actions and the nodes correspond to states in
the state space of the problem. The generation of
new states from the current state is called expansion: The parent node generates some child nodes.
Leaf nodes are nodes with no children. The set of
all leaf nodes available for expansion at any given
point is called the frontier (sometimes referred to
as open list). The way an algorithm moves through
the tree is called the search strategy. A state can be
a repeated state when it occurs multiple times, e.g.
because of a loopy path. Loopy paths are a special case of the more general concept of redundant
paths, which exist whenever there is more than one
way to get from one state to another. In games
(but not exclusively in games), rectangular grids
are used. To avoid visiting states twice, the treesearch algorithm is augmented with the explored
set (also known as the closed list), becoming the
graph-search algorithm. graph-search separates
the state-space graph into the explored region and
the unexplored region.
3.1

Infrastructure for search algorithms

Search algorithms require a data structure to keep

track of the search tree that is being constructed.
For each node n of the tree, we have a structure
that contains four components:
n.state: the state in the space to which the
node corresponds;

insert(element, queue): inserts an element and

returns the resulting queue.
Queues are characterized by the order in which
they store the inserted nodes. Three comments are
the first-in, first-out or FIFO queue, which pops the
oldest element; the last-in first-out or LIFO queue
(also known as stack), which pops the newest element of the queue; and the priority queue, which
pops the element of the queue with the highest priority according to some ordering function. To allow for efficient checking of repeated states, it is
possible to use a hash table (which remembers the
visited states) or by using the more general method
of storing repeated states in canonical form, which
means that logically equivalent states should map
to the same data structure.
3.2

Measuring problem-solving performance

An algorithms performance can be evaluated in

four ways:
completeness: does the algorithm always
find a solution if one exists?
optimality: does the algorithm find the optimal solution (lowest path cost)?
time complexity: how long does it take to
find a solution?
space complexity: how much memory is
needed to perform the search?
In AI, complexity is expressed in terms of three
quantities:
b, the branching factor or maximum number
of successors of any node;
d, the depth of the shallowest goal node;

n.parent: the node in the search tree that generated this node;

m, the maximum length of any path in the

state space.

n.action: the action that was applied to the

parent to generate the node;

Time is measured in number of nodes generated,

memory in the number of nodes stored in memory.
Specifically for search algorithms, either search
cost is used, which typically depends on time complexity, or total cost, which combines the search
cost and the path cost, to evaluate effectiveness.

n.path-cost: the cost, traditionally denoted

by g(n), of the path from the initial state to
the node, as indicated by the parent pointers.
1 This

is a direct quote. I dont they intend to imply toy problems have useless outcomes...

Uninformed Search Strategies

Uninformed search or blind search are searches

that are not provided with any information besides the information in the problem definition.
This in contrast with informed search or heuristic search, which know whether a non-goal state is
more promising than another.

4.1

Breadth-first search

4.3

Depth-first search

Depth-first search always expands the deepest

node in the current frontier of the search tree, i.e.
it gets deeper in the tree until it finds a leaf node.
It uses a LIFO queue: the most recently generated
node is chosen for expansion. Its implementation
is basically the same as the graph-search algorithm. Alternatively, it can be implemented recursively. The properties differ for the version used.
For graph-search:

Breadth-first search is a search algorithm in which

every node on a level is expanded before the nodes
on the next level are expanded, starting at the root
note. It uses a FIFO queue. As for its performance:

it is complete in finite spaces;

it is nonoptimal;

it is complete;

its time complexity is bound by the size of the

state space (can be infinite);

it is not necessarily optimal, unless the cost is

a nondecreasing function of the depth of the
node;

its space complexity is the same as breadth-first

search.
for tree-search (the recursive implementation):

its time complexity is poor: O(bd );

it is not complete;
its space complexity is similarly poor: O(bd ).

it is nonoptimal;

This shows that exponential-complexity search

problems cannot be solved by uninformed methods
for any but the smallest instances.

4.2

Uniform-cost search

Uniform-cost search is basically breadth-first

search, but instead of expanding the shallowest
nodes, it expands the node with the lowest path
cost. Another difference is that the goal test is applied to a node when it is selected for expansion
rather than when it is first generated. The last difference is that a test case is added in case a better
path is found to a node currently on the frontier.
As for its performance:
it is complete, as long as step cost > e;
it is generally optimal;
its time complexity is O(b1+(C/e) ), which can
be greater than bd if the step costs vary, but
otherwise is equal to O(bd+1 ), cf. breadthfirst search;

its time complexity is O(bm )

its space complexity is O(b m)
A variant of depth-first search is backtracking
search, which uses even less memory: only one
successor is generated at a time instead of all successors. This means that only O(m) memory is
needed, and because it modifies rather than copies
the current state description, it only requires one
state space.
4.4

Depth-limited search

As depth-first search fails unpreventably in infinite

state spaces, it is useful to have a depth limit l, even
though this introduces even more incompleteness.
It is not optimal if l 6= d (with d being the depth
limit). As for its performance:
it is incomplete, as is depth-first search;
it is nonoptimal unless l = d;
its time complexity is O(bl );

its space complexity is the same as its time

complexity.
Uniform-cost search expands more nodes in order
of their optimal path cost.

its space complexity is O(b l )

Sometimes, it is possible to determine the depth
limit from some information.

4.5

4.6

Iterative deepening depth-first search

Iterative deepening search or Iterative deepening

depth-first search is a general strategy, often used in
combination with depth-first tree search that finds
the best depth limit. As for its performance:

Bidirectional search is basically a search with two

root nodes, motivated by the fact that bd/2 + bd/2 
bd . A solution is found when the frontiers intersect.
As for its performance:
it is complete when the branching factor is finite;

it is complete when the branching factor is finite;

it is optimal when the cost is a nondecreasing

function of the depth of the node;

it is optimal when the cost is a nondecreasing

function of the depth of the node;

its time complexity is O(bd/2 );

its time complexity is O(bd );

its space complexity is O(bd/2 ) (can be halved

by using iterative deepening).

its space complexity is O(b d)

In general, iterative deepening is the preferred uninformed search method when the each space is
large and the depth of the solution is not known. Iterative lengthening works on the same principles,
except for that it uses increasing path-cost limits instead of increasing depth limits. It is not effective.
4.7

BFS
Yes
O(bd )
O(bd )
Yes

Uniform-cost
Yes
O(b1+(C/e) )
O(b1+(C/e) )
Yes

DFS
No
O(bm )
O(bm)
No

Informed (Heuristic) Search Strategies

Informed search strategies use problem-specific

knowledge beyond its definition. They find solutions more efficiently than uninformed search
strategies. The general approach for informed
search is called best-first search, i.e. graph-search
or tree-search in which a node is selected for expansion based on an evaluation function f (n), a
cost estimate. A component of this function is h(n),
a heuristic function. h(n) = the estimated cost of
the cheapest path from the state at node n to a goal
state. If n is a goal node, h(n) = 0.

5.1

As bi-directional search requires searching backward. To accomplish this, the predecessors of a

state x must be all those states that have x as a
successor. When all actions in the state space are
reversible, the predecessors of x are just its successors.

Comparing uninformed search strategies

Complete
Time Complexity
Space Complexity
Optimal

Bidirectional search

Greedy best-first search

Greedy best-first search tries to expand the node

that is closest to the goal, on the grounds that
this is likely to lead to a solution quickly. Thus,
f (n) = h(n). This can be used with the straightline distance heuristic. It is called greedy because
at every step, it tries to get as close to the goal as it
can.

Depth-limited
No
O(bl )
O(bl )
No
5.2

Iterative Deepening
Yes
O(bd )
O(bd)
Yes

Bidirectional
Yes
O(bd/2 )
O(bd/2 )
Yes

A* search: Minimizing the total estimated

solution cost

A* search evaluates nodes by combining the cost to

reach the node and the cost to get from the node
to the goal: f (n) = g(n) + h(n): the estimated cost
of the cheapest solution through n. A* search is
complete and optimal.
Conditions for optimality: Admissibility and consistency A heuristic is an admissible heuristic if
the heuristic never overestimates tho cost to reach
the goal. These are by their nature optimistic because they think the cost of solving the problem
is less than it actually is. Straight-line distance
is an admissible heuristic. A heuristic is consistent or monotonous if, for every node n and every successor n of n generated by any action a,
the estimated cost of reaching the goal from n is
no greater than the step cost of getting to n plus
the estimated cost of reaching the goal from n:
h(n) c(n, a, n0 ) + h(n0 ). This is a form of the general triangle inequality, which stipulates that each
side of a tringale cannot be longer than the sum of

the other two side. Here, the triangle is formed by

n, n and Gn closest to n.
Optimality of A* The tree-search version of A*
is optimal if h(n) is admissible, while the graphsearch version is optimal if h(n) is consistent. If
h(n) is consistent, then the values of f (n) along
any path are nondecreasing. Whenever A* selects
a node for expansion, the optimal path has been
found. Whenever the costs are nondecreasing, it
is possible to draw contours like in a topographic
map, as any node next to another is either higher
or equal to the current node. Pruning occurs when,
because the heuristic is admissible, subtrees can be
ignored. Eliminating possibilities from consideration without having to examine them is important in AI. A* is optimally efficient for any given
consistent heuristic: no other optimal algorithm is
guaranteed to expand fewer nodes than A*, because
any algorithm that does not expand all nodes with
f (n) < C runs the risk of missing the optimal solution. The absolute error of the heuristic is defined
as h h, where h os the actual cost of getting
from the root to the relative error is the defined as
e (h h)/h.
5.3

Memory-bounded heuristic search

To reduce memory requirements for A*, the simplest way is to adapt the idea of iterative deepening the the heuristic search context, resulting in
the iterative-deepening A* (IDA*) algorithm. The
main difference between IDA* and standard iterative deepening is that the cutoff used is the f -cost
rather than the depth. Recursive best-first search is
a simple recursive algorithm that attempts to operation of standard best-first search, but using only
linear space. It keeps track of the best alternative path available from any ancestor of the current node. RBFS replaces the f-value of each node
along the path with a backed-up valuethe best
f-value of its children. In this way, RBS remembers
the f -value of the best leaf in the forgotten subtree and can therefore decide whether its worth reexpanding the subtree at some later time. It is sensible to have A* use all available memory. Two algorithms that do this are MA* (memory-bounded A*)
and SMA* (simplified MA*). SMA* performs A*
until memory is full, and then drops the worst leaf
node (i.e. the one with the highest f -value.) The
forgotten value is backed up in its parent. The complete algorithm is too complicated to reproduce,
but a subtlety worth mentioning is that SMA* expands the newest best leaf and deletes the oldest
worst leaf. SMA* is complete if there is any reachable solution, and it is optimal if any optimal solution is reachable. A problem for SMA* is that

memory limitations can make a problem intractable

from the point of view of computation time.

5.4

Learning to search better

Meta-level state space allows an agent to learn how

to search better. Each state in a meta-level state
space captures the internal state of a program that
is searching in an object-level state space. Metalevel learning captures mistakes made by algorithms, and attempts to avoid exploring unpromising subtrees. The goal of learning is to minimize
the total cost of problem solving.

Heuristic Functions

The Manhattan distance or city block distance is

the sum of distances from tiles on a rectangular grid
from one point to another.

6.1

The effect of heuristic accuracy on performance

One way to characterize the quality of a heuristic

is the effective branching factor b*. If the total
number of nodes generated by A* for a particular problem N and the solution depth is d, then b*
is the branching factor that a uniform tree would
have to have in order to contain N + 1 nodes. Thus:
N + 1 = 1 + b +(b)2 + + (b)d .
A heuristic dominates another heuristic if one
heuristic is always better than another, that is, for
any node n: h2 (n) > h1 (n).
6.2

Generating admissible heuristics from relaxed problems

A problem with fewer restrictions on the actions

is called a relaxed problem. The cost of an optimal solution to a relaxed problem is an admissible
heuristic for the original problem. It is also by default consistent

6.3

Generating admissible heuristics from subproblems: Pattern databases

Admissible heuristics can also be derived from the

solution cost of a subproblem of a given problem.
The cost for a solution to a subproblem is a lower
bound on the cost of the complete problem. The
idea behind pattern databases is to store the exact solution costs for every possible subproblem
instance. Disjoint pattern databases are pattern
databases that have no overlapping entries.

6.4

Learning heuristics from experience

It is possible (with luck) for a learning agent to

construct an admissible heuristic using subprob-

lems. However, inductive learning methods work

best when supplied with features of a state a that
are relevant to predicting the states value, rather
than with just the raw state description.

Overview of pseudocode for agent types

Table-driven agent
function table-driven-agent(percept) returns an action
persistent: percepts, a sequence, initially empty
table, a table of actions, indexed by percept sequences, initially fully specified
append percept to the end of percepts
action lookup(percepts, table)
return action
Simple reflex agent
function simple-reflex-agent(percept)
persistent: rules, a set of condition-action rules
state interpret-input(percept)
rule rule-match(state, rules)
action rule.action
return action
Model-based reflex agent
function model-based-reflex-agent(percept) returns an action
persistent: state, the agents current conception of the world state
model, a description of how the next state depends on current state and action
rules, a set of condition-action rules
action, the most recent action, initially none
state update-state(state, action, percept, model)
rule rule-match(state, rules)
action rule.action
return action
Model-based reflex agent
function model-based-reflex-agent(percept) returns an action
persistent: seq, an action sequence, initially empty
state, some description of the current world state
goal, a goal, initially null
problem, a problem formulation
state update-state(state, percept)
if seq is empty then
goal formulate-goal(state)
problem formulate-problem(state, goal)
seq search(problem)
action first(seq)

seq rest(seq)
return action
Simple problem-solving agent
function simple-problem-solving-agent(percept) returns an action
persistent: seq, an action sequence, initially empty
state, some description of the current world state
goal, a goal, initially null
problem, a problem formulation
state update-state(state, percept)
if seq is empty then
goal formulate-goal(state)
problem formulate-problem(state, goal)
seq search(problem)
if seq = failure then return a null action
action first(seq)
seq rest(seq)
return action

Overview of pseudocode for search algorithms

Tree search
function tree-search(percept) returns a solution, or failure
initialize the frontier using the initial state of problem
loop do
if the frontier is empty then return failure
choose a leaf node and remove it from the frontier
if the leaf node contains a goal state then return the corresponding solution
expand the chosen node, adding the resulting nodes to the frontier

Graph search or depth-limited graph search

function graph-search(percept) returns a solution, or failure
initialize the frontier using the initial state of problem
initialize the explored set to be empty
loop do
if the frontier is empty then return failure
choose a leaf node and remove it from the frontier
if the leaf node contains a goal state then return the corresponding solution
add the node to the explored set
expand the chosen node, adding the resulting nodes to the frontier
add the node to the explored set
only if not in the frontier or explored set
The child node function
function child-node(problem, parent, action) returns a node
return a node with
state = problem.result(parent.state, action),

parent = parent, action = action,

path-cost = parent.path-cost + problem.step-cost(parent.state, action)
Breadth-first search
function breadth-first-search(problem) returns a solution, or failure
node a node with state = problem.initial-state, path-cost = 0
if problem.goal-test(node.state) then return solution(node)
frontier a FIFO queue with node as the only element
explored an empty set
loop do
if empty(frontier) then return failure
node pop(frontier) /*chooses the shallowest node in frontier*/
add node.state to explored
for each action in problem.actions(node.state) do
child child-node(problem, node, action)
if child.state is not in explored or frontier then
if problem.goal-test(child.state) then return solution(child)
frontier insert(child, frontier)
Uniform-cost search
function uniform-cost-search(problem) returns a solution, or failure
node a node with state = problem.initial-state, path-cost = 0
frontier a priority queue ordered by path-cost, with node as the only element
explored an empty set
loop do
if empty(frontier) then return failure
node pop(frontier) /*chooses lowest-cost node in frontier*/
if problem.goal-test(node.state) then return solution(node)
add node.state to explored
for each action in problem.actions(node.state) do
child child-node(problem, node, action)
if child.state is not in explored or frontier then
frontier insert(child, frontier)
else if child.state is in frontier with higher path-cost then
replace that frontier node with child
Depth-limited tree search
function depth-limited search(problem, limit) returns a solution, of failure/cutoff
return recursive-dls(make-node(problem.initial-state), problem, limit)
function recursive-dls(node, problem, limit) returns a solution, or failure/cutoff
if problem.goal-test(node.state) then return solution(node)
else if limit = 0 then return cutoff
else
cutoff-occured false
for each action in problem.actions(node.state) do
child child-node(problem, node, action)
result recursive-dls(child, problem, limit - 1)
if result = cutoff then cutoff-occured gets true
else if results 6= failure then return results
if cutoff-occured then return cutoff else return failure

Iterative deepening search

function iterative-deepening-search(problem) returns a solution, or failure
for depth = 0 to do
result depth-limited-search(problem, depth)
if result 6= cutoff then return result
Best-first search
function best-first-search(problem) returns a solution, or failure
node a node with state = problem.initial-state, path-cost = 0
frontier a priority queue ordered by f(n), with node as the only element
explored an empty set
loop do
if empty(frontier) then return failure
node pop(frontier) /*chooses lowest-cost node in frontier*/
if problem.goal-test(node.state) then return solution(node)
add node.state to explored
for each action in problem.actions(node.state) do
child child-node(problem, node, action)
if child.state is not in explored or frontier then
frontier insert(child, frontier)
else if child.state is in frontier with higher path-cost then
replace that frontier node with child
A* search
function best-first-search(problem) returns a solution, or failure
node a node with state = problem.initial-state, path-cost = 0
frontier a priority queue ordered by f(n) = g(n) + h(n), with node as the only element
explored an empty set
loop do
if empty(frontier) then return failure
node pop(frontier) /*chooses lowest-cost node in frontier*/
if problem.goal-test(node.state) then return solution(node)
add node.state to explored
for each action in problem.actions(node.state) do
child child-node(problem, node, action)
if child.state is not in explored or frontier then
frontier insert(child, frontier)
else if child.state is in frontier with higher path-cost then
replace that frontier node with child

Glossary
1
1.1

Introduction
What is AI?
to act humanly
to act rationally
to think humanly
to think rationally
rationality
Turing test
natural language
processing
knowledge
representation
automated reasoning
machine learning
total Turing test
computer vision
robotics
cognitive science
syllogisms
logic
logicist tradition
agent
rational agent
limited rationality

1.2

to act in a way that resembles the way humans act

to act in a way that maximizes some (objective) performance measure
to think in a way that resembles the way humans think
to think in a way that maximizes some (objective) performance measure
an ideal measure of performance, based on some objective ideal
a test to measure intelligence; if an AI passes it, it is considered intelligent
to understand and produce spoken language
storage of knowledge and knowledge acquisition
use the knowledge representation to answer questions and to draw new
conclusions
to adapt to new circumstances and detect and extrapolate patterns
the Turing test, but including a video signal and requiring the manipulation of physical objects
the processing and understanding of visual information from the real
world
a medium for an agent to modify its environment
an interdisciplinary field combining experimental psychological data and
computer models
patterns for argument structures, e.g. if A = B and B = C, then A = C
the study of the laws of thought
within AI, the attempt to create intelligent agents based on logic
something that acts
an agent that acts to achieve the best outcome
the implementation of rationality while keeping in mind the computational demands

The Foundations of Artificial Intelligence

rationalism
dualism
materialism
empiricism
induction
logical positivism

confirmation theory

algorithm
incompleteness theorem

a movement that advocates reasoning as the method for understanding

the world
the theory that part of the human mind/soul/spirit is outside of (the laws
of) nature
the theory that argues that the brains operation according to the laws of
physics constitutes the mind
a movement that argues that everything can be understood by experiencing it
the idea that general rules are acquired by exposure to repeated associations between elements
a doctrine that holds that all knowledge can be characterized by logical
theories connected, ultimately, to observation sentences that correspond
to sensory inputs
an attempt to analyze the acquisition of knowledge from experience by
defining an explicit computational procedure for extracting knowledge
from elementary experiences
a series of operations based on logic and mathematics to find a specific
sort of output based on a specific sort of input
any formal theory as strong as the elemental theory of natural numbers
there are true statements that are undecidable.

computable
tractability
probability
utility
decision theory

game theory

operations research
satisficing
neuroscience
neurons
singularity
behaviorism
cognitive psychology
control theory
homeostatic
computational
linguistics

1.3

something is computable if it can be represented by an algorithm

a problem must not take too long, otherwise it is intractable
the changes of a certain outcome for an event
the mathematics of preferred outcomes
a combination of probability theory and utility theory that provides a
formal and complete framework for decisions made under uncertainty (in
large domains)
decision theory in small domains: individual actions have great consequences, and rational agents should adopt policies that are or appear to
be randomized
research that deals with situations with the situations in which a payoff is
not immediately clear
to make decisions that are good enough, rather than calculating the
optimal decision
the study of nervous systems, in particular the brain
simple nerve cells which, in their large numbers, can lead to thought,
action and consciousness
the point at which computers reach a superhuman level of performance
a movement that rejected any theory involving mental processes on the
grounds that introspection could not provide reliable evidence
the branch of psychology that focuses on the brain as an information processing device
the study of having an artifact (a non-living object in the real world) controlling itself to main a stable internal state
having an internal balance
see natural language processing

The History of Artificial Intelligence

Hebbian learning

physical symbol system

hypothesis
LISP
micro-worlds
adalines
perceptron convergence
theorem
machine evolution
genetic algorithms

weak methods
expert systems
certainty factors
frames

back-propagation
learning
connectionist models

a system of artificial learning based on a computational network based on

neurons, drawing from the basic physiology of the brain, logic, and the
theory of computation
a physical symbol system has the necessary and sufficient means for general intelligent action
one of the first high-level programming languages, mainly used by early
AI programmers
limited (computational) domains on which operations could be performed
an enhanced version of Hebbs learning methods
the learning algorithm can adjust the connection strengths of a perceptron
to match any inout data, provided such a match exists
another name for machine learning
using inspiration from biological meiosis and mutation to have different
versions of algorithms compete, and thus, by means of artificial selection find a well-performing version of an algorithm
methods that do not scale well
systems putting from a large knowledge based, specialized to solve very
specific problems
a form of calculus of certainty factors
assembling facts about particular object- and event types and arranging
the types into a large taxonomic hierarchy, analogous to a biological taxonomy
neural networks that learn using a comparison of the input and the output
to weigh the edges
the kind of models back-propagation learning creates

hidden Markov models

data mining
bayesian network
human-level AI
artificial general
intelligence
friendly ai

2
2.1

Agents and Environments

percepts
percept sequence
agent function
agent program

everything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators
the agents perceptual inputs at any given instant
the complete history of everything the agent has ever perceived
the function that maps any given percept sequence to an action; a mathematical/symbolic representation
the means to achieve the agent function, i.e. the actual implementation

Good Behavior: The Concept of Rationality

rational agent

omniscience
information gathering
exploration

2.3

a subfield of AI which looks to find a universal algorithm for learning and

acting in any environment
a concern regarding the creation of AI

Intelligent Agents

agent

2.2

some model which relies heavily on mathematical rigor and statistical

analysis
a probabilistic approach to neural networks
a formalism invented to allow efficient representation of, and rigorous
reasoning with, uncertain knowledge
an AI with human-like intelligence

for each possible percept sequence, a rational agent should select an action
that is expected to maximize its performance measure, given the evidence
provided by the percept sequence and whatever built-in knowledge the
agent has.
having complete knowledge of every state of ones environment and internal state
doing actions in order to modify future percepts
moving in ones environment to acquire knowledge about said environment

The Nature of Environments

fully observable
partially observable
unobservable
single agent
multi-agent
competitive
cooperative
deterministic
stochastic
uncertain
nondeterministic

an agents sensors give it access to the complete (relevant) state of the

environment at each point in time
an agents sensors might be noisy, or part of the environment might simply not be visible to the agents sensors
the agent has no sensors
the agent is the only agent to be taken into consideration
the agent might not be the only agent in the environment, but this depends on the definition of agent
the performance measure of another agent decreases the performance
measure of the own agent
the agents share a (somewhat) common performance measure
the next state of the environment is completely determined by the current
state and the action executed by the agent
not deterministic
an environment is uncertain if it is not fully observable or not fully deterministic
an environment is nondeterministic if actions are characterized by their
possible outcomes, but no probabilities are attached to them

episodic

sequential
static
dynamic
semi-dynamic
discrete
continuous
known
unknown

2.4

The Structure of Agents

architecture
simple reflex agents
condition-action rules
internal state
model
model-based agent
utility
utility function
expected utility
learning element
performance element
critic
problem generator
reward
penalty
expressiveness
atomic representation
factored representation
structured
representation

the medium on which the agent program runs

these agents select actions based on the current percept, ignoring the rest
of the percept history
if-then rules
keeping track of the previous percepts to reflect some unobserved aspects
of the current state
some representation of the world
an agent using a model
assigning values to different options to enable easier choosing
the function to optimize utility
the realistic version of utility, for agents are not omniscient
the element in a learning agent responsible for making improvements to
the performance element
the element in a learning agent responsible for the actions the agent performs
the element in a learning agent responsible for the way the learning element knows what actions are good and which are bad
the element in a learning agent responsible for finding new and informative experiences
a form of direct (positive) feedback on the quality of the agents behavior
a form of direct (negative) feedback on the quality of the agents behavior
expressive power
each state of the world is indivisiblethere is no internal structure
each state of the world is split up into a fixed set of variables or attributes,
each of which have a value
a representation which allows for explicit descriptions of relationships
between objects

Solving Problems by Searching

problem-solving agents
planning agents
uninformed algorithms
informed algorithms

3.1

in an atomic task environment, the agents experience is divided into

atomic episodes, in which it receives a percept and then performs a single
action, which does not depend on the actions taken in previous episodes.
the current decision could affect all future decisions.
the environment does not change while the agent is deliberating
it does
the environment does not change, but the performance score does
finite number of distinct states and set of percepts and actions
infinite number of distinct states or set of percepts or actions
the agents knowledge of all details of the environment represent the entire environment
it does not

a kind of goal-based agents using atomic representation

a kind of goal-based agents using factored or structured representation
algorithms that are given no information about the problem other than its
definition
algorithms that do receive additional information

Problem-Solving Agents
problem formulation

the process of deciding what actions and states to consider, given a goal

search
execution
open-loop system
problem

transition model
goal test
path cost
successor
state space
graph
path
step cost
solution
optimal solution
abstraction

3.2

Example Problems
toy problems
incremental formulation

3.3

the process of looking for a sequence of actions that reaches the goal
the phase that is entered once a solution is found
the agent ignores the percepts during the execution of the solution
the problem is defined by the initial state; the possible actions that are
applicable in a state s; the transition model; the goal test; and the path
cost function that assigns a numeric cost to each path.
a description of what each action does
test to check whether the goal has been reached
a function that assigns a numeric cost to each path.
any applicable state
the combination of the initial state, actions and transition model
directed network
a sequence of states connected by a sequence of actions
the cost of taking action a in state s to reach state s: c(s, a, s0 )
a sequence of actions that leads from the initial state to a goal state
a solution that has the lowest path cost among all solutions
the process of removing detail from a representation

a problem intended to illustrate or exercise various problem-solving methods

a problem with solutions that apply in the real world

Searching for Solutions

search tree
expansion
parent node
child nodes
leaf nodes
frontier
open list
search strategy
repeated state
loopy path
redundant paths
explored set
closed list
queue
fifo queue
lifo queue
stack
priority queue
canonical form
completeness
optimality
time complexity
space complexity
branching factor
depth
maximum length

a representation of a solution to a problem

the generation of new states from the current state
a node that generates child nodes
nodes generated by a parent node
nodes with no children
the set of all leaf nodes
frontier
the way an algorithm moves through the tree
a state that occurs multiple times
a kind of repeated state that is cyclical
a state that exists whenever there is more than one way to get from one
state to another
the set of nodes that have been visited
explored set
a data structure used to represent nodes
first-in first-out queue: the oldest element is popped
last-in first-out queue: the newest element is popped
lifo
the element of the queue with the highest priority according to some ordering function is popped
a more general method of storing repeated states in canonical form, meaning that logically equivalent states should map to the same data structure
does the algorithm always find a solution if one exists
does the algorithm find the optimal solution, i.e. the solution with the
lowest path cost
how long does it take to find a solution
how much memory is needed to perform the search
b, the maximum number of successors of any node
d, the depth of the shallowest goal node
m, the maximum length of any path in the state space

search cost
total cost

3.4

Uninformed Search Strategies

uninformed search
blind search
informed search
heuristic search
breadth-first search

uniform-cost search
depth-first search

depth-limited search
backtracking search

iterative-deepening
search
iterative lengthening
bidirectional search
predecessors

3.5

a cost that typically depends on time complexity

a cost that combines search cost and path cost

searches that are not provided with any information besides the information in the problem definition
uninformed search
searches that know whether one non-goal state is more promising than
another
informed search
a search algorithm in which, using a fifo queue, every node on a level is
expanded before the nodes on the next level are expanded, starting at the
root node
is basically breadth-first search, but instead of expanding the shallowest
nodes, it expands the node with the lowest path cost
a search algorithm in which, using a lifo queue, which always expands
the deepest node in the current frontier of the search tree, i.e. it gets
deeper in the tree until it finds a leaf node.
depth-first search, but limited to a depth l
a variant of depth-first search which uses less memory; only one successor is generated at a time, and instead of copying the the current state
description, it modifies the current
iterative deepening search or iterative deepening depth-first search performs depth-first, but initially explores only the root node, then also
searches in the first layer, then the second, etc.
iterative deepening, but using increasing path-cost limits instead of increasing depth limits. Not effective.
search process that basically performs two searches simultaneously; when
the frontier meets, the solution is found
all states x that have x as a successor

Informed (Heuristic) Search Strategies

informed search
best-first search
heuristic function
greedy best-first search
straight-line distance
A* search
admissible heuristic
consistent

monotonous
triangle inequality
contours

a search strategy that uses problem-specific knowledge beyond the definition of the problem
graph-search or tree-search in which a node is selected for expansion
based on some evaluation function
a component of the evaluation function that estimates cost of the cheapest
path from the state at node n to a goal state
a search process that tries to expand the node that is closest to the goal
a heuristic that uses the distance between two points. can be used with
greedy best-first search
a search process that evaluates odes by combining the cost to reach the
node ant the cost to get from the node to the goal
a heuristic is admissible if the heuristic never overestimates the cost to
reach the goal
for every node n and every successor nA Z of n generated by any action a,
the estimated cost of reaching the goal from n is no greater than the step
cost of getting to nA Z plus the estimated cost of reaching the goal from
h(n) c(n, a, n) + h(n).
nA Z:
consistent
each side of a triangle cannot be longer than the sum of the other two
sides.

pruning
optimally efficient
absolute error
relative error
total cost

3.6

Heuristic Functions
manhattan distance
city block distance
effective branching
factor
dominates
relaxed problem
pattern databases
disjoint pattern
databases
features