Algorithms and Data

Structures
Lecture III
Simonas altenis
Aalborg University
simas@cs.auc.dk

September 22, 20
03

This Lecture

Divide-and-conquer technique for

algorithm design. Example problems:

Tiling
Searching (binary search)
Sorting (merge sort).

September 22

Tiling
A tromino tile:
And a 2nx2n board
with a hole:

September 22

A tiling of the
board with
trominos:

Tiling: Trivial Case (n = 1)

Trivial case (n = 1): tiling a 2x2 board

with a hole:

Idea try somehow to reduce the size

of the original problem, so that we
eventually get to the 2x2 boards
which we know how to solve
September 22

Tiling: Dividing the

Problem

To get smaller square boards lets

divide the original board into for boards

September 22

Great! We have one

problem of the size 2n1
x2n-1!
But: The other three
problems are not similar
to the original problems
they do not have holes!
5

Tiling: Dividing the

Problem

Idea: insert one tromino at the center

to get three holes in each of the three
smaller boards

September 22

Now we have four boards

with holes of the size 2n1
x2n-1.
Keep doing this division,
until we get the 2x2
boards with holes we
know how to tile those
6

Tiling: Algorithm
n
n
INPUT:
INPUT:nn the
theboard
boardsize
size(2
(2nx2
x2nboard),
board),LLlocation
locationof
ofthe
thehole.
hole.
OUTPUT:
OUTPUT:tiling
tilingof
ofthe
theboard
board

Tile(n,
Tile(n, L)
L)
if
if nn == 11 then
then
Trivial
Trivial case
case
Tile
Tile with
with one
one tromino
tromino
return
return
Divide
Divide the
the board
board into
into four
four equal-sized
equal-sized boards
boards
Place
Place one
one tromino
tromino at
at the
the center
center to
to cut
cut out
out 33 additional
additional
holes
holes
Let
Let L1,
L1, L2,
L2, L3,
L3, L4
L4 denote
denote the
the positions
positions of
of the
the 44 holes
holes
Tile(n-1,
Tile(n-1, L1)
L1)
Tile(n-1,
Tile(n-1, L2)
L2)
Tile(n-1,
Tile(n-1, L3)
L3)
Tile(n-1,
Tile(n-1, L4)
L4)

September 22

Divide and Conquer

Divide-and-conquer method for

algorithm design:

If the problem size is small enough to solve

it in a straightforward manner, solve it. Else:

Divide: Divide the problem into two or more

disjoint subproblems
Conquer: Use divide-and-conquer recursively to
solve the subproblems
Combine: Take the solutions to the subproblems
and combine these solutions into a solution for
the original problem

September 22

Tiling: Divide-and-Conquer

Tiling is a divide-and-conquer algorithm:

Just do it trivially if the board is 2x2, else:

Divide the board into four smaller boards
(introduce holes at the corners of the three
smaller boards to make them look like
original problems)
Conquer using the same algorithm
recursively
Combine by placing a single tromino in the
center to cover the three introduced holes

September 22

Binary Search

Find a number in a sorted array:

Just do it trivially if the array is of one element

Else divide into two equal halves and solve each half
Combine the results

INPUT:
INPUT:A[1..n]
A[1..n]aasorted
sorted(non-decreasing)
(non-decreasing)array
arrayofofintegers,
integers,ssan
aninteger.
integer.
OUTPUT:
an
index
j
such
that
A[j]
=
s.
NIL,
if
j
(1jn):
A[j]
s
OUTPUT: an index j such that A[j] = s. NIL, if j (1jn): A[j] s
Binary-search(A,
Binary-search(A, p,
p, r,
r, s):
s):
if
if pp == rr then
then
if
if A[p]
A[p] == ss then
then return
return pp
else
else return
return NIL
NIL
q
(p+r)/2
q (p+r)/2
ret
ret
Binary-search(A,
Binary-search(A, p,
p, q,
q, s)
s)
if
if ret
ret == NIL
NIL then
then
return
return Binary-search(A,
Binary-search(A, q+1,
q+1, r,
r, s)
s)
else
else return
return ret
ret

September 22

10

Recurrences

Running times of algorithms with Recursive

calls can be described using recurrences
A recurrence is an equation or inequality
that describes a function in terms of its
value on smaller inputs
solving_trivial_problem
if n 1
T ( n)
num_pieces T (n / subproblem_size_factor) dividing combining if n 1

Example: Binary Search

(1)
if n 1
T ( n)
2T (n / 2) (1) if n 1

September 22

11

Binary Search (improved)

T(n) = (n) not better than brute force!

Clever way to conquer:

Solve only one half!

INPUT:
INPUT:A[1..n]
A[1..n]aasorted
sorted(non-decreasing)
(non-decreasing)array
arrayofofintegers,
integers,ssan
aninteger.
integer.
OUTPUT:
an
index
j
such
that
A[j]
=
s.
NIL,
if
j
(1jn):
A[j]
s
OUTPUT: an index j such that A[j] = s. NIL, if j (1jn): A[j] s
Binary-search(A,
Binary-search(A, p,
p, r,
r, s):
s):
if
if pp == rr then
then
if
if A[p]
A[p] == ss then
then return
return pp
else
else return
return NIL
NIL
q
q (p+r)/2
(p+r)/2
if
if A[q]
A[q] ss then
then return
return Binary-search(A,
Binary-search(A, p,
p, q,
q, s)
s)
else
else return
return Binary-search(A,
Binary-search(A, q+1,
q+1, r,
r, s)
s)

September 22

12

Running Time of BS
(1)
if n 1
T ( n)
T (n / 2) (1) if n 1

T(n) = (lg n) !

September 22

13

Merge Sort Algorithm

Divide: If S has at least two elements (nothing

needs to be done if S has zero or one elements),
remove all the elements from S and put them
into two sequences, S1 and S2 , each containing
about half of the elements of S. (i.e. S1 contains
the first n/2 elements and S2 contains the
remaining n/2 elements).
Conquer: Sort sequences S1 and S2 using Merge
Sort.
Combine: Put back the elements into S by
merging the sorted sequences S1 and S2 into one
sorted sequence
September 22

14

Merge Sort: Algorithm

Merge-Sort(A,
Merge-Sort(A, p,
p, r)
r)
if
if pp << rr then
then
q
q (p+r)/2
(p+r)/2
Merge-Sort(A,
Merge-Sort(A, p,
p, q)
q)
Merge-Sort(A,
Merge-Sort(A, q+1,
q+1, r)
r)
Merge(A,
Merge(A, p,
p, q,
q, r)
r)
Merge(A,
Merge(A, p,
p, q,
q, r)
r)
Take
Take the
the smallest
smallest of
of the
the two
two topmost
topmost elements
elements of
of
sequences
sequences A[p..q]
A[p..q] and
and A[q+1..r]
A[q+1..r] and
and put
put into
into the
the
resulting
resulting sequence.
sequence. Repeat
Repeat this,
this, until
until both
both sequences
sequences
are
are empty.
empty. Copy
Copy the
the resulting
resulting sequence
sequence into
into A[p..r].
A[p..r].

September 22

15

MergeSort (Example) - 1

September 22

16

MergeSort (Example) - 2

September 22

17

MergeSort (Example) - 3

September 22

18

MergeSort (Example) - 4

September 22

19

MergeSort (Example) - 5

September 22

20

MergeSort (Example) - 6

September 22

21

MergeSort (Example) - 7

September 22

22

MergeSort (Example) - 8

September 22

23

MergeSort (Example) - 9

September 22

24

MergeSort (Example) - 10

September 22

25

MergeSort (Example) - 11

September 22

26

MergeSort (Example) - 12

September 22

27

MergeSort (Example) - 13

September 22

28

MergeSort (Example) - 14

September 22

29

MergeSort (Example) - 15

September 22

30

MergeSort (Example) - 16

September 22

31

MergeSort (Example) - 17

September 22

32

MergeSort (Example) - 18

September 22

33

MergeSort (Example) - 19

September 22

34

MergeSort (Example) - 20

September 22

35

MergeSort (Example) - 21

September 22

36

MergeSort (Example) - 22

September 22

37

Merge Sort Summarized

To sort n numbers

if n=1 done!
recursively sort 2 lists of
numbers n/2 and n/2
elements
merge 2 sorted lists in (n)
time

Strategy

break problem into similar

(smaller) subproblems
recursively solve
subproblems
combine solutions to
answer

September 22

38

Running time of MergeSort

Again the running time can be

expressed as a recurrence:

solving_trivial_problem

T ( n)

if n 1

num_pieces T ( n / subproblem_size_factor) dividing combining if n 1

(1)
if n 1
T ( n)
2T (n / 2) (n) if n 1

September 22

39

Repeated Substitution
Method

Lets find the running time of merge sort (lets

assume that n=2b, for some b).
1
if n 1

T ( n)
2T (n / 2) n if n 1
T ( n) 2T n / 2 n substitute
2 2T n / 4 n / 2 n expand
22 T ( n / 4) 2n substitute
22 (2T (n / 8) n / 4) 2n expand
23 T (n / 8) 3n

observe the pattern

T ( n) 2i T (n / 2i ) in

September 22

2lg n T (n / n) n lg n n n lg n

40

Example: Finding Min and

Max

Given an unsorted array, find a

minimum and a maximum element in
the array

INPUT:
INPUT:A[l..r]
A[l..r]an
anunsorted
unsortedarray
arrayofofintegers,
integers,l lr.
r.
OUTPUT:
OUTPUT:(min,
(min,max)
max)such
suchthat
thatj
j(ljr):
(ljr):A[j]
A[j]min
minand
andA[j]
A[j]max
max

MinMax(A,
MinMax(A, l,
l, r):
r):
if
Trivial
if ll == rr then
then return
return (A[l],
(A[l], A[r])
A[r])
Trivial case
case
q
q (l+r)/2
(l+r)/2
Divide
Divide
(minl,
(minl, maxl)
maxl) MinMax(A,
MinMax(A, l,
l, q)
q)
Conquer
(minr,
maxr)
MinMax(A,
q+1,
r)
(minr, maxr) MinMax(A, q+1, r)
if
if minl
minl << minr
minr then
then min
min == minl
minl else
else min
min == minr
minr Combine
if
if maxl
maxl >> maxr
maxr then
then max
max == maxl
maxl else
else max
max == maxr
maxr
return
return (min,
(min, max)
max)

September 22

41

Next Week

Analyzing the running time of

recursive algorithms (such as divideand-conquer)

September 22

42