Greedy Programming

Deliverables
Greedy Paradigm

Coin Change Problem

Interval Scheduling

Optimal Merge Sort

Huffman compression

0/1 and partial knapsack

Minimizing Lateness

6/7/2012 8:57 PM Copyright @ gdeepak.Com 2

 Greedy paradigm
Primarily used in Optimization problems to find
the maximum or minimum value satisfying
given constraints.

It helps in constructing solution for a problem by

a sequence of steps (piecewise) where each step
is considered a partial solution, which is
extended progressively to get complete solution.

6/7/2012 8:57 PM Copyright @ gdeepak.Com 3

 Choice of steps
Choice of each step in greedy is based on following

It must be feasible

It must be locally optimal

It must be irrevocable

Locally optimal choice is considered globally optimal.

6/7/2012 8:57 PM Copyright @ gdeepak.Com 4

 5 Pillars of Greedy Technique
Proof of correctness is necessary and important.
1. A candidate set, from which a solution is created
2. A selection function, which chooses the best candidate to be
added to the solution
3. A feasibility function that is used to determine if a candidate
can be used to contribute to a solution
4. An objective function, which assigns a value to a solution, or
a partial solution, and
5. A solution function, which will indicate when we have
discovered a complete solution

6/7/2012 8:57 PM Copyright @ gdeepak.Com 5

 General Greedy Algorithm
a[1:n]; sol ;
for i 1 to n do
x SELECT(a); //as per some optimization criteria
if FEASIBLE(sol, x) then solution UNION(sol, x);
return sol

Feasibility checks whether the selected element meets the

feasibility criteria and Union integrates the element x in the
solution

6/7/2012 8:57 PM Copyright @ gdeepak.Com 6

 Coin Change Problem
An amount and collection of coins to reach to that amount Objective is to
minimize the number of coins
Greedy theory says always return the largest coin
if coins are of denomination 32, 8, 1 it has the greedy choice property because
no amount over 32 can be made without omitting 32.

if coins are of denomination 30,20,5,1 then it doesnt have greedy choice

property because 40 can be made with two 20 coins but it returns 30,5,5 coins

Similarly if denominations are 1, 10, 21, 34, 70, 100, 350 140 Greedy: 100, 34,
1, 1, 1, 1, 1, 1 Optimal: 70, 70

6/7/2012 8:57 PM Copyright @ gdeepak.Com 7

 Interval Scheduling Problem
Given set S = {a1, , an} of activities and activity start and
finish times, e.g.
i 1 2 3 4 5 6 7 8 9 10 11
si 1 3 0 5 3 5 6 8 8 2 12
fi 4 5 6 7 8 9 10 11 12 13 14
Activities are compatible if their time intervals do not
overlap, i.e., ai is compatible with aj is si fj or sj fi.
To find Subset of non overlapping intervals so that number
of intervals are maximum.
6/7/2012 8:57 PM Copyright @ gdeepak.Com 8
 Earliest Start Time

6/7/2012 8:57 PM Copyright @ gdeepak.Com 9

 Shortest interval First

6/7/2012 8:57 PM Copyright @ gdeepak.Com 10

 Fewest Conflicts First

6/7/2012 8:57 PM Copyright @ gdeepak.Com 11

 Interval Scheduling Algorithm
Interval that ends first and remove the intervals that
overlap with these and recurse.
Sort jobs by finish times so that f1 f2 ... fn.
A
for j = 1 to n
{
if (job j compatible with A)
A A {j}
}
return A

6/7/2012 8:57 PM Copyright @ gdeepak.Com 12

 Efficient greedy algorithm
Once youve identified a reasonable greedy heuristic:
Prove that it always gives the correct answer
Develop an efficient solution

6/7/2012 8:57 PM Copyright @ gdeepak.Com 13

 Is our greedy approach correct?
Stays ahead argument: show that no matter what other
solution someone provides you, the solution provided by
your algorithm always stays ahead, in that no other
choice could do better

6/7/2012 8:57 PM Copyright @ gdeepak.Com 14

 Is our greedy approach correct?
Stays ahead argument
Let r1, r2, r3, , rk be the solution found by our approach

r1 r2 r3
rk
Let o1, o2, o3, , ok of another optimal solution

o1 o2 o3
ok
Show our approach stays ahead of any other solution

6/7/2012 8:57 PM Copyright @ gdeepak.Com 15

 Stays ahead

r1 r2 r3
rk

o1 o2 o3
ok

Compare first activities of each solution

6/7/2012 8:57 PM Copyright @ gdeepak.Com 16

 Stays ahead

r1 r2 r3
rk

o1 o2 o3
ok

finish(r1) finish(o1)

6/7/2012 8:57 PM Copyright @ gdeepak.Com 17

 Stays ahead

r2 r3
rk

o2 o3
ok

We have at least as much time as any other solution to schedule

remaining 2k tasks

6/7/2012 8:57 PM Copyright @ gdeepak.Com 18

 Optimal Merge Sort
Suppose there are 3 sorted lists L1, L2, and L3, of sizes 30, 20, and 10,
respectively, which need to be merged into a combined sorted list but
we can merge only two at a time.
We intend to find an optimal merge pattern which minimizes the total
number of comparisons.

Merge L1 & L2,: 30 + 20 = 50 comparisons , then merge the list & L3:
50 + 10 = 60 comparisons total number of comparisons: 50 + 60 = 110.

Alternatively, merge L2 & L3: 20 + 10 = 30 comparisons, the resulting

list (size 30) then merge the list with L1: 30 + 30 = 60 total number of
comparisons: 30 + 60 = 90.

6/7/2012 8:57 PM Copyright @ gdeepak.Com 19

 Merge sort example

6/7/2012 8:57 PM Copyright @ gdeepak.Com 20

 Huffman Compression
File f has one million characters. If only a, b, c, d, e, f chars come in the file then the
total size of the file is 8 million bits

If we represent these with 000 001 010 011 101 111 then only 3 million bits
0 1 00 11 10 11 will give us less then 2 million bits

Assume frequency of occurrence of a, b, c, d, e, f is 45%,13%,12%,16%,9%,5%

leaves of the binary tree will represent the bit strings

It will give us the prefix free code. use greedy for making the binary tree and then
assign 0 to every left node and 1 to every right node. The nodes with larger frequency
will be nearer to the root and with lesser frequency will be towards the leaves.

6/7/2012 8:57 PM Copyright @ gdeepak.Com 21

 Huffman Example

6/7/2012 8:57 PM Copyright @ gdeepak.Com 22

 Knapsack
Given n objects and a knapsack. Object i has weight wi and the
knapsack has a capacity M.

if a fraction xi, 0<= xi <= 1 of object i is placed into the knapsack

then a profit of pixi is earned. The objective is to obtain a filling of
the knapsack that maximizes the total profit earned.

Since knapsack capacity is M, total weight of all chosen objects

should be M. The profits and weights are positive

6/7/2012 8:57 PM Copyright @ gdeepak.Com 23

 Partial Knapsack-Greedy

6/7/2012 8:57 PM Copyright @ gdeepak.Com 24

 0/1 knapsack
1st is optimal but greedy will give 2nd solution

6/7/2012 8:57 PM Copyright @ gdeepak.Com 25

 Algorithm for partial knapsack
Input Set S of items with weight wi , benefit bi and total weight W
Output amount xi of each item i to maximize benefit with weight
W for each item i in S
xi=0
vi=bi/wi
w=0
while w<W
remove item i with highest vi
xi=min(wi, W-w)
w=w+min(wi, W-w)

6/7/2012 8:57 PM Copyright @ gdeepak.Com 26

 Job deadlines
Single resource processes one job at a time.
Job j requires tj units of processing time and is due at time dj.
If j starts at time sj, it finishes at time fj = sj + tj.
Lateness: lj = max { 0, fj - dj }.
Goal: Schedule all jobs to minimize maximum lateness
L=max lj.
1 2 3 4 5 6
tj 3 2 1 4 3 2
dj 6 8 9 9 14 15
6/7/2012 8:57 PM Copyright @ gdeepak.Com 27
 Minimizing lateness
Consider jobs in ascending order of processing time tj

1 2
tj 1 10
dj 100 10
Consider jobs in ascending order of slack dj - tj
1 2
tj 1 10
dj 2 10

6/7/2012 8:57 PM Copyright @ gdeepak.Com 28

 Minimizing Lateness

Consider Jobs in the sequence of earliest deadline first

T 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
J 1 1 1 2 2 3 4 4 4 4 5 5 5 6 6
D 6 6 6 8 8 9 9 9 9 9L 14 14 14 15 15

6/7/2012 8:57 PM Copyright @ gdeepak.Com 29

 Single Job-Single resource
There is no preemption and these have to be allocated
processor. Only one processor is available. Objective is to
minimize the sum of the finishing times
f1=t1
f2=t1+t2
fi=t1+t2++ti
:
fn=nt1+(n-1)t2++tn
So t1 should be smallest and so on

6/7/2012 8:57 PM Copyright @ gdeepak.Com 30

 Questions, Suggestions and Comments

6/7/2012 8:57 PM Copyright @ gdeepak.Com 31

 Question 1

Considering the following graph as a multi stage graph and using

Greedy Approach the shortest path is
A) 9
B) 23
C) 5
D) 17

6/7/2012 8:57 PM Copyright @ gdeepak.Com 32

 Question 2
Greedy Technique is related to the theory of
A) Calculus
B) Probability
C) Matroid
D) Algebra

6/7/2012 8:57 PM Copyright @ gdeepak.Com 33

 Question 3
For those problems where Greedy gives the optimal
solution, the complexity is better as compared to the
implementation of the same problem with Dynamic
Programming.
A) True
B) False

6/7/2012 8:57 PM Copyright @ gdeepak.Com 34

 ??
Given a question, Can you find whether greedy strategy
work in general is NP-Complete.

6/7/2012 8:57 PM Copyright @ gdeepak.Com 35