Professional Documents
Culture Documents
Unit V By G.M.KARTHIK
Outline
Multistage graphs Knapsack problem Flow shop scheduling N queen problem Graph Coloring
Dynamic Programming
Dynamic Programming is an algorithm design method that can be used when the solution to a problem may be viewed as the result of a sequence of decisions
MULTISTAGE GRAPH
Multistage Graph
Shortest path Shortest path in multistage graph Backward reasoning Forward reasoning Traveling salesperson problem
e.g.
D
18
13 2
The greedy method can not be applied to this case: (S, A, D, T) 1+4+18 = 23. The real shortest path is: (S, C, F, T) 5+2+2 = 9.
D
18
1 2
A B
d(A, T) d(B, T)
13 2
d(C, T)
D E
d(D, T)
11
d(E, T)
d(B, T) = min{9+d(D, T), 5+d(E, T), 16+d(F, T)} = min{9+18, 5+13, 16+2} = 18.
A
1 11 2 5 16 5 9 4
D
18
9 5
D E F
d(D, T) d(E, T)
13 2
16
d(F, T)
d(C, T) = min{ 2+d(F, T) } = 2+2 = 4 d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)} = min{1+22, 2+18, 5+4} = 9. The above way of reasoning is called backward reasoning.
D
18
13 2
16 d(S, A) = 1 5 d(S, B) = 2 C d(S, C) = 5 d(S,D)=min{d(S,A)+d(A,D), d(S,B)+d(B,D)} = min{ 1+4, 2+9 } = 5 d(S,E)=min{d(S,A)+d(A,E), d(S,B)+d(B,E)} = min{ 1+11, 2+5 } = 7 d(S,F)=min{d(S,B)+d(B,F), d(S,C)+d(C,F)} = min{ 2+16, 5+2 } = 7
d(S,T) = min{d(S, D)+d(D, T), d(S,E)+ d(E,T), d(S, F)+d(F, T)} = min{ 5+18, 7+13, 7+2 } =9
A
4 1 11 2 5 16 5 9
D
18
13 2
Principle of optimality
Principle of optimality: Suppose that in solving a problem, we have to make a sequence of decisions D1, D2, , Dn. If this sequence is optimal, then the last k decisions, 1 k n must be optimal. e.g. the shortest path problem If i, i1, i2, , j is a shortest path from i to j, then i1, i2, , j must be a shortest path from i1 to j In summary, if a problem can be described by a multistage graph, then it can be solved by dynamic programming.
Dynamic programming
Forward approach and backward approach:
Note that if the recurrence relations are formulated using the forward approach then the relations are solved backwards . i.e., beginning with the last decision On the other hand if the relations are formulated using the backward approach, they are solved forwards.
7 4
2 2 g 3 8
3 10 9 g 7
4 5 g 4 g
8 -14
Cost matrix:
1 2 3 4
g 2 4 6
A multistage graph can describe all possible tours of a directed 7 3 graph. (1,4,3) (1,4,3,2) Find the shortest path: (1, 4, 3, 2, 1) 5+7+3+2=17
8 -15
Time complexity:
n
( ),( ) o o n 2 (n-1)( n k )
(n-k)
! O(n 2 2 n )
8 -16
KNAPSACK PROBLEM
Knapsack Problem
Definition Using different techniques Brute force approach Greedy Approach Fractional Knapsack problem
Knapsack problem
Given some items, pack the knapsack to get the maximum total value. Each item has some weight and some value. Total weight that we can carry is no more than some fixed number W. So we must consider weights of items as well as their values. Item # 1 2 3 Weight 1 3 5 Value 8 6 5
Knapsack problem
There are two versions of the problem:
1. 2. 0-1 knapsack problem and Fractional knapsack problem
1.
Items are indivisible; you either take an item or not. Some special instances can be solved with dynamic programming Items are divisible: you can take any fraction of an item. Solved with a greedy algorithm We will see this version at a later time
2.
wi
2 3 4 5
bi
3 4 5 8
10
max bi subject to wi e W
iT iT
The problem is called a 0-1 problem, because each item must be entirely accepted or rejected.
Defining a Subproblem
If items are labeled 1..n, then a subproblem would be to find an optimal solution for Sk = {items labeled 1, 2, .. k} This is a reasonable subproblem definition. The question is: can we describe the final solution (Sn ) in terms of subproblems (Sk)? Unfortunately, we can t do that.
Defining a Subproblem
w1 =2 w2 =4 b1 =3 b2 =5 w3 =5 b3 =8 w4 =3 b4 =4
Weight Benefit
Item
# S4 S5
wi
2 4 5 3 9
bi
3 5 8 4 10
1 2 3 4 5
w5 =9 b5 =10
It means, that the best subset of Sk that has total weight w is:
1) the best subset of Sk-1 that has total weight e w, or 2) the best subset of Sk-1 that has total weight e wwk plus the item k
Recursive Formula
V [k 1, w] if wk " w V [ k , w] ! max{V [k 1, w],V [k 1, w wk ] bk } else
The best subset of Sk that has the total weight e w, either contains item k or not. First case: wk>w. Item k can t be part of the solution, since if it was, the total weight would be > w, which is unacceptable. Second case: wk e w. Then the item k can be in the solution, and we choose the case with greater value.
1 2 3
10 3 5
40 20 30
M=10
011
0 0
S
x1=0
30
010
0 0
001
000
3 1 2 4 n
M1
M2
Mm
FLOW SHOPS
PRODUCTION SYSTEMS FOR WHICH: A NUMBER OF OPERATIONS HAVE TO BE DONE ON EVERY JOB. THESE OPERATIONS HAVE TO BE DONE ON ALL JOBS IN THE SAME ORDER, i.e., THE JOBS HAVE TO FOLLOW THE SAME ROUTE. THE MACHINES ARE ASSUMED TO BE SET UP IN SERIES. COMMON ASSUMPTIONS: UNLIMITED STORAGE OR BUFFER CAPACITIES IN BETWEEN SUCCESIVE MACHINES (NO BLOCKING). A JOB HAS TO BE PROCCESSED AT EACH STAGE ON ONLY ONE OF THE MACHINES (NO PARALLEL MACHINES).
Open-shop scheduling
Example: Job: J1, J2, J3, J4 Machine: M1, M2, M3
J1 M1 M2 M3 OUT
J2
M1
M2
M3
OUT
J3
M3
M2
M1
OUT
J4
M2
M1
M3
OUT
Job-shop scheduling
Each job visit each machine at most once Each job has own routing
J1 M1 M2 M3 OUT
J2
M1
M2
M3
OUT
J3
M3
M2
M1
OUT
J4
M2
M1
M3
OUT
Flow-shop scheduling
Jobs are not preemptive Each job has m tasks with processing time Each job follow a same route
J1 M1 M2 M3 OUT
J2
M1
M2
M3
OUT
J3
M1
M2
M3
OUT
J4
M1
M2
M3
OUT
Scheduling Classification
Simple open shop scheduling Open shop Flexible open shop scheduling Simple Job shop scheduling Scheduling Job shop Flexible Job shop scheduling Simple Flow shop scheduling Flow shop Flexible Flow shop scheduling
Car-2
(4, 4) Center 1 Center 2
degreasing
P1
painting
P2
9
13
Car-2
(4, 4) Center 1
degreasing
P1
5 4 8 8
painting
P2
Center 2
Minimizes total idle time and the makespan Several conditions must be satisfied
Johnson s rule
1. Select a job with the shortest processing time
If the processing time is on the first workcenter Schedule the job right after the already scheduled at the beginning of the list If the processing time is on the second workcenter Schedule the job right before the already scheduled at the end of the list
MC1 MC2
15
15
20
35
8
43
12
55
13 6 4
58 64 68
15
15
25
40
18
Reservation systems
Estimates demand for service
N-QUEEN PROBLEM
N-Queen Problem
A classical combinatorial problem n x n chess board n queens on the same board Queen attacks other at the same row, column or diagonal line No 2 queens attack each other
Previous Works
Analytical solution
Direct computation, very fast Generate only a very restricted class of solutions
Backtracking Search
Generate all possible solutions Exponential time complexity Can only solve for n<100
QS1
Data Structure
The i-th queen is placed at row i and column queen[i] 1 queen per row The array queen must contain a permutation of integers {1, ,n} 1 queen per column 2 arrays, dn and dp, of size 2n-1 keep track of number of queen on negative and positive diagonal lines The i-th queen is counted at dn[i+queen[i]] and dp[i-queen[i]]
Problem remains
Resolve any collision on the diagonal lines
QS1
Pseudo-code
QS1
Gradient-Based Heuristic
QS2
Data Structure
Queen placement same as QS1 An array attack is maintained
Store the row indexes of queens that are under attack
QS2
Pseudo-code
QS2
Go through the attacking queen only
QS3
Improvement on QS2 Random permutation generates approximately 0.53n collisions Conflict-free initialization
The position of a new queen is randomly generated until a conflict-free place is found After a certain of queens, m, the remaining c queens are placed randomly regardless of conflicts
QS4
Algorithm same as QS1 Initialization same as QS4 The fastest algorithm
3,000,000 queens in less than one minute
Permutation Statistics
Swap Statistics
Swap Statistics
GRAPH COLORING
Graph coloring
Definition Algorithms Different type of graphs Results.
Sample Graph
Can you determine the minimum number of colors for this graph?
Only known solution guaranteed to be optimal requires exponential time examining all possible assignments, typically using backtracking after assigning colors
Graph Coloring
With these problems we can propose them as decision problems or as optimization problems The decision problem is slightly easier, but still is NP Complete Decision Problem: Given G and a positive integer k, is there a coloring of G using at most k colors? Optimization Problem: Given G, determine X(G), the chromatic number, and produce an optimal coloring that uses only X(G) colors.
Definition
Undirected graph of vertices connected by edges Each vertex assigned a specific label, or color No two adjacent vertices can have same color Goal: Minimize the number of colors used to fill the entire graph
*the vertices can also be edges, faces, or planes; however, all of these different possibilities can ultimately be reduced to the vertex problem
Objective
Perform extensive research on different heuristics for graph coloring Analyze the results of each heuristic or algorithm, and compare the running times and efficiency of each Manipulate the data and graphs used in experimenting with each heuristic and observe the impact of such changes Research current real-world applications and develop new possible utilizations of the Graph Coloring Problem
Inspiration
The Graph Coloring Problem is one of the most widely studied NP-Hard problems Graph coloring has many practical applications, such as timetabling, resource assignment, register allocation, and circuit testing Very versatile problem, as different limitations can be set on the graph, such as the way a color is assigned or even limitations on the color itself Among the numerous existing algorithms, we are seeking to either improve on the efficiency of a current solution or formulate an entirely new heuristic
Algorithms
1) Greedy Algorithms
Simple Greedy Algorithm
Let V = {v1 . . .vn}. Let be a permutation of {1 . . . n}. The simple greedy algorithm is Color v (1) with color 1. For i from 2 to n do Assume k colors have been used. Color v (i) with the minimum from {1 . . . k+1} Such that no conflict is created.
Algorithms
2) Partitions
Tabu Search
- Given graph G = (V, E), a feasible solution is a partition s = (V1 , V2 , V3 , , Vk ) of node set V into k subsets - E(Vi ) is collection of the edges of G with both endpoints in Vi - f(s) = (| E(Vi ) | : i = 1, .., k) the number of edges for which both endpoints are in the same Vi (have the same color) - s can be a coloring of the nodes of G with k colors iff f(s) = 0 - From s, we generate a neighbor s; after generating many neighbors that do no lead to tabu moves, we move to the best one - Tabu List: whenever node x is moved from Vi to Vj to get the new solution, the pair (x,i) becomes tabu(node x cannot be returned to node Vi for a certain number of iterations. The list of tabu moves is cyclic - continue iterations until either we get a solution s such that f(s) = 0, or until we reach the maximum number of iterations
Algorithms
Input G = (V, E) k =number of coIors [T] =size of tabu list. rep = number of nr in sample nbmax =maximum number of iterations. Initialization Generate a random solution s=(V 1 .... , Vk) nbiter: =0; choose an arbitrary tabu list T. While f(s) >0 and nbiter <nbmax generate rep neighbours s i of s with move sosir f(s~)<_ A (f(s)) (as soon as we get an sl with f(s~)<f(s) we stop the generation). Let s ' bc the be~t neighbour generated update tabu list T (introduce move s--*s' and remove oldest tabu move) S: =S r nbiter: = nbiter + 1 endwhile Output If f (s)=0, we get a coloring of G with k colors: V1, ..., V k are the coIor sets. Otherwise no coloring has been found with k colors.
Algorithms
Genetic Local Search Algorithm
- Undirected graph G = (V,E) with V = {v1 , v2 , , vn } as the set of nodes and E = { eij for each edge between vi and vj } as the set of edges - Must determine a partition V in a minimum number of color classes C1, C2, C3, .., Ck such that for each edge eij in E, vi and vj are not in the same color class. General Algorithm: First, fix k, the number of available colors, to a given value and search for a proper k-coloring by minimizing the number of violated constraints to 0 according to the equation: For all eij in E, c(vi) /= c(vj) If such a coloring is found, decrease k and repeat the above search process until no more possible k-coloring is found within the allowed iterations The goal is thus to find a proper k-coloring for a fixed k
Algorithms
Data: G. a graph Result: the number of conflicts with k fixed colors % f.f* : fitness function and its best value encountered so far % s* : best individual encountered so far % MaxIter : the current and maximum number of iterations allowed % best(P) : returns the best individual of the population P begin i =0 generate(Po) s* = best(Po) f* = f(s*) while ( f* > 0 and i < MaxIter) do Pi = crossing( Pi,Ti ); /*using UIS crossover */ Pi+1 = mutation(Pi ), /*using tabu search */ if (f(best(Pi+l)) < f*) then s* = best(Pi+l) f*=f(s*) i=i+l return f* end
Experiment
Run algorithms on different graphs, record number of colors used, and find the optimal heuristic Modify number of edges and number of cliques in a graph and analyze the effects of edge and clique density
Results
In existing literature, several experiments were carried out
Generated 60 random graphs with edge-density from 0.1 to 0.9. Tested with different algorithms
Results:
Number of colors increases as edge-density increases (for all heuristics) Backtracking sequential coloring gives the optimal results
Results
Vertices Edges Edgedensity Cliques Iterated Greedy Local Search Backtracking
30 35 40
27 31 35
25 29 33
33 41 49
Observations Iterated Greedy and local search found better solutions Number of edges increases, number of colors increases Number of cliques increases, number of colors increases Yet, as edge-density decreases, number of colors increases
Mycielski Graph
Grotzsch graph with the smallest number of vertices: Mycielski Graph k-Mycielski graph has 32n-2-1 nodes for n>1
Mycielski Graph
1.
Results
1. Base Case: 11 nodes and 20 edges
Algorithms Degree of Saturation Iterated Greedy Local Search Number of colors 4 4 4
Results
2. More cliques: 11 nodes and 25 edges
Algorithms Degree of Saturation Iterated Greedy Local Search Number of colors 4 -
Results
3. More nodes and edges: 23 nodes and 44 edges
Algorithms Degree of Saturation Iterated Greedy Local Search Number of colors 4 3 -
Exceptions
Leighton graph
Vertices Edges Iterated Greedy 18 Local Search 18 Backtracking 25 Optimal Edgedensity 0.08 450 8169 15
169
6656
13
Consolidated Results
Cases Case 1: 11 nodes 20 edges Edgedensity 0.55 Number of colors 4 Maximum cliques 2 Node-edge density 5 edges: 1 4 edges: 5 3 edges: 5 5 edges: 6 4 edges: 5 17 edges: 1 4 edges: 5 3 edges: 17
0.44 0.52
4 3
3 3