Prerequisites

Required:
Fluency in C/C++ Programming
Data structures and algorithms
Any mathematics course

Recommended:
Basic operating system concepts

Textbooks
Introduction to The Design and Analysis of
Algorithms 2nd Edition, Anany Levitin, AdisonWesley, 2007.
Data Structure and Algorithm Analysis in C++, 4th
Edition, Mark Allen Weiss

Course Objectives
This course introduces students to the analysis
and design of computer algorithms. Upon
completion of this course, students will be
able to do the following:
Analyze the asymptotic performance of algorithms.
Demonstrate a familiarity with major algorithms and data
structures.
Apply important algorithmic design paradigms and
methods of analysis.
Synthesize efficient algorithms in common engineering
design situations.
3

Where We're Going

Learn general approaches to algorithm design
Divide and conquer
Greedy method
Dynamic Programming
Basic Search and Traversal Technique
Graph Theory
Backtracking

What is Algorithm?
Algorithm
is any well-defined computational procedure that
takes some value, or set of values, as input and
produces some value, or set of values, as output.
is thus a sequence of computational steps that
transform the input into the output.
is a tool for solving a well - specified
computational problem.
Any special method of solving a certain kind of
problem (Webster Dictionary)
5

What is a Program?
A program is the expression of an algorithm in
a programming language
a set of instructions which the computer will
follow to solve a problem

Solutions to Programming Problems are formulated

as algorithms
An algorithm is a well-defined procedure consisting
of a number of instructions that are executed in
turn in order to solve the given problems

Problem

Algorithm

Input

Computer

Output

Algorithmic solution

Some Applications
Study problems these techniques can be
applied to
Sorting
Data retrieval
Network routing
Games
etc.

The Study of Algorithm

How to devise algorithms
How to express algorithms
How to validate algorithms
How to analyze algorithms
How to test a program

10

Importance of Analyze Algorithm

Need to recognize limitations of various
algorithms for solving a problem
Need to understand relationship between
problem size and running time
Need to learn how to analyze an algorithm's
running time without coding it
Need to learn techniques for writing more
efficient code
Need to recognize bottlenecks in code as well as
which parts of code are easiest to optimize
11

Why do we analyze about them?

understand their behavior, and (Job -Selection, performance, modify)
improve them. (Research)

12

What do we analyze about them?

Correctness
Does the input/output relation match algorithm
requirement?

Amount of work done (aka complexity)

Basic operations to do task

Amount of space used

Memory used

13

What do we analyze about them?

Simplicity, clarity
Verification and implementation.

Optimality
Is it impossible to do better?

14

Complexity
The complexity of an algorithm is simply the
amount of work the algorithm performs to
complete its task.

15

RAM model
has one processor
executes one
instruction at a time
each instruction takes
"unit time
has fixed-size operands,
and
has fixed size storage
(RAM and disk).
16

At least three important questions need to be answered

for each algorithm
Is it correct?
How much time does it take, as a function of n?
And can we do better?

Being problem solvers, we need to equip ourselves with

the tools and techniques to answer these questions.
We will first focus on point 2.
17

An Example
Consider the problem of sorting numbers.
INPUT: Sequence of n numbers <a1,a2,a3, .an>
OUTPUT: Permutation (reordering) <a1`,a2`,a3`,.an`> of the input
sequence such that a1`<a2`<a3`<..<an`
Many algorithms are available.

18

An algorithm for sorting.

Get a list of unsorted numbers
Set a marker for the unsorted section at the front of the list
Repeat steps 4 - 6 until one number remains in the unsorted section
Compare all unsorted numbers in order to select the smallest one
Swap this number with the first number in the unsorted section
Advance the marker to the right one position
Stop

19

Another algorithm for sorting

Divide sequence of m elements into two sequences of m/2
elements
Conquer both sub-sequences using Merge Sort
(recursively)
Combine two sorted sub-sequences using merge

20

How much time does each algorithm take, as a function of n?

Implement both algorithms

Note down the execution time for each algorithm
Compare the times

21

Possible steps?
Understand the problem
Formulate a solution / algorithm
Analyze the algorithm

Design a program
Implement the program
Execute the code
Measure the time

See if the solution is ok

Otherwise try to come up with another solution and again start from step 1

22

A Typical Result for both the

algorithms
Sorting Analysis, processor speed 500MHz
Data Size
Execution Time in Seconds
Selection Sort
Merge Sort
10 K
1
0
20 K
4
0
30 K
11
0
50 K
31
0
75 K
72
0
1 Lac
132
0
2 Lac
586
1
1 Million
10
2 Million
21
3 Million
33
5 Million
60
7.5 Million
90
10 Million
136

23

Another view of the results

Sorting Analysis
700

500
400

Data Size

300

Selection Sort
Merge Sort

200
100

illi
o

illi
o
10

M
5

Data Size

n
illi
o

La
2

K
75

K
30

10

Execution time in seconds

600

24

 How to analyze an algorithm with minimum

effort?
Predict the resources that the algorithm
requires
Memory
Communications Bandwidth
Logic gates etc
Most important is Computational Time
25

Possible steps with minimum effort?

1. Understand the problem
2. Formulate a solution / algorithm
3. Analyze the algorithm
a)
b)
c)
d)

Design a program
Implement the program
Execute the code
Measure the time

4. See if the solution is ok

5. Otherwise try to come up with another solution
and again start from step 1
26

 Important things before analysis

Model of the machine upon which the
algorithms is executed.
Random Access Machine (RAM)
(Sequential) to start with

Running Time:
No. of primitive operations or steps
executed.
27

Time efficiency is analyzed by determining the number

of repetitions of the basic operation as a function of
input size
input size

T(n) copC(n)
running time

execution time
for basic operation

Number of times
basic operation is
executed

28

 How do we write algorithms?

Pseudo Code:
Similar construct / keywords as in a
high level programming languages, e.g.
in C, Pascal etc.
Structured semantics of the high level
languages without caring about the
syntactic errors / grammatical rules

29

Sample Pseudo Code

Max-Subsequence-Sum(Array, N)
{

//Where N is size of Array

int this-sum = 0, Max-sum = 0;

for(int i = 0; i < N; i++)
{ for(int j = i; j < N; j++)
{
this-sum = 0;
for(int k = i; k <= j; k++)
this-sum = this-sum + Array[k];
if(this-sum > Max-sum)
Max-sum = this-sum;
}
}
return(Max-sum);

}
30

How much time each construct / keyword of a pseudo code takes to

execute.
Assume it takes ti (the ith construct)
Sum / Add up the execution time of all the constructs / keywords.
if there are m constructs then total time for all the constructs is
m

T =

i =1

31

 That will be the execution time for a given

input size (say n)
Running time as the function of the input size T(n)
m

Tn =

i =1

32

What are the constructs / Keywords.

Time for each construct
Total Time
Total time as a function of input size

33

 Constructs:
1. Sequence
2. Selection
3. Iterations
4. Recursion

34

Analysis Example
Max-Subsequence-Sum(Array, N)
{

//Where N is size of Array

int this-sum = 0, Max-sum = 0;

for(int i = 0; i < N; i++)
{ for(int j = i; j < N; j++)
{
this-sum = 0;
for(int k = i; k <= j; k++)
this-sum = this-sum + Array[k];
if(this-sum > Max-sum)
Max-sum = this-sum;
}
}
return(Max-sum);

}
35

A Maximum Sub Sequence Sum

Problem

Solve the summation

= c1n3 + c2n2 + c3n
= O(n3)
If n = 1000, Time constant = 1msec
=> T 109/(24*1000*60*60 )
11 days

36

Which algorithm is better?

The algorithms are correct,
but which is the best?
Measure the running time
(number of operations
needed).
Measure the amount of
memory used.
Note that the running time
of the algorithms increase
as the size of the input
increases.

37

What do we need?
Correctness: Whether the algorithm computes
the correct solution for all instances
Efficiency: Resources needed by the algorithm
1. Time: Number of steps.
2. Space: amount of memory used.
Measurement model: Worst case, Average case
and Best case.

38

What is Algorithm Analysis?

How to estimate the time required for an
algorithm
Techniques that drastically reduce the running
time of an algorithm
A mathemactical framework that more
rigorously describes the running time of an
algorithm

pp 39