RAIK 283: Data Structures & Algorithms

Analysis of Algorithms
Dr. Ying Lu ylu@cse.unl.edu
August 28, 2012

http://www.cse.unl.edu/~ylu/raik283/

Design and Analysis of Algorithms Chapter 2.1

RAIK 283: Data Structures & Algorithms

Giving

credit where credit is due:

Most of the lecture notes are based on the slides from the Textbooks companion website http://www.aw-bc.com/info/levitin Several slides are from Jeff Edmonds of the York University I have modified them and added new slides

Design and Analysis of Algorithms Chapter 2.1

The Time Complexity of an Algorithm

The Time Complexity of an Algorithm

Specifies how the running time depends on the size of the input

Purpose

Design and Analysis of Algorithms Chapter 2.1

Purpose
To estimate how long a program will run. To estimate the largest input that can reasonably be given to the program. To compare the efficiency of different algorithms. To help focus on the parts of code that are executed the largest number of times. To choose an algorithm for an application.

Design and Analysis of Algorithms Chapter 2.1

Purpose (Example)

Suppose a machine that performs a million floatingpoint operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50?
1) If the algorithm requires n2 such operations:

Design and Analysis of Algorithms Chapter 2.1

Purpose (Example)

Suppose a machine that performs a million floatingpoint operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50?
1) If the algorithm requires n2 such operations:

0.0025 second

Design and Analysis of Algorithms Chapter 2.1

Purpose (Example)

Suppose a machine that performs a million floatingpoint operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50?
1) If the algorithm requires n2 such operations:

0.0025 second 2) If the algorithm requires 2n such operations:

A) Takes a similar amount of time (t < 1 sec) B) Takes a little bit longer time (1 sec < t < 1 year) C) Takes a much longer time (1 year < t)

Design and Analysis of Algorithms Chapter 2.1

Purpose (Example)

Suppose a machine that performs a million floatingpoint operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50?
1) If the algorithm requires n2 such operations:

0.0025 second 2) If the algorithm requires 2n such operations: over 35 years!

Design and Analysis of Algorithms Chapter 2.1

10

Time Complexity Is a Function

Specifies how the running time depends on the size of the input. A function mapping size of input

time T(n) executed .

Design and Analysis of Algorithms Chapter 2.1

11

Definition of Time?

Design and Analysis of Algorithms Chapter 2.1

12

Definition of Time

# of seconds (machine, implementation dependent). # lines of code executed. # of times a specific operation is performed addition). (e.g.,

Design and Analysis of Algorithms Chapter 2.1

13

Theoretical analysis of time efficiency

Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size

Basic operation: the operation that contributes most towards the running time of the algorithm.
input size

T(n) copC(n)
running time execution time for basic operation Number of times basic operation is executed
14

Design and Analysis of Algorithms Chapter 2.1

Input size and basic operation examples

Problem
Search for key in a list of n items Multiply two matrices of floating point numbers Compute an

Input size measure

Basic operation

Graph problem
Design and Analysis of Algorithms Chapter 2.1
15

Input size and basic operation examples

Problem Input size measure Basic operation
Key comparison

Search for key in a list of Number of items in the n items list: n Multiply two matrices of floating point numbers Compute an

Graph problem
Design and Analysis of Algorithms Chapter 2.1
16

Input size and basic operation examples

Problem Input size measure Basic operation
Key comparison

Search for key in a list of Number of items in the n items list: n Multiply two matrices of Dimensions of matrices floating point numbers Compute an

Floating point multiplication

Graph problem
Design and Analysis of Algorithms Chapter 2.1
17

Input size and basic operation examples

Problem Input size measure Basic operation

Search for key in list of n Number of items in list n Key comparison items Multiply two matrices of Dimensions of matrices floating point numbers Compute an Floating point multiplication

Floating point multiplication

Graph problem
Design and Analysis of Algorithms Chapter 2.1
18

Input size and basic operation examples

Problem Input size measure Basic operation

Search for key in list of n Number of items in list n Key comparison items Multiply two matrices of Dimensions of matrices floating point numbers Compute an Floating point multiplication

Floating point multiplication

Visiting a vertex or traversing an edge
19

Graph problem

#vertices and/or edges

Design and Analysis of Algorithms Chapter 2.1

Theoretical analysis of time efficiency

Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size

Design and Analysis of Algorithms Chapter 2.1

20

Which Input of size n?

Efficiency also depends on the particular input

For

instance: search a key in a list of n letters

Problem input: a list of n letters How many different inputs?

Design and Analysis of Algorithms Chapter 2.1

21

Which Input of size n?

Efficiency also depends on the particular input

For instance: search a key in a list of n letters There are 26n inputs of size n. Which do we consider for the time efficiency C(n)?

Design and Analysis of Algorithms Chapter 2.1

22

Best-case, average-case, worst-case

Worst case: Best case:

W(n) maximum over inputs of size n B(n) minimum over inputs of size n

Average case: A(n) average over inputs of size n

NOT the average of worst and best case Under some assumption about the probability distribution of all possible inputs of size n, calculate the weighted sum of expected C(n) (numbers of basic operation repetitions) over all possible inputs of size n.

Design and Analysis of Algorithms Chapter 2.1

23

Example: Sequential search

Problem: Given a list of n elements and a search key K, find an element equal to K, if any. Algorithm: Scan the list and compare its successive elements with K until either a matching element is found (successful search) or the list is exhausted (unsuccessful search) Worst case

Best case
Average case
Design and Analysis of Algorithms Chapter 2.1
24

An example

Compute gcd(m, n) by applying the algorithm based on checking consecutive integers from min(m, n) down to gcd(m, n) Input size? Best case? Worst case? Average case?

Design and Analysis of Algorithms Chapter 2.1

25

Types of formulas for basic operation count

Exact formula e.g., C(n) = n(n-1)/2 Formula indicating order of growth with specific multiplicative constant e.g., C(n) 0.5 n2 Formula indicating order of growth with unknown multiplicative constant e.g., C(n) cn2

Design and Analysis of Algorithms Chapter 2.1

26

Types of formulas for basic operation count

Exact formula e.g., C(n) = n(n-1)/2 Formula indicating order of growth with specific multiplicative constant e.g., C(n) 0.5 n2 Formula indicating order of growth with unknown multiplicative constant e.g., C(n) cn2 Most important: Order of growth within a constant multiple as n
Design and Analysis of Algorithms Chapter 2.1
27

Asymptotic growth rate

A way of comparing functions that ignores constant factors and small input sizes O(g(n)): (g(n)): (g(n)):

Design and Analysis of Algorithms Chapter 2.1

28

Asymptotic growth rate

A way of comparing functions that ignores constant factors and small input sizes O(g(n)): class of functions f(n) that grow no faster than g(n) (g(n)): class of functions f(n) that grow at same rate as g(n) (g(n)): class of functions f(n) that grow at least as fast as g(n)

Design and Analysis of Algorithms Chapter 2.1

29

Table 2.1

Design and Analysis of Algorithms Chapter 2.1

30

Classifying Functions

Giving an idea of how fast a function grows without going into too much detail.

Which are more alike?

Design and Analysis of Algorithms Chapter 2.1

32

Which are more alike?

Mammals

Design and Analysis of Algorithms Chapter 2.1

33

Which are more alike?

Design and Analysis of Algorithms Chapter 2.1

34

Which are more alike?

Dogs

Design and Analysis of Algorithms Chapter 2.1

35

Classifying Animals
Vertebrates Fish Birds

Mammals Dogs
Giraffe

Design and Analysis of Algorithms Chapter 2.1

Reptiles

36

Which are more alike?

n1000

n2

2n

Design and Analysis of Algorithms Chapter 2.1

37

Which are more alike?

n1000

n2
Polynomials

2n

Design and Analysis of Algorithms Chapter 2.1

38

Which are more alike?

1000n2

3n2

2n3

Design and Analysis of Algorithms Chapter 2.1

39

Which are more alike?

1000n2
Quadratic

3n2

2n3

Design and Analysis of Algorithms Chapter 2.1

40

Classifying Functions?
Functions

Design and Analysis of Algorithms Chapter 2.1

41

Classifying Functions
Functions

Constant

Poly Logarithmic
(log n)5

Exponential

5 log n

Logarithmic

Design and Analysis of Algorithms Chapter 2.1

Polynomial
n5

Factorial
n!
42

25n

Classifying Functions?
Polynomial

Design and Analysis of Algorithms Chapter 2.1

43

Classifying Functions
Polynomial Linear
5n

5n2

Design and Analysis of Algorithms Chapter 2.1

Quadratic

5n3

Cubic

5n4

44

Logarithmic

log10n

= # digits to write n log2n = # bits to write n = 3.32 log10n log(n1000) = 1000 log(n)

Differ only by a multiplicative constant

Design and Analysis of Algorithms Chapter 2.1

45

Poly Logarithmic

(log n)5 = log5 n

Design and Analysis of Algorithms Chapter 2.1

46

Which grows faster?

log1000 n

n0.001

Design and Analysis of Algorithms Chapter 2.1

47

Poly Logarithmic << Polynomial

log1000 n << n0.001

For sufficiently large n

Design and Analysis of Algorithms Chapter 2.1

48

Which grows faster?

10000 n

0.0001 n2

Design and Analysis of Algorithms Chapter 2.1

49

Linear << Quadratic

10000 n << 0.0001 n2

For sufficiently large n

Design and Analysis of Algorithms Chapter 2.1

50

Which grows faster?

n1000

20.001 n

Design and Analysis of Algorithms Chapter 2.1

51

Polynomial << Exponential

n1000 << 20.001 n

For sufficiently large n

Design and Analysis of Algorithms Chapter 2.1

52

Ordering Functions
Functions

5 << 5 log n << (log n)5 << n5

Constant

Poly Logarithmic

Exponential

Logarithmic

Polynomial

Factorial

<<

<<

<<

<<

<<

<<

25n <<

n!
53

Design and Analysis of Algorithms Chapter 2.1

Which Functions are Constant?

5 1,000,000,000,000 0.0000000000001 -5 0 8 + sin(n)

Design and Analysis of Algorithms Chapter 2.1

54

Which Functions are Constant?

Yes 5 Yes 1,000,000,000,000 Yes 0.0000000000001 Yes -5 Yes 0 No 8 + sin(n)

Design and Analysis of Algorithms Chapter 2.1

55

Which Functions are Constant?

The running time of the algorithm is a Constant if it does not depend significantly on the size of the input. 5 1,000,000,000,000 0.0000000000001 -5 0 8 + sin(n)
Design and Analysis of Algorithms Chapter 2.1
56

Which Functions are Constant?

The running time of the algorithm is a Constant It does not depend significantly on the size of the input. Yes Yes Yes No No Yes 5 1,000,000,000,000 9 0.0000000000001 7 -5 0 Lie in between 8 + sin(n)
Design and Analysis of Algorithms Chapter 2.1
57

Which Functions are Quadratic?

n2 0.001 n2 1000 n2 5n2 + 3n + 2log n

Design and Analysis of Algorithms Chapter 2.1

58

Which Functions are Quadratic?

n2 0.001 n2 Lie in between 1000 n2 5n2 + 3n + 2log n

Design and Analysis of Algorithms Chapter 2.1

59

Which Functions are Quadratic?

n2 0.001 n2 1000 n2 5n2 + 3n + 2log n Ignore low-order terms Ignore multiplicative constants. Ignore "small" values of n. Write (n2).
Design and Analysis of Algorithms Chapter 2.1
60

Examples
f(n) 1) 2) 3) 4) ln2n nk g(n) ne cn nsinn 2n/2 O(g(n))? (g(n))? (g(n))?

n
2n

5)
6)

nlgc

clgn

lg(n!) lg(nn)
61

Design and Analysis of Algorithms Chapter 2.1