I NDIAN I NSTITUTE OF T ECHNOLOGY, K ANPUR D EPARTMENT OF C OMPUTER S CIENCE AND E NGINEERING

CS698D: Special Topics in Data Compression Assignment : 01

Muktinath Vishwakarma (13111034) January 28, 2014

Question 01: Write a decompressor algorithm for Move-to-Front coding. Answer : D ECOMPRESSOR A LGORITHM FOR M OVE - TO -F RONT CODING : Input File: InputFile : Contain the Compressed data, which is going to be decompressed. Initialization : OutputFile : To store the output of decompressed input le. SymbolTable : To store all the symbol present in the input le. Token1, Token2 and Token3: To hold the current word from the Input le Token1 = Read(InputFile); Token2 = Read(InputFile); while Token1 != EOF do if Token1 == Numeric and Token2 == String then AddInFront(SymbolTable, Token2); Write(OutputFile,Token2); Token1 = Read(InputFile); Token2 = Read(InputFile); end if if (Token1 == Numeric) and (Token2 == Numeric)) then Write(OutputFile,SymbolTable[Token1]); Token3=Delet(SymbolTable[Token1]; Token1 = Token2; Token2 = Read(InputFile); AddInFront(SymbolTable,Token3); end if if (Token2 == EOF) then Write(OutputFile,SymbolTable[Token1]; Token3=Delet(SymbolTable[Token1]); Token1=Token2; AddInFront(SymbolTable,Token3); end if end while Return OutputFile; Question 02: Suppose you have the binary alphabet, and the bits are drawn uniformly at random. What is the expected length of the rst run? What happens to the expected run length for larger alphabets? Solution : Let, X be the random variables and n be the length of alphabet.

1 Probability of same alphabet repeated is = P (R ) = n 1 1 Probability of same alphabet is not repeated is = P (N R ) = 1 n = nn . We need to nd E [ X ],

E [ X ] = P (R ) (1 + E [ X ]) + P (N R ) 1 1 1 (1 + E [ X ]) + (1 ) 1 n n 1 + E [X ] + n 1 E [X ] = n E [X ] + n E [X ] = n n E [X ] = E [X ] + n n E [X ] = n 1 So, for larger alphabet of length n, Expected length of the rst run will be, E [X ] = n n 1 For Binary alphabet (i.e. n = 2), Expected length of the rst run will be E [X ] = E [X ] = . 2 =2 21

Question 03: Suppose a sequence of words is lexicographically sorted. Show that the longest common prex of a word with its ancestors is obtained when comparing with its immediately preceding word. Proof : We will prove it by contradiction, Let A,B and C are the three lexicographically sorted words and assume that A and C are having the Longest Common Prex of length K . Now, length of prex between the A and B should be less than K , Let consider it as (K 1) then it implies that K t h character of A and B are different. Since they are lexicographically sorted, it means that K t h character of B is greater than K t h character of A. Similarly, length of prex between the B and C should be less than K , Let consider it as (K 1) then it implies that K t h character of B and C are different. Since they are lexicographically sorted, it means that K t h character of C is greater than K t h character of B. By Transitivity Rule K t h character of A is smaller than K t h character of C, which contradict our assumption.

Hence, Longest common prex of a word with its ancestors is obtained when comparing with its immediately preceding words, in lexicographically sorted words.

Question 04: Let X be any arbitrary random variable taking values in a nite set . Show that H [Y | X ] = 0 if and only if Y is a function of X . Proof : Let rst consider H (Y | X ) = 0, Then from the denition of the joint entropy of two random variables X and Y (i.e. H (Y | X )), H (Y | X ) =
x,y

p X ,Y (x , y ).l og p Y | X ( y |x ) = 0

This will true only when, if in the above expression, any one of the term will be zero. i.e. p X ,Y ( x , y ) = 0 Or, p Y | X ( y |x ) = 1 It means that Y is certain if X is known or, Y is dependent on X . Hence, Y is function X . Now consider, Y is function of X . This implies that, if X is known then Y is certain. That means that p Y | X ( y |x ) = 1, so, from the, H (Y | X ) =
x,y

p X ,Y (x , y ).l og p Y | X ( y |x )

H (Y | X ) = 0 Hence, H (Y | X ) = 0 if and only if Y is function of X .

Question 05: (A) Show that a monotone decreasing sequence of real numbers always has a limit. Proof : If we can prove that decreasing sequence of real number is converges then we can say that it has nite limit.

Thus, suppose {x n } is decreasing and bounded below. Then the set X = {ax n : n N} is a non-empty set which is bounded below. By the completeness axiom, this set has a least lower bound x . That is inf x n = inf X = x
n

is nite. We will show that x is the limit of the sequence {x n }. Given > 0, the number x + is greater than x . So, x is a lower bound for X . This means that there is some natural number N such that x N > (x + ). If n > N , then x n x N since {x n } is a decreasing sequence of real numbers. This implies that x n < (x + ). We also have x x n < (x + ), since x is lower bound for {x n }. Combining these inequalities yields. x < x n (x + ), n > N This is equivalent to, |x n x | > , n > N We conclude that lim x n = x . Hence {x n } converges, so decreasing sequence of real numbers always has a limit.

Question 05: (B) Show that any sequence of real numbers has a lim sup and lim inf. Proof : As we prove in 5(A) that every real decreasing sequence has to have subsequential limit in R. Then either this set in unbounded, in which case the lim inf is , or it is bounded below, in which case inf of set of subsequential limits is well dened and lim inf. Similarly We can show that in the case of real increasing sequence has to have subsequential limit in R. Then either this set in unbounded, in which case the lim sup is , or it is bounded upper, in which case sup of set of subsequential limits is well dened and lim sup. Hence every any sequence of real numbers has a lim sup and lim inf.

Question 06: f = O (g ). Proof :

Using any of the notion lim sup, lim inf or lim, dene the notion

We will use lim sup to dene the notion f = O (g ). To dene the above notion we need one denition and one lemma, which i am going to state here. Denition : Let f : N R. we say that = lim supn f (n ), if the following two conditions holds, (1) : ( > 0)(N)(n > N )( f (n ) < ( + )) (2) : ( > 0)(N)(n > N )( f (n ) > ( )) Lemma: Suppose that ( f (n ))n 0 is bounded by a number C > 0. Then there exist an upper limit of this sequence and lim supn f (n ) C . Now we will start the proof: Let f , g : N R then, ( f = O (g )) lim sup | f (n )| 0 n |g (n )|

Suppose f = O (g ), Let us x C and N such that | f (n )| C .|g (n )| for all n > N. So let us consider n > N, if g (n ) = 0. Then | f (n )| C |g (n )| if g (n ) = 0 and also f (n ) = 0,and accordance with the convention | f (n )| =1 |g (n )| So in both cases for n > N. We have. | f (n )| max {C , 1} = C |g (n )| This means that the sequence | g (n ) | is bounded from above by the number C , Hence from Lemma, We obtain | f (n )| C < | g (n )| Suppose now that lim supn |g (n )| C , we put = 1 into the rst condition of denition, and nd N, such that for all n > N we have, | f (n )| (C + 1) |g (n )|
| f (n )| f (n ) 0 0

= 1. We have,

Hence for all n > N, we have | f (n )| (C + 1)|g (n )| f = O (g ) .

1 Question 07: Show that f (x ) = x has a discontinuity at 0 and is continuous on all points in R {0}. Proof : Denition : A function f : R R is continuous, if for every x R, for any sequence of reals (x n ) n =1 with limit x , we have that limn f (x n ) = f (x ). Given function,

f (x ) = Let rst evaluate at x = 0. For x = 0,

x 0

1 x

lim f (x ) = lim

1 = x 0 x

and, f (x ) = f (0) = So, for x = 0,

x 0

lim f (x ) = f (x )

Hence, at x = 0, f (x ) is discontinuous. Now, Let consider any point k , where k R {0}, then, for x = k , lim f (x ) = lim 1 1 = x k x k

x k

and, f (x ) = f (k ) = So, for x = k ,

x k

1 k

lim f (x ) = f (x )

Hence, at x = k , f (x ) is continuous. 1 has a discontinuity at x = 0 and is continuous on all Hence it is proved that, f (x ) = x points in R {0}.

Question 08: Show that every continuous bounded function f : [0, 1] R is Riemannintegrable. Proof : We know that f is continuous over a closed and bounded interval. Therefore f must be uniformly continuous over [0, 1], i.e. for any given > 0 we can nd a > 0 such that | f (x ) f ( y )| < for all x and y with |x y | < . Now take any > 0 and choose a partition P with |P | < . Then
n

|U ( f , P ) L ( f , P )|
j =1

|c j d j |(x j x j 1 )

where c j is the sup of f over [x j 1 , x j ] and d j is the i n f over that interval. Since the function is continuous, it assumes its maximum and minimum over each of the subintervals, so that i n f and sup can be replaced with mi n and max . Since the norm of the partition is less than , we know that |c j d j | < , j . But Then,
n

|U ( f , P ) L ( f , P )|
j =1 n

(x j x j 1 )

=
j =1

(x j x j 1 ) = (x n x 0 ) = (1 0) =

Because the last sum is telescopic. That nishes the Proof.

Question 09: Compute 0 x 2 d x using the denition of Riemann integral. Answer: For this problem we will rst calculate the lower (i.e L n ( f )) and upper (i.e. U n ( f )) Riemann sum. Where, L n ( f ) is dened as lower Riemann sum of n (where n=1,2,3...) equal size interval over

[0, 1]. U n ( f ) is dened as upper Riemann sum of n (where n=1,2,3...) equal size interval over [0, 1]. Given function is f (x ) = x 2 , Let consider n=1; L 1 ( f ) = 1. f (0) = 1.(0)2 = 1.0 = 0 U 1 ( f ) = 1. f (1) = 1.(1)2 = 1.1 = 1 For n=2, 1 1 1 1 1 1 1 L 2 ( f ) = . f (0) + . f ( ) = .0 + . = 2 2 2 2 2 4 8 1 1 1 1 5 1 1 U 2 ( f ) = . f ( ) + . f (1) = . + .1 = 2 2 2 2 4 2 8

1 1 1 1 2 1 1 1 1 4 5 L 3 ( f ) = . f (0) + . f ( ) + . f ( ) = .0 + . + . = 3 3 3 3 3 3 3 9 3 9 27 1 2 1 1 1 1 4 1 14 1 1 U 3 ( f ) = . f ( ) + . f ( ) + . f (1) = . + . + .1 = 3 3 3 3 3 3 9 3 9 3 27 Similarly we can nd for n=4,5,6... To generalize this notion, we have to divide the interval [0, 1] in n equal interval, such 1 k that k t h interval be [ k n , n ] (where k=0,1,2,3,....). 2 For this interval supremum value of function will be f (x k ) and inmum will be 2 f (x k +1 ). We know the formula for the sum of n squared number term : 12 + 22 + 32 + 42 + ....... + n 2 = Now we have, 1 2 1 n 2 1 1 1 1 xk = 3 k = n (n + 1)(2n + 1) = (1 + )(2 + ) U (f ) = 3 n k =1 6.n 6 n n k =1 n
n n

For n=3,

1 n (n + 1)(2n + 1) k =1 6

Ln ( f ) =

n 1

1 2 1 n 1 2 1 1 1 1 x k 1 = 3 k = n (n 1)(2n 1) = (1 )(2 ) 3 n k =1 6.n 6 n n k =1 n

Take limit to innity for both Lower and Upper Riemann sum. 1 1 1 1 1 (1 + )(2 + ) = .(1 + 0).(2 + 0) = x x 6 n n 6 3 1 1 1 1 1 lim L n f (n ) = lim (1 )(2 ) = .(1 0).(2 0) = x x 6 n n 6 3 lim U n f (n ) = lim

So,
x

lim U n f (n ) = lim L n f (n )
x

Thus in the interval [0, 1] the Riemann Integral

1 0

x 2d x =

1 3

Question 10: Show that if A 1 , A 2 F , then ( A 1 A 2 ) = 0 implies that ( A 1 ) = ( A 2 ). Proof : It is given that ( A 1 A 2 ) = 0 It implies that, (( A 1 A 2 ) ( A 2 A 1 )) = 0 It implies that, ( A 1 A 2 ) + ( A 2 A 1 ) = 0 (1) Equation (1) is possible only when, ( A 1 A 2 ) = 0 and, ( A 2 A 1 ) = 0 So, we can say that, ( A 1 A 2 ) = ( A 2 A 1 ) (2) Add, ( A 1 A 2 ) both side of Equation (2).. ( A 1 A 2 ) + ( A 1 A 2 ) = ( A 2 A 1 ) + ( A 1 A 2 ) (3) Since we know that, ( A 1 A 2 ) + ( A 1 A 2 ) = ( A 1 )

10

Hence, From Equation (3), It is clear that, ( A 1 ) = ( A 2 ) Hence Proof is complete.

11