i Hc Khoa Hc T Nhin-i Hc Quc Gia H Ch Minh Khoa in T Vin Thng

Lp Cao Hc in T - Vin Thng - My Tnh Kha 21

.ooOoo.

Mn hc : Phng php phn tch thng k

ti : Thut ton m ha Viterbi

GVHD : PGS.TS Nguyn Hu Phng SV : Bi Thanh Sn

Ni Dung
1. Gii thiu l thuyt m ha .................................................................................................3 1.1 L thuyt m ha ...............................................................................................................3 1.2. M ha knh truyn ..........................................................................................................3 1.3. M khi tuyn tnh............................................................................................................4 1.4. M kt hp .......................................................................................................................5 2. 3. C s ton hc ca thut ton Viterbi ..................................................................................7 Ti liu tham kho ............................................................................................................. 14

1. Gii thiu l thuyt m ha

1.1 L thuyt m ha (ting Anh: coding theory) l mt ngnh ca ton hc (mathematics)
v khoa hc in ton (computer science)) nhm gii quyt tnh trng li d xy ra trong qu trnh truyn thng s liu trn cc knh truyn c nhiu cao (noisy channels)), dng nhng phng php tinh xo khin phn ln cc li xy ra c th c chnh sa. N cn x l nhng c tnh ca m (codes)), v do vy gip ph hp vi nhng ng dng c th. C hai loi m hiu: 1. M ha dng ngun (M ha entrpi (Entropy encoding)) 2. M ha trn knh truyn (Sa li pha trc (Forward error correction)) Ci u tin chng ta ni n l m ha dng ngun. nh ca phng php ny l nn d liu t chnh ngun ca n, trc khi truyn i, gip cho vic truyn thng c hiu qu hn. Chng ta chng kin thi quen ny hng ngy trn Internet, nht l trong cch dng "zip" nn d liu gim lng d liu phi truyn, gim nh gnh nng cho mng li truyn thng, ng thi thu nh c tp tin. Ci th hai l m ha trn knh truyn. Bng vic cng thm nhng bit mi vo trong d liu c truyn, cn gi l bit chn l (parity bits), k thut ny gip cho vic truyn thng tn hiu chnh xc hn trong mi trng nhiu lon ca knh truyn thng. C nhiu chng trnh ng dng, m ngi dng trung bnh khng n, s dng m ha trn knh truyn. K thut Reed-Solomon c dng nhm sa li do nhng vt xc v bi trn b mt a m nhc CD thng thng. Trong ng dng ny, knh truyn thng li chnh l bn thn ci a CD. in thoi di ng "Cell phones" cng dng k thut m ha c hiu ng cao (powerful coding technique) sa cc li trong vic truyn sng rai tn s cao b yu m v b nhiu. Modem x l s liu, vic truyn thng qua ng in thoi, v ng nhin ngay c chnh NASA, tt c u s dng k thut m ha trn knh truyn hiu ng truyn nhng bit s liu qua ng dy.

1.2. M ha knh truyn

Mc ch ca l thuyt M ha trn knh truyn (channel encoding theory) l tm nhng m c th truyn thng nhanh chng, cha ng nhiu m k (code word) hp l v c th sa li (error correction) hoc t nht pht hin cc li xy ra (error detection). Cc mc ch trn khng ph thuc vo nhau, v mi loi m c cng dng ti u cho mt ng dng ring bit. Nhng c tnh m mi loi m ny cn cn tu thuc nhiu vo xc sut li xy ra trong qu trnh truyn thng. i vi mt a CD thng thng, li trong m thanh xy ra ch yu l do bi v nhng vt xc trn mt a. V th, cc m c lng vo vi nhau. D liu c phn b trn ton b mt a. Tuy khng c tt cho lm, song mt m ti din n gin c th c dng lm mt v d d hiu. Chng hn, chng ta ly mt khi s liu bit (i din cho m thanh) v truyn gi chng ba ln lin. Bn my thu, chng ta kim tra c ba phn lp li trn, tng bit tng bit mt, ri ly ci no c s bu cao nht. im tri khoy y l, chng ta khng ch truyn gi cc bit theo th t. Chng ta lng n vo vi nhau. Khi d liu ny, trc tin, c chia ra lm 4 khi nh. Sau chng ta gi mt bit khi u tin, tip theo mt bit khi th hai v.v tun t qua cc khi. Vic ny c lp i lp li ba ln phn b s liu ra trn b mt a. Trong ng cnh ca m ti din n

gin trn, vic lm ny hnh nh khng c hiu qu cho lm. Song hin nay c nhng m c hiu ng cao, rt ph hp vi vic sa li xy ra t ngt do mt vt xc hay mt vt bi, khi dng k thut lng s liu ni trn. Mi m thng ch thch hp cho mt ng dng nht nh. Vin thng trong v tr (deep space) b gii hn bi nhiu nhit (thermal noise) trong thit b thu. Hin trng ny khng xy ra mt cch t pht bt thng, song xy ra theo mt chu trnh tip din. Tng t nh vy, modem vi di tn hp b hn ch v nhiu m tn ti trong mng li in thoi. Nhng nhiu m ny c th c biu hin r hn bng mt m hnh m tp tip din. in thoi di ng "Cell phones" hay c vn do s suy sng nhanh chng xy ra. Tn s cao c dng c th gy ra s suy sng tn hiu mt cch nhanh chng (rapid fading), ngay c khi my nhn ch di ch vi phn Anh (inches) 1. Mt ln na, ngi ta hin c mt loi thuc hng M ha trn knh truyn c thit k i u vi tnh trng suy sng. T "L thuyt m ha i s" m ch mt chi nhnh ca l thuyt m ha trn knh truyn, trong c tnh ca m c biu hin bng cc i s v da vo m nghin cu su hn. L thuyt m ha i s c chia ra lm 2 loi m chnh 1. M khi tuyn tnh (Linear block codes) 2. M kt hp (Convolutional codes) Chng phn tch ba c tnh sau ca m -- ni chung l:

Chiu di ca m (code word length) Tng s cc m k hp l (total number of valid code words) Khong cch Hamming ti thiu gia hai m k hp l (the minimum Hamming distance between two valid code words)

1.3. M khi tuyn tnh

M khi tuyn tnh mang tnh nng tuyn tnh (linearity), chng hn tng ca hai m k no y li chnh l mt m k; v chng c ng dng vo cc bit ca ngun trn tng khi mt; ci tn m khi tuyn tnh l v vy (linear block codes). C nhng khi m bt tuyn tnh, song kh m chng minh c rng mt m no l mt m tt nu m y khng c c tnh ny. Bt c m khi tuyn tnh no cng c i din l (n,m,dmin), trong 1. n, l chiu di ca m k, trong k hiu (symbols), 2. m, l s k hiu ngun (source symbols) c dng m ha tc thi, 3. dmin, l khong cch hamming ti thiu ca m (the minimum hamming distance for the code) C nhiu loi m khi tuyn tnh, nh 1. M tun hon (Cyclic codes) (M Hamming l mt b phn nh (subset) ca m tun hon) 2. M ti din (Repetition codes)

3. 4. 5. 6. 7.

M chn l (Parity codes) M Reed-Solomon (Reed Solomon codes) M BCH (BCH code) M Reed-Muller M hon ho (Perfect codes)

M khi c gn lin vi bi ton "ng gi ng xu" l bi ton gy mt s ch trong nhiu nm qua. Trn b din hai chiu, chng ta c th hnh dung c vn mt cch d dng. Ly mt nm ng xu, nm trn mt bn, ri dn chng li gn vi nhau. Kt qu cho chng ta mt mu hnh lc gic tng t nh hnh t ong. Cc m khi cn da vo nhiu chiu khc na, khng d g m hnh dung c. M Golay 2 c hiu ng cao, dng trong truyn thng qua khong khng v tr, s dng nhng 24 chiu. Nu c dng l m nh phn (thng thy), cc chiu m ch n chiu di ca m k nh nh ngha trn. L thuyt v m s dng m hnh hnh cu vi s chiu "N". Ly v d, bao nhiu ng xu phi cn ph kn mt mt bn, hay trong khong khng 3 chiu, bao nhiu hn bi phi cn nhi kn mt hnh cu. Nhng cn nhc khc bao gm vic chn la m. Ly v d, do nhi nhmg hnh lc lng vo trong mt ci hp hnh ch nht, chng ta li nhng khong trng cc gc. Khi cc chiu ca hp c tng ln, t l phn trm so snh ca cc khong trng nh i, cho n mt c no y, nhng phn nhi chim ht cc khong khng v m ny c gi m hon ho. S m kiu ny tng i him (Hamming [n,k,3], Golay [24,12,8],[23,12,7],[12,6,6]) Mt iu thng b b qua l s lng nhng hng xm k cn (neighbors) m mt m k c th c. Chng ta c th dng li v d cc ng xu y. u tin, chng ta gp cc ng xu li theo cc hng hnh ch nht. Mi mt ng xu c 4 ng k cn (v 4 ci bn gc xa hn). Trong b cc ca hnh lc gic, mi ng xu c 6 ng k cn. Khi chng ta tng s chiu ln, s lng cc ng k cn tng ln mt cch nhanh trng. Kt qu l s lng cc m tp, bn cch cc m chnh, m my thu c th chn, cng tng ln, v do m gy ra li. y chnh l khuyt im cn bn ca m khi, v cng l khuyt im ca tt c cc loi m. C th vic gy li tr nn kh khn hn, nu ch c mt hng xm k cn m thi, song con s cc hng xm k cn c th ln n lm cho chnh tng xc sut li b nh hng (total error probability actually suffers).

1.4. M kt hp
M kt hp (Convolutional codes) c s dng trong cc modem di tn m (voiceband modems) (V.32, V.17, V.34) v trong cc in thoi di ng GSM, cng nh trong cc thit b truyn thng ca qun i v trang v trong cc thit b truyn thng vi v tinh. nh y l lm cho tt c cc k hiu m k (codeword symbol) tr thnh tng trng s (weighted sum) ca nhiu loi k hiu thng ip trong nhp liu (various input message symbols). N tng t nh ton kt hp c dng trong cc h tuyn tnh bt bin (linear time invariant systems) dng tm xut liu (output) ca mt h thng, khi chng ta bit nhp liu (input) v cc p ng xung (impulse response). Ni chung chng ta tm xut liu ca b m ha kt hp h (system convolutional encoder), tc s kt hp ca nhp liu bit, i chiu vi trng thi ca b m ha kt hp (convolution encoder), hoc trng thi ca cc thanh bin (registers).

V c bn m ni, m kt hp khng gip thm g trong vic chng nhiu hn mt m khi tng ng. Trong nhiu trng hp, chng ni chung cho chng ta mt phng php thc thi n gin hn, hn hn mt m khi c hiu qu tng ng (a block code of equal power). B m ha thng l mt mch in n gin, c mt b nh (state memory), mt vi bin php truyn thng tin phn hi bo tnh hnh (some feedback logic), thng l cc cng loi tr XOR (XOR gates). B m ha c th c thc thi trong phn mm hay phn sn (firmware). Thut ton Viterbi (Viterbi algorithm) l mt thut ton ngn gn nht (optimum algorithm) c dng gii m cc m kt hp. Hin c nhng phng php gim c gip vo vic gim khi lng tnh ton phi lm. Nhng phng php ny phn ln da vo vic tm tuyn ng c kh nng xy ra cao nht (most likely paths). Tuy khng ngn gn, song trong mi trng nhiu thp hn, ngi ta thng thy chng cho nhng kt qu kh quan. Cc b iu hnh vi x l hin i (Modern microprocessors) c kh nng thc hin nhng thut ton tm gim c ni trn vi t l trn 4000 m k trong mt giy. Trong bt k mt h thng truyn dn v tuyn no t tng t n h thng s, cc tn hiu u phi c iu ch ln mt tn s sng mang nht nh truyn dn qua khng gian. Do , d l cc tu v tr, cc tn la hay h thng thng tin t bo th vn tch tn hiu ra khi nhiu ng vai tr ct li th hin tnh u vit ca mi h thng thng tin. Trong v tr, tn hiu lun b chm su vo trong nhiu do nhiu v tuyn t cc thin h hoc qu trnh pht x t mt tri. Hay nh trn mt t, tn hiu ca mt my v tuyn t bo b gy nhiu bi rt nhiu my pht v tuyn t bo khc xung quanh n. Vy lm th no c th tch tn hiu cn thit ra khi nhiu? thc hin iu , ngi ta dng gii php m ha knh cc tn hiu thm vo thng tin gc cc bit d bo v trc khi chng c pht qua knh truyn c nhiu. Bng cch m ha knh nh vy, cc bit thng tin (bit 0 hay 1) c th c biu din khng ch biu din bng mt gc dch pha (iu ch kha pha M-PSK) m cn c th c biu din bng 4, 8 hoc nhiu hn na cc k hiu. Vi h thng dng m ha knh, pha thu chng ta khng tch cc bit thng tin thc t m thay vo l cc k hiu m ha t cc k hiu chng ta c th khi phc li cc bit thng tin gc. tng c bn ca gii php la chn m ha knh l nu nh mt trong s cc k hiu pht qua knh b mo do nhiu th nhng k hiu b nh hng bi nhiu t hn s gip gii m cc bit thng tin thc t vi tin cy cao hn. Trong mi trng nhiu, c mt thc t l nu ngay khi nhn c tn hiu thu chng ta thc hin gii m cc bit 0 hoc 1 th hiu qu s khng cao. Thay vo , chng ta ch nh gi tin cy, (rt tin cy, tin cy, tin cy thp , tin cy rt thp), khi gii m bit 1 hoc bit 0 da trn tn hiu thu c. Vi cch gii m nh vy ngi ta gi l quyt nh mm (soft decision). Nhng thng tin th hin tin cy ca quyt nh mm c a n b gii m tm ra bit thng tin gc l 0 hay 1 thng qua vic so snh kt qu vi nhng bit thng tin ln cn. Gii thut mm nh nu trn s rt phc tp, phc tp s tng theo hm m khi di cc thanh ghi dch trong b to m tng ln. Cng trnh nghin cu trong vng ba thng ca Viterbi lm gim phc tp thng qua vic kt hp cc quyt nh mm quyt nh chui bit thng tin no l ging vi chui bit c pht i nht. Gii php ny ca Viterbi c chng minh l cho kt qu ging vi kt qu ca phng php gii m ti u vi phc tp cao trn. Vi u im , JPL ngay lp tc s dng thut ton gii m

Viterbi trong h thng thng tin v tinh cng nh v tr ca mnh. Sau ny cc h thng thng tin qun s cng nh cc mng thng tin s nh cc tuyn truyn dn viba hay h thng TTD t bo GSM, CDMA u dng thut ton gii m ny gii m m xon.

2. C s ton hc ca thut ton Viterbi

Vi b m c u vo k bit song song, u ra n bit, tc m r = k/n. Chui tn hiu vo k hiu x, x= (x 0 (1) , x 0 (2) , ..., x 0 (k) , x 1 (1) , ..., x 1 (k) , x L+m-1 (1) , ..., x L+m-1 (k) ) T m c, c =(c 0 (1) , c 0 (2) , ..., c 0 (n) , c 1 (1) , ..., c 1 (n) , c L+m-1 (1) , ..., c L+m-1 (n) ) Trong L l di ca chui tn hiu vo, m l di ln nht trong s cc thanh ghi. T m nhn c r r =( r 0 (1) , r 0 (2) , ..., r 0 (n) , r 1 (1) , ..., r 1 (n) , r L+m-1 (1) , ..., r L+m-1 (n) ) Gii m ra y y =(y 0 (1) , y 0 (2) , ..., y 0 (n) , y 1 (1) , ..., y 1 (n) , y L+m-1 (1) , ..., y L+m-1 (n) ) y l mt t m trong b m. Thut ton s tm y sao cho xc sut p(r|y) l ln nht. Thut ton gi s knh truyn l khng c nh, tc l li trn 1 bit bt k khng ph thuc vo cc bit trc n. T l thuyt xc sut, ta c xc sut chung ca cc s kin c lp bng tch xc sut ca cc s kin ring bit.

c lng max(p(r|y)) tng ng max(log p(r|y)) do c:

tnh tng ny n gin ta nh ngha thang o bit nh sau:

Vi a, b c xc nh sao cho thang o bit l 1 s nguyn dng nh Gi tr a v b c nh ngha l knh i xng nh phn (Binary Symmetric Channel BSC) hoc gii m cng (hard-decision decoding) V d mt knh i xng nh phn:

iu ny ging vi khi nim khong cch Hamming m chng ta hc. Gii thut Viterbi chn t m y thng qua cy trellis sao cho t m ny c khong cch Hamming nh nht vi t m nhn c r. Khong cch Hamming ln nht Knh bt k: a v b c th tnh da trn cc li v gi tr kim th t c bit metric chp nhn c

Biu thc ny ch ra tng chi ph cho c lng t chui nhn c r vi t m sau gii m y trn cy trellis. Gi tr nhnh th k c nh ngha:

V mt phn nhnh th k l:

Gi tr nhnh th k a ra chi ph cho vic chn mt nhnh t cy trellis. Gi tr mt phn nhnh th k a ra chi ph cho chn mt t m y vi ch s thi gian l k. Gii thut Viterbi s dng cy trellis tnh ton gi tr ng dn - Mi trng thi (node) trong cy trellis c gn mt gi tr, mt phn gi tr ng dn. - Mt phn gi tr ng dn c xc nh t trng thi 0 ti thi im 0 n trng tip k ti thi im t>=0. - Ti mi trng thi, mt phn gi tr ng dn tt nht hoc l gi tr ln hn hoc nh hn ph thuc vo a v b c chn theo cch 2 hay cch 1. - Gi tr c la chn biu din ng dn cn li v ng dn ny c lu tr cn cc ng khc b loi b khi cy trellis. Gii thut Viterbi chn ng dn cui cng cn li nh l ng dn Maximum Likelihood . Ln tr li theo ng dn ny trn cy trellis s thu c t m cn tm. t = t+1 Tnh mt phn gi tr ng dn trong tt c ng dn bt u trng thi Sk ti thi im t. u tin, tm gi tr nhnh th t

Gi tr ny c tnh t khong cch Hamming

Tip theo, tnh mt phn gi tr ng dn th t

Gi tr ny c tnh t V(S k,t-1 ) + M(r t |y t ) 3. Thit lp V(S k,t ) l gi tr ng dn tt nht t trng thi S k ti thi im t. Mt phn gi tr ng dn tt nht l ng dn c gi tr nh nht. Nu c nhiu ng dn c gi tr nh nht nh nhau th c th chn bt k ng no 4. Lu ng ny v bit cn li v trng thi ca n 5. Nu t11. 1. t=0 V(S 0,0 ) = 0; V(S k,t ) = + < L+m-1 th quay li bc 2 Kt qu ca gii thut Viterbi l ng dn trn cy trellis duy nht tng ng vi t m ML

V d : Chui input x = {1010100} vi 2 bit cui 00 l bit xa thanh ghi Sau khi qua b m ha c c = { 11 , 10, 00, 10, 00, 10, 11}, sau khi truyn qua knh th chui nhn c r = { 10 , 10, 00, 10, 00, 10, 11} c li (gch chn). Hnh sau th hin s dch chuyn trng thi trn cy trellis ca b m ny

T cy trellis trong hnh, chui m c lng y = {11, 10, 00, 10, 00, 10, 11}. Tn dng th trng thi dch chuyn trong hnh trc ta c chui thng tin c lng x = {1010100}

Phng php 1: s dng khong cch Euclidean thay v khong cch Hamming. Nhng bit nhn c s dng trong trng hp khong cch Euclidean c x l bng lng t ha nhiu bit. Phng php 2: s dng gi tr tng quan vi bit nhn c s dng trong trng hp gi tr ny cng c x l bi lng t ha nhiu bit. Trong gii m soft-decision, pha thu khng gn gi tr 0 hoc 1 cho mi bit nhn c m s dng lng t ha s bit khng xc nh. Chui nhn c r c s dng trc tip trong gii m Viterbi soft-decision. Gii thut ny tng ng HDVA, ch khc l s dng khong cch Euclidean. Gii thut SDVA1 nh sau: 1. t=0 V(S0,0) = 0; V(Sk,t) = + 2. t = t+1 Tnh mt phn gi tr ng dn trong tt c ng dn bt u trng thi Sk ti thi im t. u tin , tm gi tr nhnh th t

Gi tr ny c tnh t khong cch Euclidean

Tip theo , tnh mt phn gi tr ng dn th t Gi tr ny c tnh t

V(S k,t-1 ) + M(r t |y t ) Thit lp V(S k,t ) l gi tr ng dn tt nht t trng thi S k ti thi im t Mt phn gi tr ng dn tt nht l ng dn c gi tr nh nht. Nu c nhiu ng dn c gi tr nh nht nh nhau th c th chn bt k ng no. Lu ng ny v bit cn li v trng thi ca n .Nu t< L+m-1 th quay li bc 2.

i vi phng php 2, gii thut SDVA2 c pht trin nh sau. Hm (likelihood function) c biu din bi hm phn b xc sut chun:

Vi E b l nng lng nhn c trn mi bit ca t m. N 0 mt nhiu ph trn mt mt. Bit nhn c l bin ngu nhin chun vi gi tr trung bnh /2. log 2 v ca biu thc trn ta c: v s khc bit N 0

= C1 (r y) + C2 Vi C1 v C2 khng l hm ca y T y, bit c nh ngha nh sau

SDVA2 thc hin nh sau: S k,t l trng thi trn cy Trellis tng ng trng thi S k ti thi im t. Mi trng thi trn cy Trellis c gn mt gi tr V(S k,t ) t=0 => V(S0,0) = 0; V(Sk,t) = - (a) t = t+1

(b) Tnh mt phn gi tr ng dn trong tt c ng dn bt u trng thi S k ti thi im t.22. 1. t=0 V(S 0,0 ) = 0; V(S k,t ) = u tin, tm gi tr nhnh th t \

Gi tr ny c tnh t s tng quan gia Tip theo, tnh mt phn gi tr ng dn th t c tnh t V(S k,t-1 ) + M(r t |y t )

v . Gi tr ny

Thit lp V(S k,t ) l gi tr ng dn tt nht t trng thi S k ti thi im t Mt phn gi tr ng dn tt nht l ng dn c gi tr ln nht. Nu c nhiu ng dn c gi tr ln nht nh nhau th c th chn bt k ng no Lu ng ny v bit cn li v trng thi ca n Nu t< L+m-1 th quay li bc 2

3. Ti liu tham kho

1. http://www.comp.leeds.ac.uk/roger/HiddenMarkovModels/html_dev/viterbi_algorithm/s1 _pg1.html 2. http://en.wikipedia.org/wiki/Viterbi_algorithm 3. http://pw1.netcom.com/~chip.f/viterbi/tutorial.html 4. William Cary Huffman, Vera Pless. Fundamentals of error-correcting codes. Cambridge University Press, 2003. 646 5. Todd K. Moon. Error correction coding: mathematical methods and algorithms. John Wiley and Sons, 2005. 756 6. George Cyril Clark, J. Bibb Cain. Error-correction coding for digital communications. Springer, 1981. 422