GT Ly Thuyet Thong Tin

Gio trnh: L thuyt thng tin.
MC LC
GII THIU TNG QUAN.............................................................................................................6
1. MC CH ...........................................................................................................................6
2. YU CU .............................................................................................................................6
3. NI DUNG CT LI...........................................................................................................7
4. KT THC TIN QUYT ..................................................................................................7
5. TI LIU THAM KHO.....................................................................................................8
6. PHNG PHP HC TP.................................................................................................8
CHNG 1: GII THIU ...............................................................................................................9
1. Mc tiu.................................................................................................................................9
2. i tng nghin cu............................................................................................................9
3. M hnh l thuyt thng tin theo quan im Shannon ........................................................10
4. Lng tin bit v cha bit .................................................................................................10
5. V d v lng tin bit v cha bit ....................................................................................10
6. nh l c s ca k thut truyn tin ..................................................................................11
7. M t trng thi truyn tin c nhiu ....................................................................................11
8. Minh ha k thut gim nhiu.............................................................................................12
9. Chi ph phi tr cho k thut gim nhiu ............................................................................13
10.
Khi nim v dung lng knh truyn ............................................................................13
11.
Vn sinh m ................................................................................................................13
12.
Vn gii m.................................................................................................................13
CHNG 2: O LNG TIN ...............................................................................................15
BI 2.1: ENTROPY .......................................................................................................................15
1. Mc tiu...............................................................................................................................15
2. V d v entropy..................................................................................................................15
3. Nhn xt v o lng tin ................................................................................................15
4. Khi nim entropy ...............................................................................................................16
5. Entropy ca mt s kin......................................................................................................16
6. Entropy ca mt phn phi .................................................................................................16
7. nh l dng gii tch ca Entropy......................................................................................16
8. V d minh ha ....................................................................................................................17
9. Bi ton v cy tm kim nh phn-t vn ...................................................................17
10.
Bi ton v cy tm kim nh phn - Din gii................................................................17
11.
Bi tp .............................................................................................................................18
BI 2.2: CC TNH CHT CA ENTROPY .............................................................................19
1. Mc tiu: .............................................................................................................................19
2. Cc tnh cht c bn ca Entropy........................................................................................19
3. Minh ha tnh cht 1 v 2....................................................................................................19
4. Minh ha tnh cht 3 v 4....................................................................................................19
5. nh l cc i ca entropy ................................................................................................20
6. Chng minh nh l cc i ca Entropy............................................................................20
7. Bi tp .................................................................................................................................21
BI 2.3: ENTROPY CA NHIU BIN .....................................................................................22
1. Mc tiu...............................................................................................................................22
2. nh ngha Entropy ca nhiu bin.....................................................................................22
3. V d Entropy ca nhiu bin..............................................................................................22
4. nh ngha Entropy c iu kin.........................................................................................22
Bin son: TS. L Quy t Thng, ThS. Phan Tn Ti & Ks. Dng Vn Hiu.

5. V d Entropy c iu kin .................................................................................................23
6. Quan h gia H(X,Y) vi H(X) v H(Y) khi X, Y c lp.................................................23
7. Quan h gia H(X,Y) vi H(X) v H(Y) khi X, Y tng quan ..........................................24
8. Bi tp .................................................................................................................................25
BI 2.4: MINH HA CC ENTROPY ........................................................................................26
1. Mc tiu...............................................................................................................................26
2. Yu cu ca bi ton ...........................................................................................................26
3. Xc nh cc phn phi ngu nhin ca bi ton ................................................................26
4. Minh ha Entropy H(X), H(Y) v H(X,Y)..........................................................................27
5. Minh ha Entropy H(X/Y) v H(Y/X) ................................................................................27
6. Minh ha quan h gia cc Entropy....................................................................................27
BAI 2.5: O LNG TIN (MESURE OF INFORMATION) ......................................................28
1. Mc tiu...............................................................................................................................28
2. t vn bi ton..............................................................................................................28
3. Xc nh cc phn phi ca bi ton...................................................................................28
4. Nhn xt da theo entropy ..................................................................................................28
5. nh ngha lng tin ...........................................................................................................29
6. Bi tp .................................................................................................................................29
CHNG 3: SINH M TCH C (Decypherable Coding)...................................................31
BI 3.1: KHI NIM V M TCH C..............................................................................31
1. Mc tiu...............................................................................................................................31
2. t vn bi ton sinh m ................................................................................................31
3. Khi nim v bng m khng tch c .............................................................................32
4. Bng m tch c..............................................................................................................32
5. Khi nim bng m tc thi ................................................................................................33
6. Gii thut kim tra tnh tch c ca bng m..................................................................33
7. Bi ton 1- yu cu..............................................................................................................33
8. Bi ton 1 - p dng gii thut ...........................................................................................34
9. Bi ton 2 ............................................................................................................................34
10.
Bi tp .............................................................................................................................35
BI 3.2: QUAN H GIA M TCH C V DI M................................................36
1. Mc tiu...............................................................................................................................36
2. nh l Kraftn(1949)...........................................................................................................36
3. nh ngha cy bc D c k. .................................................................................................36
4. Vn sinh m cho cy bc D c k ....................................................................................37
5. Chng minh nh l Kraft (iu kin cn) .........................................................................37
6. Chng minh nh l Kraft (iu kin )...........................................................................38
7. V d minh ha nh l Kraft ..............................................................................................38
8. Bi tp .................................................................................................................................39
BI 3.3: TNH TI U CA DI M..................................................................................40
1. Mc tiu...............................................................................................................................40
2. nh l Shannon (1948) ......................................................................................................40
3. Bng m ti u tuyt i .....................................................................................................40
4. Bng m ti u tng i....................................................................................................41
5. iu kin nhn bit mt bng m ti u .............................................................................41
6. nh l Huffman .................................................................................................................41
7. Phng php sinh m Huffman...........................................................................................42
8. Minh ha phng php sinh m Huffman ..........................................................................42
9. Nhn xt tnh ti u ca bng m Huffman ........................................................................43
10.
Bi tp .............................................................................................................................43

CHNG 4: KNH TRUYN ......................................................................................................45
BI 4.1: KNH TRUYN RI RC KHNG NH ...................................................................45
1. Mc tiu...............................................................................................................................45
2. Gii thiu.............................................................................................................................45
3. M hnh vt l .....................................................................................................................45
4. M hnh ton hc.................................................................................................................46
5. V d xc nh phn phi u nhn.....................................................................................47
6. Lng tin trn knh truyn..................................................................................................47
7. nh ngha dung lng knh truyn....................................................................................48
BAI 4.2: CC DNG KNH TRUYN........................................................................................49
1. Mc tiu...............................................................................................................................49
2. Hiu nh l v dung lng knh truyn,Knh truyn khng mt tin.................................49
3. Knh truyn xc nh ..........................................................................................................49
4. Knh truyn khng nhiu ....................................................................................................50
5. Knh truyn khng s dng c. ......................................................................................50
6. Knh truyn i xng..........................................................................................................50
7. Xy dng cng thc tnh dung lng knh truyn i xng ..............................................51
8. nh l v dung lng knh truyn.....................................................................................52
9. Bi tp .................................................................................................................................52
BI 4.3: LC GII M .......................................................................................................53
1. Mc tiu...............................................................................................................................53
2. t vn bi ton gii m .................................................................................................53
3. V d bi ton gii m .........................................................................................................53
4. Cc khi nim c bn ca k thut truyn tin .....................................................................54
5. V d minh ha cc khi nim c bn .................................................................................54
6. Cc dng sai s c bn ........................................................................................................55
7. Phng php xy dng lt gii m ti u....................................................................55
8. Minh ha xy dng lc gii m ti u .........................................................................56
9. Minh ha cch tnh cc sai s..............................................................................................57
10.
Bi tp 1 ..........................................................................................................................58
11.
Bi Tp 2 .........................................................................................................................58
CHNG 5: SA LI...................................................................................................................59
BI 5.1: NGUYN L KHONG CCH NH NHT HAMMING .........................................59
1. Mc tiu: .............................................................................................................................59
2. Khong cch Hamming .......................................................................................................59
3. Knh truyn i xng nh phn v lc gii m ti u..................................................59
4. V d knh truyn i xng nh phn..................................................................................60
5. Quan h gia xc sut gii m v khong cch Hamming..................................................60
6. Nguyn l Hamming ...........................................................................................................60
7. Bi tp .................................................................................................................................61
BI 5.2: B V T SA LI V CN HAMMING ...........................................................62
1. Mc tiu...............................................................................................................................62
2. B v t sa li...............................................................................................................62
3. Chng minh v minh ha b ..........................................................................................62
4. Cn Hamming. ....................................................................................................................63
5. Phn cc dng li.................................................................................................................64
6. Bi tp .................................................................................................................................64
BI 5.3: M KIM TRA CHN L .............................................................................................64
1. Mc tiu: .............................................................................................................................64
2. B m kim tra chn l........................................................................................................65

3. Phng php kim tra chn l .............................................................................................65
4. Phng php sinh m kim tra chn l ...............................................................................66
5. V d sinh m kim tra chn l............................................................................................66
6. nh l quan h gia di m n, s bit kim tra m v s li t sa e ..............................67
7. V d tm m nh nht t n v e...........................................................................................68
8. V d tm e ln nht t m v n.............................................................................................68
9. Bi tp .................................................................................................................................68
BI 5.4: NHM CNG TNH V B T M CHN L .........................................................69
1. Mc tiu...............................................................................................................................69
2. Khi nim nhm cng tnh. .................................................................................................69
3. Tnh cht ca b m chn l................................................................................................69
4. V d minh ha ....................................................................................................................70
5. Phng php sinh m kim tra chn l nhanh.....................................................................71
6. V d sinh m kim tra chn l nhanh .................................................................................71
7. Bi tp .................................................................................................................................72
BI 5.5: LC SA LI TI U.........................................................................................73
1. Mc tiu...............................................................................................................................73
2. t vn ............................................................................................................................73
3. nh ngha Hip hp ...........................................................................................................73
4. Lc sa li theo cc hip hp ......................................................................................74
5. Lc sa li thong qua b li.........................................................................................74
6. V d minh ha lc sa li 1 bit...................................................................................74
9. Xc sut truyn ng...........................................................................................................76
10.
Bi tp .............................................................................................................................76
BI 5.6: M HAMMING ..............................................................................................................76
1. Mc tiu...............................................................................................................................76
2. M Hammin.........................................................................................................................77
3. Tnh cht..............................................................................................................................77
4. V d minh ha ....................................................................................................................77
5. Bi tp .................................................................................................................................78
BI 5.7: THANH GHI LI TNG BC ...................................................................................79
1. Mc tiu...............................................................................................................................79
2. t vn ............................................................................................................................79
3. Biu din vt l ca thanh ghi .............................................................................................79
4. Biu din ton hc ca thanh ghi ........................................................................................80
5. V d thanh ghi lui tng bc .............................................................................................80
6. Chu k ca thanh ghi...........................................................................................................81
7. V d tm chu k ca thanh ghi ...........................................................................................81
8. Bi tp .................................................................................................................................82
BI 5.8: M XOAY VNG ..........................................................................................................82
1. Mc tiu...............................................................................................................................82
2. Ma trn kim tra chn l m xoay vng ..............................................................................83
3. nh ngha m xoay vng ...................................................................................................83
4. Phng php sinh nhanh b m xoay vng.........................................................................83
5. V d sinh nhanh b m xoay vng.....................................................................................84
6. Bi tp .................................................................................................................................85
BI 5.9: A THC C TRNG CA THANH GHI ...............................................................86
1. Mc tiu...............................................................................................................................86

2. nh ngha a thc c trng ca thanh ghi .......................................................................86
3. Quan h gia chu k n, a thc c trng v a thc (xn + 1)............................................86
4. Th tc sinh thanh ghi li tng bc ..................................................................................87
5. V d minh ha ....................................................................................................................87
6. Bi tp .................................................................................................................................87
Bi 5.10: PHNG PHP SINH M XOAY VNG ..................................................................88
1. Mc tiu...............................................................................................................................88
2. t vn ............................................................................................................................88
3. Phng php sinh bng m xoay vng................................................................................88
4. V d minh ha 1.................................................................................................................89
5. V d minh ha 2.................................................................................................................89
6. V d minh ha 3.................................................................................................................90
7. Bng lit k mt s a thc c trng.................................................................................90
8. Bi tp .................................................................................................................................90
BI TP TNG HP ....................................................................................................................91
1. Mc tiu...............................................................................................................................91
2. Bi 1 ....................................................................................................................................91
3. Bi 2 ....................................................................................................................................91
4. Bi 3 ....................................................................................................................................92
5. Bi 4 ....................................................................................................................................93
TI LIU THAM KHO...............................................................................................................95
GII THIU TNG QUAN

GIO TRNH L THUYT THNG TIN
MC CH
Gio trnh ny s cung cp cho ngi c nhng khi kin thc c bn ca l thuyt thng tin
nh: do lng tin (Measure of Information), Sinh m tch c (Decypherable Coding),
Knh truyn tin ri rc khng nh (Discrete Memoryless Channel) v Sa li trn knh truyn
(Error Correcting Codings).
Lin quan n o lng tin, gio trnh s trnh by cc khi nim c bn v thng tin,
entropy, mt s cng thc, tnh cht, cc nh l quan trng ca entropy v cch tnh
lng tin.
V Sinh m tch c, gio trnh s gii thiu n ngi hc cc vn v yu cu ca

bi ton sinh m, gii m duy nht, cng nh m tc thi v gii thut kim tra m tch
c. Cc nh l quan trng c cp trong ni dung ny l: nh l Kraft (1949),
nh l Shannon (1948) v nh l sinh m Huffman.
V knh truyn tin ri rc khng nh, gio trnh s gii thiu m hnh knh truyn theo
2 kha cnh vt l v ton hc. Cc khi nim v dung lng knh truyn, phn lp knh
truyn, nh l v dung lng knh truyn, cng nh cc khi nim trong k thut truyn
tin v phng php xy dng lc gii m ti u cng c trnh by trong mn hc
ny.
Vn Sa li (hay x l m sai) trn knh truyn l mt vn rt quan trng v

c quan tm nhiu trong mn hc ny. Cc ni dung c gii thiu n cc bn s l
Nguyn l Khong cch Hamming, cc nh l v Cn Hamming, phng php kim tra
chn l, cc lc sa li, Bng m Hamming v Bng m xoay vng.
Hn na, hu ht cc vn nu trn u c a vo ni dung ging dy cc bc i hc

ca mt s ngnh trong c ngnh Cng ngh thng tin. Do , c mt ti liu phc v
cng tc ging dy ca gio vin cng nh vic hc tp v nghin cu ca sinh vin, chng ti
mnh dn bin son gio trnh ny nhm gip cho sinh vin c mt ti liu t hc v nghin
cu mt cch hiu qu.
YU CU
Sau khi hc xong mn ny, sinh vin phi c c nhng kh nng sau:
Hiu cc khi nim v v thng tin, Entropy, Entropy ca mt phn phi, Entropy ca
nhiu phn phi, Entropy c iu kin, o lng tin. Vn dng gii quyt cc bi ton
v xc nh lng tin.
Bit khi nim v m tch c, m khng tch c, bng m ti u. Hiu nh l Kraft
(1949), nh l Shannon (1948), nh l sinh m Huffman v phng php sinh m
Huffman. Vn dng sinh bng m tch c ti u, nhn bit c bng m nh th
no l bng m ti u v c th vn dng vit cc chng trnh sinh m, gii m (hay
vit chng trnh nn v gii nn). T y, cc sinh vin c th t nghin cu cc loi
bng m khc vn dng cho vic m ha v bo mt thng tin mt cch hiu qu.
Bit cc khi nim v knh truyn tin ri rc khng nh, dung lng knh truyn v phn
lp knh truyn. Hiu nh l v dung lng knh truyn, phng php xy dng lc
gii m ti u v cch tnh xc sut truyn sai trn knh truyn.
Bit cc khi nim v khong cch Hamming, nguyn l khong cch Hamming, cc nh
l v Cn Hamming, phng php kim tra chn l, cc lc sa li, Bng m
Hamming v Bng m xoay vng.
Vn dng cc kin thc hc c thit k mt h thng truyn nhn d liu vi quy
trnh c bn: m ha, gii m v bo mt thng tin.
L thuyt thng tin cng l mt trong cc mn hc kh ca ngnh Cng ngh thng tin v n
i hi ngi hc phi c kin thc c bn v ton v xc sut thng k. Do , i hi ngi
hc phi t b sung cc kin thc c bn v ton v xc sut thng k cho mnh (nu thiu),
tham gia lp hc y v lm cc bi tp theo yu cu ca mn hc th mi tip thu kin
thc mn hc mt cch hiu qu.
NI DUNG CT LI
Gio trnh gm 5 chng c trnh by trong 45 tit ging cho sinh vin chuyn ngnh Cng
ngh thng tin, trong c khong 30 tit l thuyt v 15 tit bi tp m gio vin s hng dn
cho sinh vin trn lp.
Chng 1: Gii thiu. Chng ny trnh by cc ni dung c tnh tng quan v mn hc bao
gm: cc i tng nghin cu, m hnh l thuyt thng tin theo quan im ca nh ton hc
Shannon, khi nim v lng tin bit v cha bit, nh l c bn ca k thut truyn tin.
Chng 2: o lng tin. Chng ny trnh by cc vn c bn v entropy, cc tnh cht
ca entropy, entropy ca nhiu bin, entropy c iu kin, cc nh l v quan h gia cc
entropy v lng tin ca mt s kin.
Chng 3: Sinh m tch c. Ni dung chnh ca chng ny bao gm cc khi nim v m
tch c, quan h gia m tch c v di m, tnh ti u ca di m.
Chng 4: Knh truyn. Cc ni dung c trnh by trong chng ny bao gm khi nim v
knh truyn tin ri rc khng nh, cc m hnh truyn tin kha cnh vt l v ton hc, dung
lng trn knh truyn, phn lp cc knh truyn. Phng php xy dng lc gii m ti u
v cch tnh xc sut truyn sai cng c gii thiu trong chng ny.
Chng 5: Sa li. Chng ny trnh by cc ni dung ct li sau: khi nim v khong cch
Hamming, nguyn l khong cch nh nht Hamming, b v t sa li v nh l Cn
Hamming. Chng ny cng gii thiu v b m kim tra chn l, phng php kim tra chn l,
lc sa li ti u, m Hamming v m xoay vng.
KT THC TIN QUYT

hc tt mn hc ny, i hi sinh vin phi nm vng cc mn hc c lin quan nh: xc sut
thng k, i s boole (php ton Modulo 2 v a thc nh phn). Cc mn hc c lin quan v c
th tham kho thm nh k thut s, h iu hnh, mng my tnh.
TI LIU THAM KHO

1. David J.C. Mackey, Information Theory, Infernce, and Learning Algorithms, CamBridge
University Express-2003.
2. G.J.ChaiTin, Algorithmic Information Theory, CamBridge University Express-1992.
3. Sanford Goldman, Information Theory.
4. http://www.inference.phy.cam.ac.uk/mackay/info-theory/course.html.
5. http://en.wikipedia.org/wiki/Information_theory.
6. http://www-2.cs.cmu.edu/~dst/Tutorials/Info-Theory/.
7. http://cscs.umich.edu/~crshalizi/notebooks/information-theory.html.
8. http://www.lecb.ncifcrf.gov/~toms/paper/primer/primer.pdf.
9. http://www.cs.ucl.ac.uk/staff/S.Bhatti/D51-notes/node27.html.
10. http://guest.engelschall.com/~sb/hamming/.
11. http://www2.rad.com/networks/1994/err_con/hamming.htm
PHNG PHP HC TP
phc v cho mc tiu nng cao kh nng t hc tp v t nghin cu ca sinh vin, gio trnh
ny c bin son cng vi cc gio trnh khc thuc chuyn ngnh Cng ngh thng tin ca
Khoa Cng ngh thng tin v Truyn thng i Hc Cn Th theo d n ASVIET002CNTT
Tng cng hiu qu o to v nng lc o to ca sinh vin khoa Cng ngh Thng tini hc Cn Th. Chng ti c gng trnh by gio trnh ny mt cch c h thng cc ni
dung theo b cc cc chng ng vi cc khi kin thc nu trn, mi chng c c trnh
by theo b cc ca cc bi hc v mi bi hc gii thiu n ngi hc mt vn no trong
s cc vn ca mt khi kin thc tng ng vi mt chng. Khi hc xong cc bi hc ca
mt chng, ngi hc s c mt khi kin thc cn thit tng ng cho mn hc. Ni dung ca
cc bi hc u c a vo cc v d ngi hc d hiu, ty theo tng vn m ngi hc
cn phi hc v nghin cu trong thi lng t 1 n 2 tit t hc cho mt bi hc trong mt
chng. Nh vy, hc tt mn hc ny, trc ht sinh vin cn phi:
Hc y cc mn hc tin quyt, b sung nhng kin thc c bn v ton v xc sut

thng k (nu thiu).
Hc v nghin cu k tng chng theo trnh t cc chng c trnh by trong gio
trnh ny. Trong tng chng, hc cc bi theo th t c trnh by, sau mi bi phi lm
bi tp y (nu c).
Tham gia lp y , tho lun cc vn tn ti cha hiu trong qu trnh t hc.
Sau mi chng hc, phi nm vng cc khi nim, cc nh ngha, cc cng thc tnh
ton v vn dng gii cc bi ton c tnh cht tng hp c gii thiu cui chng.
Vn dng kin thc c c sau khi hc xong cc chng gii mt s bi tp tng hp
cui gio trnh, t gip cho ngi hc hiu su hn v mn hc v c th gii quyt
cc vn tng t trong thc t.
Vic cho ra i mt gio trnh vi nhng mc ch nh trn l khng n gin khi kh nng v
kinh nghim ca ngi son cn c hn, nhiu khi nim, thut ng dng trong gio trnh cha
c nh ngha mt cch chnh thng. V vy gio trnh ny chc khng trnh khi nhng khim
khuyt, rt mong nhn c s gp ca cc ng nghip v ngi c.
CHNG 1: GII THIU

1: Mc tiu
Sau khi hon tt bi hc ny bn c th bit:
- i tng nghin cu,
- M hnh l thuyt thng tin theo quan im Shannon,
- Cc khi nim v Lng tin bit v lng tin cha bit,
- nh l c s ca k thut truyn tin,
- Khi nim chung v dung lng knh truyn,
- Vn sinh m v gii m.
i tng nghin cu
L thuyt thng k v thng tin c xy dng trn hai hng khc nhau bi hai nh ton hc
Shannon (1948) v Wiener (1949). L thuyt thng tin nghin cu qu trnh x l tn hiu nh
sau:
u vo (input): nhn tn hiu t mt lnh vc c th, tc l tn hiu xut hin theo cc k hiu
(symbol) t mt tp hp cho trc v theo phn phi xc sut bit.
Tn hiu c truyn i trn knh truyn (channel) v c th b nhiu cng theo mt phn phi
xc sut no . Knh truyn c th c hiu di hai ngha:
Di ngha vt l: knh truyn l mt h thng truyn tn hiu (dy dn, mch, sng, ...) v gy
nhiu ty thao cht lng ca h thng.
Di ngha ton hc: knh truyn l cc phn phi xc sut xc nh trn lp cc tn hiu ang xt
u nhn tn hiu (output).
u ra (output): dng li tn hiu chn tht nht c th c so vi tn hiu u vo.
Shannon xy dng m hnh l thuyt thng tin trn c s gii quyt bi ton: sinh m di ti
u khi nhn tn hiu u vo. Tn ti u c xt trn 3 yu t sau:
Phn phi xc sut ca s xut hin ca cc tn hiu.
Tnh duy nht ca m v cho php t iu chnh m sai nu c vi chnh xc cao nht. Gii m
ng thi t ng iu chnh m hoc xc nh on m truyn sai.
Trong kh , Wiener li nghin cu phng php x l tn hiu u ra: c lng ti u chui
tn hiu so vi chnh n khi nhn u vo khng qua qu trnh sinh m. Nh vy phng php
Wiener c p dng trong nhng trng hp con ngi khng kim sot c qu trnh truyn
tn hiu. Mn x l tn hiu cp n vn ny.
M hnh l thuyt thng tin theo quan im Shannon

L thuyt thng tin c xt y theo quan im ca Shannon. i tng nghin cu l mt h
thng lin lc truyn tin (communication system) nh s di y:
Ngun
M ha
Knh
Gii m
Nhn
Nhiu
B ch ci
B ch ci
Din gii:
-
Ngun (source) thng tin cn gi l thng bo cn c truyn u vo (Input).

M ha (encode) l b sinh m. ng vi mt thng bo, b sinh m s gn cho mt i
tng (object) ph hp vi k thut truyn tin. i tng c th l:
o Dy s ngh phn (Digital) dng: 01010101, cng ging nh m my tnh.
o Sng lin tc (Analog) cng ging nh truyn radio.
Knh (channel) l phng tin truyn m ca thng tin.
Nhiu (noise) c sinh ra do knh truyn tin. Ty vo cht lng ca knh truyn m
nhiu nhiu hay t.
Gii m (decode) u ra (output) a dy m tr v dng thng bo ban u vi xc sut
cao nht. Sau thng bo s c chuyn cho ni nhn. Trong s trn, chng ta quan
tm n 2 khi m ha v gii m trong ton b mn hc.
Lng tin bit v cha bit

Mt bin ngu nhin (BNN) X lun mang mt lng tin no . Nu X cha xy ra (hay ta cha
bit c th thng tin v X) th lng tin ca n l cha bit, trong trng hp ny X c mt lng
tin cha bit. Ngc li nu X xy ra (hay ta bit c th thng tin v X) th lng tin v bin
ngu nhin X coi nh bit hon ton, trong trng hp ny X c mt lng tin bit.
Nu bit thng tin ca mt BNN X thng qua BNN Y xy ra th ta c th ni: chng ta ch bit
mt phn lng thng tin ca X trn c s bit Y.
V d v lng tin bit v cha bit

Ta xt v d v mt ngi t chc tr chi may ri khch quan vi vic tung mt ng tin c
u hnh khng c u hnh. Nu ngi chi chn mt khng c u hnh th thng khi kt
qu tung ng tin l khng c u hnh, nguc li th thua. Tuy nhin ngi t chc chi c th
n gian bng cch s dng 2 ng tin Tht- Gi khc nhau sau:
+ ng tin loi 1 (hay ng tin tht): ng cht c 1 mt c u hnh.
+ ng tin loi 2 (hay ng tin gi ): ng cht, mi mt u c 1 u hnh.
Mc d ngi t chc chi c th n gian nhng qu trnh trao i 2 ng tin cho nhau l ngu
nhiu, vy liu ngi t chc chi c th n gian hon ton c khng? Hay lng tin bit v
cha bit ca s kin ly mt ng tin t 2 ng tin ni trn c hiu nh th no?
10

Ta th xt mt trng hp sau: nu ngi chi ly ngu nhin 1 ng tin v sau thc hin
vic tung ng tin ly c 2 ln. Qua 2 ln tung ng tin, ta m c s u hnh xut hin.
Da vo s u hnh xut hin, ta c th phn on c ngi t chc chi ly c ng
tin no.
Chng hn: Nu s u hnh m c sau 2 ln tng l 1 th ng tin ly c l ng tin
tht. Ngc li nu s u hnh m c l 2 th ng tin ly c c th l tht hay cng c
th l gi. Nh vy, ta nhn c mt phn thng tin v loi ng tin qua s u hnh m
c sau 2 ln tung. Ta c th tnh c lng tin bng bao nhiu? (Vic tnh lng tin ny s
c tho lun sau). Di y l mt s bng phn phi ca bi ton trn:
Gi BNN X v loi ng tin (X=1 nu ly c ng tin loi 1 v X=1 nu ly c ng tin
loi 2 c ly).
Khi phn phi ca X c dng:
X
P
1
0.5
2
0.5
t BNN Y l BNN v s u hnh m c sau 2 ln tung. Khi ta c th xc nh c phn

phi ca Y vi iu kin xy ra ca X trong 2 trng hp sau.
Phn phi ca Y khi bit X=1 c dng:
Y/X=1
P
0
0.25
1
0.5
2
0.25
Phn phi ca Y khi bit X=2 c dng:

Y/X=2
P
0
0
1
0
2
1
nh l c s ca k thut truyn tin

Trong A New Basic of Information Theory (1954), Feinstein a ra nh l sau: Trn mt
knh truyn c nhiu, ngi ta lun c th thc hin mt phng php truyn sao cho t c sai
s nh hn sai s cho php (nh bt k) cho trc i vi knh truyn.
Chng ta s khng chng minh nh l, thay vo , chng ta s tham kho n cc minh ha
gim nhiu trong cc ni dung tip theo ca bi hc.
M t trng thi truyn tin c nhiu

Gi s, mt thng bo c truyn i trn mt knh truyn nh phn ri rc. Thng bo cn
truyn c m ha thnh dy s nh phn (0,1) v c di c tnh theo n v bit. Gi s 1
bit truyn trn knh nhiu vi xc sut 1/4 (hay tnh trung bnh c truyn 4 bit th c th nhiu 1
bit).
Ta c s trng thi truyn tin sau:
ng
Ngun
M ha
Truyn tng bit
ng
1
11
Minh ha k thut gim nhiu

Trong k thut truyn tin, ngi ta c th lm gim sai lm khi nhn tin bng cch truyn lp li 1
bit vi s l ln.
V d: truyn lp li 3 cho 1 bit cn truyn (xc sut nhiu 1 bit bng 1/4). Khi nhn 3 bit lin
nhau cui knh c xem nh l 1 bit. Gi tr ca bit ny c hiu l 0 (hay 1) nu bit 0 (bit 1)
c s ln xut hin nhiu hn trong dy 3 bit nhn c lin nhau (hay gii m theo nguyn tc a
s). Ta cn chng minh vi phng php truyn ny th xc sut truyn sai tht s < 1/4 (xc sut
nhiu cho trc ca knh truyn).
S truyn tin:
Bit truyn
0
Tuyn lp 3 ln
000
000
000
000
000
000
000
000
111
111
111
111
111
111
111
111
Nhn 3 bit
000
001
010
100
101
011
110
111
000
001
010
100
011
110
111
111
Gii m
0
0
0
0
1
1
1
1
0
0
0
0
1
1
1
1
Tht vy:
Gi s Xi xc nh gi tr ng hay sai ca bit th i nhn c cui knh truyn vi Xi =1 nu
bit th i nhn c l sai v Xi =0 nu bit th i nhn c l ng. Theo gi thit ban u ca
knh truyn th phn phi xc sut ca Xi c dng Bernoulli b(1/4):
Xi
P
1
3/4
0
1/4
Gi Y ={X1 + X2 + X3 } l tng s bit nhn sai sau 3 ln truyn lp cho 1 bit. Trong trng hp
ny Y tun theo phn phi Nh thc B(p,n), vi p=1/4 (xc sut truyn sai mt bit) v q =3/4 (xc
sut truyn ng 1 bit):
Y ~ B(i,n) hay
12
p (Y = i ) = C ni . p i q n i
Trong : C ni =
n!
i!( n i )!
Vy truyn sai khi Y {2, 3} c xc xut l:

Psai= P(y2) = P(Y=2) + P(Y=3) = B(2,3) + B(2,3)
10 1
1 3
3
1
Hay Psai = (C 32 ( ) 2 .( )1 ) + (C 33 ( ) 3 ( ) 0 ) =
< (pcm).
64 4
4 4
4
4
Chi ph phi tr cho k thut gim nhiu

Theo cch thc lp li nh trn, ta c th gim sai lm bao nhiu cng c (lp cng nhiu th
sai cng t), nhng thi gian truyn cng tng ln v chi ph truyn cng s tng theo.
Hay ta c th hiu nh sau:
Lp cng nhiu ln 1 bit => thi gian truyn cng nhiu => chi ph cng tng.
Khi nim v dung lng knh truyn

V d trn cho chng ta thy cn phi xc nh mt thng s cho truyn tin m bo sai s
chp nhn c v ng thi tc truyn cng khng qu chm.
Khi nim dung lng knh truyn l khi nim rt c bn ca l thuyt truyn tin v l mt i
lng vt l ng thi cng l i lng ton hc (c n v l bit). N cho php xc nh tc
truyn ti a ca mi knh truyn. Do , da vo dung lng knh truyn, ngi ta c th ch ra
tc truyn tin ng thi vi mt phng php truyn c sai s cho php.
Vn sinh m
T k thut truyn tin trn cho ta thy qu trnh sinh m v gii m c m t nh sau: mt n
v thng tin nhn c u vo s c gn cho mt k hiu trong b k hiu sinh m. Mt k
hiu m c gn n ln lp li (da vo dung lng ca knh truyn, ta c th xc nh c n).
Thit b sinh m (Coding device/ Encoder) s thc hin qu trnh sinh m.
Nh vy, mt n v thng tin t ngun pht tin s c thit b sinh m gn cho mt dy n k
hiu m. Dy k hiu m ca 1 n v thng tin c gi l mt t m (Code word). Trong trng
hp tng qut, ngi ta c th gn mt khi k t m cho mt khi thng tin no v c gi
l mt t m.
Vn gii m
cui knh truyn, mt thit b gii m (Decoding device/ Decoder) s thc hin qu trnh ngc
li nh sau: kim tra dy k hiu m quyt nh gii m v mt t m v a n v dng khi
tin ban u.
V d:
Khi tin ban u
: 01010101
Khi k hiu m u truyn (lp 3 ln): 000111000111000111000111.
Khi k hiu m u nhn
: 001110100111011001000111
Khi tin nhn c cui cng
: 01011001 (sai 2 bit so vi khi tin ban u)
Do lm sao ua khi tin nhn c v khi tin ban u 01010101, y chnh l cng vic
ca b gii m (Decoder).
13

Mt vn quan trng cn lu l phi ng b gia tc np thng tin (pht tn hiu) vi tc
truyn tin. Nu tc np thng tin bng hoc ln hn so vi tc truyn tin ca knh, th
cn phi gim tc np thng tin sao cho nh hn tc truyn tin.
14
CHNG 2: O LNG TIN

Mc tiu: trnh by cc khi nim v o lng tin cha bit v bit v mt bin ngu nhin
X. Tnh ton cc lng tin ny thng qua nh ngha v cc tnh cht ca Entropy t mt hay
nhiu bin ngu nhin.
BI 2.1: ENTROPY
Mc tiu
Sau khi hon tt bi hc ny bn c th:
- Hiu c cc khi nim Entropy,
- Bit Entropy ca mt s kin v Entropy ca mt phn phi,
- Hiu nh l dng gii tch ca Entropy,
- Bit Bi ton v cy tm kim nh phn v
- Lm kin thc c s hiu v hc tt cc bi hc tip theo.
V d v entropy
Trc ht, ta cn tm hiu mt v d v khi nim do ca mt lng tin da vo cc s kin
hay cc phn phi xc sut ngu nhin nh sau:
Xt 2 BNN X v Y c phn phi sau:
X={1, 2, 3, 4, 5} c phn phi u hay p(X=i) = 1/5.
Y={1, 2} cng c phn phi u hay p(Y=i) = 1/2.
Bn thn X v Y u mang mt lng tin v thng tin v X v Y cha bit do chng l ngu
nhin. Do , X hay Y u c mt lng tin khng chc chn v lng tin chc chn, tng ca 2
lng tin ny l khng i v thc t n bng bao nhiu th ta cha th bit. Lng tin khng chc
chn ca X (hay Y) c gi l Entropy.
Tuy nhin, nu X v Y c tng quan nhau th X cng c mt phn lng tin khng chc chn
thng qua lng tin bit ca Y (hay thng tin v Y c bit). Trong trng hp ny, mt
phn lng tin khng chc chn ca thng qua lng tin bit ca Y c gi l Entropy c
iu kin.
Nhn xt v o lng tin

R rng, ta cn phi xy dng mt i lng ton hc rt c th c th o c lng tin cha
bit t mt bin ngu nhin. Mt cch trc quan, lng tin phi th hin c cc vn sau:
Mt s kin c xc sut cng nh th s kin t xy ra, cng c ngha l tnh khng chc chn
cng ln. Nu o lng tin ca n th n cho mt lng tin khng bit cng ln.
Mt tp hp cc s kin ngu nhin (hay Bin ngu nhin) cng nhiu s kin c phn phi cng
u th tnh khng chc chn cng ln. Nu o lng tin ca n th s c lng tin khng bit
cng ln. Hay lng tin chc chn cng nh.
15

Mt phn phi xc sut cng lch nhiu (c xc xut rt nh v rt ln) th tnh khng chc chn
cng t v do s c mt lng tin cha bit cng nh so vi phn phi xc sut u hay lng
tin chc chn ca n cng cao.
Khi nim entropy

Trong ting vit ta cha c t tng ng vi t Entropy, tuy nhin chng ta c th tm hiu
hiu thong qua trc khi i vo nh ngha chc ch v mt ton hc ca Entropy nh sau:
Entropy l mt i lng ton hc dng o lng tin khng chc (hay lng ngu nhin) ca
mt s kin hay ca phn phi ngu nhin cho trc. Hay mt s ti liu ting anh gi l
Uncertainty Measure.
Entropy ca mt s kin
Gi s c mt s kin A c xc sut xut hin l p. Khi , ta ni A c mt lng khng chc
chn c o bi hm s h(p) vi p [0,1]. Hm h(p) c gi l Entropy nu n tho 2 tiu
ton hc sau:
Tin 01: h(p) l hm lin tc khng m v n iu gim.
Tin 02: nu A v B l hai s kin c lp nhau, c xc sut xut hin ln lt l pA v pB. Khi
, p(A,B) = pA.pB nhng h(A,B) = h(pA) + h(pB).
Entropy ca mt phn phi

Xt bin ngu nhin X c phn phi:
X
P
x1
p1
x2
p2
x3
p3
xM
pM
Nu gi Ai l s kin X=xi, (i=1,2,3,..) th Entropy ca Ai l: h(Ai)= h(pi)

Gi Y=h(X) l hm ngu nhin ca X v nhn cc gi tr l dy cc Entropy ca cc s kin X=xi,
tc l Y=h(X)={h(p1), h(p2), , h(pn)}.
Vy, Entropy ca X chnh l k vng ton hc ca Y=h(X) c dng:
H(X)=H(p1, p2, p3, ,pn) = p1h(p1)+ p2h(p2)++pnh(pn).
Tng qut:
n
H ( X ) = p i h( pi )
i =1
nh l dng gii tch ca Entropy

M
nh l: Hm H(X) = H(p1, p2,...,pM) = C pi log( pi )

i =1
C = const >0
C s logarithm l bt k.
B : h(p)=-Clog(p).
Trng hp C=1 v c s logarithm = 2 th n v tnh l bit.
Khi :
h(p)=-log2(p) (vt: bit) v
16
H(X) = H(p1 , p 2 ,..., p M ) = pi log 2 ( pi )

i =1
Qui c trong cch vit: log(pi)= log2(pi)
V d minh ha
Nu s kin A c xc sut xut hin l 1/2 th h(A)=h(1/2)= -log(1/2) = 1 (bit)
Xt BNN X c phn phi sau:
X
P
x1
x2
1/2 1/4
x3
1/4
H(X) = H(1/2, 1/4, 1/4) = -(1/2log(1/2)+1/4log(1/4)+1/4log(1/4)) =3/2 (bit)
Bi ton v cy tm kim nh phn-t vn

Gi s, tm 1 trong 5 ngi c tn bit trc s xut hin theo phn phi sau:
X x1
P 0,2
x2
0,3
x3
0,2
x4
0,15
x5
0,15
Trong : x1, x5 ln lt l tn ca 5 ngi m ta cn nhn ra vi cch xc nh tn bng cu

hi ng sai (yes/no).
S di y minh ha cch xc nh tn ca mt ngi:
Yes
x1
Yes
No
x2
No
Yes
x3
X=x1?
X=x1/x2?
X=x3?
No
X=x4?
Yes
x4
No
x5
Bi ton v cy tm kim nh phn - Din gii

Theo s trn:
tm x1, x2, x3 vi xc sut tng ng l 0.2, 0.3, 0.2 ta ch cn tn 2 cu hi.
tm x4, x5 vi xc sut tng ng 0.15, 0.15 th ta cn 3 cu hi.
Vy:
S cu hi trung bnh l: 2 x (0,2+0,3+0,2) + 3 x (0,15+0,15) = 2.3
Mt khc: Entropy ca X: H(X)= H(0.2, 0.3, 0.2, 0.15, 0.15)=2.27.
Ta lun c s cu hi trung bnh lun H(X) (theo nh l Shannon s trnh by sau). V s cu
hi trung bnh trong trng hp ny xp s H(X) nn y l s cu hi trung bnh ti u tm ra
tn chnh xc ca mt ngi. Do , s tm kim trn l s ti u.
17
Sinh vin t cho thm 1 hay 2 s tm kim khc v t din gii tng t - xem nh bi tp.
Bi tp
Tnh H(X) vi phn phi sau:
X
P
x1
x2
1/3 1/3
x3
1/3
Tnh H(Y) vi phn phi sau:

Y
P
x1
x2
1/6 2/6
x3
1/6
x4
2/6
18
BI 2.2: CC TNH CHT CA ENTROPY

Mc tiu:
- Hiu cc tnh cht c bn ca Entropy,
- Hiu nh l cc i ca Entropy,
- Vn dng gii mt s bi ton v Entropy,
- Lm c s vn dng gii quyt cc bi ton tnh dung lng knh truyn.
Cc tnh cht c bn ca Entropy

Xt bin ngu nhin X = {x1, x2, , xM}. Entropy ca bin ngu nhin X c cc tnh cht:
1
1
1. Hm s f(M) = H( ,, ) n iu tng.
M
M
2. Hm s f(ML) = f(M)+f(L).
3. H(p1, p2, , pM) = H(p1 + p2 ++pr, pr+1+ pr+2++ pM)
p
p
+ (p1 + p 2 + + p r )H( r 1 ,..., r r )
i=1 pi i =1 pi
+ (p r +1 + p r + 2 + + p M )H(
pr +1
i=r +1 pi
M
,...,
pM
i=r +1 pi
M
4. H(p, 1-p) l hm lin tc theo P.
Minh ha tnh cht 1 v 2

Minh ha tnh cht 1:
Trong trng hp bin ngu nhin X c phn phi u
Entropy ca X nh sau :
1
1
1
1
1
1
1
1
1
1
1
H (X ) = H
,
,L ,
log
log
,...,
log
= M
log
=
M
m
M
M
M
M
M
M
M
M M
=> H(X) = log
1
= log M l hm n iu tng
M
Minh ha tnh cht 2:

Trong trng hp 2 bin ngu nhin X, Y c lp c phn phi u vi BNN X c M s
kin v BNN Y c L s kin.
Gi f(M), f(L) ln lt l Entropy ca X, ca Y. Theo tnh cht 2 ca Entropy ta c
f(ML)=f(M)+f(L)
Minh ha tnh cht 3 v 4

Minh ha tnh cht 3:
Xt con xc sc c 6 mt vi xc sut xut hin cc mt c cho trong bng sau:
X
P
x1
10%
x2
20%
x3
25%
x4
25%
x5
15%
x6
5%
Ta c th gom cc s kin x1, x2, x3 li thnh mt s kin mi l x123 c xc sut xut hin
l 55%, gom s kin x5 v x6 li thnh s kin x56 c xc sut 20%.
19

Ta c mt nhin ngu nhin mi X* c phn phi sau:
X*
P
x123
55%
x4
25%
X56
20%
n y cc bn c th p dng cng thc tnh, so snh cc Entropy v nhn xt tnh

cht 3. Phn ny xem nh bi tp cho sinh vin.
Minh ha tnh cht 4:

hiu tnh cht th 4, ta xt dng th ca hm s H(p, 1-p ):
R rng H(p, 1-p) l mt hm lin tc theo p.
nh l cc i ca entropy
nh l: H(p1, p2, ,pM) log(M)
Trong : ng thc xy ra khi v ch khi p1== pM= 1/M
B : cho 2 b {p1, p2, ,pM} v {q1, q2,,qM} l cc b s dng bt k v
M
i =1
i =1
pi = qi
M
i =1
i =1
Khi , ta c H(p1, p2, ,pM)= p i log 2 p i p i log 2 q i (*)

ng thc xy ra khi pi=qi vi i=1,..,M.
Chng minh nh l cc i ca Entropy

Chng minh b :
Theo ton hc ta lun c th chng minh c ln(x) x-1 vi x>0 v ng thc ng khi x=1.
t x= qi/pi Suy ra ln(qi/pi) qi/pi 1 (v ng thc ng khi qi=pi vi mi i).
M
M
q
pi ln i (qi pi ) = 1 1 = 0
pi i =1
i =1
M
i =1
i =1
pi ln pi pi lnqi (ng thc xy ra khi qi=pi).

Theo ton hc ta c lnx = log2x / log2e
M
i =1
i =1
(1)
(2)
T (1) v (2), ta c p i log pi p i logq i (ng thc xy ra khi qi=pi.)

Chng minh nh l:
t qi 1 , i
M
T b , ta c:
M
i =1
i =1
p i log 2 pi pi log 2
M
1
= log 2 M pi = log 2 M
M
i =1
20
v ng thc ch xy ra khi pi= 1 , i (pcm).

M
Bi tp
Bi 1: Cho 2 bin ngu nhin X, Y c lp nhau c phn phi sau:
X
P
x1
1/2
x2
1/2
Y
P
y1
1/4
y2
1/4
y3
1/4
y4
1/4
Tnh H(X), H(Y).

Bi 2: Kim tra li kt qu ca ca bi 1 bng tnh cht 2.
Bi 3: Cho bin ngu nhin X c phn phi sau:
X
P
x1
10%
x2
20%
x3
25%
x4
25%
x5
15%
x6
5%
Ta c th gom cc s kin x1, x2, x3 li thnh mt s kin mi l x123 c xc sut xut hin l 55%,
gom s kin x5 v x6 li thnh s kin x56 c xc sut 20%.
Ta c mt nhin ngu nhin mi X* c phn phi sau:
X*
P
x123
55%
x4
25%
x56
20%
- Tnh entropy ca X, X* v kim tra li tnh cht 3.

- Kim tra li nh l cc i t d liu cho trn.
21
BI 2.3: ENTROPY CA NHIU BIN

Mc tiu
- Hiu bit cc nh ngha Entropy ca nhiu bin v Entropy c iu kin,
- Hiu mi quan h gia H(X,Y) vi H(X) v H(Y) khi X, Y c lp,
- Hiu mi quan h gia H(X,Y) vi H(X) v H(Y) khi X, Y tng quan,
- Vn dng mi quan h ga cc Entropy tnh cc Entropy mt cch hiu qu,
- Vn dng Entropy c iu kin lm c s tnh lng tin trong bi hc k tip
nh ngha Entropy ca nhiu bin

Gi s: X v Y l 2 bin ngu nhin cho trc vi pj = p(X=xi,Y=yj) ( i=1,..,M v j=1,,L).
Khi , Entropy H(X,Y) c dng:
M
H(X, Y) = p ( xi , y j ) log 2 p ( xi , y j )
i =1 j =1
Hay
M
H(X, Y) = p ij log 2 p ij
i =1 j =1
Mt cch tng qut:

H(x 1 , , x n ) = -
p( x ,..., x
1
) log 2 p ( x1 , x 2 ,..., x n )
X 1 ,L, X n
V d Entropy ca nhiu bin

Cho 2 BNN X v Y c lp nhau v c cc phn phi:
X=1
P
0
0.5
Y
P
0
0.25
1
0.5
1
0.5
2
0.25
Tnh H(X,Y).
- Lp phn phi ca P(X,Y)
X,Y
P(X,Y)
X=0,Y=0
0.125
X=0,Y=1
0.25
X=0,Y=2
0.125
X=1,Y=0
0.125
X=1,Y=1
0.25
X=1,Y=2
0.125
- H(X,Y) =H(0.125, 0.25, 0.125, 0.125, 0.25, 0.125)=2.5 (Bit)
nh ngha Entropy c iu kin

Entropy ca Y vi iu kin X=xi (i=1,..,M) c nh ngha l:
L
H (Y / X = xi ) = p ( y j / xi ) log p ( y j / xi )
j =1
22

Entropy ca Y vi iu kin X xy ra c nh ngha l:
M
H (Y / X ) = p ( xi ) H (Y / X = xi )
i =1
V d Entropy c iu kin
Xt bin ngu nhin X v bin ngu nhin Y c tng quan nhau. Cc phn phi nh sau:
X
P
1 .
0.5
2
0.5
Phn phi ca Y c iu kin X:

Y/X=1
P
0
0.25
1
0.5
2
0.25
Y/X=2
P
0
0
1
0
2
1
Entropy ca Y/X=1 v Y/X=2 nh sau :

H(Y/X=1)=H(0.25, 0.5 , 0.25)= -0.25 log0.25 0.5 log0.5-0.25 log0.25
=0.5 + 0.5 + 0.5= 1.5 (Bit)
H(Y/X=2)= H(0; 0; 1)= 0 (Bit)
Entropy ca Y khi X xy ra:
H(Y/X)=P(X=1) H(Y/X=1)+ P(X=2) H(Y/X=2)=(0.5x1.5) + ((0.5x0)=0.75 (Bit).
Quan h gia H(X,Y) vi H(X) v H(Y) khi X, Y c lp

nh l 1: H(X,Y) H(X)+H(Y) v ng thc xy ra khi X, Y c lp
Chng minh:
Ta c:
L
P ( xi ) = p ( xi , y j )
j =1
M
P ( y i ) = p ( xi , y j )
i =1
i =1
i =1 j =1
H ( X ) = p ( xi ) log 2 p ( xi ) = p ( xi , y j ) log 2 p ( xi )
L
H (Y ) = p ( y j ) log 2 p ( y j ) = p ( xi , y j ) log 2 p ( y j )
j =1
i =1 j =1
H ( X ) + H (Y ) = p ( xi , y j )[log 2 p ( xi ) + log 2 p ( y j )]
i =1 j =1
M
H ( X ) + H (Y ) = p ( xi , y j )[log 2 p ( xi ) p ( y j )] (1)
i =1 j =1
t qij =p(xi)p(yj)
23
p ij log 2 q ij p ij log 2 p ij
i =1 j = i
(2)
i =1 j =1
ng thc xy ra khi p(xi, yj)=pij =qij =p(xi)p(yj) hay X , Y c lp nhau.

(Theo b nh l cc i)
Mt khc:
M
H ( X , Y ) = p ( xi , y j ) log 2 p ( xi , y j ) = pij log 2 pij

i =1 j =1
(3)
i =1 j =1
T (1), (2) v (3), ta c H(X,Y) H(X)+H(Y) v ng thc xy ra khi X, Y c lp (pcm)
H qu:
H(X1, , Xn) H(X1)++H(Xn)
H(X1,Xn; Y1,,Yn) H(X1,Xn)+ H(Y1,,Yn)
Quan h gia H(X,Y) vi H(X) v H(Y) khi X, Y tng quan

nh l 2: H(X,Y)=H(X)+H(Y/X)=H(Y)+H(X/Y).
nh l 3: H(Y/X) H(Y) v Du ng thc xy ra khi v ch khi X v Y c lp nhau.
Chng minh nh l 2:
M
H(X, Y) = - p ( xi , y j ) log 2 p ( xi , y j )
i =1 j =1
M
= - p ( xi , y j ) log 2 [ p ( xi ). p ( y j / xi )]
i =1 j =1
M
= p ( xi , y j ) log 2 p ( xi ) p ( xi , y j ) log 2 p ( y j / xi )
i =1 j =1
i =1 j =1
= H(X) + H(Y/X)
Tng t ta c: H(X,Y)=H(Y)+H(X/Y)
24

Chng minh nh l 3:
T nh l 1 v nh l v quan h gia cc Entropy, ta c:
H(X,Y)=H(X)+H(Y/X) H(X)+ H(Y) => H(Y/X) H(Y).
H(X)
H(Y)
H(X/Y)
H(Y/X)
Sinh vin t chng minh
Bi tp
Xt BNN X v BNN Y c tng quan nhau. Cc phn phi nh sau:
X
1 .
2
P
0.5
0.5
Phn phi ca Y c iu kin X:
Y/X=1
0
P
0.25
Y/X=2
P
0
0
1
0.5
2
0.25
1
0
2
1
1. Tnh cc Entropy sau: H(X), H(Y).

2. Tnh cc Entropy c iu kin sau: H(X/Y), H(Y/X).
3. Tnh cc Entropy sau: H(X,Y).
4. T kt qu cu 1,2 v 3 hy minh ha cc nh l 1, 2 v 3 cho bi hc.
25
BI 2.4: MINH HA CC ENTROPY

Mc tiu
- Bit c Yu cu ca bi ton,
- Bit cch xc nh cc phn phi ngu nhin ca bi ton,
- Vn dng cc bi hc trc tnh cc Entropy H(X), H(Y) v H(X,Y),
- Vn dng cc bi hc trc tnh cc Entropy c iu kin H(X/Y) v H(Y/X),
- Nhn xt v so snh quan h gia cc Entropy
- Ngoi ra cn gip bn n tp v hiu r hn cc cng thc tnh Entropy.
Yu cu ca bi ton
Mc d ngi t chc chi c th n gian nhng qu trnh trao i 2 ng tin cho nhau l ngu
nhiu, vy liu ngi t chc chi c th n gian hon ton c khng? Hay lng tin bit v
cha bit ca s kin ly mt ng tin t 2 ng tin ni trn c hiu nh th no?
Ta th xt mt trng hp sau: nu ngi t chc chi ly ngu nhin 1 ng tin v sau thc
hin vic tung ng tin ly c 2 ln. Qua 2 ln tung ng tin, ta m c s u hnh xut
hin. Da vo s u hnh xut hin, ta c th phn on c ngi t chc chi ly c
ng tin no.
Chng hn: Nu s u hnh m c sau 2 ln tng l 1 th ng tin ly c l ng tin
tht, ngc li nu s u hnh m c l 2 th ng tin ly c c th l tht hay cng c
th l gi. Nh vy, ta nhn c mt phn thng tin v loi ng tin qua s u hnh m
c sau 2 ln tung. Ta c th tnh c lng tin bng bao nhiu? (Vic tnh lng tin ny s
c tho lun sau).
Xc nh cc phn phi ngu nhin ca bi ton

t X l bin ngu nhin v loi ng tin.
Phn phi ca X:
X
P
1
0.5
2
0.5
t bin ngu nhin Y l s u hnh m c sau 2 ln tung:

Phn phi ca Y khi nhn c ng tin c 1 mt c u hnh (Y/X=1)
Y/X=1
P
0
0.25
1
0.5
2
0.25
26
Phn phi ca Y khi nhn c ng tin c 2 mt u c u hnh (Y/X=2)

Y/X=2
0
1
2
P
0
0
1
Tm phn phi ca Y:
P(Y=0) = p(X=1)p(Y=0/X=1)+p(X=2)p(Y=0/X=2) = 0,5 x 0,25 +0,5 x 0 =0.125
P(Y=1) = p(X=1)p(Y=1/X=1)+p(X=2)p(Y=1/X=2) = 0,5 x 0,5 +0,5 x 0 =0.250
P(Y=2) = p(X=1)p(Y=2/X=1)+p(X=2)p(Y=2/X=2) = 0,5 x 0,25 + 0,5 x 1=0.625
Y
0
1
2
P
0.125 0.25
0.625
Minh ha Entropy H(X), H(Y) v H(X,Y)

Entropy ca X:
H(X) = H(0.5, 05)
= -(0.5)log(0.5) -(0.5)log(0.5) = 1 (bit)
Entropy ca Y:
H(X) = H(0.125, 0.25, 0.625)
= -(0.125)log(0.125) + (0.25)log(0.25) + (0.625)log(0.625) = 1.2988 (bit)

Entropy ca X v Y: H(X,Y)
Xem nh bi tp dnh cho cc bn sinh vin
Entropy ca Y/X l trung bnh ca cc entropy Y/X=xi.
M
Vy, Entropy ca Y c iu kin X: H(Y/X)= p ( xi ).H (Y / X = xi )

i =1
Tng t: H(Y,Z/X), H(Z/X,Y)
Minh ha Entropy H(X/Y) v H(Y/X)

Tnh Entropy ca Y khi bit X: H(Y/X)
H(Y/X=1) = H(0.25, 0.5 , 0.25)
= -(0.25log0.25 + 0.5log0.5 + 0.25log0.25)= 1.5 (bit)
H(Y/X=2)= H(0, 0, 1)= 0
H(Y/X)= p(X=1)H(Y/X=1)+ p(X=2)H(Y/X=2)= 0.5 x 1.5 + 0.5 x 0= 0.75 (bit)
Tnh Entropy ca X khi bit Y: H(X/Y)

Xem nh bi tp dnh cho cc bn sinh vin (Gp : bn nn lp cc phn phi cho cc
trng hp (X/Y=0), (X/Y=1) v (X/Y=2).
Minh ha quan h gia cc Entropy

Xem nh bi tp dnh cho cc bn sinh vin.
Gi : sau khi bn tnh H(X,Y) v H(X/Y), bn da vo cc nh l 1,2 v 3 cng vi cc kt qu
tnh c so snh v minh ha.
27
BAI 2.5: O LNG TIN (MESURE OF INFORMATION)

Mc tiu
- Bit bi ton tnh lng tin,
- Hiu nh ngha lng tin,
- Bit cch tnh lng tin,
- C th vn dng tnh lng tin cho cc bi ton tng t.
t vn bi ton
Mc d ngi t chi c th n gian nhng qu trnh trao i 2 ng tin cho nhau l ngu
nhiu, vy liu ngi t chc chi c th n gian hon ton c khng? Ta th xt mt trng
hp sau: nu ngi chi ly ngu nhin 1 ng tin v sau thc hin vic tung ng tin ly
c 2 ln. Qua 2 ln tung ng tin, ta m c s u hnh xut hin. Da vo s u hnh
xut hin, hy tnh lng tin v loi ng tin ly c l bao nhiu?
Xc nh cc phn phi ca bi ton

t bin ngu nhin X l loi ng tin, khi phn phi ca X c dng :
X
P
1
0.5
2
0.5
t bin ngu nhin Y l s u hnh m c sau 2 ln tung. Khi ta c th xc nh c

phn phi ca Y trong 2 trng hp sau.
Trng hp 1: Phn phi ca Y khi bit ng tin l tht (X=1) c dng:
Y/X=1
P
0
0.25
1
0.5
2
0.25
Trng hp 2: Phn phi ca Y khi bit ng tin l gi (X=2) c dng:

Y/X=2
P
0
0
1
0
2
1
Ta c th tnh d dng phn phi ca Y nh sau:

Y
0
1
2
P
0.125 0.25 0.625
Nhn xt da theo entropy

T cc bng phn phi trn, ta c:
28

Entropy ca Y:
H(Y) = H(0.125, 0.25, 0.625) = 1.3 (bit)
Entropy ca Y khi bit X
H(Y/X=1) = H(0.25, 0.5 , 0.25)= 1.5 (bit)
H(Y/X=2)= H(0, 0, 1)= 0
H(Y/X)= p(X=1)H(Y/X=1)+ p(X=2)H(Y/X=2) = 0.75 (bit)
Vy, H(Y) > H(Y/X)
nh ngha lng tin

T nhn xt v quan h gia cc entropy trn, ta c th nh ngha lng tin nh sau:
nh ngha: Lng tin (hay thng lng) ca X khi Y xy ra l lng chnh lch gia lng
khng chc chn ca X v lng khng chc chn ca X khi Y xy ra c quan h vi X.
Ta c th hiu khi nim ny nh sau: X v Y l 2 bin ngu nhin nn chng c 2 lng tin
khng chc chn. Nu X v Y c lp, th X xy ra khng nh hng ti Y nn ta vn khng bit
g thm v X v X gi nguyn lng khng chc chn ca n. Trong trng hp ny lng tin v
X khi Y xy ra l bng 0. Nu Y c tng quan vi X th khi Y xy ra ta bit hon ton v Y v
mt phn thng tin v X. Phn thng tin chnh l lng tin bit v X nhng vn cha bit
ht v X. Bi ton y l tnh lng tin bit v X khi Y xy ra.
K hiu: I(X/Y) = H(X)-H(X/Y) l lng tin bit v X khi Y xy ra.
Ch : ta lun c I(X/Y) = I(Y/X)
V d: xt li v d trn, ta c lng tin v X khi bit Y l
I(X/Y)= I(Y/X)= H(Y) H(Y/X) = 1.3 0.75=0.55 (bit).
Bi tp
1. Thc hin mt php th con xc sc ng cht ng thi vi mt ng tin cng ng cht.
Trong , con xc sc c cc mt im t 1 n 6, ng tin mt mt c u hnh v mt kia
khng c u hnh. Trc tin th con xc sc, nu s im 4 th tung ng tin mt ln, ngc
li th tung ng tin hai ln. Tnh lng tin v s im con xc sc khi bit thng tin v s u
hnh m c.
2. Ngi ta thc hin mt kho st trn cc sinh vin i hc v mi quan h gia kh nng hc
tp vi s hu phng tin i li v tinh thn i hu. Kt qu cho thy:
Trong tng s sinh vin c 3/4 sinh vin hon thnh chng trnh hc v 1/4 khng hon thnh.
Trong s sinh vin hon thnh chng trnh hc, 10% c xe con. Ngc li, trong s sinh vin
khng hon thnh chng trnh hc c ti 50% c xe con.
Tt c sinh vin c xe con u tham gia hi i hu sinh vin. Trong s sinh vin khng c xe con
(k c hon thnh hay khng hon thnh kha hc) th 40% sinh vin tham gia hi i hu sinh
vin.
a. Tm thng tin v trng thi hc tp ca sinh vin khi bit iu kin v phng tin i li ca
h.
b. Tm thng tin v tnh trng hc tp ca sinh vin khi bit tinh thn i hu ca h.
29

3. Nhng ngi dn ca mt lng c chia lm 2 nhm A v B. Mt na nhm A chuyn ni
tht, 3/10 ni di v 2/10 t tri tr li. Trong nhm B: 3/10 ni tht, 1/2 ni di v 2/10 t tri
tr li. Gi s p l xc sut chn 1 ngi thuc nhm A v I(p) = I(Y/X) l lng tin v ngi ni
tht sau khi chn nhm, tnh I(p), tm p* sao I(p*) = Max(I(p) v tnh I(p*).
30
CHNG 3: SINH M TCH C

(Decypherable Coding)
Mc tiu:
Phn ny cp n bi ton m ha (coding) cc gi tr ca mt bin X. Khi m cc gi tr
ca X ngi ta phi s dng bng k t m (Coding Character Table) hay bng ch ci (Code
Alphabet). Nh vy, mt gi tr x ca X s c m thnh mt t m (Code Word) w di dng
mt dy cc k t m vi di l n k t. Trong truyn tin, mt dy cc gi tr ca X c pht
sinh v c m thnh mt dy lin tc cc t m hay mt dy cc k t m ly t bng k t
m. Vn cn gii quyt l:
1. Khi nhn mt dy k t m lin tc th ta c th gii m thnh mt dy cc gi tr duy
nht ca X hay khng ? Ni cch khc, dy k t m ny c tch c thnh cc t m
mt cch duy nht hay khng ?
2. Ch ra phng php xy dng m tch c ti u.
BI 3.1: KHI NIM V M TCH C

Mc tiu
- Bit yu cu ca bi ton sinh m,
- Hiu khi nim v bng m tch c v bng m khng tch c,
- Hiu khi nim v bng m tc thi,
- Hiu gii thut kim tra tnh tch c ca mt bng m,
- Vn dng gii thut kim tra tnh tch c ca mt bng m kim tra xem mt bng
m c phi l bng m tch c hay khng.
t vn bi ton sinh m
Gi s ngun tin X xut hin v c ghi li thng qua mt thit b c bit. Chng hn nh nh
c ghi li bng my nh, m thanh c ghi li bng my ghi m, Qua knh truyn, nhng
thng tin ny cn phi c m ha cho ph hp. c th m ha ngi ta cn mt bng ch ci
gm cc ch ci quy nh trc (chng hn bng ch ci la tinh, bng m nh phn, ). Mi gi
tr ca X sau c m di dng mt dy hu hn cc ch ci v ta gi dy hu hn cc ch
ci gn cho mt gi tr ca x l mt t m.
Ta xt BNN X={x1, x2, ,xn} c phn phi {p1, p2, , pn} c quan st lin tc v c lp. Dy
cc gi tr nhn c gi l thng bo (Message) c dng xi1xi2xin. Tp hp A={a1, a2, , an} l
tp hp k t m (Code Characters) hay l bng ch ci (Code Alphabet) dng sinh m. Mt
gi tr xi X c gn bi mt dy hu hn cc k t m c gi l t m (Code word). Tp
hp gm tt c cc t m gn cho tt c cc gi tr ca X c gi l b m hay bng m (Code).
Cc t m phi khc nhau tng i mt.
31

B m c gi l tch c nu nh t mt dy cc k t m nhn c lin tc (c m ha
t b m ny), ta lun lun gii m c vi kt qu duy nht l dy cc gi tr gc ca X.
Shannon (1948) ln u tin a ra nh l c s v sinh m tch c. Mc Millan (1956)
chng minh nh l v iu kin cn v ca bng m tch c. Nhng vn sinh m tch
c ch c xt mt cch chun mc bi Feinstein (1958), Abramson (1963) v Fano (1961).
Sardinas(1960) v Patterson (1963) a ra nh l v gii thut kim tra tnh tch c ca mt
bng m. Abramson (1963) a ra khi nim bng m tc thi.
Trong phm vi bi ging ny, bi ton sinh m ti u c t ra y l tm ra mt phng php
sinh m sao cho di trung bnh ca cc t m trong b m l nh nht. Ngha l, nu gi tr xi
c gn bi t m c di ni th bi ton sinh m phi tha:
n
pn
i =1
Min
Huffman (1950) a ra qui trnh xy dng mt bng m ti u tha yu cu ny.
Khi nim v bng m khng tch c

Bng m khng tch c l bng m m khi m ha thng bo Msg ta s nhn c mt dy cc
t m ws, v khi gii m dy cc t m ws th ta c th nhn c nhiu thng bo Msg khc
nhau.
V d: Xt bin ngu nhin X={x1, x2, x3, x4} c bng m W={w1=0, w2=1, w3=01, w4=10}.
Gi s thng bo ngun c ni dung: x1x2x3x4x3x2x1. Khi dy m tng ng vit t W c
dng: 0101100110.
Nu gii m tun t t tri qua phi ta nhn kt qu: x1x2x1x2x2x1x1x2x2x1. Nhng nu bng
phng php khc ta c th nhn c kt qu: x3x3x4x3x4 v nhiu thng bo khc na.
Nhn xt: Bng m gii m khng tch c l bng m m trong tn ti t nht mt t m
ny l m kha ca mt hay nhiu t m khc trong b m (v d t m w1=0 hay w2=1 l m
kha ca w3).
Bng m tch c
Bng m tch c l bng m m khi m ha thng bo Msg ta s nhn c dy cc t m ws,
v khi gii m dy cc t m ws th ta ch nhn c mt thng bo duy nht l Msg ban u.
V d: Xt bin ngu nhin X={x1, x2} c bng m tng ng W={w1=0, w2=01}.
Phng php gii m c s dng nh sau: ch gii m khi no nhn c on m vi
di bng di ca t m di nht.
Gi s dy m nhn c (cn gii m) l: 0010000101001.
S dng phng php gii m trn ta nhn c duy nht dy thng bo gc:
x1x2x1x1x1x2x2x1x2.
C th chi tit ha cc bc gii m dy t m trn nh sau:
Nhn c on 00 -> Gii ra x1 , cn li 0.
Nhn tip 1 ->01 -> Gii ra x2.
Nhn tip 00
-> Gii ra x1, cn li 0.
32

Nhn tip 0 -> 00 -> Gii ra x1, cn li 0.
Nhn tip 0 -> 00 -> Gii ra x1, cn li 0.
Nhn tip 1 -> 01 -> Gii ra x2.
Nhn tip 01
-> Gii ra x2.
Nhn tip 00
-> Gii ra x1, cn li 0.
Nhn tip 1 -> 01 -> Gii ra x2.
Kt qu dy thng bo l: x1x2x1x1x1x2x2x1x2.
Kt lun: Bng m tch c l bng m m trong khng tn li t m ny l m kha t m
khc, tuy nhin vn c th tn ti t m ny l tin t (phn u) ca t m kia.
Khi nim bng m tc thi

Bng m tc thi l bng m m khi m ha thng bo Msg ta s nhn c dy cc t m ws, v
khi gii m dy cc t m ws th ta ch nhn c mt thng bo duy nht l Msg ban u.
Abramson chng minh c kt qu sau: Bng m tc thi l bng m khng tn ti t
m ny l tin t ca t m khc.
V d 1: Bng m W={w1=10; w2=101; w3=100} khng phi bng m tc thi v w1 l tin t ca
w2 v w3.
V d 2: Bng m W={w1=0, w2=100, w3=101, w4=11} l bng m tc thi v khng tn ti t
m ny l tin t ca t m khc.
Gii thut kim tra tnh tch c ca bng m

Th tc sau y do Sardinas (1960), Patterson (1963) v Abramson (1963) a ra nhm kim tra
xem mt bng m no c phi l bng m tch c (bng m cho php gii m duy nht) hay
khng.
Input: Bng m W
Output: Kt lun bng m tch c hay khng tch c.
Gii thut:
Bc khi to: Gn tp hp S0=W.
Bc 1: xc nh tp hp S1 t S0:
- Khi to S1={}
- Vi wi, wj S0, ta xt: nu wi=wjA (wj l tin t ca wi) hoc wj=wi A (wi l tin t
ca wj) th thm A (phn hu t) vo S1.
Bc k: xc nh tp hp Sk (k2) t tp hp S0 v Sk-1:
- Khi to: Sk={}
- Vi wi S0 v vj Sk-1, ta xt: nu wi=vjA (vj l tin t ca wi) hoc vj=wi A (wi l
tin t ca vj) th thm A (phn hu t) vo Sk.
iu kin dng vng lp:
Nu Sk={} th dng v kt lun bng m tch c (k1).
Nu tn ti t m wi trong Sk hay Sk S0 th dng v kt lun bng m khng tch
c.
Nu Sk=St<k th dng v kt lun bng m tch c (k1).
Bi ton 1- yu cu
33

Bi ton: Kim tra xem bng m W={a, c, ad, abb, bad, deb, bbcde} c phi l bng m tch
c hay khng?
p dng Gii thut kim tra tnh tch c ca mt bng m:
Bc khi to: S0={a, c, ad, abb, bad, deb, bbcde}
Bc 1: Tnh S1
Khi to S1={}
V a l tin t ca ad nn a phn hu t d vo S1 => S1={d}.
V a l tin t ca abb nn a phn hu t bb vo S1 => S1={d, bb}.
Kim tra iu kin dng: khng tha -> qua bc 2.
Bc 2: Tnh S2 t S0 v S1.
Khi to S2={}.
V d S1 l tin t ca deb S0 nn a phn hu t eb vo S2
=> S2={eb}
V bb S1 l tin t ca bbcde S0 nn a phn hu t cde vo S2
=> S2={eb, cde}
Bi ton 1 - p dng gii thut
Khi to S3={}.
V c S0 l tin t ca cde S2 nn a phn hu t de vo S3
=> S3={de}
Khi to S4={}.
V de S3 l tin t ca deb S0 nn a phn hu t b vo S4
=> S4={b}
+ khi to S5={}.
+ V b S4 l tin t ca bad S0 nn a phn hu t ad vo S5 => S5={ad}
+ V b S4 l tin t ca bbcde S0 nn a bcde vo S5
=> S5={ad, bcde}
Kim tra iu kin dng: V S5 c cha t m ad nn dng li v kt lun y l bng m
khng tch c.
Bi ton 2
Bi ton: Kim tra xem bng m W={010, 0001, 0110, 1100, 00011, 00110, 11110, 101011} c
phi l bng m tch c khng?
p dng Gii thut kim tra tnh tch c ca mt bng m:
Bc khi to v bc 1
- Tp hp S0 ={010, 0001, 0110, 1100, 00011, 00110, 11110, 101011}
- Tp hp S1 ={1}
Dnh cho sinh vin t lm cc buc tip theo.
Kt qu gi :
Tp hp S2 ={100, 1110, 01011}
Tp hp S3={11}
34

Tp hp S4={00, 110}
Tp hp S5={01, 0, 011, 110}
Tp hp S6={0, 10, 001, 110, 0011, 0110}
Tp hp S6 cha t m 0110 nn bng m ny khng phi l bng m tch c.
Bi tp
1. Hy cho bit bng m sau c phi l bng m tch c hay khng?
W={w1=00, w2=01, w3=0010, w4=0111, w5=0110}
2. Hy ly v d mt bng m tch c, v chng minh n l bng m tch c.
35
BI 3.2: QUAN H GIA M TCH C V DI

M
Mc tiu
Sau khi hon tt bi hc ny bn c th hiu:
- nh l Kraft (1949),
- nh ngha cy bc D c K,
- Vn sinh m cho cy bc D c K,
- Vn dng nh l Kraff kim tra s tn ti bng m tch c v sinh bng m tch
c.
nh l Kraftn(1949).
Gi X={x1, x2,, xM} l bin ngu nhin cha cc gi tr cn truyn c phn phi l P={p1, p2,
, pM}.
A={a1, a2,,aD} l b k t sinh m c D ch ci (D c gi l c s sinh m).
Gi tr xi c m ha thnh t m wi c di l ni.
t N={n1, n2,,nM} l tp hp di cc t m.
nh l (Kraft- 1949):
iu kin cn v tn ti bng m tc thi vi di N={n1,n2,,nM} l
M
ni
i =1
V d 1: B m W={w1, w2, w3} vi M=3; n1=1; n2=2; n3=3; D=2

M
1
1
1 7
D ni = 1 + 2 + 3 = < 1
8
2
2
2
i =1
=> Tn ti bng m tc thi.
V d 2: B m W={w1, w2, w3} vi M=3; n1=n2=1; n3=2; D=2

M
1
1
1
5
+ 1 + 2 = >1
1
4
2
2
2
i =1
=> Khng tn ti bng m tc thi.
ngh: sinh vin tm hiu ni dung tip theo v tr li gii thch 2 v d trn.
ni
nh ngha cy bc D c k.
nh ngha: Cy bc D c k l cy c h thng nt, cnh tha iu kin:
- T 1 nt c s cnh i ra khng vt qu D hay mt nt c khng qu D nt con.
- Nt cui cng (Nt l) cch nt gc khng vt qu k cnh.
36

V d: cy bc D=2 v c k=3
Vn sinh m cho cy bc D c k
Sinh m cho cc nt ca cy bc D c K (tr nt gc):
n gin ha: mi nt (tr nt gc) c k hiu bi dy k hiu ca nt cha lm tin t +
mt k t b sung ly t tp hp {0, 1, 2, , D-1} thay cho bng ch ci A={a1, a2, , aD}.
V d 1: Cy bc D=2 c k=3
00
0
01
V d 2: Cy bc D=3 c k=2.
000
01
010
02
1
110
10
11
100
101
11
00
001
011
10
12
2
111
20
21
22
Tnh cht:
+ Cc nt (tr nt gc) ca cy u c m ha t bng ch ci {0, 1, 2,, D-1}
+ Mi nt ( m ha) c m ca nt k trc l tin t.
+ Tng s cc nt l bng Dk = tng s cc m tc thi c th c.
Chng minh nh l Kraft (iu kin cn)

Gi s, cho trc bng m tc thi W={w1, w2,, wM} vi N={n1 n2 nM}. Ta cn c/m:
M
ni
i =1
Xy dng cy bc D c nM v sinh m cho cc nt tr nt gc vi cc k t m ly t bng ch

ci A = {0, 1, 2,, D-1}. M ti mi nt (tr nt gc) u c kh nng c chn l t m.
Nh vy, ta tin hnh chn cc t m cho bng m tc thi vi qui tc l: mt nt no c
chn gn mt t m th tt c cc nt k sau nt gn t m phi c xa. C th nh sau:
37

Chn mt nt c m vi di m l n1 gn cho n mt t m w1.
n n
=> Tng s nt l c xa tng ng l D M 1
Chn mt nt c m vi di m l n2 gn cho n mt t m w2.
n n
=> Tng s nt l c xa tng ng l D M 2
Chn mt nt c m vi di m l nn gn cho n mt t m wn.

n n
=> s nt l c gn t m l D M M
Vy s nt l b xa hoc c gn t m l:
=> D
=>
nM n1
ni
+D
nM n1
+L+ D
nM n M
= D
nM ni
nM
= tng s nt l.
i =1
1 (pcm)
i =1
Chng minh nh l Kraft (iu kin )

Gi s:
ni
1 , cn chng minh tn ti bng m tc thi vi N={n1, n2, , nM}, ta ch
i =1
cn ch ra th tc xy dng bng m tc thi nh sau:
Th tc to m tc thi:
Xt N={n1, n2, ,nM} v c s sinh m l D:
Bc 1: Ta xp th t n1 n2 nM, xy dng cy bc D c k=nM v sinh m cho cc nt .
Bc 2: Chn nt bt k trn cy c di n1 gn cho t m w1 v xa tt c cc nt k sau n.
Bc 3: Lp li cc bc 2 i vi vic chn cc t m cn li w2, , wM ng vi n2, , nM.
=> Bng m W={w1, w2, , wM} l bng m tc thi.
V d minh ha nh l Kraft
V d 1: Xt bng m tha M=3, D=2, n1=1, n2=2, n3=3. Vy ta kim tra xem c to c bng
m tc thi hay khng?
3
7
n
Ta c 2 i = 2 1 + 2 2 + 2 3 = < 1
8
i =1
=> W= {w1, w2, w3} l bng m tc thi
Ta Xy dng bng m nh sau:
000
00
001
w1= 0
010
01
011
1
- Chn w1=0 , ct b cc nt con ca nt w1.

- Chn w2=10, ct b cc nt con ca nt
w2.
- Chn w3=111
100
w2=10
101
110
11
w3=
111
38
Ch : ngoi bng m tc thi chn c trn, ta cn c th sinh c nhiu bng m tc thi

khc. ngh sinh vin a ra bng m tc thi khc.
Bi tp
1. Tm 1 bng m tch c tha tnh cht D = 2, k = 4?
2. Tm tt c cc bng m tch c tha tnh cht D=2, k=3?
3. Hy ch ra bng m sau y l bng m khng tch c:
W={w1=00, w2=1, w3=100, w4=110, w5=111}
4. Hy tm mt bng m nh phn tch c c t nht 5 t m tha iu kin
M
ni
=1
i =1
39
BI 3.3: TNH TI U CA DI M
Mc tiu
- Hiu nh l Shannon (1948),
- Bit c cc tiu chun nh gi bng m ti u tuyt i v bng m ti u tng i,
- iu kin nhn bit mt bng m ti u,
- Hiu nh l Huffman,
- Bit Phng php sinh m Huffman,
- Vn dng phng php sinh m Huffman sinh m Huffman cho mt thng bo,
- Vn dng phng php sinh m Huffman vit chng trnh nn.
nh l Shannon (1948)
Pht biu nh l:
M
t n = pi ni l di trung bnh ca bng m.

i =1
Khi n
H(X )
log 2 D
Du ng thc xy ra khi v ch khi pi = D ni hay
ni
=1
i =1
H (X )
. Nu m
log 2 D
khng tch c di trung bnh ca n c th nh hn cn di. Nu m tch c khng ti
u th di ca n s ln hn nhiu so vi cn di, cn nu m tch c ti u th di
trung bnh ca n gn vi cn di.
Bi ton t ra s l tm phng php xy dng bng m tch c ti u.
Ch :
H D (X) = pi log D pi
Din gii: i vi m tch c di trung bnh ca m s c cn di l
H ( X ) pi log 2 pi
=
log 2 D
log 2 D
l entropy ca X vi c s D.
(X ) =
Bng m ti u tuyt i
nh l: Bng m c gi l ti u tuyt i khi n =
H (X )
hay pi = D ni
D
log 2
V d: xt bin ngu nhin X={x1, x2, x3, x4}

C phn phi: P={1/2, 1/4, 1/8, 1/8}
C bng m W={w1= 0, w2=10, w3=110, w4=111}
1
1
1
1
12
Ta tnh c di trung bnh t m: n = *1 + * 2 + * 3 + * 3 = = 1.75
2
4
8
8
8
Tnh Entropy ca X: H(X)= H(0.5, 0.25, 0.125, 0.125) = 0.5 +0.5 + 0.375 + 0.375 =1.75
Log2D=1.
40

000
00
W= {w1, w2, w3, w4} l bng m ti u

tuyt i v tha iu kin:
H (X )
n=
log 2D
001
0
010
01
w1
011
10
1
w2
11
100
101
w3
111 w4
110
Bng m ti u tng i
nh l: Bng m c gi l ti u tng i khi:
H(X )
H (X )
n<
+1
log 2 D
log 2 D
iu kin nhn bit mt bng m ti u

nh l (vi D=2):
- Xc sut pi cng ln th di ni ca t m wi cng nh.
- Gi s p1 p2 pM. Nu pi pi+1 pi+r th ni ni+1 ni+r th 2 t m tng ng vi 2
gi tr c xc sut nh nht c di m bng nhau nM-1 =nM.
- Trong cc t m c di bng nhau v cng bng nM (di nht) th tn ti t nht 2 t m
wM-1 v wM c M-1 k t u ging nhau v k t th M khc nhau.
V d: xt bng m W={w1=0, w2=100, w3=101, w4=1101, w5=1110}
Bng m trn khng phi l bng m ti u v 2 t m w4, w5 c di ln nht =4 k t nhng 3
k t u khc nhau.
Ghi ch: D > 2 c xt tng t.
nh l Huffman
nh l: Gi s X c phn phi xc sut vi th t gim dn sau:
X
P
x1
p1
x2
p2
xM
pM
Gi s bng m ca X l W={w1, w2, , wM-1, wM}.

t xM-1,M={xM-1, xM} c xc sut l pM-1,M=pM-1+pM.
v X* = { x1, x2,, xM-1,M} c phn phi sau:
X*
P
x1
P1
x2
p2
x*M-2 x*M-1,M
p*M-2 p*M-1,M
Gi s W* ={w1, w2, , wM-2, x*M-1,M} l bng m ti u ca X*. Khi :

- wM-1=w*M-1,M + 0.
- wM =w*M-1,M + 1.
41
Phng php sinh m Huffman

Gi s X c phn phi xc sut vi th t gim dn sau:
X
P
x1
p1
x2
p2
xM
pM
Th tc li (D=2):
Khi to: t M0=M
Bc 1:
- t x M 0 1, M 0 = {x M 0 1 , x M 0 } c xc sut p M 0 1, M 0 = p M 0 1 + p M
- Sp xp li theo t t gim dn ca xc sut ta nhn c dy phn phi mi c M0-1

phn t nh sau: p1 , p 2 , L , p M 0 2 , p M 0 1, M 0
Bc 2: Lp li bc 1 vi s lu vt
wM 0 1 = wM 0 1,M 0 +"0"
wM 0 = wM 0 1,M 0 +"1"
Gim M0: M0=M0-1, vng lp kt thc khi M0=2
(Ch : trong trng hp tng qut, vong lp kt thc khi M0 D.)
Th tc tin:
i ngc li so vi th tc li ng thi xc nh t m mi bc t s lu vt th tc
li.
Minh ha phng php sinh m Huffman
V d 1: sinh bng m nh phn Huffman cho X c phn phi sau:
X
P
x1 x2
x3 x4
x5 x6
0.3 0.25 0.2 0.1 0.1 0.05
42

Th tc li:
Bc 1
Bc 2
Bc 3
Bc 4
Bc 5
X P
X P
X
P
X
P
X
P
x1 0.3
x1 0.3
x1 0.3 x23 0.45
x1564 0.55
x2 0.25
x2 0.25
x564 0.25
x1
0.3 0 x23
x3 0.2
x3 02
x2 0,25 0
x564 0.25 1
x4 0.1
x56 0.15 0 x3 0.2
1
x5 0.1 0 x4 0.1
1
x6 0.05 1
Th tc tin:
Bc 1
Bc 2
X W
x1564 0
x23
1
X W
x23
1
x1
00
x564 01
Bc 3
X
W
x1
00
x564 01
x2
10
x3
11
Bc 4
X
W
x1
x2
x3
x56
x4
0
0.45
Bc 5
X W
00
x1
10
x2
11
x3
010
x4
011
x5
x6
00 = w1
10 = w2
11 = w3
011 = w4
0100 = w5
0101 = w6
Nhn xt tnh ti u ca bng m Huffman

V cy Huffman ca bng m trn:
00=w1
010
0
01
0100=w5
0101=w6
10=w2
011=w
di trung bnh ca t m: 11=w

n =(0.3 x 2)+ (0.25 x 2)+ (0.2 x 2) + (0.1 x 3) +(0.1 x 4) + (0.05 x 4) = 2.4
Entropy ca X: H(X) = H(0.3, 0.25; 0.2, 0.1,0.1, 0.05)
= 2.4
Nhn xt: Do D = 2 v log2D=1, Ta c n = H(X) nn bng m trn ti u tuyt i.
Bi tp
1. Cho bin ngu nhin X c phn phi sau:

X
P
x1 x2
0.4 0.3
x3 x4
0.2 0.1
43

2. Cho bin ngu nhin Y c phn phi sau:
Y
P
y1 y2
0.3 0.2
y3 y4 y5
y6
y7
y8 y9
0.2 0.1 0.05 0.05 0.04 0.03 0.03
3. Cho on vn bn thoi the thi thoi thi the thoi thi the. Tm bng m nh phn Huffman
dng m ha on vn bn trn.
4. Thay tng k t trong on vn bn trn thnh mt t m, ct tng on 8 bits i thnh
s thp phn. Cho bit dy s thp phn kt qu.
44
CHNG 4: KNH TRUYN

Mc tiu:
Trnh by m hnh truyn tin ri rc tng k t m c lp ln nhau (ph hp vi c im
ca knh). M hnh ny cn gi l knh truyn ri rc khng nh (Memoryless Discret Channel).
T m hnh ny ngi ta c th xy dng cch tnh dung lng knh truyn v phng php phn
loi u nhn c th gii m tt nht.
BI 4.1: KNH TRUYN RI RC KHNG NH

Mc tiu
- Bit m hnh knh truyn tin ri rc khng nh 2 kha cnh vt l v ton hc.
- Khi nim v lng tin trn knh truyn
- nh ngha dung lng knh truyn
Gii thiu
Trc ht, ta c th hiu khi nim knh truyn ri rc v khng nh bi hc ny nh sau: khi
nim truyn ri rc y l truyn tun t cc k t c lp nhau (hay truyn tng k t mt),
cn khi nim khng nh y l ch xt mi quan h gia k t truyn v k t nhn c
tng ng, khng xt n mi quan h gia k t nhn c vi k t nhn c trc .
Khi nim v mt knh truyn ri rc da vo phn b xc sut ca tn hiu ra ph thuc vo tn
hiu vo v trng thi ca knh truyn c chun ha bi Feinstein (1958) v Wolfowitz
(1961). Dung lng knh (Channel Capacity) c xc nh chnh xc nh Muroya (1953) v
Fano (1961). Gii thut v chng trnh tnh dung lng knh c vit bi Eisenberg (1963).
nh l c bn v truyn tin ch ra rng vi dung lng knh cho trc lun c th tm ra mt
phng php truyn tin vi lng tin nh hn dung lng knh v t sai s nh hn sai s cho
php bt k. nh l c bn v truyn tin c Feinstein (1954, 1958) kho st. Cc nh khoa
hc Blackwell, Breinan (1958, 1959) v Thomasian (1961) ln lt chnh l t chun tt
hn. Trong cc ni dung tip theo ca bi hc, cc bn s tm hiu v m hnh knh truyn tin ri
rc khng nh khia cnh vt l v ton hc. c bit m hnh ton hc s ch ra cch xc nh
phn phi u ra da vo phn phi u vo.
M hnh vt l
Mt thng bo c cu to t cc k hiu ca mt bng ch ci u truyn (input) v c
truyn trn knh. Thng bo c nhn cui knh (hay u nhn-output) v c gii m theo
bng ch ci u truyn. Mt khc, tng k t u nhn c th quan h vi cc k t u
nhn trc , cc k t u truyn v trng thi ca knh truyn. n gin, y chng ta
ch xt m hnh vt l nh sau:
45

Xt tng k t u nhn ch ph thuc vo k t u truyn tng ng vi n, nu knh
truyn c nhiu th mt k t u truyn c th c din gii (nhiu) ra nhiu k t khc nhau
u nhn v do to ra mt phn phi xc sut c iu kin cho k t u nhn. M hnh
truyn tin ri rc khng nh l m hnh truyn tin ch xt mi quan h gia k t truyn v k t
nhn c tng ng, khng xt mi quan h gia k t nhn c v k t nhn c trc .
M hnh:
X
u truyn
Knh truyn
u nhn
P(e)
nhiu
X
Cc qui c:
- X: l bin ngu nhin c gi tr cn truyn u truyn.
- Y: l bin ngu nhin cha gi tr c th nhn c u nhn.
- X: l bng ch ci sinh m u truyn.
- Y: l bng ch ci gii m u nhn.
- X, Y, X, Y: u hu hn v ri rc.
- Truyn ri rc tng k t v nhn cng ri rc tng k t.
- K t nhn sau khng ph thuc vo k t nhn trc.
M hnh ton hc
Ta gi:
- X={x1, x2, , xM} l b k t sinh m u truyn (input).
- Y={y1, y2,,yL} l b k t gii m u nhn (output).
- Bin ngu nhin X ly gi tr ( m ha) trn X v c phn phi p(X=xi)=p(xi) vi
i=1,..,M.
- Bin ngu nhin Y ly gi tr (gii m) trn Y v c phn phi xc sut c iu kin:
P(Y=yj/X=xi)=p(yj/xi)=pij vi j=1,..,L.
Gi A=||pij|| l ma trn truyn tin hay m hnh truyn tin ca knh truyn ri rc khng nh.
Vi i= 1, M , j= 1, L v pij = p(Y=yj/X=xi) = p(yj/xi) l xc sut nhn c gi tr yj khi truyn
gi tr xi.
Tnh phn phi u nhn:
M
Ta c: p(Y=yj) = p(yj) = p( xi ). p ( y j / xi )
i =1
p(yj) = p ( xi ). p ( y j / xi )
i =1
M
= p ( xi ). pij
i =1
Vy p(yj)= PX .Aj
(1)
Mt cc tng qut: PY = PX.A

Trong :
(2)
46

- Aj l ct th j ca A
- PX = [p(x1), p(x2),., p(xM)].
- PY = [p(y1), p(y2),., p(yM)].
V d xc nh phn phi u nhn

Cho ma trn truyn tin nh sau:
x1 0.5 0.2 0.3
A = x 2 0.3 0.5 0.2
x3 0.2 0.3 0.5
y1
y 2 y3
Xc sut truyn: p(x1)=0.5 v p(x2)=p(x3)= 0.25.
Ta tm phn phi ca Y :
Ta c: PX =(0.5, 0.25, 0.25)
p dng cng thc (1) trn ta c:
p(y1) = Px .A1 = 0.375
p(y2) = Px .A2 = 0.3
p(y3) = Px .A3 = 0.325
PY =(0.375, 0.3, 0.325)
Lng tin trn knh truyn

V d: cho ma trn truyn tin nh sau:
x1 0.5 0.2 0.3
A = x 2 0.3 0.5 0.2
x3 0.2 0.3 0.5
y1
y 2 y3
Xc sut truyn: p(x1)=0.5 v p(x2)=p(x3)= 0.25.
X = {x1, x2, x3} c xem nh tp cc k t truyn v Y ={y1, y2, y3} l tp cc k t nhn.
Tnh lng tin trn knh truyn:

Ta tm phn phi ca Y :
Ta c: PX =(0.5, 0.25, 0.25)
p dng cng thc (1) trn ta c:
p(y1) = Px .A1 = 0.375
p(y2) = Px .A2 = 0.3
p(y3) = Px .A3 = 0.325
PY =(0.375, 0.3, 0.325)
Tnh cc Entropy:
H(Y) = H(0.375, 0.3, 0.325) = 1.58 (bit)
H(Y/X=x1) = H(0.5, 0.2, 0.3)= 1.49 (bit)
H(Y/X=x2) = H(0.3, 0.5, 0.2)= 1.49 (bit)
H(Y/X=x1) = H(0.2, 0.3, 0.5)= 1.49 (bit)
H(Y/X)= p(x1).H(Y/X=x1) + p(x2).H(Y/X=x2) + p(x3).H(Y/X=x3) = 1.49 (bit)
Lng thng tin truyn trn knh: I (X/Y)= I (Y/X)= H(Y) - H(Y/X) = 0,09 (bit)
47
nh ngha dung lng knh truyn

Da vo ma trn truyn tin A, ta c th d dng tnh lng tin trn knh truyn.
I(X/Y)= H(X)-H(Y/X)
= H(Y)-H(X/Y)
= I(Y/X)
Ta c I(X/Y)= H(Y)-H(Y/X), trong :
H(Y)= H(PX .A) ph thuc vo PX.
H(Y/X) ph thuc vo PX
Vy: I(Y/X) ph thuc hon ton vo PX v do I(Y/X) c th t Max vi PX xc nh no .
Ta nh ngha: C = Max I ( X / Y ) l dung lng ca knh truyn (VT: bit).
p ( X )
48
BAI 4.2: CC DNG KNH TRUYN

Mc tiu
Bit knh truyn khng mt tin,
Bit knh truyn xc nh,
Bit knh truyn khng nhiu,
Bit knh truyn khng s dng c,
Hiu knh truyn i xng,
Hiu nh l v dung lng knh truyn,Knh truyn khng mt tin

M hnh: t tp hp cc gi tr c th nhn c u nhn Y={y1, y2, , yL} c phn thnh
M nhm Bi tng ng vi cc gi tr xi u truyn v xc sut truyn xi vi iu kin nhn
yj l p(X= xi /Y=yj Bi)=1 ( vi M < L ).
u truyn
x1
u nhn
y1
Nhm B1
yk
x2
yk+1
Nhm B2
yh
xM
yt
Nhm BM
yL
c trng ca knh truyn khng mt tin l H(X/Y)=0. C ngha l lng tin cha bit v X khi
nhn Y l bng 0 hay ta c th hiu khi nhn c Y th ta hon ton c th bit v X.
Dung lng: C=log2M (Sinh vin t chng minh, xem nh bi tp)
Knh truyn xc nh
M hnh: t tp hp cc gi tr c th truyn u truyn c phn thnh L nhm Bj tng ng
vi cc gi tr c th nhn c yj u nhn v xc sut nhn yj vi iu kin truyn xi l
p(Y=yj/X=xi Bj)=1 (M>L).
u truyn
u nhn
x1
Nhm B1
y1
xk
xk+1
Nhm B2
y2
xh
xt
Nhm BL
yL
xL
c trng: ca knh truyn xc nh l H(Y/X)=0. C ngha l lng tin cha bit v Y khi
truyn X bng 0 hay khi truyn X th ta bit s nhn c Y.
49

Dung lng: C=log2L (Sinh vin t chng minh, xem nh bi tp)
Knh truyn khng nhiu

M hnh: l s kt hp ca knh truyn xc nh v knh truyn khng mt thng tin, truyn k
t no s nhn c ng k t .
u truyn
u nhn
x1
x1
x2
x2
xM
xM
c trng: H(X/Y)=H(Y/X)=0.
Dung lng: C=log2L=log2M (Sinh vin t chng minh, xem nh bi tp)
V d: ma trn truyn tin ca knh truyn khng nhiu vi M=L=3:
x1 1 0 0
x 2 0 1 0
A=
x3 0 0 1
y1 y 2 y 3
Knh truyn khng s dng c.

M hnh: l knh truyn m khi truyn gi tr no th mt gi tr hoc xc sut nhiu thng tin
trn knh truyn ln hn xc sut nhn c.
c trng: H(X/Y)=H(Y/X)= max

Dung lng: C=0 (Sinh vin t chng minh, xem nh bi tp)
V d: knh truyn c ma trn truyn tin nh sau:
1
A=
1
Knh truyn i xng

M hnh: l knh truyn m ma trn truyn tin c c im sau:
+ Mi dng ca ma trn A l mt hon v ca phn phi P={p1, p2, , pL}
+ Mi ct ca ma trn A l mt hon v ca Q={q1, q2, , qM}
V d: cho knh truyn i xng c ma trn truyn tin nh sau:
x1 1 / 2 1 / 3 1 / 6
x 2 1 / 3 1 / 6 1 / 2
A =
x3 1 / 6 1 / 2 1 / 3
y1
y2
y3
50
Xy dng cng thc tnh dung lng knh truyn i xng

Do H(Y/X) khng ph thuc vo phn phi ca X => Max ca I(X/Y) c quy v m ca H(Y).
Hay
C = Max I ( X / Y ) = Max( H (Y ) H (Y / X ))
Ta c th tnh d dng:
L
H (Y / X ) = p ' j log p ' j = const

j =i
Do :
L
C = Max I ( X / Y ) = MaxH (Y ) + p ' j log p ' j

j =i
Do H(Y)<= logL => ta cn chng t = xy ra khi p1=p2=...=pL=1/L

Xt trng hp P(X=xi)=1/M, vi mi i => chng minh P(Y=yj)=1/L vi mi j
Tht vy :
M
P (Y = y j ) = P(Y = y j , X = xi )
i =1
= P( X = xi )P(Y = y j / X = xi ) =
i =1
1
M
P
i =1
ij
1
qi
M
T A ta nhn thy:
p11 ...
A = ... ...
p
M 1 ...
Do
= A
p1L
... => A = tng cc phn t ca A.

p ML
+ hang
= A
+ cot
i =i
i =i
=> M = L qi => qi =
M
L
1 M 1
=> P (Y = y j ) =
= => H (Y ) = p ' P (Y = y j ) log P (Y = y j ) = log L = Max
M L
L
=> H(Y) t max l logL khi P(Y=yj)=1/L hoc P(X=xi)=1/M
L
Vy: C= log L H(p1, p2, , pL ) hay C = log L + p j log p j

j =1
Ch : trng hp knh 1 bit vi nhiu

1
Ma trn truyn tin A =

1

Dung lng C=1+(1-) log(1-)+log = 1- H(, 1-)
H( , 1-)
51
1 H(,1-)
nh l v dung lng knh truyn

Gi s ma trn A c dng vung v c ma trn nghch o l A-1
K hiu A=||pij|| vi i=1,2,...,M v j =1,2,...,M
A-1=||qij|| vi i=1,2,...,M v j =1,2,...,M
M
M
t tham s dk= q jk exp 2 q ji H (Y / X = xi ), k = 1, M
j =1
i =1
Nu dk>0 th dung lng knh truyn c dng:
M
M
C = Log exp 2 q ji H (Y / X = xi )
i =1
j =1
Gi tr cc i t khi tn hiu vo X=X* tha phn phi P(X*=xk)=2-Cdk
Hay C=max I(X/Y)=I(X*/Y)
Ch :
- iu kin dk>0 cho php hm I(X/Y) l hm li => Tn ti Max tuyt i ti phn phi ca
X* vi p(X*=xk)=2-C dk =pk (vi mi k).
- Nu iu kin ma trn vung hoc ma trn ngch o khng tha th gi tr cc i max s
nm trn ng bin ca min xc nh {pk>0 v -pk=1}
Bi tp
1. Cho mt knh truyn c ma trn truyn tin nh sau:
x1 1 / 2 1 / 3 1 / 6
x 2 1 / 3 1 / 6 1 / 2
x3 1 / 6 1 / 2 1 / 3
y1 y 2
y3
Tnh dung lng knh truyn.
2. Chng minh cc cng thc tnh dung lng knh truyn trn.
52
BI 4.3: LC GII M
Mc tiu
- Bit t vn bi ton gii m,
- Hiu cc khi nim c bn ca k thut truyn tin,
- Bit v hiu cc dng sai s c bn ca k thut truyn tin,
- Hiu phng php xy dng lc gii m ti u,
- Vn dng xy dng lc gii m ti u v tnh cc dng xc sut truyn sai.
t vn bi ton gii m
Phn tch yu cu gii m:
Khi truyn gi tr xi, ta s nhn c yj.
i vi knh truyn khng nhiu th yj chnh l xi. i vi knh truyn c nhiu th yj c
th khc xi. Do ta cn tm cch gii m yj v gi tr xi khi knh truyn c nhiu.
Php phn hoch cc gi tr u nhn:
Php phn hoch tp cc gi tr u nhp yj Y l php phn chia tp Y thnh cc tp
con Bi sao cho:
Bi I B j =
1. M
( i j)
U Bi = Y
i =1
2. Khi nhn yj Bi th gii m v xi.
V d bi ton gii m
Cho tp cc t m truyn X v tp cc dy n bit nhn c Y nh sau:
X={0000, 0101, 1110, 1011}
Y={0000, 0001, 0010, 0011, 0100, 0101, 0110, 0111,
1000, 1001, 1010, 1011, 1100, 1101, 1110, 1111}
Gi s ta c th phn hoch tp Y thnh cc tp con Bi nh sau:
B1={0000, 1000, 0001, 0010}
B2={0101, 1101, 0100, 0111}
B3={1110, 0110, 1111, 1100}
B4={1011, 0011, 1010, 1001}
Gi s nhn yj = 0011 th gii m v x4 = 1011 v yj B4.
53
Cc khi nim c bn ca k thut truyn tin

Xt s truyn tin nh sau:
R
Ngun
Knh
B to m
k t
m
B gii m
nhiu
P(e)
X{x1, , xM}
Nhn
k t
gii m
Y{y1, , yL}
Din gii:
- Ngun pht tn hiu (hay thng bo) vi vn tc R (tn hiu/giy).
- Tn hiu c m ha t b k t m.
- Tn hiu m ha c truyn trn knh vi vn tc C (k t/giy), C ng thi l dung lng
ca knh truyn.
- Tn hiu truyn trn knh c th b nhiu vi xc sut P(e).
- Trc khi nhn, tn hiu m ha c gii m theo mt phng thc ti u v chnh xc
cao nht c th c.
Bi ton t ra y: tm gii php to m sao cho sai s u nhn c xc sut nh hn bt k
( < P(e)) ng thi vi ng b ha: vn tc pht thng bo ngun R v vn tc truyn ti C
(C l dung lng knh).
Cc khi nim c bn:
T m: l dy n k t truyn hay dy n k t nhn ng.
B m (S,n): l tp hp gm S t m vi di mi t m u bng n v c k hiu l x(1), ,
x(s).
Lc gii m: l mt hm gn cho mt dy n k t nhn c yj mt t m ca b m W =
{w1, w2, , ws}. K hiu: g(yj) = wi
Lc gii m ti u: l lc gii m sao cho tng xc sut truyn sai l nh nht hay tng
xc sut truyn ng l ln nht.
Ngha l: khi nhn yj th ta gii m v wi* sao cho:
P(wi*/yj) = Max{P(wk/yj)}
wk W
V d minh ha cc khi nim c bn

Gi s knh truyn tng bit vi C=1, ngun pht thng bo vi tc R=2/5 bit/giy (R<C).
thun li cho m ha v gim nhiu, ta xt tng khong thi gian n = 5 giy. Nh vy trong
khong thi gian n = 5 giy, ta c:
Tp hp cc tn hiu khc nhau l 2nR = 4. Gi s 4 tn hiu l m1, m2, m3, m4.

S bit c pht ra l nR=2 bit v mt tn hiu dng mi c kt cu bi mt dy
cc bit.
54
Qu trnh m ha cc tn hiu m1, m2, m3, m4 cn ch l: mi mi cn c m ha

vi s bit ti a l nC=5 bit. Vy, ta c th m ha cc tn hiu mi theo 2 cch sau:
Cch 1:
Cch 2:
m1=00000
m2=01101
m3=11010
m4=10111
m1=00
m2=01
m3=10
m4=11
Nu s dng cch 1 vi di 5 bit, trong 5 bit c th hiu l c 2 bit thng tin cn truyn v 3
bit con li l 3 bit c b sung pht hin nhiu theo mt phng php no s c cp
cc ni dung tip theo sau. Vi cch m ha ny, ta c nhiu kh nng pht hin v sa sai do
nhiu.
Nu s dng cch 2 th trng hp c 1 bit truyn sai s dn n trng lp sang mt trong cc tn
hiu khc. V d truyn m1=00 v nhn 2 bit l 01 (do nhiu), trong trng hp ny 01 chnh l
m2, y l mt tn hiu ng nn ta khng th pht hin c nhiu hay khng nhiu.
Nh vy, trong khong thi gian truyn v dung lng knh cho php, ta cn m ha mi tn hiu
cng di cng tt nhng khng c vt qu di m cho php. Trng hp vi thi gian n=5
v c= 1 bit th nC=5 l s bit ti a c th truyn nn ta ch m ha tn hiu vi di m ti a l
5 bit.
Cc dng sai s c bn
Xc sut truyn sai t m xi: p(e/xi)= p(Y=yj Bi/X=xi)
M
Xc sut truyn sai trung bnh: p(e) = p ( X = xi ) p (e / xi )

i =1
Xc sut truyn sai ln nht: p m (e) = Max p(e / xi )

i =1, M
Phng php xy dng lt gii m ti u

Theo cng thc Bayes:
Ta c: P(wk/yj) = [p(wk).p(yj/wk)] / p(yj) vi (wk W)
T nh ngha lc gii m ti u:
tm wk sao cho P(wk/yj) Max p(wk).p(yj/wk) Max.
Nh vy, ta c th xy dng lc gii m ti u theo cc bc sau:
Bc 0: Khi to cc Bi = (i)
Bc lp: xt vi mi yj Y
+ Tnh:
p(w1).p(yj/w1)
p(w2).p(yj/w2)
p(wM).p(yj/wM)
55

+ So snh cc gi tr tnh trn v chn gi tr w*i sao cho p(w*i).p(yj/w*i)= Max
{p(wk).p(yj/wk)} (wk W)
+ Bi = Bi + {yj} v g(yj) = w*i.
Minh ha xy dng lc gii m ti u

Bi ton:
Cho ma trn truyn tin A v xc sut u truyn nh sau:
x1 1 / 2 1 / 3 1 / 6
x 2 1 / 3 1 / 6 1 / 2
x3 1 / 6 1 / 2 1 / 3
y1 y 2
y3
Vi p(x1)=1/2; p(x2)=p(x3)=1/4. Hy xy dng lc gii m ti u.
p dng phng php xy dng lc gii m ti u:

Bc 0: B1={}; B2={}; B3={};
Bc 1: Nhn gi tr y1, ta tnh:
+ p(x1).p(y1/x1)= 1/2.1/2 = 1/4 (Max)
+ p(x2).p(y1/x2)= 1/4.1/3 = 1/12
+ p(x3).p(y1/x3)= 1/4.1/6 = 1/24
Do p(x1).p(y1/x1) ln nht nn lit k y1 vo tp hp B1 tng ng vi x1.
=> B1={y1}.
+ p(x1).p(y2/x1)= 1/2 . 1/3 = 1/6 (Max)
+ p(x2).p(y2/x2)= 1/4 . 1/6 = 1/24
+ p(x3).p(y2/x3)= 1/4 . 1/2 = 1/8
=> B1={y1, y2}.
+ p(x1).p(y3/x1)= 1/2 . 1/6 = 1/12
+ p(x2).p(y3/x2)= 1/4 . 1/2 = 1/8 (Max)
+ p(x3).p(y3/x3)= 1/4 . 1/3 = 1/12
=> B2={y3}.
56

Kt qu:
Phn hoch: B1={y1, y2}, B2={y3} v B3={}.
Lc gii m ti u:
Nhn
Gii m
x1
y1
Nhm B
1
x2
y2
Nhm B2
x3
y3
Minh ha cch tnh cc sai s
Xt li v d minh ha xy dng lc gii m ti u trn, ta c:
- Ma trn truyn tin A:
x1 1 / 2 1 / 3 1 / 6
x 2 1 / 3 1 / 6 1 / 2
x3 1 / 6 1 / 2 1 / 3
y1 y 2
y3
- Xc sut u truyn: p(x1)=1/2; p(x2)=p(x3)=1/4.
- Lc gii m ti u:
Nhn
y1
Nhm B
1
Gii m
x1
x2
y2
Nhm B2
x3
y3
- Phn hoch: B1={y1, y2}, B2={y3} v B3={}.
Tnh cc xc sut truyn sai:
Xc sut truyn sai mt t m:
Xc sut truyn sai t m x1: p(e/x1)= p(Y=yj B1/X=x1)
= p(y3/x1) =1/6
= p(y1/x2) + p(y2/x2) =1/3+1/6=1/2
= p(y1/x3) + p(y2/x3) + p(y3/x3) =1/6+1/3+1/2=1
M
Xc sut truyn sai trung bnh: p(e) = p ( X = xi ) p (e / xi )

i =1
p(e)=p(x1).p(e/x1) + p(x2).p(e/x2) + p(x3).p(e/x3) = 1/2.1/6 + 1/4.1/2 + 1/4.1 = 11/24

Xc sut truyn sai ln nht: p m (e) = Max p(e / xi )
i =1, M
pm(e) = Max{ p(e/x1), p(e/x2), p(e/x3)} = p(e/x3) =1
57
Bi tp 1
1. Cho ma trn truyn tin sau:
x1 1 / 2 1 / 3 1 / 6
x2 1 / 3 1 / 6 1 / 2
x3 1 / 6 1 / 2 1 / 3
y1 y2 y3
Bit xc sut u truyn: p(x1)=5/10, p(x2)=3/10, p(x3)=2/10.
- Tnh dung lng knh truyn.
- Xy dng lc gii m ti u.
- Tnh cc sai s p(e) v pm(e).
x1 7 / 12 3 / 12 2 / 12
x2 2 / 12 7 / 12 3 / 12
x3 3 / 12 2 / 12 7 / 12
y2
y3
y1
Bit xc sut u truyn: p(x1)=1/3, p(x2)=1/3, p(x3)=1/3.
- Xy dng lc gii m ti u
- Tm cc sai s p(e) v pm(e).
Bi Tp 2
x1 1 2 13 1 6
1
1
x2 1
2
3
6
1
1
x3 1
6
2
3
y1 y2 y3
Bit p(x1)=1/2, p(x2)=1/4, p(x3)=1/4.
x1 7 / 10 2 / 10 1 / 10
x2 1 / 10 7 / 10 2 / 10
x3 2 / 10 1 / 10 7 / 10
y2
y3
y1
Bit xc sut truyn p(x1)=0.4, p(x2)=0.4, p(x3)=0.2.
58
CHNG 5: SA LI
Mc tiu: Xy dng nguyn tc sa li da vo khong cch Hamming. Trn nguyn tc ny,
phng php sa li kim tra chn l (parity check) c xy dng v to ra quy trnh sa li
ti u v ph hp vi cng ngh truyn tin hin nay.
BI 5.1: NGUYN L KHONG CCH NH NHT

HAMMING
Mc tiu:
Sau khi hon tt bi hc ny bn c th hiu:
- nh ngha khong cch Hamming
- Knh truyn i xng nh phn v lc gii m ti u
- Quan h gia xc sut gii m v khong cch Hamming
- Nguyn l khong cch nh nht ca Hamming.
Khong cch Hamming

nh ngha: cho v1 v v2 l 2 dy nh phn di n bit, ta gi khong cch Hamming gia 2 dy v1,
v2 l s bit tng ng khc nhau. K hiu: d(v1, v2).
V d:
v1=10101010
v2=10101111
Ta nhn thy rng bit th 6 v bit th 8 gia gia v1 v v2 l khc nhau nn s bit tng ng khc
nhau gia v1 v v2 l 2. Do , ta ni khong cch Hamming gia v1 v v2 l 2 hay d(v1, v2) = 2
Knh truyn i xng nh phn v lc gii m ti u

Xt knh truyn i xng nh phn. Gi s ta truyn cc dy t m nh phn c di n bits vi
xc sut truyn sai 1 bit l .
1-
0
0
1-
Gi W = {w1, w2,,ws} l tp s t m truyn, di mi t m u bng n bit.

V = {v1, v2,., v2n} l tp cc dy n bit nhn c cui knh vi W c phn phi u, xc
sut nhn vj khi truyn wi l p(vj/wi) = pij.
Theo lc gii m ti u ta c: khi nhn vj th gii m v wi* sao cho:
P(wi*/vj) = Max{P(wk/vj)}
(wi W)
Ta c: P(wk/yj) = [p(wk).p(yj/wk)] / p(yj) vi (wk W)
P(wk/yj) Max p(wk).p(yj/wk) Max.
59

Do W c phn phi u nn P(wk/yj) Max p(yj/wk) Max
Vy: tm wi* sao cho P(wi*/vj) = Max{P(wk/vj)} ta ch cn tm wi* sao cho
P(vj/ wi*) = Max{P(vj/ wk)} (ch da vo ma trn truyn tin A)
V d knh truyn i xng nh phn

Xt ma trn truyn tin A v xc sut u truyn nh sau:
w1 1 / 2 1 / 3 1 / 6
w2 1 / 3 1 / 6 1 / 2
v p(w1) = p(w2) = p(w3) = 1/3.
A=
w3 1 / 6 1 / 2 1 / 3
v1 v 2
v3
da vo lc gii m ti u ta c:
Nhn v1 gii m v w1
Nhn v2 gii m v w3
Nhn v3 gii m v w2.
Quan h gia xc sut gii m v khong cch Hamming

Gi s nhn c v:
Xt 2 t m w1 v w2 cn chn gii m cho v.
+ Gi d1=d(v, w1), d2=d(v,w2).
d
nd
+ Ta c:
p(v/w1)= 1 (1 ) 1 (xc sut nhn v khi truyn w1).
P(v/w2)=
d2
(1 )
nd2
d1
(1
(xc sut nhn v khi truyn w2).

nd
d d
p(v / w1 )
) 1 1 2 1
= d
=
So snh xc sut:
p(v / w2 ) 2 (1 ) n d 2
1
>1
Nu nhiu 0 < < th
Do : P(v/w1)>p(v/w2) d1 <d2
Nhn xt: xc sut gii m cng ln th khong cch Hamming cng nh.
Nguyn l Hamming
nh l: trn knh truyn i xng nh phn vi s t m u truyn c di n bit, lc gii
m ti u c th thay th bng lc gii m theo khong cch Hamming vi nguyn l: nu
nhn c v, ta s gii ra w*i
sao cho d(v,w*i)=Min d(v,wk) (vi wk W).
V d:
xt b m W={w1=00000, w2=10011, w3=11100, w4=01111}
Gi s nhn c dy v=01011.
ta c: d(v,w1)=3; d(v,w2)=2; d(v,w3)=4; d(v,w4)=1.
vy v c gii v w4 v khong cch Hamming gia v v w4 l nh nht.
60
Bi tp
1. Cho b m W={w1=000000, w2=101010, w3=111000, w4=111111} v nhn c dy
v=010111, khi gii m v t m no? din gii?
2. Cho b m W={w1=000000, w2=010101, w3=000111, w4=111111} v Nhn c dy
v=010111, khi gii m v t m no? din gii?
61
BI 5.2: B V T SA LI V CN HAMMING
Mc tiu
- Bit c B v t sa li,
- Hiu nh l v cn Hamming,
- Bit phn loi c cc dng li,
- Lm c s l thuyt cho cc phng php sa li c trnh bi trong cc bi hc tip
theo.
B v t sa li
t vn : mt t m w di n bit khi c truyn tun t tng bit c th sai e bit. Vn t ra
l khong cch (Hamming) gia cc t m v sai s e quan h vi nhau nh th no c th
phn bit tt nht ng thi tt c cc t m? B sau xc nh quan h ny.
B :
Xt b m W={w1, w2, , ws} gm c s t m nh phn di n bit v 1 s nguyn dng e.
1. Nu d(wi, wj) 2e+1 (vi ij )
Khi : tt c cc dy nhn c v c s bit li e th v c th t iu chnh (hay t sa li).
2. Nu d(wi, wj) 2e (vi ij )
Khi : tt c cc dy nhn c v c s bit li < e th v c th t iu chnh. Tt c cc dy
nhn c c s bit li = e th ta ch pht hin l v c li v khng th t iu chnh c.
3. Ngc li;
Nu v c s ch s bit li e v c th t iu chnh th d(wi, wj) 2e+1 (vi ij ).
Nu v c s ch s bit li e-1 t iu chnh c v tt c cc tn hiu vi s ch s bit li
e c pht hin th khong cch gia cc t m lun tha: d(wi,wj) 2e (vi ij ).
Chng minh v minh ha b

a. Gi s: d(w, w) 2e+1 vi ij . Nu w v w c cng khong cch i vi dy v th
d(v,w)=d(v,w) e+1. Vy , nu d(v, w*) e th v c th c gii m ra w*.
b. Nu d(wi,wj) 2e vi ij, c kh nng c v, w v w vi s ch s li l:
d(v,w)=d(v,w)=e (d(v,w)+ d(v,w) d(w,w) 2e). C th pht hin ra cc t m gn v,
nhng do tn ti cng lc nhiu t m gn nht vi v dn n khng gii m c, ngc li
hon ton tng t.
62

Minh ha:
a. d(wi, wj)= 2e+1= 7, e=3
Nu vBi th v c gii m v wi
Nu vBj th v c gii m v wj
wi *
* wj
v
b. d(wi, wj) = 2e = 8 (e = 4, e - 1=3)

nu vBi , vBj => cc im cch tm khong cch 3 th lun c gii m, cn cc
im cch tm 4 th ch pht hin li ch khng th gii m c.
c. M 3 chiu (x, y, z) bt u t gc 000. C mt tn hiu thay i th m b y i theo 1 cnh,
chng hn:
000 cch 010, 001 bi 1 cnh,
011 cch 010, 111 v 001 bi 1 cnh.
Nh vy, nu ta chn w1=010, w2=001, w3=111 th khong cch gia chng l 2
d(w1, w2)=d(w1, w3)=d(w2, w3)=2
vy nu c li pht sinh th ch pht hin ch khng sa c.

y
x
110 w3=111
w1=010
100
000
w2=001
101
z
Cn Hamming.
t vn : trong tng s 2n dy nh nhn di n bit c th chn ra bao nhiu dy to thnh mt
b m c th t iu chnh c e bit li. nh l cn Hamming cho chng ta xc nh s t m
c di n bit vi gi thit: c kh nng t sa c e bit li (iu kin cn t sa li).
nh l: Nu b m W c s t m c di n bit c th t sa c e bit li th
2n
s e
C ni
i =1
Ghi ch: Cni = n!/(i!*(n-i)!)
63

Chng minh:
Xt t m nh phn wi c di n bit v c kh nng t sa c e bit li.

e
S dy vj sai khc vi wi t 0 n e bit l : C n0 + C n1 + C n2 + ... + C ne = C ni

i =0
Tng ng vi s t m, tng s dy vj c th t sa li l : s. C ni 2 n
(2n l tng s dy nh phn di n bits).
=> s
i =0
2n
e
C
i =1
i
n
Phn cc dng li
Gi s ta truyn t m n bit wi W ( 1 i s) v nhn c dy n bit vj ( 1 j 2n).
Cc loi li c th pht hin sau:
Li c th t iu chnh:
Trong trng hp ny tn ti duy nht t m w*i sao cho d(vj, w*i)= Min d(vj, wk) vi wk W.
=> vj c gii m v w*i
Li ch pht hin khng iu chnh c:
Trong trng hp ny tn ti t m w*i v w**i sao cho
d(vj, w*i)= d(vj, w**i)=Min d(vj, wk) vi wk W
=> vj khng th gii m chnh xc.
Li khng pht hin c.
Trong trng hp ta gii m ra w*i nhng khc vi wi truyn.
Bi tp
1. Cho n=7 v e=2, hy p dng nh l cn Hamming cho bit s t m ti a ca b m W.
2. Cho n=7 v e=2, hy p dng nh l cn Hamming cho bit s t m ti a ca b m W.
3. Hy cho mt v d c th minh ha cc trng hp phn loi li.
BI 5.3: M KIM TRA CHN L

Mc tiu:
- Hiu b m kim tra chn l,
- Hiu phng php kim tra chn l,
64

- Bit tnh cht c bn ca phng php kim tra chn l,
- Hiu v vn dng tt phng php sinh m kim tra chn l,
- Hiu v vn dng tt nh l quan h gia di m n, s bit kim tra m v s li t sa
e,
- Vn dng cho cc bi hc tip theo.
B m kim tra chn l

B m kim tra chn l l b m gm s t m, trong mi t m c dng sau:
w=r1r2r3rm rm+1rm+2rm+k (vi n = m+k).
m bit kim tra
k bit thng tin
Ghi ch: trong mt s trng hp sinh m theo phng php kim tra chn l, th t cc bit kim
tra v cc bit thng tin c th xen k nhau (theo mt th t no , chng hn nh m
Hamming,) hay cng c th theo mt th t khc (theo quy c khc). y, ta chn th t
cc bit kim tra chn l v cc bit thng tin nh trn d tnh ton nhng vn mt tnh tng qut
ha.
Trong : w vit theo dong l chuyn v ca w (w c vit theo ct)

+ ri: l bit th i ca t m ( 1 i n).
+ n: di ca t m hay s bit ca t m chn l.
+ m: s bit kim tra.
+ k = n-m: s bit thng tin s=2k (v vi k bit thng tin th ta ch c th biu din ti a
2k trng thi thng tin k bit).
+ on kim tra: gm m bit dng kim tra m sai.
+ on thng tin: gm k bit thng tin.
Mi on m thng tin c duy nht mt on m kim tra v c xc nh bi h phng trnh
tuyn tnh nh phn sau:
a11 r1 + a12 r2 + ... + a1n rn = 0
a r + a r + ... + a r = 0
21 1
22 2
2n n
...
...
... ...
...
a n1 r1 + a n 2 r2 + ... + a nn rn = 0
Gi A=||aij|| =Am x n , aij {0,1}, i= 1, m , j= 1, n . Ma trn A c gi l ma trn kim tra chn l c
hng l m (hay Rank(A) = m).
Cc php ton trong Modulo 2 (+,-):
0 + 1 = 1 + 0 = 1; 0 1 = 1 0 = 1;
1 + 1 = 1 1 = 0;
Phng php kim tra chn l

Gi w=r1r2rn l t m truyn (hay dy n bit truyn) v v=r1r2rn l dy n bit nhn c.
Qui c: v, w (ln lt l chuyn v ca v v w) c vit theo dng. Cn v, w c vit theo
ct.
Nu A.v = 0 th v = w, ta gi v l chn (trng hp nhn ng)

65

Nu A.v 0 th v w, ta gi v l l (trng hp nhn sai).
Ta gi z = v-w l b li gia v v w. Ngha l ti cc v tr z = {0} th bit nhn c tng ng l
bit ng v ti cc v tr z = {1} th bit nhn c tng ng l bit sai (hay bit li).
Ta gi C = A.v l b sa li (hay b iu chnh li).
Ta c C = A.z = A.(v-w) = A.v-A.w = A.v C = A.v = A.z
Tnh cht ca b sa li: dy n bit nhn c v v b li tng ng c cng b iu chnh.
Phng php sinh m kim tra chn l

Gi s: cho trc ma trn kim tra chn l A vi Rank(A) = m.
Tm b m chn l W={w1, w2, w3,,ws}
Bc 0: Xc nh cc gi tr n, m, k, s
di ca t m n= s ct ca ma trn A.
S bit kim tra m= s dng ca ma trn A.
S bit thng tin: k = n-m.
S t m s=2k ca b m.
Bc i: Tm cc t m th i (1 i s):
Gi kpi l trin khai nh phn k bit ca s i
T m cn tm l: wi=r1r2..rmkpi
Gii h phng trnh A.wi=0 tm m bit kim tra ng vi k bit thng tin (kpi) bit
=> t m wi
V d sinh m kim tra chn l

Xy dng b m kim tra chn l c sinh t ma trn kim tra A nh sau:
1 0 0 1 1 0
A= 0 1 1 1 0 1 Rank(A) = 3
1 0 1 1 0 1
Bc 0:
n=6 (= s dng ca ma trn A)
m=3 (= s ct ca ma trn A)
S bit thng tin k = n m = 3 => S t m s=2k=8 t m.
66

Bc i: Tm t m th i (1 i s):
w1=r1r2r3000 (000 l trin khai nh phn k=3 bits ca s i=0)
Gii h phng trnh A.w1=0

1 0 0 1 1 0
0 1 1 1 0 1
1 0 1 1 0 1
r1
r
2 0
r1 = 0
r1 = 0
r3
= 0 => r2 + r3 = 1 => r2 = 0 => w1=001001

0 0
r + r = 1
r = 1
1 3
3
0

1
Gii h phng trnh A.w2=0
r1
r
2
r1 = 1
r1 = 1
1 0 0 1 1 0 0
0 1 1 1 0 1 r3 = 0 => r + r = 0 => r = 1 =>w =111010
2
2
2 3
0
1 0 1 1 0 1
r3 = 1
r1 + r3 = 0
1

0
Gii tng t cho cc trng hp cn li ta c:
w0=000000, w3=110011, w4=110100,
w5=111101, w6=001110, w7=000111.
W={000000, 001001, 111010, 110011, 110100, 111101, 001110, 000111}
nh l quan h gia di m n, s bit kim tra m v s li t sa e

iu kin cn (Cn Hamming):
iu kin cn b m chn l c di n bit c th t sa c e bit li vi k bit thng tin v m
bit kim tra l:
e
2 m C ni
i =0
iu kin ( K Vasharmov-Gilbert-Sacks):
iu kin b m kim tra chn l c di n bit vi m bit kim tra chn l c th t sa
c e bit li l:
2m >
i
2 e 1
C
i =0
i
n 1
Ghi ch: Cn = n!/(i!*(n-i)!)
67
V d tm m nh nht t n v e
Gi s bit trc n=7 v e=1. Tm s bit kim tra ti thiu cn thit ca b m chn l.
Theo nh l iu kin cn (Cn Hamming):
e
Ta c: 2 m C ni
i =0
e =1
2 m C 7i (*)
i =0
m = 1 (*) sai.
m = 2 (*) sai.
m 3 (*) ng.
Vy s bit kim tra ti thiu cn thit l m = 3.
V d tm e ln nht t m v n
Gi s cho trc m=3, k=2. Tm s bit li ln nht c th t sa e?
Theo nh l iu kin (K Vassharmov-Gilbert-Sacks):
2m
2 e 1
2 e 1
i =0
i =0
C ni 1 2 3
i
51
(*)
e =1 (*) ng.
e > 1 (*) sai.
Vy s bit li ln nht c th t sa l e = 1.
Bi tp
1. Xy dng b m kim tra chn l c sinh t ma trn kim tra A nh sau:
1 0 0 1 1 1
A = 0 1 1 1 0 1
1 0 1 1 0 1
2. Tm b m kim tra chn l c sinh t ma trn kim tra A nh sau:
1 0 0 1 1 1
A = 0 1 0 1 0 1
1 0 1 1 0 1
Gi gii bi tp 1 & 2: da vo phng php sinh m kim tra chn l v tham kho v
d sinh m kim tra chn l.
3. Xt b m kim tra chn l di 15 bit c th t sa c 1 bit li trn ng truyn, hy
cho bit s bit kim tra chn l ti thiu?
4. Xt b m kim tra chn l di 8 bit vi 4 bit kim tra chn l. Hy cho bit s li t sa
ti a ca b m?
Gi gii bi tp 3 & 4: da vo inh l iu kin cn (Cn Hamming) v iu kin
(K Varshamov-Gilbert-Sacks).
68
BI 5.4: NHM CNG TNH V B T M CHN L

Mc tiu.
Hiu Khi nim nhm cng tnh,
Bit cc tnh cht ca b m chn l,
Vn dng sinh ma trn kim tra chn l t b m kim tra chn l.
Vn dng tt phng php sinh b m kim tra chn l t cc t m c lp tuyn
tnh ca b m.
Khi nim nhm cng tnh.

t vn :
Nh chng ta bit, phng php sinh m kim tra chn l gip ta sinh b m kim tra chn l
vi s t m tng ng l s=2k. Vi phng php ny, ta phi xc nh tng t m mt (bng
cch gii h phng trnh tuyn tnh nh phn). Gi s: k=5 ta phi xc nh s=25 =32 t m hay
k=10 ta phi xc nh s=210=1024 t m,iu ny s mt nhiu thi gian nu k cng ln. Vn
t ra y l tm ra mt phng php sinh b m kim tra chn l nhanh hn v mt thi
gian. Phng php sinh m kim tra chn l da theo l thuyt nhm s gii quyt vn ny.
Khi nim nhm cng tnh:
Nhm G c gi l mt nhm cng tnh nu G c cc tnh cht:
- a, b G a+b G ( tnh cht cng).
- a, b, c G a + (b + c)= (a + b) + c ( tnh cht kt hp).
- G sao cho + a = a + = a, a G ( l Identity Element ca G).
- a G -aG : a + (-a)=
Nhm G l nhm hon v (nhm Aben) nu a,b G=> a + b = b + a.
V d:
- Tp hp cc s nguyn vi php + thng thng l nhm Aben.
- Tp hp cc s nh phn c di n bit cng vi php + trong Modulo 2 to thnh nhm
Aben.
Tnh cht ca b m chn l

Tnh tng ng ca b m nhm cng tnh v b t m kim tra chn l c th hin qua 2
nh l sau:
nh l 1: tp hp cc t m trong b m kim tra chn l l mt nhm cng tnh.
( ngh sinh vin chng minh nh l ny da vo cc tnh cht ca nhm cng tnh)
nh l 2: Nu tp hp W l tp cc dy nh phn vi di cc dy cng bng n v W l mt
nhm Aben vi php cng Modulo 2 th W c th xem nh mt b m kim tra chn l c sinh
ra t ma trn A c dng nh sau:
69
b11
b
A = I m 21
...
bm1
b12
b22
...
bm 2
... b1k
... b2 k
... ...
... bmk
Trong :
- Ma trn A c m dng v n ct.
- Im : l ma trn n v cp m.
- k: l s dy nh phn (hay t m) c lp tuyn tnh ln nht.
- n: l di ca t m v m = n-k:
- bij: c xc nh bng cch da vo h phng trnh tuyn tnh (*) v k t m c lp
tuyn tnh nh sau:
wi=r1r2r3rm rm+1rm+2rn. (i = 1, k )
on kim tra on thng tin
r1 = b11rm +1 + ... + b1k rm + k
(*)....................................
r = b r + ... + b r
m1 m +1
mk m + k
m
Th k t m c lp tuyn tnh vo h pt (*) tm cc bij ma trn A.
V d minh ha
Xt tp hp M gm c 8 dy nh phn di 6 bits nh sau:
r1 r2 r3 r4 r5 r6
w0 = 0
w1 = 1
w2 = 1
w3 = 0
w4 = 0
w5 = 1
w6 = 1
w7 = 0
0
0
1
1
1
1
0
0
0
1
0
0
1
1
0
1
0
0
0
1
0
1
1
1
0
0
1
0
1
0
1
1
0
1
0
1
1 (w1+w2)
0 (w1+w3)
1 (w2+w3)
0 (w1+w2+w3)
Ta thy {w1, w2, w3} l tp hp ln nht cc t m c lp tuyn tnh t tp hp M:

w1 = 1 0 1 0 0 1
w2 = 1 1 0 0 1 0
w3 = 0 1 0 1 0 1
n=6 v k=3. => m = n k = 3.
70

Nh vy: ma trn kim tra chn l c dng nh sau:
b11 b12 b13
b21 b22 b23
A = I3
b31 b32 b33
Cc bij ( i, i = 1,3 ) c xc nh t h phng trnh tuyn tnh nh phn sau:

1
0
1
0

+
+
1
0
1
=
b
b
b
11
12
13 0
0
0

1
0
0
1
0

=> 1 = b21 0 + b22 1 + b23 0
1
0
1
1

1
0
0
1

0 = b31 0 + b32 1 + b33 0
1
0
1
0

b11 = 1 b12 = 1 b13 = 1
=>
b21 = 1 b22 = 1 b23 = 0
b31 = 1 b32 = 0 b33 = 1
1 0 0 1 1 1
=> A= 0 1 0 1 1 0
0 0 1 1 0 1
Vy ta c th s dng nhm M nh l mt b m kim tra chn l.
Phng php sinh m kim tra chn l nhanh

Bc khi to: xc nh cc gi tr n, m, k, s.
Bc 1: sinh k t m c lp tuyn tnh (ltt).
Bc 2: cng t hp cc t m:
+ Cng cc t hp ca 2 t m t k m ltt => c C k2 t m.
---+ Cng cc t hp ca k t m t k t m ltt => c C kk t m.
Bc 3: Cng s-1 t m tm c tm t m cui cng => C k0 = 1 t m.
k
Tng s t m s= C ki = 2 k t m.
i =0
V d sinh m kim tra chn l nhanh

1 0 0 1 1 0
Tm b m nhm khi bit trc ma trn kim tra A = 0 1 1 1 0 1
1 0 1 1 0 1
k
Bc khi to: n = 6, m = 3, k = 3, s = 2 = 8.
71

Bc 1: Sinh k = 3 t c lp truyn tnh: w1=001001, w2=111010, w3=110100
Bc 2: Cng t hp cc t m.
+ Cng cc t hp 2 t m ltt:
w4=w1+w2=110011
w5=w1+w3=111101
w6=w2+w3=001110
+ Cng cc t hp 3 t m ltt:
w7=w1+w2+w3=001111
Bc 3: xc nh t m cui cng:
w0=w1+w2+w3+w4+w5+w6+w7=000000
Bi tp
1. S dng phng php sinh m nhanh cho b m t ma trn kim tra A nh sau:
1 0 0 1 1 1
A = 0 1 1 1 0 1
1 0 1 1 0 1
2. S dng phng php sinh m nhanh cho b m t ma trn kim tra A trong cc trng hp
sau:
0
A=
0
0
0
1
0
0
1
0
1
1
0
1
0
0
1
0
0
1
0
0
0
; A=
1
0
0
1
0
1
1
1
1
1
1
1
1
1
1
0
1
1
0
0
1
1
0
0
; A=
0
0
1
0
0
1
0
0
0
0
1
0
0
0
0
1
1
1
1
0
0
1
72
BI 5.5: LC SA LI TI U
Mc tiu
- Bit c vn ca bi ton,
- Hiu nh ngha Hip hp,
- Vn dng xy dng lc sa li theo cc hip hp,
- Vn dng xy dng lc sa li thng qua b sa li,
- Vn dng tnh Xc sut truyn ng cho lc sa li,
- Kin thc t c s l c s cc bn c th ng dng cho vic thit k mt h, thng
m ha, gii m v bo mt thng tin.
t vn
Trong mt h thng lin lc truyn tin, bn cnh cc yu cu thit b (nh ngun pht, b m ha,
knh truyn, b gii m,) m bo tt cho vic truyn v nhn d liu th cn c cc kha cnh
khc nh phng php m ha v gii m sao cho ti u l phn rt quan trng trong h thng.
Vn lun c t ra y l lm th no ch ra mt phng php gii m ti u, c ngha
l h thng phi c kh nng pht hin v sa li mt cch chnh xc nht c th c khi nhiu xy
ra. y chnh l vn chnh c tho lun trong sut bi hc ny.
nh ngha Hip hp
Gi W={w1, w2, ,ws} l b m kim tra chn l.
V ={v1, v2, , v 2 n } l tp hp cc dy n bit c th nhn c cui knh.
Ta gi mt hip hp ca W trong V l tp hp c dng z + W (z l b li)
V d: Cho ma trn kim tra chn l sau:
1 0 1 0
A=
0 1 1 1
T A, ta c th xy dng c b m tng ng sau: W={w0=0000, w1=0101, w2=1110,

w3=1011}.
Ta c th thy rng, cc b li mt bit khc nhau c th c l z={1000, 0100, 0010, 0001}. Do
cc hip hp ng vi cc b li 1 bit s l:
w0
w1
w2
w3
0000 0101 1110 1011
Hip hp 1
Hip hp 2
Hip hp 3
Hip hp 4
1000
0100
0010
0001
1101
0001
0111
0100
0110
1010
1100
1111
0011
1111
1001
1010
(vi z1=1000)
(vi z2=0100)
(vi z3=0010)
(vi z4=0001)
Trong : hip hp i = wi + zi, cc bn c th xt thm cc b li sai 2 bit, 3 bit, c cc

hip hp ng vi cc b li sai 2 bit, 3bit,.
73
Lc sa li theo cc hip hp
Bc 1: Lp bng cc hip hp ng vi cc b li cn thit
- Dng u tin vit cc t m wi W.
- Cc dng tip theo ng vi ct w0 = 0000 vit cc b li z (cc b li 1 bit, 2 bit,).
- Cc dng ct th i c xc nh bi z + wi
Bc 2: Qu trnh gii m
Gii m: khi nhn v, ta xc nh ct th i cha v v gii m v wi tng ng.
V d: xy dng lc sa li theo cc hip hp cho b m c sinh t ma trn kim tra chn
l sau:
1 0 1 0
A=
0 1 1 1
T A, ta c th xy dng c b m tng ng sau: W={w0=0000, w1=0101, w2=1110,

w3=1011}.
Bc 1: Lp bng cc hip hp ng vi cc b li cn thit:
Ta xy dng cc hip hp ng vi cc b li sai 1 bit. Vy z={1000, 0100, 0010, 0001}.
w0
w1
w2
w3
0000 0101 1110 1011
Hip hp 1
Hip hp 2
Hip hp 3
Hip hp 4
1000 1101 0110

0100 0001 1010
0010 0111 1100
0001 0100 1111
(Bng cc hip hp)
Bc 2: Qu trnh gii m:
Gi s nhn v = 0111. Tra tm v trn
gii m v w1 = 0101.
0011
1111
1001
1010
(vi z1=1000)
(vi z2=0100)
(vi z3=0010)
(vi z4=0001)
bng cc Hip hp ta c v ct 1. Do , v c
Gi s nhn v = 1010. Tra tm v trn bng cc Hip hp ta c v ct 2 hay ct 3. Do , v c

gii m v w2 hay w3, trong trng hp ny gii m khng chnh xc. ngh cc bn lu v
cho kin ca bn v cc trng hp gii m khng chnh xc ny.
Lc sa li thong qua b li
xy dng lc sa li thng qua b sa li, ta da vo tnh cht ca b sa li. Nh vy ta
c th thy lc gii m gm 2 bc sau:
Bc 1: Lp bng sa li: B li (Z) B sa li (C=A*Z).
Bc 2: Qu trnh sa li
- Khi nhn c dy n bit v V, ta xc nh b iu li C cho v vi C=A.v
- Tra bng sa li tm b li z0 ng vi C.
- Gii m w=v+z0.
V d minh ha lc sa li 1 bit
1 0 0 0 1 1
0 1 0 0 1 0
Xt b m c sinh t ma trn A =
0 0 1 0 1 1
0 0 0 1 0 1
74
B m tng ng c xc nh l: w1=000000, w2=101101, w3=111010, w4=010111

( ngh cc bn tham kho phng php sinh m chn l v xy dng li b m t ma trn kim
tra chn l A).
Lc sa li:
Bc 1: Lp bng sa li: B li- B iu chnh (e = 1)
B li (z)
B iu chnh (C=A.z)
B 0 li
B li 1 bit
000000
100000
010000
001000
000100
000010
000001
0000
1000
0100
0010
0001
1110
1011
1 B
6 B
Bc 2: Qu trnh sa li
- Gi s nhn v=001101, tnh C = A.v = 1000
- Tra bng sa li tm b li z0 ng vi C, ta c z0 = 100000
Gii m w = v + z0 = 001101 + 100000 = 101101 = w2
-
Lc sa li:
B li (z)
B iu chnh (C=A.z)
B li 2 bit
110000
101000
100100
100010
100001
011000
010100
1100
1010
1001
0110
0011
0110
0101
7 B
(Tt c cc b 2 li cn li c trng b iu chnh vi cc b trn)

Bc 2: Quy trnh sa li
- Gi s nhn v=100111, tnh C = A.v = 1100
- Tra bng sa li tm b li z0 ng vi C, ta c z0 = 110000
- Gii m w = v + z0 = 100111 + 110000 = 010111 = w4
75
Lc sa li:
z
B li 3 bit
C=A.z
110100 1101
110001 0111
2 B
(Tt c cc b 3 li cn li c trng b iu chnh vi cc b trn)

Bc 2: Quy trnh sa li
Gi s nhn v=011001, tnh C = A.v = 1101
Tra bng sa li tm b li z0 ng vi C, ta c z0 = 110100
Gii m w=v + z0 = 011001 + 110100 = 101101 = w2
Ch :
Tng s b iu chnh = 2m. Trong mt s trng hp, b m chn l ch cho php pht hin li
trn ng truyn v khng th gii m chnh xc do tng s b iu chnh = 2m v s b li c
th ln hn nhiu (so vi tng s b iu chnh).
Xc sut truyn ng
Gi Ni l s b li ng vi i li c th t sa, khi xc sut truyn ng v t iu chnh s l:
n
P (e' ) = Ni. i.(1 i ) n i

i =0
Vi n l di t m
V d: xt trng hp cc v d trn vi n= 6 v t sa e = 3 bit li. p dng cng thc trn ta c:
3
P (e' ) = Ni. i.(1 i ) n i = (1 ) 6 + 6 (1 ) 5 + 7 2 (1 ) 4 + 2 3 (1 ) 3

i =0
Bi tp
1. Cho ma trn kim tra chn l sau:
1 0 0 1 1 1
A = 0 1 1 1 0 1
1 0 1 1 0 1
-
Xy dng b m kim tra chn l.

Minh ha quy trnh sa li 1 bit.
2. T kt qu ca bi tp 1, hy minh ha lc sa li thng qua b iu chnh trong cc

trng hp li 1 bit, 2 bit. Tnh xc sut truyn ng cho cc trng hp c th t sa c.
BI 5.6: M HAMMING
Mc tiu
76

-
Hiu M Hamming,
Hiu tnh cht ca m Hamming.
M Hammin
M Hamming l mt dng m nhm (m kim tra chn l) c xc nh t ma trn kim tra chn
l A c dng sau:
- Ct th j ca ma trn A l biu din nh phn m bit (m l s bit kim tra chn l) ca
s j theo qui c biu din nh phn ca s j c vit theo th t t di ln trn (vit
theo ct), tng ng vi vit t tri sang phi (vit theo dng).
- Cc bit v tr 2i ( i = 0, 1, 2, ) c chn lm bit kim tra.
V d 1: biu din nh phn ca s j = 3 c m = 3 bit nh sau:
Vit theo dng: 011 (vit t tri sang phi)
1

Vit theo ct: 1 (vit t di ln)
0

V d 2: ma trn kim tra chn l vi n=6, m=3 c th vit nh sau:
1 0 1 0 1 0
A = 0 1 1 0 0 1
0 0 0 1 1 1
T m Hamming c dng: w=r1r2r3r4r5r6. Trong , r1r2r4 l cc bit kim tra v r3r5r6 l cc bit
thng tin (v cc bit v tr 2i (vi i = 0, 1, 2, ) c chn lm bits kim tra).
Tnh cht
Nu cho trc s bit (m) v s bit li t sa (e) th s bit ti a ca b m Hamming (n) c th
c c lng t bt ng thc sau:
e
2 m C ni
i =o
V d minh ha
Tm b m Hamming vi n = 6 v m =3
Ta c th vit ngay ma trn kim tra chn l cho b m Hamming
1 0 1 0 1 0
A = 0 1 1 0 0 1
0 0 0 1 1 1
T A k = n m = 3.
Cc bit cc v tr 1, 2, 4 c chn lm cc bit kim tra.
=> s t m ca b m Hamming l s = 2k = 8
Tm k t m c lp tuyn tnh c dng:
w1=r1r20r401
w2=r1r20r410
w3=r1r21r400
Gii cc h phng trnh: A.w1=0, A.w2=0, A.w3=0
Cc t m cn li c xc nh theo phng php sinh m nhanh.
77

Ghi ch: Kt qu chi tit xy dng bng m Hamming dnh cho sinh vin t lm.
Bi tp
1. Vit ma trn kim tra chn l cho b m Hamming vi n = 15.
2. T kt qu bi tp 1, hy tm cc t m Hamming c lp tuyn tnh tng ng.
3. Xt b m Hamming vi s bit kim tra cho trc l m, khi :
- di m ti thiu l bao nhiu?
- di m ti a l bao nhiu?
78
BI 5.7: THANH GHI LI TNG BC

Mc tiu
Sau khi hon tt bi hc ny bn c th bit:
- t vn v thanh ghi li tng bc,
- Cch biu din vt l ca thanh ghi,
- Cch biu din ton hc ca thanh ghi,
- Tm chu k ca thanh ghi.
t vn
Nh chng ta bit, phng php sinh b m kim tra chn l da trn l thuyt nhm cho php
chng ta sinh m nhanh bng cch ch sinh ra k t m c lp tuyn tnh trong tng s s=2k t m,
t k t m ny ta c th xc nh cc t m cn li (bng cch cng t hp cc t m). Vn t
ra y l lm sao tm ra mt phng php sinh m khc sao cho s t m sinh ban u nh
hn k (k l s t m c lp tuyn tnh ca b m kim tra chn l) v t y ta c th xc nh
nhanh cc t m cn. C th da trn m hnh ca thanh ghi li tng bc c th gii quyt c
vn ny.
Biu din vt l ca thanh ghi

gi mt cch ngn gn, ta qui c gi thanh ghi thay v goi thanh ghi li tng bc. Biu din
vt l ca thanh ghi c th thy nh hnh v di y:
+
Fm-1
Fm-2
F1
F0
am-1
am-2
a1
a0
Fm-1, Fm-2, , F1, F0 : cc bit lu tr d liu nh phn.

am-1, am-2, , a1, a0 : cc cng tc (switch) dng ng (=1) hay m ( =0).
+ : l b lm tnh cng trong php ton mudulo 2 sau mi xung ng h vi d liu

do cc bit ca thanh ghi gi v.
Qu trnh dch chuyn li tng bc: sau mi xung ng h th d liu trong bit Fi s c
chuyn v lu tr bit Fi-1 (F1-> F0; F2->F1; ; Fm-2->Fm-3; Fm-1->Fm-2). Tt c cc gi tr trn
cc Fi (trc khi c xung in) s c chuyn v b cng (ty theo cc cng tc ng hay m),
tng ca cc gi tr ny s c a vo lu tr bit Fm-1.
Ta s nghin cu thanh ghi ny c th hn trong cc ni dung tip theo nhm tm ra mt phng

php sinh m m ta c th gi l m xoay vng. y cng l mt dng m kim tra chn l.
79
Biu din ton hc ca thanh ghi

Mc tiu ca vic biu din ton hc l tm ra cc m hnh tnh ton phc v cho vic nghin
cu sinh m xoay vong chn l t thanh ghi.
Gi x = (x0, x1, , xm-2, xm-1) l gi tr cc bit ca thanh ghi ti thi im trc khi c nhp xung
ng h.
x = (x0, x1, , xm-2, xm-1) l gi tr cc bit ca thanh ghi sau khi c nhp xung ng h.
Khi ta c:
x0=x1
x1=x2
x2=x3
----------
xm-2=xm-1
xm-1=a0x0 + a1x1 + + am-1xm-1.
Hay vit theo dng ma trn ta c x = T.x
Trong :
0
0
0 1 0 0 0
0 0 1 0 0
0
0
0 0 0 1 0
0
0
T= 0 0 0 0 ...
0
0
... ... ... ... ... ...
...
0
1
0 0 0 0 ..
a a a a ... a
a m 1
1
2
3
m2
0
T: Ma trn vung cp m.
Dng cui ca ma trn: l cc h s: a0,
a1, , am-1.
Gc trn bn phi: l ma trn n v cp
m-1.
T c gi l ma trn c trng ca thanh ghi li tng bc.

Qu trnh dch chuyn li tng bc ca thanh ghi:
x0
x2
Gi x(0)= x3 l vc t ch gi tr ca thanh ghi ti thi im ang xt.
M
x
m 1
Gi tr ca thanh ghi sau 1 xung ng h l x(1)=T.x(0)
Gi tr ca thanh ghi sau 2 xung ng h l x(2)=T.x(1)=T2.x(0)
-----------
V d thanh ghi lui tng bc

Cho thanh ghi lui tng bc sau:
+
F3
F2
F1
F0
T thanh ghi, ta c: m=4, a0=1, a1=0, a2=1, a3=0.
80
0
0
Ma trn c trng ca thanh ghi: T=
0
1
0
0
0
0
1
0
1
0
0
1
Chu k ca thanh ghi

Nh trnh by trn v qu trnh dch chuyn li tng bc ca thanh ghi:
x0
x2
Nu ta gi x(0)= x3
l vc t ch gi tr ca thanh ghi ti thi im khi to th cc gi
tr ca thanh ghi cc thi im tip theo nh sau:

M
x
m 1
Gi tr ca thanh ghi sau 1 xung ng h l x(1)=T.x(0)
----------------
Gi tr ca thanh ghi sau n xung ng h l x(n)=T.x(n-1)=Tn.x(0) (bi v s trng thi thng tin khc
nhau c th c l 2m)
Vy chu k ca thanh ghi l s xung nhp ng h thanh ghi lp li trng thi ban u. Ngha l
nu x(0)0 v n>0 sao cho x(n) = x(0) th ta ni n l chu k ca thanh ghi.
Lu :
Cch vit biu din nh phn cho gi tr ca x(i) theo th t t trn xung (theo ct), tng ng vi
vit t tri sang phi (theo dng). V d: biu din nh phn ca x(i) = 3 c m = 3 bit nh sau:
Vit theo dng: x(i) = 011 (vit t tri sang phi)
0

(i)
Vit theo ct: x = 1 (vit t trn xung)
1

V d tm chu k ca thanh ghi

Cho thanh ghi lui tng bc nh hnh sau:
+
F3
F2
F1
F0
T thanh ghi ta c: m=4, a0=1, a1=0, a2=1, a3=0.

0 1 0 0
0 0 1 0

0 0 0 1
1 0 1 0
81
x0 0

x 0
(0)
c gi tr khi to ca thanh ghi x =1= 1 =
x
0
2
x 1
3
Tm chu k:
0
0
1

0
1
X(1)=T.x(0)= x(2)=T.x(1)= x(3)=T.x(2)=

1
0
1

0
1
0

0
1

1
x(4)=T.x(3)= x(5)=T.x(4)= x(6)=T.x(5)=

0
0

0
0

Tng t:
+ Khi chn x(0) = 3 thi ta cng c chu k n = 6.
+ Khi chn x(0) = 6 th ta c chu k n = 3.
+ Khi chn x(0) = 0 th ta c chu k n = 1.
0

0 (0)
0 = x

1

12
15
0
11
13
10
Chu k n=6
Thanh ghi trn c 4 chu k.
Chu k n=6
14
Chu k n=3
Chu k n=1
Bi tp
1. Tm cc chu k ca thanh ghi lui tng bc nh hnh sau:
+
F2
F2
F1
F0
2. Tm cc chu k ca thanh ghi lui tng bc nh hnh sau:

+
F2
F1
F0
BI 5.8: M XOAY VNG

Mc tiu
82

-
Bit cch xc nh ma trn kim tra chn l cho m xoay vng (hay cn gi l m
vng),
Hiu nh ngha m xoay vng,
Vn dng xy dng b m xoay vng,
Vn dng phng php sinh nhanh b m xoay vng sinh b m kim tra chn l.
Ma trn kim tra chn l m xoay vng

nh ngha: ma trn kim tra chn l c thit k t thanh ghi li tng bc l ma trn c dng
sau:
A=[x(0)| T x(0)|T2 x(0) ||Tn-1 x(0)] vi n l chu k ca thanh ghi (n > m)
Trong :
- T l ma trn c trng ca thanh ghi.
- x(0) 0: l gi tr khi to ca thanh ghi.
- n : l chiu di ca t m v cng l chu k ca thanh ghi.
- m: l s bit kim tra hay s bit ca thanh ghi.
V d: xt li v d tm chu k thanh ghi, nu chn gi tr khi to ca thanh ghi l x(0) = 1 th ta
c ma trn kim tra vi chu k n=6 nh sau:
0 0 0 1 0 1
0 0 1 0 1 0
(0)
(1)
(2)
(3)
(4)
(5)
A =[ x x x x x x ]=
0 1 0 1 0 0
1 0 1 0 0 0
nh ngha m xoay vng

M xoay vng l m kim tra chn l c sinh ra t ma trn kim tra chn l ng vi chu k n
ca thanh ghi li tng bc c dng nh:
A=[x(0)| Tx(0)|T2x(0) ||Tn-1x(0) ]
V d: xt li ma trn kim tra chn l trn
0 0 0 1 0 1
0 0 1 0 1 0
(chu k n = 6)
A=
0 1 0 1 0 0
1 0 1 0 0 0
Ta c n = 6, m = 3, k = 2 s = 2k = 22 = 4 t m.
p dng Phng php sinh m nhanh b m kim tra chn l ta c b m kim tra chn l gm 4
t m sau : w0 = 000000, w1 = 101010, w2 = 010101, w4 = 111111, y chnh l mt trong cc b
m xoay vng sinh t thanh ghi li tng bc nu trn (Cc bc sinh m nhanh ngh cc
bn t lm)
Phng php sinh nhanh b m xoay vng

Cch sinh nhanh k t m c lp tuyn tnh ca b m vng t a0, a1, a2, , am-1:
Bc 1: sinh m xoay vng u tin
Sinh m xoay vng u tin c dng w1=a0a1a2am-1 100000
k-1 bit 0
Bc 2: sinh k -1 t m c lp tuyn tnh cn li
83

w2= 0a0a1a2am-110000 (dch w1 sang phi 1 bit).
k-2 bit 0
.
wk= 00000a0a1a2am-11 (dch t wk-1 sang phi 1 bit).
k-1 bit 0
Bc 3: xc nh cc t m cn li ca b m
Cc t m cn li gm (2k k t m) c xc nh bng cch cng t hp ca 2, 3, , k t m
t k t m c lp tuyn tnh trn.
V d sinh nhanh b m xoay vng

Cho thanh ghi lui tng bc nh hnh sau:
+
F3
F2
F1
F0
T thanh ghi, ta c: m=4, n=6, a0=1, a1=0, a2=1, a3=0.

Bc 1: Sinh m xoay vng u tin
w1=101010
Bc 2: Sinh k -1 t m c lp tuyn tnh cn li
w2=010101
Bc 3: Xc nh cc t m cn li ca b m
w3 =111111 (w1+w2), w0 =000000 (w1+w2 + w3)
B m vng va sinh l W={000000, 101010, 010101, 111111)
84
Bi tp
1. Cho thanh ghi li tng bc sau:
F2
F1
F0
- Tm ma trn kim tra chn l c s ct n > 4

- T kt qu cu a, xc nh b m xoay vng tng ng.
- Tm b m xoay vng theo phng php sinh nhanh b m xoay vng
2. Cho thanh ghi li tng bc sau:
+
F3
F2
F1
F0
Tm ma trn kim tra chn l c s ct n > 4

T kt qu cu a, xc nh b m xoay vng tng ng.
Tm b m xoay vng theo phng php sinh nhanh b m xoay vng.
85
BI 5.9: A THC C TRNG CA THANH GHI

Mc tiu
- Hiu nh ngha a thc c trng ca thanh ghi,
- Hiu Quan h gia chu k n, a thc c trng v a thc (xn + 1),
- Vn dng sinh thanh ghi li tng bc,
- Lm c s vn dng sinh b m vng.
nh ngha a thc c trng ca thanh ghi

nh ngha: a thc c trng ca thanh ghi c ma trn c trng l T l a thc c dng
gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm.
vi a0, a1, a2,, am-1 l cc cng tc ca thanh ghi v m l s bit ca thanh ghi
V d: xt li thanh ghi nh hnh sau:
F3
F2
F1
F0
a0 = 1, a1= 0, a2 = 1, a3 = 0
a thc c trng ca thanh ghi c dng: gm(x)=1 + x2 + x4.
Quan h gia chu k n, a thc c trng v a thc (xn + 1)

a thc c trng ca thanh ghi gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm lun chia ht a thc (xn
+ 1).
V d: xt li thanh ghi lui tng bc nh hnh sau:
+
F3
F2
F1
F0
T thanh ghi ta c th xc nh cc kt qu sau:

- a0 = 1, a1= 0, a2 = 1, a3 = 0
- a thc c trng ca thanh ghi c dng: g4(x)=1 + x2 + x4.
- Thanh ghi ny c chu k n = 6.
Thc hin php chia a thc (x6 + 1) : (1 + x2 + x4) = (x2 + 1) chia ht.
Ghi ch: php ton trn a thc nh phn vn l php ton Modulo 2.
V d: xt li thanh ghi lui tng bc nh hnh sau:

+
F3
F2
F1
F0
a0 = 1, a1= 0, a2 = 1, a3 = 0
a thc c trng ca thanh ghi c dng: g4(x)=1 + x2 + x4.
thanh ghi ny c chu k n = 6 v (x6 + 1) : 1 + x2 + x4 = x2 + 1.
86
Th tc sinh thanh ghi li tng bc

sinh thanh ghi li tng bc vi s bit l m v c chu k n, ta c th thc hin theo cc bc
sau:
Bc 1: xc nh a thc c trng ca thanh ghi
- Tm 2 a thc gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm
v hk(x)=h0 + h1x+ h2x2 + +hk-1xk-1 + xk sao cho (xn + 1) = gm(x)* hk(x).
- Nu (xn + 1) = gm(x)* hk(x) th ta chn gm(x) lm a thc c trng cho thanh ghi (v
s bit kim tra ca b m l m) v thc hin bc 2.
- Ngc li: khng tn ti thanh ghi theo yu cu.
Bc 2: v thanh ghi
T gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm a0, a1, a2,, am-1 thanh ghi c dng:
Fm-1
Fm-2
F1
F0
+
am-1
am-2
a1
a0
V d minh ha
Thit k thanh ghi c m=3 bit v chu k n=7, ta thc hin theo 2 bc sau:
Bc 1: Xc nh a thc c trng ca thanh ghi
Ta c (x7 + 1) : (1 + x2 + x3) = (1 + x2 + x3 + x4)
Do m=3 nn chn g3(x) = (1 + x2 + x3) lm a thc c trng ca thanh ghi.
Bc 2: V thanh ghi
T g3(x) = (1 + x2 + x3) ta c, a0=1, a1=0, a2=1
F2
F1
F0
Bi tp
1. Trong cc thanh ghi sau y, thanh ghi no sinh ra b m vng c di n=15 bit?
(R1):
F3
F2
F1
F0
(R2):
F3
F2
F1
F0
(R3):
F3
F2
F1
F0
2. Nu cc bc cn thit thit k b m xoay vng di 15 bit vi s bit kim tra l 4.

V s thanh ghi dng tng qut.
87
Bi 5.10: PHNG PHP SINH M XOAY VNG

Mc tiu
- Hiu cc phng php sinh m vng,
- Bit bng lit k mt s a thc c trng,
- Vn dng sinh m vng theo nhiu cch khch nhau.
t vn
sinh b m kim tra chn l, ta c th da theo nhiu phng php khc nhau nh: sinh m
da theo l thuyt nhm, m Hamming,... Vn t ra y l lm sao sinh b m xoay vng
vi di n bit v m bit kim tra chn l. Phng php sinh m xoay vng da trn l thuyt v
a thc c trng nh phn ca thanh ghi gip ta c ci nhn tng qut v vn sinh b m xoay
vng theo nhiu cch khc nhau.
Phng php sinh bng m xoay vng

sinh m xoay vng di n bit vi m bit kim tra v k bit thng tin, ta c th thc hin theo
cc bc sau:
Bc 1: tm 2 a thc gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm
v hk(x)=h0 + h1x+ h2x2 + +hk-1xk-1 + xk sao cho (xn + 1) = gm(x)* hk(x).
Nu (xn + 1) = gm(x)* hk(x) th chuyn sang bc 2
Ngc li khng th sinh b m vng theo yu cu.
Bc 2: ta c th sinh b m xoay vng theo cc cch nh di y:
Cch 1: Chn a thc gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm
a0, a1, a2,, am-1
thanh ghi ma trn c trng T
chu k n ma trn kim tra chn l A.
B m xoay vng.
Cch 2: chn a thc gm(x)=a0 + a1x+ a2 x2+ +am-1xm-1 + xm
a0, a1, a2,, am-1
Sinh nhanh k t m c lp tuyn tnh vi t m sinh c lp tuyn tnh u tin c
dng: w1=a0a1a2am-1100000 B m xoay vng.
k-1 bit 0
Cch 3: chn hk(x)=h0 + h1x+ h2x2 + +hk-1xk-1 + xk lm a thc sinh ma trn kim tra
chn l cho b m vng c dng:
0 0
0 1
1 h
k 1
hk 1
hk 1
k k 1
h1
h1
h0
h1
h1

h0
h0
h0
0
0
(m-1) bits
Sinh b m xoay vng theo Phng php sinh nhanh b m xoay vng.
88
Nhn xt: kt qu theo 3 cch sinh b m xoay vng ni trn la nh nhau (cho cng b m).
V d minh ha 1
Thit k thanh ghi v sinh ma trn kim tra chn l.
Chn a thc gm(x)= 1+x+x4 a0 = 1, a1 = 1, a2 = 0, a3 = 0
F3
F2
F1
F0
+
0
0
0
1
Tm chu k ca thanh ghi:
0

0
(0)
Chn gi tr khi to x =1=
0

1

1
0
0
1
0
1
0
0
0
0
1
0
0
1
0
0

(1)
(0) 0
(2)
(1) 1
(3)
(2) 0
(4)
(3) 0
(5)
(4) 1
x =T.x = ; x =Tx = ; x =Tx = ; x =Tx = ; x =Tx =
1
0
0
1
1

0
0
1
1
0

1
1
0
1
0

(6)
(5) 1
(7)
(6) 0
(8)
(7) 1
(9)
(8) 0
(10)
(9) 1
x =Tx = ; x =Tx = ; x =Tx = ; x =Tx = ; x =Tx =
0
1
0
1
1

1
0
1
1
1

1
1
1
1
0

(11)
(12) 1 (12)
(11) 1 (13)
(12) 1 (14)
(13) 0
(15)
(14) 0
x =Tx = ;x =Tx = ;x =Tx = ;x =Tx = ; x =T.x = = x(0)
1
1
0
0
0

1
0
0
0
1

Ma trn kim tra chn l :
0 0 0 1 0
0 0 1 0 0
A=
0 1 0 0 1
1 0 0 1 1
0
1
1
0
1
1
0
1
1
0
1
0
0
1
0
1
1
0
1
1
0
1
1
1
1
1
1
1
1
1
1
0
1
1
0
0
0
0
B m xoay vng vin=14, m=4, k=11.
V d minh ha 2
Chn a thc gm(x)= 1+x+x4 a0 = 1, a1 = 1, a2 = 0, a3 = 0.
89
Bc 1: Sinh m xoay vng u tin

w1 =110010000000000
Bc 2: Sinh k -1 t m c lp tuyn tnh cn li
w2 =011001000000000
w3 =001100100000000
w4 =000110010000000
w5 =000011001000000
w6 =000001100100000
w7 =000000110010000
w8 =000000011001000
w9 =000000001100100
w10=000000000110010
w11=000000000011001
Bc 3: Xc nh cc t m cn li ca b m
(215 - 11) t m cn li c xc nh bng cch cng t hp 2, 3, 4,.., k = 11 t
m t k=11 t m c lp tuyn tnh.
V d minh ha 3
Chn hk(x)= 1+ x + x2 + x3 +x5 + x7 + x8 + x11 lm a thc sinh ma trn kim tra chn l cho b m
vng h0 = 1, h1 = 1, h2 = 1, h3 = 1, h4 = 0, h5 = 1, h6 = 0, h7 = 1, h8 =1, h9 = 0, h10 = 0.
0 0 0 1 0 0 1 1 0 1 0 1 1 1 1
0 0 1 0 0 1 1 0 1 0 1 1 1 1 0
B m xoay vng
A=
0 1 0 0 1 1 0 1 0 1 1 1 1 0 0
1 0 0 1 1 0 1 0 1 1 1 1 0 0 0
Bng lit k mt s a thc c trng

M
3
4
5
6
7
8
9
10
11
12
13
a thc
1+x+x3
1+x+x4
1+x2+x5
1+x+x6
1+x3+x7
1+x2+x3+x4+x8
1+x4+x9
1+x3+x10
1+x2+x11
1+x+x4+x6+x12
1+x+x3+x4+x13
M
14
15
16
17
18
19
20
21
22
23
24
a thc
1+x+x6+x10+x14
1+x+x15
1+x+x3+x12+x16
1+x3+x7
1+x7+x18
1+x+x2+x5+x19
1+x3+x20
1+x2+x21
1+x+x22
1+x3+x23
1+x+x2+x7+x24
Bi tp
1. Tm b m vng c di 7 bit.
2. Tm thanh ghi sinh b m vng c di 15 bit.
3. Tm thanh ghi sinh b m vng c di 31 bit.
90
BI TP TNG HP
Mc tiu
- Hiu r hn v ni dung mn hc.
- Vn dng ni dung mn hc gii quyt mt s bi tp tng hp.
Bi 1
Xt mt m hnh chn on bnh t cc triu chng: A, B v C; chn on 1 trong 4 bnh: 1,
2, 3 v 4 vi ma trn chn on (hay ma trn truyn tin).
Bnh
Triu chng
A
B
C
0,6
0,2
0
0,3
0,6
0
0
0,2
0,3
0,1
0
0,7
Yu cu:
Cu 1: V s m t m hnh chn on bnh trn v din gii cc ngha ca s .
Cu 2: Nu phn phi ca Triu chng c dng:
Triu chng
P
A
0,5
B
0,3
C
0,2
Tnh cc lng sau :

Lng ngu nhin (Entropy) ca Triu chng .
Lng ngu nhin ca Bnh.
Lng ngu nhin ca Bnh khi bit Triu chng.
Lng chn on ng.(Lng thng tin bit v Bnh thng qua Triu chng) v t l
chn on ng l bao nhiu phn trm.
Cu 3: By gi ngi ta s dng 2 bit m thng tin v Triu chng (c 1 triu chng d tr) v
5 bit m cc triu chng khi chn on bnh trc tuyn. M t cc on ca dy 5 bit trong
phng php kim tra chn l.
Cu 4: Nu s dng ma trn kim tra chn l dng:
A=
1
0
1
1
1
0
1
0
0
0
1
1
1
1
1
Tnh cc t m.
Xy dng B sa li 1 bit dng cho t ng sa li ti u trong qu trnh chn on trc tuyn.
Cho mt v d.
Bi 2
Xt mt knh truyn tin c bit dng : Truyn X Nhn Y.
91

Truyn mt gi tr ca X c th nhn c nhiu gi tr khc nhau ca Y vi cc xc sut khc
nhau. Bng xc sut truyn X v nhn cc Y khc nhau c cho di y:
Y
X
x0
x1
x2
x3
y1
y2
y3
y4
y5
y6
0,6
0,1
0,05
0,1
0,1
0,05
0,1
0,05
0,1
0,6
0,1
0,05
0,05
0,1
0,05
0,1
0,05
0,1
0,6
0,1
0,1
0,05
0,1
0,6
Yu cu:
Cu 1: V s m t knh truyn tin trn v din gii cc ngha ca s .
Cu 2: Nu phn phi ca X c dng :
X
P
x0
0.5
x1
0.25
x3
0.15
x4
0.1
tnh thng lng v X truyn trn knh.

Cu 3: Phn phi ca X cn c dng nh th no thng lng truyn trn knh l ln nht.
Tnh dung lng knh truyn.
Cu 4: By gi ngi ta s dng 2 bit m thng tin v X v 4 bit m cc gi tr truyn trn
knh. M t cc on ca dy 4 bit trong phng php kim tra chn l.
A=
1
0
1
1
1
0
0
1
Tnh cc t m.
Xy dng B sa li dng cho t ng sa li ti u trong qu trnh truyn tin. Cho mt v d.
Bi 3
Ngi ta cn nh gi knh truyn tin v chun b thc hin truyn mt loi tn hiu c bit: X =
{x0, x1, x2, x3}
Cng vic u tin l phi kho st knh truyn. Kt qu kho st cho thy:
Knh c th truyn nhn c 8 gi tr khc nhau, c kh nng pht hin li hoc iu chnh
li. Ma trn truyn tin c dng:
y1
y2
y3
y4
y5
y6
y7
y8
Y
X
x0
0,6
0,1
0,05 0,05 0,05 0,05 0,05 0,05
x1
0,05 0,05 0,6
0,1
0,05 0,05 0,05 0,05
x2
0,05 0,05 0,05 0,05 0,6
0,1
0,05 0,05
x3
0,05 0,05 0,05 0,05 0,05 0,05 0,6
0,1
Yu cu:
Cu 1: V s m t knh truyn tin trn v din gii cc ngha ca s . Nu phn phi ca
X c dng :
X
P
x0
0.5
x1
0.25
x3
0.15
x4
0.1
tnh thng lng v X truyn trn knh.

92

Cu 2: Phn lp cc gi tr ca Y v cc lp B0, B1, B2, v B3 dng gii m ti u Y tt nht
v cc gi tr tng ng ca X.
Cu 3 : By gi ngi ta s dng 2 bit m thng tin v X v 4 bit m cc gi tr truyn trn
knh. M t cc on ca dy 4 bit trong phng php kim tra chn l.
A=
1
0
0
1
0
1
1
1
Tnh cc t m.
Xy dng B sa li dng cho t ng sa li ti u trong qu trnh truyn tin. Cho mt v d.
Bi 4
Xt mt m hnh chn on bnh t cc triu chng: A, B v C; chn on 1 trong 4 bnh: 1,
2, 3 v 4 vi ma trn chn on (hay ma trn truyn tin)
Bnh
Triu chng
A
B
C
0,5
0,1
0
0,3
0,2
0,1
0
0,7
0,3
0,2
0
0,6
Yu cu:
Cu 1: Gi s ngi ta bit thm 3 triu chng gy bnh khc l : D, E v F v mun ghi li
cc triu chng ny thng qua bng k hiu A = {+, - }.
Hy kim tra tnh tch c ca bng m sau :
Triu chng : X
M : W
A
+
B
-+
C
++-
D
--+-
E
++-+
F
--
B
0.2
C
0.2
D
0.05
E
0.03
F
0.2
Cu 2: Nu cc triu chng cu 1 c phn phi :

Triu chng : X
P
A
0.5
Gi s c mt ngi bnh vi 1 trong 5 triu chng trn n khm bnh v bc s s hi bnh vi

nguyn tc, sao cho ngi bnh ch tr li bng 2 cu : ng hoc Sai.
Tm phng php hi bnh vi s cu hi trung bnh t nht.
Tnh s cu hi trung bnh.
Tnh lng ngu nhin ca Triu chng.
Nhn xt g v s cu hi trung bnh v lng ngu nhin ca triu chng.
Cu 3: By gi s dng m hnh 3 triu chng {A, B, C} v 4 bnh. V s m t m hnh
chn on bnh v din gii cc ngha ca s .
93

Cu 4: T kt qu cu 3, ngi ta s dng 2 bit m thng tin v Triu chng (c 1 triu chng
d tr) v 5 bit m cc triu chng khi chn on bnh trc tuyn. M t cc on ca dy 5
bit trong phng php kim tra chn l.
A=
1
0
1
1
1
0
1
0
0
0
1
1
1
1
1
Tnh cc t m.
94
TI LIU THAM KHO

12. G.J.ChaiTin, Algorithmic Information Theory, CamBridge University Express-1992.
13. David J.C. Mackey, Information Theory, Infernce, and Learning Algorithms, CamBridge
University Express-2003.
14. Sanford Goldman, Information Theory.
15. http://www.inference.phy.cam.ac.uk/mackay/info-theory/course.html.
16. http://en.wikipedia.org/wiki/Information_theory.
17. http://www-2.cs.cmu.edu/~dst/Tutorials/Info-Theory/.
18. http://cscs.umich.edu/~crshalizi/notebooks/information-theory.html.
19. http://www.lecb.ncifcrf.gov/~toms/paper/primer/primer.pdf.
20. http://www.cs.ucl.ac.uk/staff/S.Bhatti/D51-notes/node27.html.
21. http://guest.engelschall.com/~sb/hamming/.
22. http://www2.rad.com/networks/1994/err_con/hamming.htm.
95

GT Ly Thuyet Thong Tin

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

GT Ly Thuyet Thong Tin

Uploaded by

Copyright:

Available Formats

Gio trnh: L thuyt thng tin.

Gio trnh: L thuyt thng tin.

Gio trnh: L thuyt thng tin.

Gio trnh: L thuyt thng tin.

Gio trnh: L thuyt thng tin.

Gio trnh: L thuyt thng tin.

GII THIU TNG QUAN

V Sinh m tch c, gio trnh s gii thiu n ngi hc cc vn v yu cu ca

Vn Sa li (hay x l m sai) trn knh truyn l mt vn rt quan trng v

Hn na, hu ht cc vn nu trn u c a vo ni dung ging dy cc bc i hc

Gio trnh: L thuyt thng tin.

KT THC TIN QUYT

Gio trnh: L thuyt thng tin.

TI LIU THAM KHO

Hc y cc mn hc tin quyt, b sung nhng kin thc c bn v ton v xc sut

Gio trnh: L thuyt thng tin.

CHNG 1: GII THIU

Gio trnh: L thuyt thng tin.

M hnh l thuyt thng tin theo quan im Shannon

Ngun (source) thng tin cn gi l thng bo cn c truyn u vo (Input).

Lng tin bit v cha bit

V d v lng tin bit v cha bit

Gio trnh: L thuyt thng tin.

t BNN Y l BNN v s u hnh m c sau 2 ln tung. Khi ta c th xc nh c phn

Phn phi ca Y khi bit X=2 c dng:

nh l c s ca k thut truyn tin

M t trng thi truyn tin c nhiu

Truyn tng bit

Gio trnh: L thuyt thng tin.

Minh ha k thut gim nhiu

Gio trnh: L thuyt thng tin.

Vy truyn sai khi Y {2, 3} c xc xut l:

Chi ph phi tr cho k thut gim nhiu

Khi nim v dung lng knh truyn

Gio trnh: L thuyt thng tin.

Gio trnh: L thuyt thng tin.

CHNG 2: O LNG TIN

Nhn xt v o lng tin

Gio trnh: L thuyt thng tin.

Khi nim entropy

Entropy ca mt phn phi

Nu gi Ai l s kin X=xi, (i=1,2,3,..) th Entropy ca Ai l: h(Ai)= h(pi)

nh l dng gii tch ca Entropy

nh l: Hm H(X) = H(p1, p2,...,pM) = C pi log( pi )

Gio trnh: L thuyt thng tin.

H(X) = H(p1 , p 2 ,..., p M ) = pi log 2 ( pi )

Qui c trong cch vit: log(pi)= log2(pi)

H(X) = H(1/2, 1/4, 1/4) = -(1/2log(1/2)+1/4log(1/4)+1/4log(1/4)) =3/2 (bit)

Bi ton v cy tm kim nh phn-t vn

Trong : x1, x5 ln lt l tn ca 5 ngi m ta cn nhn ra vi cch xc nh tn bng cu

Bi ton v cy tm kim nh phn - Din gii

Gio trnh: L thuyt thng tin.

Tnh H(Y) vi phn phi sau:

Gio trnh: L thuyt thng tin.

BI 2.2: CC TNH CHT CA ENTROPY

Cc tnh cht c bn ca Entropy

4. H(p, 1-p) l hm lin tc theo P.

Minh ha tnh cht 1 v 2

=> H(X) = log

Minh ha tnh cht 2:

Minh ha tnh cht 3 v 4

Gio trnh: L thuyt thng tin.

n y cc bn c th p dng cng thc tnh, so snh cc Entropy v nhn xt tnh

Minh ha tnh cht 4: