1

Cha bi tp


3.1. Thnh ph n c 1% dn s l sinh vin. Trong s sinh vin c
50% l nam thanh nin. S nam thanh nin trong thnh ph l 32%.
Gi s ta gp mt nam thanh nin. Hy tnh l-ng thng tin cha
trong tin khi bit rng l mt sinh vin

Bi Im 3.1
Gi
1
x la su kin gp mt nam thanh nin F
1
( ) 0, 32 p x

2
x la su kin gp mt sinh vinF
2
( ) 0, 01 p x
Khi do xac sut s sinh vin co trong s nam thanh nin la
1 2
( / ) 0, 5 p x x
Bai toan dt ra la phai tim xac sut d co mt sinh vin trong s nam thanh
nin
2 1
( / ) ? p x x
Tu du bai ta co nhn xet sau :
1
x va
2
x la 2 bin c ( 2 su kin ) phuc thuc nhau, do do xac sut d
1
x va
2
x xay ra dng thoi la
1 2 1 2 1 2 1 2
( , ) ( ). ( / ) ( ). ( / ) p x x p x p x x p x p x x
(1)
Tu (1) suy ra
2 1 2
2 1
1
( ). ( / ) 0, 01.0, 5 1
( / )
( ). 0,32 64
p x p x x
p x x
p x


Luong thng tin

6
2 1 2 1
1
( / ) log ( / ) log log2 6, 0
64
I x x p x x bit


2 1
( / ) I x x
6 bit



2
3.2. hai hp ng bt ch, mi hp ng 20 bt ch. Hp th nht
c 10 bt trng, 5 bt en v 5 bt . Hp th hai c 8 bt trng,
8 bt en v 4 bt . Ta ly h ho mt bt ch t mi hp. Hi
rng php th no trong hai php th ni trn c bt nh ln.
Bi lm 3.2
p A co : 10 trng, 05 den, 05 do _ 20
p B co : 08 trng, 08 den, 04 do _ 20
Ly ngu nhin mi hp 01 but chi :
Hp A :
Xac sut ly duoc trng : p(a
i
)
20
10

2
1
; Xac sut ly duoc den : p(a
2
)
20
5

4
1

Xac sut ly duoc do : p(a
3
)
20
5

4
1

Hp B :
Xac sut ly duoc trng : p(b
1
)
20
8

5
2
; Xac sut ly duoc den : p(b
2
)
20
8

5
2

Xac sut ly duoc do : p(b
3
)
20
4

5
1

Vy A va B co th biu din :
A

4
1
4
1
2
1
3 2 1
, , ,
B

5
1
5
2
5
2
3 2 1
b b b

Ta co :
(A) - ) a ( log ) a (
i 2
3
1
i
p p
i

-
2
1
2
log
2
1
-
4
1
2
log
4
1
-
4
1
2
log
4
1
1,5
(B) - ) ( log ) b (
j 2
3
1
j
b p p
f

-
5
2
2
log
5
2
-
5
2
2
log
5
2
-
5
1
2
log
5
1
-
5
4

2
log 5 1,52
Ta thy (B)~(A), va Entrpy la d bt dinh trung binh cua ngun tin nn phep thu o hp
B co d bt dinh cao hon.



3
3.3. c tn hiu
1
x ,
2
x vi cc xc sut tin nghim
1
p x 3/ 4 ,

2
p x 1/ 4 -c truyn theo knh nh phn i xng c nhiu nh-
hnh v. Do c nhiu nn xc sut thu ng mi tn hiu gim i ch
bng 7/8. Hy tm:
a. L-ng tin tc ring c iu kin
2 2
x / y
b. L-ng tin tc cho
2 2
x , y
c. c l-ng tin tc trung bnh
2
X,y , H(X), H(X/Y), (X,Y)

Bi Im 3.3

a. Tinh luong tin ring co diu kin
2 1
( / ) I x y

Ta co :
2 2 2 2 2 2 2 2
( , ) ( ). ( / ) ( ). ( / ) p x y p x p y x p y p x y


2 2 2
2 2
2 2 2
1 7 7
.
( ). ( / )
4 8 32
( / )
( ) ( ) ( )
p x p y x
p x y
p y p y p y

(1)

2 1 2 1 2 2 2
3 1 1 7 10
( ) ( ). ( / ) ( ). ( / ) . .
4 8 4 8 32
p y p x p y x p x p y x

1
y
2
y
2
x
1
x

1 1
p y / x 7/ 8

2 2
p y / x 7/ 8

1 2
p y / x 1/ 8

2 1
p y / x 1/ 8

1
p x 3/ 4

2
p x 1/ 4
4
Thay
2
10
( )
32
p y vao (1) ta co
2 2
2
7 7
7
32 32
( / )
10
( ) 10
32
p x y
p y


Suy ra
2 2
7
( / ) log 0, 514573
10
I x y
bit
b. Tnh lung tin cho
2 2
( , ) I x y

2 2
2 2
2
7
( / )
10
( , ) log log log2,8 1, 5
1
( )
4
p x y
I x y bit
p x


c. Tnh cc lung tin trung bnh
2
( / ) I X y H(X) H(X/Y) I(XY)
`
2
i 2
2 i 2 i 2
i1
(x / )
( , ) |I(x , )| (x / ).log
( )
i
p y
I X y M y p y
p x

1 2 2 2
2 1 2 2 2
1 2
(x / ) (x / )
( , ) (x / ).log (x / ).log
( ) ( )
p y p y
I X y p y p y
p x p x

(2)
Thay 1 2 2 2
7 3
(x / ) 1 (x / ) 1
10 10
p y p y
vao (2)
Ta c
2
( / ) I X y 06534 bit
(X)
2
1
( ).log ( ) 0,811278
i i
i
p x p x

bit
(X/Y)
2 2
1 1
( , ).log ( / )
i f i f
i f
p x y p x y

1 1 1 1 1 2 1 2 2 1 2 1 2 2 2 2
( , ).log ( / ) ( , ).log ( / ) ( , ).log ( / ) ( , ).log ( / ) p x y p x y p x y p x y p x y p x y p x y p x y (3)
5
Tinh
1 1 1 1 1
3 7 21
( , ) ( ). ( / ) .
4 8 32
p x y p x p y x ;
1 2 1 2 1
3 1 3
( , ) ( ). ( / ) .
4 8 32
p x y p x p y x
2 1 2 1 2
1 1 1
( , ) ( ). ( / ) .
4 8 32
p x y p x p y x
2 2 2 2 2
1 7 7
( , ) ( ). ( / ) .
4 8 32
p x y p x p y x ;
2 2
7
( / )
10
p x y
;
1 2
3
( / )
10
p x y
;
1 1 1 1 1 1 1 1
( , ) ( ). ( / ) ( ). ( / ) p x y p x p y x p y p x y
1 1 1
1 1
1
3 7
.
( ). ( / ) 21
4 8
( / ) ;
22
( ) 22
32
p x p y x
p x y
p y

2 1
1
( / )
22
p x y

Thay cac phep tinh trn vao (3) ta co
(X/Y)0,459 bit
Tinh I(X,Y)(X)-(X/Y)0,352 bit

3.5. Mt in i v tuyn in gm 16 khi c gi tr nh- nhau
v tin cy v -c mc ni tip v mt thit b kim tra -
thng bo s hng hc ca cc khi. Hy tnh s In th t
nht tin hnh bng thit b kim tra - thng bo c
th pht hin bt c s hng hc no ca tt c cc khi.

Theo gi thit bt nh ca khi hng l:

i i
1
I a logp a log 4bit
16

L-ng thng tin nhn -c sau mi php o:
B plogp 1 p log 1 p
Vi p : xc sut c tn hiu
6
1 p : xc sut khng c tn hiu
xc nh -c khi hng (kh ht bt nh) s php o
cn thit n l:



i
min
max
I a
n
B

B max F khi
1
p 1 p
2


max
B 1bit

min
4bit
n 4
1bit
ln o
n
min
thut ton o phi m bo B max F Mi ln o
phi o im gia ca cc khi cn xc nh nhm m bo
1
p 1 p
2
.
Gi s khi hng l khi 6. c php o cn thit l:
- Ln 1: o u ra khi 8: Khng c tn hiu, khi hng nm
trong cc khi t 1 F 8.
- Ln 2: o u ra khi 4: Khng c tn hiu, khi hng nm
trong cc khi t 5 F 8.
- Ln 3: o u ra khi 6: Khng c tn hiu, khi hng nm
trong khi 5 hoc 6.
- Ln 4: o u ra khi 5: tn hiu. Vy khi hng l khi 6

3.9. A chn mt trong cc s t 0 7. Hi B phi dng trung bnh
bao nhiu cu hi tm ra s A m A chn ?

bt nh ca s -c chn ngu nhin:

i i
1
I a logp a log 3bit
8

tm -c s chn ca A, B phi t cc cu hi cho A
thu -c mt l-ng thng tin cn thit l 3 bt.
7
Mi cu hi ca B (dng la chn) c th xem l ngun ri rc nh
phn, bi vy l-ng thng tin nhn -c sau mi cu tr li t-ng ng l:
B plogp 1 p log 1 p
Vi p : xc sut nhn cu tr li Jng J
1 p : xc sut nhn cu tr li JsaiJ
Vy s cu hi cn thit n l :


i
I a
n
B

S cu hi trung bnh ti thiu l:


i
min
max
I a
n
B


max
B khi
1
p 1 p
2


min
3bit
n 3
1bit
ln hi
Gi s A chn s 3. c cu hi B c th t cho A l:
- u 1 - S A chn ln hn 3? Tr li: Sai
- u 2 - S A chn ln hn 1? Tr li: ng
- u 3 - S A chn ln hn 2? Tr li: Sai
Vy s A chn l 3

iai bi 3.10 trang 117

i mt dim anh (phn tu anh) co 8 muc sang dng xac sut, do do luong
thng tin trung binh cua mi tin (Entropy) la cuc dai va bng log83
Voi 5.10`5 yu t (phn tu) thi Luong thng tin la 5.10`5x 31,5.10`6 bit
Kha nng thng qua cua knh VTT la : C`25x1,5.10`6 (b/s)
Theo cng thuc 3.52 (trang 111)
s
n
P
C' Flog 1
P
3





C`Flog164F -~ FC`/49,375.10`6

8






3.13 : Trong 27 ng xu c 1 ng xu gi nh hn. tm -c
ng xu gi ng-i ta s dng mt cn a thng bng. Hy
tnh s In cn trung bnh ti thiu xc nh -c ng
xu gi. Nu thut ton cn ?

Theo gi thit bt nh cha trong s kin ng xu gi l :
(x
i
) = - log p(x
i
) = - log 1/27 = log 27 bit
Khi s dng cn a thng bng, sau mi ln cn cc s kin c
th c l :
- n thng bng vi xc sut p
- n lch tri vi xc sut q
- n lch phi vi xc sut 1-p-q
L-ng thng tin nhn -c sau mi ln cn :
H(B) = -plog p qlog q (1-p-q)log (1-p-q)
xc nh -c ng xu gi tng l-ng thng tin nhn -c sau
cc ln cn phi khng nh hn bt nh ca ng xu gi. Nh-
vy s ln cn cn thit l : n = (x
i
)/H(B)
n c gi tr nh nht th H(B) phi t gi tr cc i.
Ta c H(B) = H(B)
max
= log 3 khi p = q = 1-p-q = 1/3.
Khi n
min
= (x
i
)/H(B)
max
= log27/log 3 = 3 ln cn.
Thut ton cn nh- sau( m bo p = q = 1-p-q )
- Ln 1 : hia 27 ng xu thnh 3 phn, mi phn c 9 ng xu.
Ly 2 phn bt k t ln mi bn cn 1 phn . Nu cn
thng bng th ng xu gi nm trong 9 ng xu ch-a cn.
Ng-c li, tu theo cn lch tri hay lch phi ta cng xc nh
-c phn c cha ng xu gi.
9
- Ln 2 : hia 9 ng c cha ng xu gi thnh 3 phn nh-
nhau, mi phn c 3 ng xu. t 2 phn bt k ln 2 bn cn.
Kt qu ca php cn s gip ta xc nh -c 3 ng xu c
cha ng xu gi.
- Ln 3 : Ly 2 ng xu bt k trong 3 ng xu c cha ng xu
gi t ln 2 a cn. Sau ln cn ny ta s xc nh -c
ng xu gi.




3.14: Trong b t l kh 52 qun (khng k fng teo), A rt ra mt
qun bi bt k. Tnh s cu hi trung bnh ti tiu m B cn t
cho A xc nh -c qun bi m A rt. Nu thut ton hi? Gi
s A rt ra 7 qun r, hy nu cc cu hi cn thit?
bt nh v qun bi m A rt:

i i
1
I a logp a log 6bit
52

Gi s B t cho A cu hi dng la chn, khi l-ng thng tin
nhn -c sau mi cu tr li ca A l:
B plogp 1 p log 1 p
Vi p : xc sut nhn cu tr li l Jng J
1 p : xc sut nhn cu tr li l JsaiJ
S cu hi cn thit xc nh -c qun bi A rt l:


i
I a
n
B

Ta thy n min F khi B max F

max
B B 1bit khi
1
p 1 p
2

S cu hi trung bnh ti thiu l:
min
log52 bit
n 6
1bit
ln
10
Thut ton phi m bo
1
p 1 p
2
.
Gi s A rt ra 7 r. c cu hi cn thit c th nh- sau:
- u 1: Qun A rt l qun ? ng
- u 2: Qun A rt l qun c? Sai
- u 3: Qun A rt c gi tr A 7? ng (gi s J = 11, Q = 12, K
= 13, At=1)
- u 4: Qun A rt c gi tr A 3? Sai
- u 5: Qun A rt c gi tr A 5? Sai
- u 6: Qun A rt l 6 r? Sai
Vy qun A rt l 7 r