Li cm n

Xin chn thnh cm n cc thy, cc c thuc khoa in- in t trng i Hc Tn

c Thng tn tnh truyn t cho chng em nhiu kin thc qu bu.
Chng em xin gi li cm n su sc n Thy nh Thun, Thy hng dn,
gip chng em trong sut thi gian thc hin ti.
Em xin cm n!

Thnh ph H Ch Minh, thng 5/2012

Nhm sinh vin thc hin:
Kiu Anh Dng Nguyn Thanh
Lim- Trn Nguyn Vn oi

Trnh by bo co NCKH
Ni dung ca NCKH c t chc v trnh by trong 6 chng:
Chng 0: M u: Gii thiu v ti.
Chng 1: Tng quan v bi ton nhn dng mt ngi v cc cch tip cn.
Chng 2:D tm khun mt trong nh.
Chng 3: Trch chn c trng cho nh khun mt.
Chng 4:Nhn dng nh khun mt.
Chng 5: ng dng th nghim.
Chng 6: nh gi v hng pht trin.

Mc lc

Danh sch cc hnh

Hnh 1.3.1 1 : M hnh tng qut ca bi ton nhn dng bin s xe........................................14
Hnh 2.2.1 1 : S h d tm khun mt bng AdaBoost NN..............................................19
Hnh 2.2.2 1 : Minh ho h d tm khun mt bng AdaBoost-Adaboost.................................19
Hnh 2.3.1 3 : V d v nh ngc sng......................................................................................19
Hnh 2.3.1 4: V d v nh b che khut thnh phn quan trng................................................20
Hnh 2.3.1 5 : V d v nh c cm xc c bit........................................................................20
Hnh 3.3.2 2: Cc nh vi phng php rt trch PCA b nhn dng sai vi b d liu nc
ngoi................................................................................................................................................22
Hnh 3.3.2 3 : Cc nh vi phng php rt trch ICA b nhn dng sai vi b d liu nc
ngoi................................................................................................................................................23
Bng 3.3.4 1 : Kt qu so snh PCA v ICA trn b d liu trong nc...................................24
Hnh 3.3.4 2 : Cc nh vi phng php rt trch PCA b nhn dng sai vi b d liu t to
.........................................................................................................................................................24
Hnh 4.2.1 1 :S h thng nhn dng mt ngi dng SVM................................................26
Hnh 4.2.2 1 :S h thng nhn dng mt ngi dng mng Nron.....................................27
Bng4.3.2 1 : Thi gian hun luyn SVM v NN b d liu nc ngoi..................................27
Bng 4.3.2 2 : Thi gian nhn dng th mc test nc ngoi bng SVM v NN ....................28
Bng 4.3.2 3 : Kt qu so snh nhn dng SVM v NN trn b d liu nc ngoi.................29
Hnh 4.3.2 1 : Biu kt qu th nghim hai phng php nhn dng SVM v NN trn.....29
b d liu nc ngoi: thng k trn b test.................................................................................29
Bng 4.3.4 - 1: Thi gian hun luyn SVM v NN b d liu trong nc...................................30

Bng 4.3.4 - 2: Thi gian nhn dng th mc test trong nc bng SVM v NN ......................30
Bng 4.3.4 - 3: Kt qu so snh SVM v NN trn b d liu trong nc.....................................31
Hnh 4.3.4 - 1 :Biu kt qu th nghim hai phng php nhn dng SVM v NN...............31
trn b d liu trong nc: thng k trn b test..........................................................................31
Hnh A.2.2 1 : Strong classifier H(x) c xy dng bng AdaBoost.......................................41
Hnh A.2.2 - 2: V d minh ho s kt hp ca 3 phn lp tuyn tnh.........................................42
Bng A.2.2 1 : Thut ton AdaBoost...........................................................................................43
Bng A.2.2 2 : Mt phin bn khc ca thut ton AdaBoost....................................................46
Hnh A.2.3 - 1 : Cc c trng Haar-like c s.............................................................................47
Hnh A.2.3 - 2: Cc min hnh hc c trng Haar like............................................................47
Hnh A.2.3 - 3: ngha hnh hc ca o hm nh......................................................................48
Hnh A.2.3 - 4: Cch tnh gi tr mt c trng.........................................................................48
Hnh A.2.3 - 5: D tm bn tay bng c trng Haar like .........................................................49
Hnh A.2.3 - 6: D tm khun mt bng c trng haar like.....................................................49
.........................................................................................................................................................50
Hnh A.2.4 - 1: Cascade Classifier................................................................................................50
Hnh A.3.3 - 1 Hng ca vc t ring.........................................................................................53
Hnh A.4.2 - 1: Hai tn hiu ngun ( khng quan st trc tip c, tc cc tnh hiu n l s1(t)
v s2(t)), hai tn hiu trn (quan st c l x1(t) v x2(t) ).........................................................58
Hnh A.4.3 - 1: Cc k hiu trong ICA...........................................................................................60
Hnh A.4.6 - 1: Phn b kt hp ca hai thnh phn c lp s1, s2 c phn b ng nht (trc
ngang: s1, trc ng s2)..................................................................................................................63
Hnh A.4.6 - 2: Phn b kt hp ca cc trn ln x1, x2 (trc ngang x1, trc ng x2)..............63
Hnh A.4.6 - 3: Phn phi kt hp ca hai bin Gauss..................................................................64
Hnh A.4.7.2.1 - 1: Hm mt ca phn phi Laplace, mt in hnh ca phn phi siu Gauss,
so vi phn phi Gauss ng gch nt, c hai mt c chun ha phng sai n v....66
Hnh A.4.10.2 - 1:Minh ha kin trc 1 trong m hnh ICA.........................................................75

Hnh A.4.10.2 - 2: vct c trng cho mi k thut. Hng u cha 8 vct ring vi 8 tr ring
ln nht trong PCA. Hng 2 cha cc vect c trng trong ICA vi kin trc I, hng 3 ch ra 8
vct c trng trong ICA vi kin trc 2.....................................................................................76
Hnh A.4.10.2 - 3:M hnh tng hp nh cho kin trc 1 ca ICA...............................................77
Hnh A.4.10.3 - 1:minh ha kin trc 2 trong m hnh ICA..........................................................78
Hnh A.4.10.3 - 2:M hnh tng hp nh cho kin trc 2 ca ICA...............................................79
Hnh A.4.10.3 - 3:M hnh tng hp nh cho kin trc 2 ca ICA trn pixel...............................79
Hnh A.4.11.1- 2. S phn phi d liu trong khng gian 3 chiu v cc trc tng ng ca
PCA v ICA. Mi trc l mt ct ca ma trn nghch o ca ma trn trn W-1 tm thy bi
PCA v ICA. Cc trc ca PCA trc giao trong khi ICA th khng, do khong cch cc im
d liu s thay i khi chiu xung khng gian mi ny..............................................................81
Hnh A.5.1 - 1 : Siu mt phn cch tuyn tnh cho trng hp phn cch c v....................82
k hiu cc support vc tr chnh l cc im c bao bng vin trn........................................82
Hnh A.6.1-1: M hnh mt Nron thn kinh.................................................................................87
Hnh A.6.2.1 - 1: M hnh Nron nhn to....................................................................................89
Hnh A.6.2.1.1 - 1: M hnh ton hc tng qut ca mt Nron...................................................90
Hnh A.6.2.1.3 - 1: Cc x l tng ng trong mt Nron.......................................................91
Hnh A.6.2.1.4 - 1: Hot ng tnh ton ca Nron.......................................................................93
Hnh A.6.2.2 - 1: Nm s lin kt c bn ca mng Nron: (a) m hnh mngtruyn thng
mt lp; (b) m hnh mng truyn thng a lp; (c) m hnh: mt Nron n vi lin kt phn
hi n chnh n; (d) m hnh: mng lp mt lp; (e) m hnh: mng lp a lp........................96
Hnh A.6.2.2 - 2: Lin kt bn trong ca phn hi.........................................................................97
Hnh A.6.2.3 - 1: Ma trn trng s ni kt.....................................................................................98
Hnh A.6.2.3 - 2 : Hc c gim st................................................................................................99
Hnh A.6.2.3 - 3 : Hc tng cng.................................................................................................99
Hnh A.6.2.3 - 4: Hc khng gim st.........................................................................................100
Hnh A.6.2.3 - 5: Lut hc pht sinh trng s (di khng c cung cp trong trng hp hc
khng gim st).............................................................................................................................102
Hnh A.6.2.4 - 1: Mt s dng hm dng trong nh x t u vo -> u ra..............................104

Bng A.6.2.4 1 : Mt s hm truyn thng dng trong mng Nron......................................105

Hnh A.6.2.5 1 : Cc loi lin kt ca Nron............................................................................106
Hnh A.6.3.1 - 1 : Cu hnh mng RBF tiu biu.........................................................................107
Hnh A.6.3.1 - 2 : Nhng tp phn lp trong khng gian 2 chiu.............................................109
HnhA.6.3.1 - 3 : nh x cc tp phn lp ln neuron RBF.......................................................110
Hnh A.6.4.1-1: Minh ha mng lan truyn thng ba lp............................................................112
Hnh B.3.1 1: Giao din chnh ca chng trnh......................................................................121
Hnh B.3.2 -1: Mn hnh minh ha chc nng test trn video.....................................................122
Hnh B.3.2 2 : Giao din ca chng trnh khi hot ng........................................................123
Hnh B.3.3 1 : Mn hnh tham s hun luyn cho mng nron................................................124
Hnh B.3.4 -1 : Mn hnh c d liu hun luyn...................................................................125
Hnh B.3.5 1: Mn hnh test th mc........................................................................................126
Hnh B.3.6 1 : Mn hnh nhn dng trn nh tnh.....................................................................127

Danh sch cc bng

Hnh 1.3.1 1 : M hnh tng qut ca bi ton nhn dng bin s xe........................................14
Hnh 2.2.1 1 : S h d tm khun mt bng AdaBoost NN..............................................19
Hnh 2.2.2 1 : Minh ho h d tm khun mt bng AdaBoost-Adaboost.................................19
Hnh 2.3.1 3 : V d v nh ngc sng......................................................................................19
Hnh 2.3.1 4: V d v nh b che khut thnh phn quan trng................................................20
Hnh 2.3.1 5 : V d v nh c cm xc c bit........................................................................20
Hnh 3.3.2 2: Cc nh vi phng php rt trch PCA b nhn dng sai vi b d liu nc
ngoi................................................................................................................................................22
Hnh 3.3.2 3 : Cc nh vi phng php rt trch ICA b nhn dng sai vi b d liu nc
ngoi................................................................................................................................................23
Bng 3.3.4 1 : Kt qu so snh PCA v ICA trn b d liu trong nc...................................24
Hnh 3.3.4 2 : Cc nh vi phng php rt trch PCA b nhn dng sai vi b d liu t to
.........................................................................................................................................................24
Hnh 4.2.1 1 :S h thng nhn dng mt ngi dng SVM................................................26
Hnh 4.2.2 1 :S h thng nhn dng mt ngi dng mng Nron.....................................27
Bng4.3.2 1 : Thi gian hun luyn SVM v NN b d liu nc ngoi..................................27
Bng 4.3.2 2 : Thi gian nhn dng th mc test nc ngoi bng SVM v NN ....................28
Bng 4.3.2 3 : Kt qu so snh nhn dng SVM v NN trn b d liu nc ngoi.................29
Hnh 4.3.2 1 : Biu kt qu th nghim hai phng php nhn dng SVM v NN trn.....29
b d liu nc ngoi: thng k trn b test.................................................................................29

Bng 4.3.4 - 1: Thi gian hun luyn SVM v NN b d liu trong nc...................................30
Bng 4.3.4 - 2: Thi gian nhn dng th mc test trong nc bng SVM v NN ......................30
Bng 4.3.4 - 3: Kt qu so snh SVM v NN trn b d liu trong nc.....................................31
Hnh 4.3.4 - 1 :Biu kt qu th nghim hai phng php nhn dng SVM v NN...............31
trn b d liu trong nc: thng k trn b test..........................................................................31
Hnh A.2.2 1 : Strong classifier H(x) c xy dng bng AdaBoost.......................................41
Hnh A.2.2 - 2: V d minh ho s kt hp ca 3 phn lp tuyn tnh.........................................42
Bng A.2.2 1 : Thut ton AdaBoost...........................................................................................43
Bng A.2.2 2 : Mt phin bn khc ca thut ton AdaBoost....................................................46
Hnh A.2.3 - 1 : Cc c trng Haar-like c s.............................................................................47
Hnh A.2.3 - 2: Cc min hnh hc c trng Haar like............................................................47
Hnh A.2.3 - 3: ngha hnh hc ca o hm nh......................................................................48
Hnh A.2.3 - 4: Cch tnh gi tr mt c trng.........................................................................48
Hnh A.2.3 - 5: D tm bn tay bng c trng Haar like .........................................................49
Hnh A.2.3 - 6: D tm khun mt bng c trng haar like.....................................................49
.........................................................................................................................................................50
Hnh A.2.4 - 1: Cascade Classifier................................................................................................50
Hnh A.3.3 - 1 Hng ca vc t ring.........................................................................................53
Hnh A.4.2 - 1: Hai tn hiu ngun ( khng quan st trc tip c, tc cc tnh hiu n l s1(t)
v s2(t)), hai tn hiu trn (quan st c l x1(t) v x2(t) ).........................................................58
Hnh A.4.3 - 1: Cc k hiu trong ICA...........................................................................................60
Hnh A.4.6 - 1: Phn b kt hp ca hai thnh phn c lp s1, s2 c phn b ng nht (trc
ngang: s1, trc ng s2)..................................................................................................................63
Hnh A.4.6 - 2: Phn b kt hp ca cc trn ln x1, x2 (trc ngang x1, trc ng x2)..............63
Hnh A.4.6 - 3: Phn phi kt hp ca hai bin Gauss..................................................................64
Hnh A.4.7.2.1 - 1: Hm mt ca phn phi Laplace, mt in hnh ca phn phi siu Gauss,
so vi phn phi Gauss ng gch nt, c hai mt c chun ha phng sai n v....66
Hnh A.4.10.2 - 1:Minh ha kin trc 1 trong m hnh ICA.........................................................75

Hnh A.4.10.2 - 2: vct c trng cho mi k thut. Hng u cha 8 vct ring vi 8 tr ring
ln nht trong PCA. Hng 2 cha cc vect c trng trong ICA vi kin trc I, hng 3 ch ra 8
vct c trng trong ICA vi kin trc 2.....................................................................................76
Hnh A.4.10.2 - 3:M hnh tng hp nh cho kin trc 1 ca ICA...............................................77
Hnh A.4.10.3 - 1:minh ha kin trc 2 trong m hnh ICA..........................................................78
Hnh A.4.10.3 - 2:M hnh tng hp nh cho kin trc 2 ca ICA...............................................79
Hnh A.4.10.3 - 3:M hnh tng hp nh cho kin trc 2 ca ICA trn pixel...............................79
Hnh A.4.11.1- 2. S phn phi d liu trong khng gian 3 chiu v cc trc tng ng ca
PCA v ICA. Mi trc l mt ct ca ma trn nghch o ca ma trn trn W-1 tm thy bi
PCA v ICA. Cc trc ca PCA trc giao trong khi ICA th khng, do khong cch cc im
d liu s thay i khi chiu xung khng gian mi ny..............................................................81
Hnh A.5.1 - 1 : Siu mt phn cch tuyn tnh cho trng hp phn cch c v....................82
k hiu cc support vc tr chnh l cc im c bao bng vin trn........................................82
Hnh A.6.1-1: M hnh mt Nron thn kinh.................................................................................87
Hnh A.6.2.1 - 1: M hnh Nron nhn to....................................................................................89
Hnh A.6.2.1.1 - 1: M hnh ton hc tng qut ca mt Nron...................................................90
Hnh A.6.2.1.3 - 1: Cc x l tng ng trong mt Nron.......................................................91
Hnh A.6.2.1.4 - 1: Hot ng tnh ton ca Nron.......................................................................93
Hnh A.6.2.2 - 1: Nm s lin kt c bn ca mng Nron: (a) m hnh mngtruyn thng
mt lp; (b) m hnh mng truyn thng a lp; (c) m hnh: mt Nron n vi lin kt phn
hi n chnh n; (d) m hnh: mng lp mt lp; (e) m hnh: mng lp a lp........................96
Hnh A.6.2.2 - 2: Lin kt bn trong ca phn hi.........................................................................97
Hnh A.6.2.3 - 1: Ma trn trng s ni kt.....................................................................................98
Hnh A.6.2.3 - 2 : Hc c gim st................................................................................................99
Hnh A.6.2.3 - 3 : Hc tng cng.................................................................................................99
Hnh A.6.2.3 - 4: Hc khng gim st.........................................................................................100
Hnh A.6.2.3 - 5: Lut hc pht sinh trng s (di khng c cung cp trong trng hp hc
khng gim st).............................................................................................................................102
Hnh A.6.2.4 - 1: Mt s dng hm dng trong nh x t u vo -> u ra..............................104

Bng A.6.2.4 1 : Mt s hm truyn thng dng trong mng Nron......................................105

Hnh A.6.2.5 1 : Cc loi lin kt ca Nron............................................................................106
Hnh A.6.3.1 - 1 : Cu hnh mng RBF tiu biu.........................................................................107
Hnh A.6.3.1 - 2 : Nhng tp phn lp trong khng gian 2 chiu.............................................109
HnhA.6.3.1 - 3 : nh x cc tp phn lp ln neuron RBF.......................................................110
Hnh A.6.4.1-1: Minh ha mng lan truyn thng ba lp............................................................112
Hnh B.3.1 1: Giao din chnh ca chng trnh......................................................................121
Hnh B.3.2 -1: Mn hnh minh ha chc nng test trn video.....................................................122
Hnh B.3.2 2 : Giao din ca chng trnh khi hot ng........................................................123
Hnh B.3.3 1 : Mn hnh tham s hun luyn cho mng nron................................................124
Hnh B.3.4 -1 : Mn hnh c d liu hun luyn...................................................................125
Hnh B.3.5 1: Mn hnh test th mc........................................................................................126
Hnh B.3.6 1 : Mn hnh nhn dng trn nh tnh.....................................................................127

Chng 0: Gii thiu

M u, chng ti s trnh by v hin trng thc t v cch tip cn ca ti:

10

Nhng nm gn y, bi ton nhn dng c rt nhiu tc gi quan tm v xut

nhiu phng php gii khc nhau. Tuy nhin chnh xc ca cc phng php cha
cao, cc kt qu chng thc cha tt.
Di y tng kt mt s phng php trong v ngoi nc tin hnh:
Nc ngoi [3]:

Dng phng php SVM nhn dng khun mt, s dng chin lc
kt hp nhiu b phn loi nh phn xy dng b phn loi a lp.

S dng phng php PCA kt hp LDA (phn tch c lp tuyn

tnh). Bc 1, chiu nh khun mt t khng gian nh th sang cc
khng gian khun mt (Mi lp khun mt c nhn dng s c
m hnh ha bng mt khng gian khun mt) dng PCA. Bc 2, s
dng phng php LDA to b phn loi tuyn tnh c kh nng
phn lp cc lp khun mt
S dng phng php mng Neural nhn to x l v nhn dng
khun mt
Trong nc:
Nhn dng mt ngi da vo thng tin da vo thng tin khun mt
xut hin trn nh. S dng phng php SVM v HMM [2].

Nhn dng mt ngi da trn FSVM v AdaBoost [1].

xut phng php rt trch c trng hnh hc (pht hin mt,

ming) cho bi ton nhn dng mt ngi [3].

giai on d tm khun mt phn ln ch p dng phng php Adaboost, tuy

nhin phng php Adaboost c nhc im l pht hin ra i khi sai nh khun mt.
C nhiu phng php chng thc li nh khun mt sau khi c pht hin bi
Adaboost, trong lun vn chng ti so snh cc phng php c thc hin l
Adaboost, Adaboost + NN, Adaboost + RBF v xut ca chng ti l Adaboost +
Adaboost.

11

Sau khi d tm c khun mt, phng php trch chn c trng truyn thng
c s dng l phng php chia li thng thng, hoc phng php PCA, tuy
nhin cc phng php ny cn c nhng yu im v vy chng ti p dng phng
php rt c trng mi ICA. Sau , qu trnh phn lp s c thc hin bng phng
php SVM, Mng Nron v so snh kt qu ca hai phng php phn lp ny. Sau ,
t cc kt qu l thuyt v thc nghim chng ti la chn ra phng php tt nht trong
tng giai on to thnh mt h thng ti u.
H thng xut cng c kim chng thng qua mt ng dng thc t: truy tm i
tng trong video. C th, bi ton c th pht biu ngn gn nh sau: Cho trc mt
CSDL nh mt ngi cc i tng cn truy tm v mt on video quay li khu vc cn
kim sot. Vn t ra l: xy dng h thng truy tm t ng tr li cu hi: nh mt
ngui cc i tng cn truy tm c xut hin trong on video quay hay khng? Bi
ton ny c p dng rng ri cho nhiu lnh vc khc nhau: Pht hin khng b, ngn
chn truy cp bt hp php,
chnh l ni dung ti kha lun ca chng ti: xut phng php truy tm nh
mt ngi trn video
V ni dung, lun vn tp trung vo cc vn sau:
Tm hiu c s l thuyt p dng cho bi ton nhn dng: AdaBoost, PCA,

ICA, SVM, Mng Nron.

D tm v chng thc khun mt: Adaboost, Mng Nron truyn thng ba lp,

mng nron RBF.

p dng phng php trch chn c trng mi ICA.
Chun b c s d liu ring cho ng dng: b d liu chun nc ngoi v b

d liu t to.
Bo co kt qu th nghim. Tin ti xy dng m hnh nhn dng ti u

nht.
Xy dng ng dng truy tm i tng trn video.

12

Chng 1 Tng quan v bi ton nhn dng mt ngi

1.1 Mt s ng dng ca nhn dng bin s xe trong thc t:
1.2. Nhng kh khn ra trong bi ton nhn dng bin s:
-

13

1.3. xut m hnh gii quyt ng vi tng khu ca bi ton nhn dng mt
ngi.
1.3.1 M hnh tng qut ca bi ton nhn dng mt ngi:

Hnh 1.3.1 1 : M hnh tng qut ca bi ton nhn dng bin s xe

1.3.2 Cc cng trnh nghin cu v phng php d tm v nhn dng khun mt:
Bi ton nhn dng bin s xe cn xc nh bn vn chnh:
Ngoi nc:

Wenyi Zhao, Arvindh Krishnaswamy, Rama Chellappa, Danie L.Swets, John Weng
(1998) s dng phng php PCA (phn tch thnh phn chnh) kt hp LDA (phn
tch c lp tuyn tnh). Bc 1, chiu nh khun mt t khng gian nh th sang
khng gian cc khng gian khun mt (Mi lp khun mt c nhn dng s c
m hnh ha bng mtkhng gian khun mt) dng PCA. Bc 2, s dng phng

14

php LDA to b phn loi tuyn tnh c kh nng phn lp cc lp khun mt.

John Daugnman (1998) , a ra phng php dng c trng v trng ca mt

phn bit cp (trai/gi) song sinh.

Emmanuel Viennet v Francoise Fogelman Soulie (1998) , s dng phng php

mng neural nhn to x l v nhn dng khun mt.

Antonio J.Colmenarez v Thomas S.Huang (1998), s dng k thut hc th gic v

ph hp mu 2-D. ng quan nim bi ton d tm khun mt l thao tc phn loi
khun mt trong khun mt thuc v mt lp v cc i tng khc thuc v lp
cn li bng cch c lng m hnh xc sut cho mi phn lp, v vic d tm s
dng lut quyt nh Maximum-likelihood.

Kazunori Okada, Johannes Steffens, Thomas Maurer, Hai Hong, Egor Elagin,
Hartmut Neven, and Christoph (1998), nhn dng khun mt da vo sng Gabor v
phng php ph hp th b. Vi tng dng th biu din khun mt,
nh khun mt c nh du ti cc v tr c xc nh trc trn khun mt,
gi cc v tr ny chnh l cc v tr chun. Khi thc hin thao tc so khp th vi
mt nh, cc im chun (Jets) s trch ra t nh v so snh cc im chun ny vi
tt c cc im chun tng ng trong cc th khc nhau, v th no ph hp
nht vi nh s c chn.

Baback Moghaddam v Alex Pentland (1998) , a ra phng php ph hp th gic

trc tip t cc nh cn s dng cho mc ch nhn dng khun mt v dng o
xc sut tnh tng t.

Massimo Tistaelli v Enrico Grosso (1998), a ra k thut th gic ng. V kh

nng quan st cc chuyn ng ca khun mt v x l cc tnh hung theo d nh
l thng tin rt quan trng, t nhn c m t y hn v khun mt cho mc
ch thu thp mu v nhn dng.

Jeffrey Huang, Chengjun Liu, v Harry Wechsler (1998), xut thut ton cn c
trn tnh tin ha (Evolutionary computation) v di truyn (Genetic) cho cc tc v

15

nhn dng khun mt. i vi cch tip cn ny, hai mt s c d tm trc tin
v thng tin ny c xem l vt quan st khun mt, trnh x l d tip mt bng
cch s dng mt thut ton lai kt hp thao tc hc v tin ha trong qu trnh
hc.

Daniel Bgraham v Nigel M Allinson (1998), s dng phng php c gi l to

bn sao khng gian c trng biu din v nhn dng hng di chuyn ca khun
mt.

Ara V.Nefian v Monson H.Hayes III (1998) trnh by hng tip cn theo m hnh
m hnh Markov n (HMM) trong nh mu khun mt c lng ha thnh
chui quan st trn khun mt theo quan nim da trn th t xut hin cc c trng
khun mt {hai chn my, hai lng mi, mi, ming, cm}. Trong chui quan st ,
mi quan st li l mt vector nhiu chiu v mi vector quan st ny c s dng
c trng cho mi trng thi trong chui trng trng thi ca HMM. Mi ngi
c c lng bng mt m hnh ca HMM.

Guodong Guo, Stan Z.Li, Kap Luk Chan (17 January 2001), dng phng php
SVM nhn dng khun mt. S dng chin lc kt hp nhiu b phn loi nh
phn xy dng b phn loi SVM a lp.
Trong nc:

Trn Phc Long, Nguyn Vn Lng (Lun vn c nhn tin hc, H KHTN
TP.HCM, 7/2003) , nhn dng khun mt da vo cc thng tin xut hin trn nh
bng SVM v HMM.

Lu Buon Vinh, Hong Phng Anh (Lun vn c nhn tin hc, H KHTN
TP.HCM, 7/2004) , nhn dng mt ngi da trn FSVM v AdaBoost.

Nguyn Anh Tun (Lun vn c nhn tin hc, H KHTN TP.HCM 7/2004), kho st
ng dng ca tp th trong la chn v rt gn c trng cho bi ton nhn dng mt
ngi.

L Minh Tr Nguyn Thy Hng (2006) [3], xut phng php trch chn c

16

trng mi cho bi ton nhn dng mt ngi, Lun vn C nhn CNTT Trng i
Hc KHTN TPHCM
1.3.3 Hng tip cn trong lun vn gii quyt bi ton nhn dng khun mt:
- Gii quyt bi ton nhn dng mt ngi l gii quyt mt bi ton ln, trong
bi ton ny ngi ta phi gii quyt bn bi ton nh hn, chnh l bn vn
chnh trong bi ton nhn dng mt ngi 1.3.2, trong bn bi ton nh c rt
nhiu phng php gii quyt, lun vn chng ti s gii quyt tng giai on
ca bi ton nhn dng mt ngi bng cch so snh cc phng php c v
cc phng php do chng ti xut.
- T kt qu so snh, tin ti xy dng m hnh hon chnh cho bi ton nhn
dng mt ngi.
- So snh, nh gi m hnh xut vi mt s m hnh truyn thng trn CSDL
ng dng c th, t ch ra tnh u vit ca m hnh xut, t a ra lp bi
ton thch hp vi m hnh xut.
C th trong tng vn :
detect khun mt trong nh, lun vn so snh bn phng php: Adaboost [3],

Adaboost + mng Nron 3 lp [12], Adaboost + Mng Nron RBF [27], Adaboost +
Adaboost.
c trng c la chn cho khun mt l c trng ton cc ca khun mt kt hp

vi cc c trng ca cc b phn: mt tri, mt phi, ming [3].

V vn rt trch c trng, lun vn so snh hai phng php rt trch c trng

cho khun mt c xem l mnh nht hin nay l phn tch thnh phn chnh
(Principal Component Analysis - PCA) [3] v

phn tch thnh phn c lp

(Independent Component Analysis - ICA).

Phng php hun luyn cho my nhn dng khi c tp vect c trng: lun vn

so snh hai phng php c xem l mnh nht hin nay l Mng Nron (NN) v
Support Vect Machine (SVM) [6].

17

Chng 2: D bin s xe trong nh

2.1 Bi ton:
nh bin s xe thng l nh bao gm c background, c th nhn dng c i
tng trong nh trc ht phi gii quyt bi ton d tm bin s xe ca i tng nm
ti v tr no trong nh.
2.2 xut phng php:
2.2.1 Cc phng php c thc hin [1][3][12][27]:

18

Hnh 2.2.1 1 : S h d tm khun mt bng AdaBoost NN

2.2.2 Phng php xut trong lun vn:

Hnh 2.2.2 1 : Minh ho h d tm khun mt bng AdaBoost-Adaboost

2.3 Cc kt qu thc nghim ca tng phng php:

2.3.1 C s d liu th nghim:

Hnh 2.3.1 3 : V d v nh ngc sng

19

Hnh 2.3.1 4: V d v nh b che khut thnh phn quan trng

Hnh 2.3.1 5 : V d v nh c cm xc c bit

20

Chng 3 Trch chn c trng cho bin s xe

3.1 Bi ton:
Sau khi pht hin c khun mt trong nh u vo ta cn phi biu din nh khun
mt thnh mt vc t c trng, tuy nhin v kch thc nh khun mt qu ln nu biu
din vc t l cc pixel ca nh khun mt th c th lm cho qu trnh hun luyn v
nhn dng rt chm, v xy ra trng hp qu khp v vy bi ton t ra l cn phi c
phng php biu din nh khun mt thnh vc t c trng m vn gi li c
nhng thnh phn quan trng ca nh.
3.2 xut phng php:
3.2.1 Cc phng php c thc hin [3][6][12]:
3.3 Cc kt qu thc nghim ca tng phng php:
3.3.1 C s d liu th nghim chun nc ngoi :
3.3.2 Kt qu :
Nhn xt:

21

Cc nh vi phng php rt trch PCA b nhn dng sai:

Hnh 3.3.2 2: Cc nh vi phng php rt trch PCA b nhn dng sai vi b d liu nc ngoi

Cc nh vi phng php rt trch ICA b nhn dng sai:

22

Hnh 3.3.2 3 : Cc nh vi phng php rt trch ICA b nhn dng sai vi b d liu nc ngoi

3.3.3 C s d liu t to:

B d liu c tng hp t hai ngun:

Hc sinh trng THPT Nguyn Hu Cu ( huyn Hc Mn, thnh ph

H Ch Minh) [6]
Mt s sinh vin trong khoa CNTT trng khoa hc t nhin[3]

Tng cng nh ca 19 ngi S nh hun luyn l 83 nh, s nh test l 76 nh

Nhn xt tp nh trong nc: Tp nh hc sinh trng THPT Nguyn Hu Cu c nhiu
nh khun mt quay tri, phi, mt s nh qu ti, hoc sng khng ng u trn
khun mt:
3.3.4 Kt qu :
Ngi
1

S nh

Train

S nh

train
6

ng
100%

test
5

PCA + FNN
ng
5

23

T L
100.00%

ICA +FNN
ng
5

T L
100.00%

2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Tng

6
5
5
5
4
5
3
4
5
4
3
3
4
4
5
4
4
4

100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%

5
3
3
4
5
6
4
4
3
4
5
4
2
3
5
4
4
3

5
3
3
3
5
5
4
4
3
4
5
4
2
3
5
4
4
3

100.00%
100.00%
100.00%
75.00%
100.00%
83.33%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%

5
3
3
4
5
6
4
4
3
4
5
4
2
3
5
4
4
3

100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%

cng

83

100%

76

74

97.37%

76

100.00%

Bng 3.3.4 1 : Kt qu so snh PCA v ICA trn b d liu trong nc

Cc nh nhn dng sai ca rt trch PCA

Hnh 3.3.4 2 : Cc nh vi phng php rt trch PCA b nhn dng sai vi b d liu t to

3.4 Kt lun:

24

T c s l thuyt ta thy phng php rt trch c trng ICA tt hn phng php

PCA v iu ny c chng ti kim chng qua thc nghim trn tp d liu chun
CalTech cng nh tp d liu do chng ti t to. Tm li ti giai on rt trch c
trng phng php ICA tt hn phng php PCA.

Chng 4 : Nhn dng bin s xe

4.1 Bi ton :
Sau khi c c vc t c trng ca khun mt, vn tip theo cn phi gii quyt l
lm sao so khp hai vc t c trng, tc l lm sao xc nh mt khun mt c
sn trong c s d liu v mt khun mt u vo ca h thng nhn dng c phi l mt
hay khng. C nhiu phng php gii quyt vn ny. Trong lun vn chng ti p
dng hai phng php nhn dng c xem l mnh nht hin nay.

25

4.2 Cc phng php :

4.2.1 Nhn dng bng phng php SVM [3][6][12][ph lc A.5]:

Hnh 4.2.1 1 :S h thng nhn dng mt ngi dng SVM

4.2.2 Nhn dng bng phng php mng Nron [6] [ph lc A.6]

26

Hnh 4.2.2 1 :S h thng nhn dng mt ngi dng mng Nron

4.3 Cc kt qu thc nghim ca tng phng php nhn dng:

Phng php rt trch c trng ICA.
Mng nron dng hun luyn v nhn dng: Dng mng nron FANN ca

Steffen Nissen [32], [33]

SVM dng hun luyn v nhn dng: th vin svmlib2.8
4.3.1 C s d liu th nghim chun nc ngoi :
M t c s d liu : tp nh CalTech m t trong 2.3.1.
4.3.2 Kt qu :
Thi gian hun luyn SVM v NN (khng bao gm thi gian to d liu, rt trch
c trng ICA) l:
FNN

SVM

Thi gian hun luyn(s) 71.2

1.6

Bng4.3.2 1 : Thi gian hun luyn SVM v NN b d liu nc ngoi

Ta thy thi gian hun luyn SVM rt nhanh.

Thi gian nhn dng ton b th mc test l:

27

Thi gian

FNN

SVM

65.1

64.9

nhndng(s
)
Bng 4.3.2 2 : Thi gian nhn dng th mc test nc ngoi bng SVM v NN

Kt qu nhn dng:
ICA + SVM
Ngi
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

S nh

Train

S nh

Train
5
5
3
5
5
5
5
3
5
4
3
3
5
5
5
5
2
5
5
7
6
6
5
3
5
5

ng
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%

Test
16
15
2
17
16
18
15
2
16
3
2
2
14
16
20
17
2
14
15
20
14
14
17
2
15
17

ng
12
15
1
15
14
18
15
1
15
3
1
0
10
15
20
17
0
14
13
20
14
14
17
1
15
17

28

ICA +FNN

T L
ng T L
75.00%
13
81.25%
100.00%
15
100.00%
50.00%
1
50.00%
88.24%
16
94.12%
87.50%
16
100.00%
100.00%
18
100.00%
100.00%
15
100.00%
50.00%
1
50.00%
93.75%
15
93.75%
100.00%
3
100.00%
50.00%
1
50.00%
0.00%
2
100.00%
71.43%
14
100.00%
93.75%
16
100.00%
100.00%
20
100.00%
100.00%
17
100.00%
0.00%
2
100.00%
100.00%
14
100.00%
86.67%
14
93.33%
100.00%
18
90.00%
100.00%
14
100.00%
100.00%
14
100.00%
100.00%
17
100.00%
50.00%
2
100.00%
100.00%
15
100.00%
100.00%
17
100.00%

Tng
cng

120

100%

321

297

92.52%

310

96.57%

Bng 4.3.2 3 : Kt qu so snh nhn dng SVM v NN trn b d liu nc ngoi

Kt qu nhn dng trn b train c hai phng php nhn dng SVM v NN

u t 100%

Phn trm nhn dng ng

So snh hai phng php nhn dng SVM

v NN (rt trch c trng bng ICA)
100
90
80
70
60
50
40
30
20
10
0

92.52

96.57

Nhn dng bng

phng php SVM
Nhn dng bng
phng php NN

Hnh 4.3.2 1 : Biu kt qu th nghim hai phng php nhn dng SVM v NN trn
b d liu nc ngoi: thng k trn b test

4.3.3 C s d liu trong nc t to:

C s d liu th nghim c m t trong 3.3.3
4.3.4 Kt qu:
Thi gian hun luyn SVM v NN (khng bao gm thi gian to d liu, rt trch
c trng ICA) l:

29

Thi gian hun

FNN

SVM

74.9

0.66

luyn(s)
Bng 4.3.4 - 1: Thi gian hun luyn SVM v NN b d liu trong nc

Ta thy thi gian hun luyn SVM rt nhanh.

Thi gian nhn dng ton b th mc test l:

Thi gian nhn

FNN

SVM

90

83.7

dng(s)
Bng 4.3.4 - 2: Thi gian nhn dng th mc test trong nc bng SVM v NN

Kt qu nhn dng:
Ngi
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

S nh

Train

S nh

train
6
6
5
5
5
4
5
3
4
5
4
3
3
4
4
5
4

ng
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%

test
5
5
3
3
4
5
6
4
4
3
4
5
4
2
3
5
4

ICA +SVM
ng
5
5
3
3
2
5
6
4
4
3
4
5
4
2
2
5
4

30

T L
100.00%
100.00%
100.00%
100.00%
50.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
66.67%
100.00%
100.00%

ICA +FNN
ng
5
5
3
3
4
5
6
4
4
3
4
5
4
2
3
5
4

T L
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%
100.00%

18
19
Tng

4
4

100%
100%

4
3

4
3

100.00%
100.00%

4
3

100.00%
100.00%

cng

83

100%

76

73

96.05%

76

100.00%

Bng 4.3.4 - 3: Kt qu so snh SVM v NN trn b d liu trong nc

Kt qu nhn dng trn b train c hai phng php SVM v NN u t

100%

Phn trm nhn dng ng

So snh hai phng php nhn dng SVM

v NN (rt trch c trng bng ICA)
100
90
80
70
60
50
40
30
20
10
0

96.05

100

Nhn dng bng

phng php SVM
Nhn dng bng
phng php NN

Hnh 4.3.4 - 1 :Biu kt qu th nghim hai phng php nhn dng SVM v NN
trn b d liu trong nc: thng k trn b test

4.4 Kt lun:
T cc kt qu thc nghim ta thy phng php SVM hun luyn v nhn dng rt
nhanh tuy nhin kt qu chnh xc khng cao, phng php mng Nron tuy hun luyn
chm v nhn dng chm hn phng php SVM nhng kt qu nhn dng chnh xc
hn nhiu. Tm li trong giai on nhn dng, chng ti la chn phng php mng
nron.

31

Chng 6 : nh gi v hng pht trin

6.1 nh gi:

Vi vic tm hiu kh k c s l thuyt v cc phng php truyn thng cho

tng giai on ca bi ton nhn dng mt ngi v ng dng vo bi ton thc
nghim, lun vn t c mt s kt qu sau y:
Trnh by r rng v c h thng cc phng php p dng cho tng giai on
ca bi ton nhn dng mt ngi t pht hin khun mt trong nh, n rt
trch c trng v nhn dng.
H thng ha bi ton nhn dng mt ngi, thng k, so snh bng l thuyt
v thc nghim cc phng php truyn thng gii quyt tng giai on
ca bi ton nhn dng mt ngi, t xy dng c m hnh tt nht cho
bi ton nhn dng mt ngi ng vi b d liu tng ng.
B d liu kh y , trong nc ln nc ngoi, cc nh trong mi trng
sng khng ng nht, v quay t m 30 n dng 30
Cc thnh phn khng quan trng ca khun mt nh trn khi b che khut vn
cho kt qu chnh xc, cc nh ti hoc qu sng, hoc quay tri phi 30
vn cho kt qu chnh xc.
Trong giai on pht hin khun mt trong nh c kt dng mng nron

chng thc nhm nng cao kt qu pht hin.

32

 Trong giai on rt trch c trng c p dng phng php kh mi l ICA

v kt qu chng minh l phng php ny tt hn phng php truyn
thng PCA.
Xy dng chng trnh th nghim Truy tm i tng trong video, v u
vo ca h thng l nh ng (video). Kt qu tt ca h thng cho thy
tnh kh thi ca m hnh xut.
Tuy vy lun vn vn cn hn ch nh sau:
Kt qu nhn dng cn nh hng nhiu khi khun mt c cc cm xc khc,
tuy nhin iu ny c th c khc phc khi nng s lng nh hun luyn
bao qut hu ht cc cm xc khun mt ca tng i tng.
C s d liu th nghim cn t.
6.2 Hng pht trin:
Th nghim trn c s d liu ln.
Tm kim cc phng php khc v d tm mt, ming trn khun mt.
p dng phng php rt trch c trng ICA kt hpvi cc phng php
khc chng hn LDA.
i vi ng dng, c th s dng thut gii di truyn tm cc mu ng
dng vi cc i tng tm c.
i vi cc khun mt c gc quay hn 30 th khng th nhn dng trong
khng gian hai chiu m phi trong khng gian ba chiu.
6.3 Kt lun:
Tm li, trong ti ny xut mt phng php cho truy tm nh mt ngi trong
video, phng php l s kt hp ca mt s cc phng php thng dng hin nay:
adaboost, ICA, mng nron. Vic la chn cc phng php c th nghim trn c s
d liu chun c nhiu nh khoa hc s dng (CalTech [4]) v c s d liu t to.

33

Cc kt qu thu c ch ra tnh kh thi ca cc phng php c chn la ng vi

tng giai on gii quyt bi ton. H thng truy tm nh mt ngi trn video c thc
hin vi 5 on video quay ngu nhin cho thy tnh kh quan ca phng php
xut.
Chng ti hi vng trong mt tng lai khng xa khi m nhng h thng nhn dng ca
chng ta t n mt tin cy nht nh th nhng ng dng nhn dng s rng ri.
V nhng nghin cu tip theo lnh vc ny s c nng ln mt cp cao hn,
hon thin hn.

Ph lc:
A. Cc c s l thuyt:
A.1 Cc Khi Nim Ton Hc Lin Quan n PCA v ICA
A.1.1 Tr ring (eigenvalue) v vc t ring (eigenvector):[7]
Cho ma trn A R n . S R c gi l gi tr ring (hay tr c trng) ca A nu tn
ti vc t x = (x1, .. ,xn) R sao cho :
n

x1
x1
x
x
A 2 = 2 (*)
...
...


xn
xn

vc t x # 0 gi l vc t ring (hay vc t c trng) ca A ng vi gi tr ring . Ma

trn A vi kch thc n x n c ti a n tr ring v n vc t ring tng ng. Mt ma trn
A nxn kh nghch s c n gi tr ring v n vc t ring tng ng.
Khai trin ng thc (*) ta uc h phng trnh tuyn tnh thun nht nh sau :
(a11 ) x1 + a12 x 2 + ... + a1n x n = 0
a 21 x1 + (a 22 ) x 2 + ... + a1n x n = 0
...................................................
a n1 x1 + a n 2 x2 + ..... + (a nn ) x n = 0

34

(**)

a thc c trng: a thc c trng ca A k hiu PA ( ), l a thc bc n theo (

l mt tr ring ca A ) nh bi:
PA ( ) = det (A - In)

=
=

a11

a12 ..............a1n

a 21

a 22 ........a 2 n

.......................................
a n1
a n 2 ..............a nn

(-1) n + an-1 n-1++ a1 + a0

Vi an-1, an-2, a0 F , a0 = det (A)

Thut ton tm gi tr ring v vc t ring:
n
Cho f l ton t tuyn tnh trong khng gian vc t n chiu R .Ton t tuyn tnh ny

c biu din bng ma trn vung A kch thc n x n . tm cc gi tr ring v vc t

ring ca f (ca A) tin hnh theo cc bc sau:
Bc1:

Lp a thc c trng PA ( ) = det (A - In)

Bc2:

Gii phng trnh c trng PA ( ) =0 tm cc gi tr ring ca A.

Bc3:

Vi mi gi tr ring (nu c) ca f, gii h phng trnh(**) tm cc

vc t ring ng ng vi gi tr ring .V d : cho ma trn A

3 2
3

A = 1
1 2
3 1 0
, tm cc gi tr ring v vc t ring ca ma trn .

Gii:
Lp a thc c trng ca ma trn A:
3
PA ( ) = 1
3

3
1
1

2
2 = (4 )(2 + 4)

35

a thc c trng PA ( ) =0 c 1 nghim duy nht = 4.Vy A c duy nht gi tr ring

= 4.

tm vc t ring ng vi = 4 gii h phng trnh ng vi = 4:

x1 + 3x2 + 2 x3 = 0
x1 3x2 2 x3 = 0
3x1 x2 4 x3 = 0
H phng trnh ny c 1 nghim l (1,1,-1)
Vy ma trn A c mt gi tr ring l = 4 ng vi vc t ring l (1,1,-1)
vc t trc giao v h trc giao: Cho X l khng gian vc t tch v hng, cc vc t x
v y ca X gi l trc giao nu tch v hng ca x v y bng 0.Tp S cc vc t ca X
gi l h trc giao nu 2 vc t phn bit bt k ca S u trc giao.
A.1.2. K vng, phng sai,hip phng sai, ma trn hip phng sai [8][9]
A.1.2.1 K vng:
Gi s X l bin ngu nhin ri rc vi min gi tr {xi, i = 1,..n }
n

Nu | xi | P[ X = xi ] hi t th i lng EX =

| x

i =1

i =1

| P[ X = xi ] c gi l k

vng ton ca X
Gi s X l bin ngu nhin lin tc vi hm mt fx(x)
+

Nu | x | f x ( x)dx < th i lng EX =

xf

( x )dx c gi l k vng ton

ca X
Trong thng k, k vng E[X] ca mt bin ngu nhin X c th c c lng
bng trung bnh mu X :

36

X =

1 n
xi
n i =1

A.1.2.2 Phng sai:

Gi tr phng sai trong thng k mt chiu l o mc phn tn ca bin
ngu nhin xung quanh k vng.
Xt bin ngu nhin X c k vng l EX, nu tn ti DX = E(X-EX)2 th ta ni
l phng sai ca X.
Tnh cht :DX = E(X2) (EX)2
Trong thng k, phng sai DX ca mt bin ngu nhin X c th c c
lng bng phng sai mu X :
2
S2 = X X

A.1.2.3 Hip phng sai:

Gi s X,Y l cc bin ngu nhin c k vng. Nu tn ti
cov (X,Y) = E(X-EX)(Y-EY) th ta gi l hip phng sai ca hai bin ngu
nhin X v Y.
nu cov(X,Y) = 0 th X,Y c gi l phi tng quan.
Tnh cht: Cov (X,Y) = E(XY) EXEY

(A.1.2.3_1)

Trong thng k, hip phng sai cov(X,Y) ca hai bin ngu nhin X,Y c th
c c lng bng hip phng sai mu:
m11 =

1 n
( xi X )( yi Y )
n i =1

A.1.2.4 Ma trn hip phng sai: [9]

t [X1, X2,, XN] l mt ma trn p x N cc php quan st. Gi tr trung bnh
mu (sample mean) ca cc vc t quan st X1,, XN c cho bi:
M =

1
(X1 + X2 + + XN)
N

Vi k = 1,N, t Xk = Xk M

37

t B = [X1, X2,, XN]

B c gi l dng lnh trung bnh, lc ma trn hip phng sai (mu) l
ma trn pxp c xc nh bi:
S=

1
BBT
N 1

Cho X l biu th mt vc t vn thay i trn tp hp cc vc t quan st v k hiu cc

ta ca X l x1, , xp. Th x1 l mt s v hng vn thay i trn tp hp cc ta
u tin ca X1,, XN.
Cho j = 1,,p phn t sij trn ng cho ca S c gi l phng sai ca xj.
Ma trn hip phng sai l mt ma trn i xng. Mi phn t Sij ca ma trn l
hip phng sai gia 2 thnh phn xi v xj trong vc t X.
Nu Sij = 0, ta ni 2 thnh phn xi v xj c lp hay khng ph thuc ln nhau.
Nu Sij 0 , ta ni xi v xj khng c lp hay gia chng c mi tng quan vi
nhau.
A.1.3 c lp thng k v phi tng quan:
A.1.3.1 c lp thng k
H hu hn cc bin ngu nhin [X1, X2,, Xn] c gi l c lp thng k nu
phn phi ng thi ca chng bng tch cc phn phi bin duyn (phn phi ring
phn) tc l:
n

FX1 , X 2 ,..., X n = FXi

i =1

Hoc nu xt trn hm mt th mt ng thi bng tch cc mt ring bit:

n

f X1 , X 2 ,..., X n = f Xi
i =1

nh ngha trn dn n tnh cht sau y ca cc bin ngu nhin c lp. Cho x1, x2,
L hai bin ngu nhin c lp, cho hai hm h1, h2, ta lun c tnh cht sau [10]:
E{ h1(x1) h2(x2) } = E{ h1(x1)} E{ h2(x2) }

38

(A.1.3.1_1)

A.1.3.2 Phi tng quan ch l mt phn ca c lp:

Ta c: nu cc bin ngu nhin l c lp th chng phi tng quan. R rng trong
cng thc (A.1.3.1_1), nu ta cho h1(x1) = x1, h2(x2) = x2 th ta c cng thc
(A.1.2.3_1) tc l x1, v x2 l phi tng quan, nhng phi tng quan th cha chc l c
lp.
V d (x1, x2) l cc gi tr ri rc v phn b vi xc sut bng vi bt k gi tr
sau: (0, 1) , (0, -1), (1, 0), (-1, 0) th x1 v x2 khng tng quan v:
x1x2 = {0,0,0,0} nn E{x1x2} = 0.
E{x1} = 0x1/4 + 0x1/4 + 1x1/4 + -1x1/ 4 = 0, tng t E{x2} = 0.
Ta c E{x1x2} = E{x1}E{x2} nn x1,x2 l phi tng quan nhng:
E{x12 x22} = 0 # = E{x12}E{x22}, vi phm iu kin ca (A.1.3.1_1) nn x1,x2
khng c lp.
A.2 Phng php AdaBoost [1][11][12][13][14]
A.2.1 Tip cn Boosting
Boosting l k thut dng tng chnh xc cho cc thut ton hc (Learning
Algorithm). Nguyn l c bn ca n l kt hp cc b phn loi yu (weak classifiers)
thnh mt b phn loi mnh. Trong weak classifiers l b phn loi n gin ch cn
c chnh xc trn 50%. Bng cch ny chng ta ni b phn loi c boost.
Xt 1 bi ton phn loi 2 lp (mu cn nhn dng c phn vo 1 trong 2 lp) vi D l
tp hun luyn gm c n mu. Trc tin, chng ta s chn ngu nhin n1 mu t tp D
(n1<n) to tp D1. Sau chng ta s xy dng weak classifier u tin C1 t tp D1.
Tip theo chng ta s xy dng tp D2 hun luyn b phn loi C2, D2 s c xy
dng sao cho mt na s mu ca n c phn loi ng bi C1 v na cn li b phn
loi sai bi C1. Bng cch ny, C2 cha ng nhng thng tin b sung cho C1. By gi
chng ta s hun luyn C2 t D2.

39

Tip theo, chng ta s xy dng tp D3 t nhng mu khng c phn loi tt bi s

kt hp gia C1 v C2: nhng mu cn li trong D m C1 v C2 cho kt qu khc nhau.
Nh vy, D3 s gm nhng mu m C1 v C2 hot ng khng hiu qu . Sau cng,
chng ta s hun luyn b phn loi C3 t D3.
By gi, chng ta c mt strong classifier: s kt hp gia C1, C2, C3. Khi tin hnh
nhn dng mt mu X, kt qu s c quyt nh bi s tho thun ca 3 b C1, C2, v
C3. Nu c C1 v C2 u phn X v cng 1 lp th lp ny chnh l kt qu phn loi X;
ngc li nu C1 v C2 phn X vo 2 lp khc nhau, C3 s quyt nh X thuc v lp no.
A.2.2 AdaBoost (Adaptive Boost)
AdaBoost l mt tip cn boosting c Freund v Schapire a ra vo nm 1995.
Adaboost cng hot ng trn nguyn tc kt hp tuyn tnh cc weak classifiers hnh
thnh mt strong classifier.
L mt ci tin ca tip cn boosting, AdaBoost s dng thm khi nim trng s
(weight) nh du cc mu kh nhn dng. Trong qu trnh hun luyn, c mi weak
classifiers c xy dng, thut ton s tin hnh cp nht li trng s chun b cho
vic xy dng weak classifier k tip: tng trng s ca cc mu b nhn dng sai v
gim trng s ca cc mu c nhn dng ng bi weak classifier va xy dng. Bng
cch ny weak classifer sau c th tp trung vo cc mu m cc weak classifiers trc
n lm cha tt. Sau cng, cc weak classifers s c kt hp ty theo mc tt ca
chng to nn strong classifier.
C th hnh dung mt cch trc quan nh sau: bit mt nh c phi l nh mt ngi
hay khng, ta hi T ngi (tng ng vi T classifiers xy dng t T vng lp ca
boosting), nh gi ca mi ngi (tng ng vi mt weak classifier) ch cn tt hn
ngu nhin mt cht (t l sai di 50%). Sau ta s nh trng s cho nh gi ca
tng ngi (th hin qua h s ), ngi no c kh nng nh gi tt cc mu kh th
mc quan trng ca ngi trong kt lun cui cng s cao hn nhng ngi ch

40

nh gi tt c cc mu d. Vic cp nht li trng s ca cc mu sau mi vng lp

boosting chnh l nh gi kh ca cc mu (mu c nhiu ngi nh gi sai l
mu cng kh).

Hnh A.2.2 1 : Strong classifier H(x) c xy dng bng AdaBoost

Cc weak classifiers hk (x) c biu din nh sau:

1,
hk =
0,

41

Trong :
1

x = ( x1 , x 2 ,..., x n ) : vct c trng ca mu.

: ngng.

f k : hm lng gi vct c trng ca mu.

p k : h s quyt nh chiu ca bt phng trnh.

Cng thc trn c th c din gii nh sau: nu gi tr vct c trng ca mu cho

bi hm lng gi ca b phn loi vt qua mt ngng cho trc th mu l object
(i tng cn nhn dng), ngc li mu l background (khng phi i tng).
V d:

Hnh A.2.2 - 2: V d minh ho s kt hp ca 3 phn lp tuyn tnh

42

Thut ton AdaBoost:

1. Cho mt tp hun luyn gm N mu c nh du (x1, y1), (x2, y2), , (xN,
yN). Vi xk X = (x1k, x2k, , xmk) l vct c trng v yk {1,0} l nhn
ca mu (1 ng vi object, 0 tng ng vi background).
2. Khi to trng s ban u cho tt c cc mu:
1
, vi k {1, 2, , N}
N

w1,k =

3. Xy dng T weak classifier.

Lp t = 1,,T

Vi mi c trng trong vct c trng, xy dng mt weak classifier h j

vi ngng j v li j
n

j = wt ,k h j ( x k ) y k
k

Chn ra hj vi j nh nht ta c ht
ht: X {1,0}

Cp nht li trng s
wt +1,k =

wt ,k
Zt

e t yk ht ( xk )

Trong :
1 1 j
o t = ln
2 j

Z t : h s dng a wt+1,k v on [0,1] (normalization

factor).
4. Strong classifier xy dng c:
T

H ( x) = sign t ht ( x )
t =1

Bng A.2.2 1 : Thut ton AdaBoost

43

Qu trnh hun luyn b phn loi c thc hin bng mt vng lp m mi

bc lp, thut ton s chn ra weak classifier ht thc hin vic phn loi vi li t nh
nht (do s l b phn loi tt nht) b sung vo strong classifier. Mi khi chn
c mt b phn loi ht, AdaBoost s tnh gi tr t theo cng thc trn. t cng
c chn trn nguyn tc lm gim thiu gi tr li t .
H s t ni ln mc quan trng ca ht.

Trong cng thc ca b phn loi H(x):

T

H (x ) = sign t ht ( x)
t =1

Ta thy tt c b phn loi ht u c ng gp vo kt qu phn loi ca

H(x), v mc ng gp ca chng ph thuc vo gi tr t tng ng: ht
v t cng ln th n cng c vai tr quan trng trong H(x).

Trong cng thc tnh t :

1 1 j
t = ln
2 j

D thy gi tr t t l nghch vi j . Bi v ht c chn vi tiu ch t

j nh nht, do n s bo m gi tr t ln nht. Cng thc ny do

Freund v Schapire a ra.

Sau khi tnh c gi tr t , AdaBoost tin hnh cp nht li trng s ca cc mu: tng
trng s cc mu m ht phn loi sai, gim trng s cc mu m ht phn loi ng. Bng
cch ny, trng s cc mu phn nh c kh nhn dng ca mu v ht+1 s u
tin hc cch phn loi cc mu ny.

44

Vng lp xy dng strong classifier s dng li sau T ln lp. Trong thc t ci t,

ngi ta t s dng gi tr T v khng c cng thc no m bo tnh c gi tr T ti u
cho qu trnh hun luyn. Thay vo ngi ta s dng gi tr max false positive hay
max false alarm (t l nhn dng sai ti a cc mu background). T l ny ca b phn
loi cn xy dng khng c php vt qu gi tr ny. Khi , qua cc ln lp, false
alarm ca strong classifier Ht(x) xy dng c (ti ln th t) s gim dn, v vng lp
kt thc khi t l ny thp hn max false alarm.
Ngi ta a ra mt s phin bn khc ca thut ton Adaboost di y l mt trong
s : [13]

45

Bng A.2.2 2 : Mt phin bn khc ca thut ton AdaBoost

ngha ca thut ton ny cng tng t nh thut ton va cp trn

46

A.2.3 Cch trch chn c trng cho AdaBoost

c trng s dng cho AdaBoost da trn c trng Haar like. c trng ny c xc
nh da vo s chnh lch gia tng cc min hnh hc. Mi c trng Haar like gm
hai, ba hoc bn min hnh hc.
Tp cc c trng Haar like c s nh sau:

Hnh A.2.3 - 1 : Cc c trng Haar-like c s

Cc c trng s dng d tm khun mt l tp m rng ca cc tp c trng Haar

like c s. C 3 tp hp min hnh hc chnh c s dng trong d tm khun mt nh
sau:
(1) c trng cnh:

(2) c trng ng:

(3)

c trng tm:

Hnh A.2.3 - 2: Cc min hnh hc c trng Haar like

Gi s min en l min dng v min trng l min m, gi tr ca c trng Haar

like bng tng gi tr cc pixel en tr cho tng gi tr cc pixel trng:

47

Cc c trng Haar like c th c tnh mt cch nhanh chng da trn o hm bc

nht ii(x,y) ca nh i(x,y). o hm ii(x,y) ca nh i(x,y) bng tng cc gi tr pixel pha
trn tri ca x,y:

Hnh A.2.3 - 3: ngha hnh hc ca o hm nh

P ( x, y ) =

i ( x' , y ' )

x ' x , y ' y

V d: tnh gi tr D

Hnh A.2.3 - 4: Cch tnh gi tr mt c trng

P1 = A, P2 = A + B, P3 = A + C , P4 = A + B + C + D
P1 + P4 P2 P3 = A + A + B + C + D A B A C = D

Vic tnh c trng Haar like ch cn l tr gi tr tng cc hnh ch nht c tnh

nh trn.
V d v d tm bn tay v d tm khun mt:

48

D tm bn tay, nh c qut bi mt ca khung s nh cha ng c trng Haar

like:

Hnh A.2.3 - 5: D tm bn tay bng c trng Haar like

D tm khun mt cng tng t nh trn:

Hnh A.2.3 - 6: D tm khun mt bng c trng haar like

A.2.4 Cascade of Classifiers:

Cc b phn loi tt phi tn rt nhiu thi gian cho ra kt qu phn loi bi v n
phi xt rt nhiu c trng ca mu. Tuy nhin, trong cc mu a vo, khng phi
mu no cng thuc loi kh nhn dng, c nhng mu background rt d nhn dng ra
(ta gi y l nhng mu background n gin). i vi nhng mu ny, ta ch cn xt
mt hay vi c trng n gin l c th nhn din c ch khng cn xt tt c cc
c trng. Nhng i vi cc b phn loi thng thng th cho d mu nhn dng l d

49

hay kh th n vn s xt tt c cc c trng m n rt ra c trong qu trnh hc. Do

chng s tn thi gian x l mt cch khng cn thit.
Cascade of Classifier c xy dng chnh l nhm rt ngn thi gian x l, gim thiu
false alarm cho b phn loi. Cascade tree gm nhiu stage (hay cn gi l layer), mi
stage ca cy s l mt stage classifier. Mi mu c phn loi l i tng th n
phi i qua ht tt c cc stages ca cy. Cc stage classifiers nhng stage sau c
hun luyn bng nhng mu negative m stage classifier trc nhn dng sai, tc l
n s tp trung hc nhng mu background kh hn, do s kt hp cc stage classifier
ny li s gip b phn loi c false alarm thp. Vi cu trc ny, nhng mu
background d nhn din s b loi ngay nhng stage u tin, gip p ng tt nht
i vi phc tp gia tng ca mu a vo, ng thi gip rt ngn thi gian x l.

Hnh A.2.4 - 1: Cascade Classifier

A.2.5 Cascade of Boosted Classifiers

Cascade of Boosted Classifiers l m hnh Cascade of Classifier vi mi classifier c
xy dng bng AdaBoost s dng Haar Feature. M hnh ny c Viola v John s
dng rt thnh cng trong bi ton d tm khun mt.

50

A.3 Phng php Phn Tch Thnh Phn Chnh (Principal Component Analysis PCA)[3][6][9]
A.3.1 Mc ch, ngha:
PCA (Principal Componens Analysis) cn c gi l Karhunen-Loeve Transform
(KLT) hay Hotelling transform. PCA tm php bin i tuyn tnh :
-

Gim s chiu ca khng gian c trng nhng vn gi c cc c trng chnh.

Cc tiu ha vic mt mt thng tin.

Phng php phn tch thnh phn chnh chiu d liu theo chiu bin i nhiu nht.
A.3.2 Phng php:
Gi s x T = [ x(1), x( 2),..., x( N )] , vc t N x 1.
Xt ma trn bin i trc giao A (kch thc n x n )chuyn x thnh y :

a1T
T
a
y = AT x = 2 x =
...

a TN

a1 ( N ) x1
a1 (1) a1 (2) ....
a (1) a (2) .....
a 2 ( N ) x 2
2
2
............................................. ...

a N (1) a N (2) ........ a N ( N ) x N

V A trc giao nn:

N

x = Ay =

y(i)a
i =1

a N (1)
x1
a1 (1)
a 2 (1)

x 2 = y (1) a1 (2) + y (2) a 2 (2) + ..... + y ( N ) a N (2)

tc:
...

...

...

...

a N ( N )
x N
a1 ( N )
a 2 ( N )

tng ng: xi = y(1)a1(i) + y(2)a2(i) ++ y(N)aN(i)

Nhn xt: ai , i [1..N ] l cc vc t c s ca php bin i.

51

Phng php Karhunen Loeve :

Cn tm php bin i A tha mn 2 mc ch:
-

gim s chiu ca khng gian c trng nhng vn gi c cc c trng chnh

cc tiu ha vic mt mt thng tin.

Cn to sinh cc y(i) c c tnh khng tng quan ( loi bt d tha thng tin ca
vct c trng):
E[y(i)y(j)]=0 vi i # j
y (1) y (1) y (1) y (2) y (1) y ( N )
y (2) y (1) y (2) y (2) y (2) y ( N )

R y = E[ yy T ] = E[ AT xx T A] = AT E[ xx T ] A = AT R x A =
................................................

y ( N ) y (1) y ( N ) y (2) .. y ( N ) y ( N )

Rx l ma trn i xng => tn ti cc vct ring i mt trc giao.

Nu ma trn A c xc nh sao cho cc ct ca ma trn l cc cc vct ring ca Rx,
th Ry = ATRxA cho. Trong cc phn t trn ng cho ca Ry l cc gi tr ring
ca ma trn Rx.
A.3.3 Gim s chiu ca khng gian c trng:
N

x = y (i ) a i ,
i =1

y i = a iT ( x)

Xp x x trong khng gian con M chiu (Mean Square Error Approximationn MSE) :
M

x = y (i )ai M << N
i =1

nh gi sai s:
2

E[ x x ] = E[

y(i)ai ] =

i = M +1

aiT E[ xx T ]ai =

i = M +1

52

aiT R x ai =

i = M +1

i = M +1

Kt lun: nu chn M vc t ring tng ng vi M gi tr ring ln nht ca

2

R x = E[ xx T ]

E[ x x ]

th sai s

l b nht, v xp x x bi

x = y (i )ai
i =1

Hnh A.3.3 - 1 Hng ca vc t ring

Phng php phn tch thnh phn chnh chiu d liu theo chiu bin i nhiu nht.
A.3.4 p dng phn tch thnh phn chnh trong rt trch c trng nh:
Gi s tp nh mu gm M nh: T1, T2, ..., TM.
K hiu vc t nh Ti c s chiu N = R (row) x C (column):
TiT = [ Ti(1) Ti(2) .... Ti(N) ] (Ti l mt vct ct)
t ATB l nh trung bnh ca tp nh mu:

1
ATB =
M

T
i =1

Xt tp nh lch so vi nh trung bnh:

X i = Ti ATB

Xt php bin i tuyn tnh A chuyn X thnh Y:

Y = AT X (A c kch thc N x N)

Cn tm A sao cho Y gi c cc c trng chnh (cc c trng khng tng quan

loi bt d tha thng tin ca vct c trng).

53

Xt ma trn vung NxN : Cy = Y YT

trong :
Y1 (1) Y2 (1) ... YM (1)
Y (2) Y (2) ... Y (2)
2
M

Y = 1
.......................................

Y1 ( N ) Y2 ( N ) ... YM ( N )

CY = AT XX T A = AT C X A
CX i xng {a i , i [1...N ]} l h vc t c s trc giao i mt.
Nu xc nh A vi ct l vc t ring ai ca CX th CY c cho ho.
Qua php bin i A, ta c cc c trng khng tng quan:
N

Y (i)Y ( j ) = 0,
k =1

Ch :

i j

CX = XXT c kch thc N x N, nn thay v tm gi tr ring ca CX, ta tm

cc gi tr ring ca CX = XT X (vi kch thc M x M), gim chi ph tnh ton (trong
trng hp s nh trong tp hc << s pixel ca nh, M << N).
Gi F v ln lt l vc t ring v gi tr ring ca ma trn CX = XT.X ta c:
XTX F = F
Nhn 2 v vi X, ta c:
XXTXF = X F = XF
Nh vy XF l vc t ring ca ma trn CX = XXT tng ng vi tr ring .Vy tm
vc t ring ca ma trn CX = XXT, ta ch cn tm M vc t ring F ca ma trn Cx =
XTX, sau F = XF, vi F l vct ring ca CX = XXT.
C th gi li ton b M vct ring hoc c th chn K vc t ring tng ng vi K
gi tr ring ln nht ca CX lm vc t c s xp x nh Xi (K<< N) lm cc tiu sai
s.

54

Tm li, cc bc cn thc hin:

Bc 1: Tnh vc t trung bnh ca tp mu:
=

1
M

k =1

Bc 2: Tnh khong cch t mi nh ti nh trung bnh:

i = i
Bc 3: To ma trn hip phng sai:

1 M
C=
n Tn = A. AT

M 1 n=1
Vi

A = [ 1 2 .... M ]

Bc 4: Tnh cc gi tr ring ca ma trn C:

1 < 2 < .... < N

Bc 5: Tnh cc vc t ring u k tng ng vi gi tr ring k

k =

1
M

(u
i =1

T
k

i )2

Bc 6: Gim s chiu: ch gi li nhng vc t ring c ngha (tng ng vi cc gi

tr ring ln nht).

y1 u1T
y T
2 = u 2 ( ) = U T ( )
... ...
T
y K u K
Trong K l s chiu trong khng gian c trng (K<<N)
V d minh ha cch rt gn vc t bng phng php phn tch thnh phn chnh [25]:

55

Gi s tp d liu c 10 mu, mi mu l vc t 2 chiu (x, y) nh sau:

x
2.5
0.5
2.2
1.9
3.1
2.3
2
1
1.5
1.1

y
2.4
0.7
2.9
2.2
3.0
2.7
1.6
1.1
1.6
0.9

Bc 1: Tnh vc t trung bnh ca tp mu:

10

x = xi = 1.81
i =1
10

y = y i = 1.91
i =1

Bc 2: Tnh khong cch t mi mu ti mu trung bnh: Ly cc gi tr trn mi chiu

tr cho gi tr trung bnh ca chiu trn tp mu:
x
.69
-1.31
.39
.09
1.29
.49
.19
-.81
-.31
-.71

y
.49
-1.21
.99
.29
1.09
.79
-.31
-.81
-.31
-1.01

Bc 3: Tnh ma trn hip phng sai:

56

D liu a vo l d liu 2 chiu. V vy ma trn hip phng sai c kch thc 2x2.
Ma trn hip phng sai c tnh bng cng thc nh bc 3 nu phn trn:
.616555556 .615444444

cov =
.615444444 .716555556

Bc 4: Tnh cc gi tr ring ca ma trn hip phng sai. Cc gi tr ring trong v d

ny:
.490833989

eigenvalues =
1.28402771

Bc 5: Tnh cc vc t ring ca ma trn hip phng sai:

.735178656
eigenvectors =
.677873399

.677873399

.735178656

Bc 6: Gim s chiu: Ch gi li cc vc t ring c ngha tng ng vi cc gi tr

ring ln nht. Trong v d ny c th ly ton b ma trn eigenvc trs hoc ch n
gin ly 1 eigenvector c tr ring ln nht.
A.4 Phng php Phn tch thnh phn c lp (Independent Component
Analysis - ICA)[10][15][20][21][22][23]
A.4.1 Gii thiu tng quan:
Mt vn nn tng trong phn tch d liu, x l tn hiu, mng nron l tm mt biu
din ph hp ca d liu, v d nh vct ngu nhin. tnh ton v quan nim n
gin, cc biu din thng nhm n mt bin i tuyn tnh ca d liu gc, theo mt
cch khc, mi thnh phn ca s biu din l s t hp tuyn tnh ca cc bin ban u.
Cc php bin i tuyn tnh ni ting bao gm, phn tch thnh phn chnh (PCA), phn
tch tha s (factor analysis) Phn tch thnh phn c lp (ICA) l mt phng php
bin i c pht trin gn y, m mc ch l tm kim mt s biu din tuyn tnh
ca d cc d liu khng c phn phi gauss cc thnh phn l c lp thng k hoc

57

c lp nht c th. S biu din ny gn nh gi li cu trc cn thit ca d liu trong

nhiu ng dng, trong c ng dng rt trch c trng v tch tn hiu.
A.4.2. ng lc thc y s pht trin ca ICA:
Bi ton cocktail party: Hy tng tng bn ang trong mt cn phng c hai ngi
ang ni chuyn cng lc. Cn phng ny c hai microphone c t hai v tr khc
nhau (hnh A.4.2 - 1). Cc microphone cho ra hai tn hiu thu li
theo thi gian l x1(t), x2(t) c biu din di dng:
x1(t) = a11s1(t) + a12s2(t)

(A.4.2_1)

x2(t) = a21s1(t) + a22s2(t)

(A.4.2_2)

trong aij, vi i, j = 1,2 l cc h s trn ph thuc vo nhiu yu t nh hng t

microphone, khong cch t ngi ni n micro, phn m ca phng . Trong m hnh
ICA cc tn hiu xi(t) v cc tn hiu ngun si(t) c xem nh l cc bin ngu nhin,
thay v l cc tn hiu thi gian thc s, nn trong cch vit ch s thi gian t c b i:

Hnh A.4.2 - 1: Hai tn hiu ngun ( khng quan st trc tip c, tc cc tnh hiu n l s1(t) v s2(t)),
hai tn hiu trn (quan st c l x1(t) v x2(t) ).

58

Bi ton c t ra l chng ta phi xp x cc tn hiu ting ni ban u s i(t) m ch s

dng cc tn hiu c thu li xi(t). n gin bi ton, ta s b qua cc tr v thi
gian cng nh cc yu t khc (nhiu).
Tht ra nu chng ta bit cc tham s aij ta c th gii cc phng trnh tuyn tnh
(A.4.2_1), (A.4.2_2) trn. Tuy nhin chng ta hon ton khng bit cc tham s aij v
vy bi ton s kh hn rt nhiu.
Mt cch tip cn gii bi ton ny l s dng thng tin trn cc tnh cht thng k
ca cc tn hiu sij c lng cc aij. Tht ra, ta hon ton c th gi s rng cc tn
hiu s1(t), s2(t) l c lp thng k mi thi im t.
ICA ban u c pht trin gii quyt cc bi ton tng t nh bi ton cocktailparty. Gn y, s quan tm n ICA ngy cng tng, ICA ngy cng c nhiu ng dng
trong nhiu lnh vc khc nhau.
V d cc tn hiu in no thu c bi mt in no (electroencephalogram - EEG).
D liu EEG cha bn ghi ca cc tn hiu in th thuc nhiu vng khc nhau trn da
u. Cc tn hiu in th ny c l c sinh ra bi s trn ln cc tn hiu ca no. Bi
ton t ra tng t nh bi ton cocktail-party. Chng ta mun tm cc thnh phn ban
u ca hot ng no b nhng chng ta ch quan st c cc tn hiu trn ln ca cc
thnh phn . ICA c th tm ra cc thng tin hu dng v hot ng ca b no bng
cch truy xut vo cc thnh phn c lp ca n.
A.4.3. M hnh ICA c bn:
Xt trng hp c n tn hiu trn xi c tng hp t n ngun si th:
xi = ai1s1 + ai2s2 + + ainsn , i = 1, n

(A.4.3_1)

trong aij, i, j = 1, n l cc h s trn gi s c tr thc. Ta gi s l s ngun si.

v s tn hiu trn quan st c xi l bng nhau c n gin. Bi ton ICA l c
lng cc tn hiu ngun si t cc tn hiu trn quan st c xi trong khi cc h s aij
cha bit.

59

K hiu: gi x l vc t ngu nhin biu din cc phn t trngi x l vc t ngu nhin

biu din cc phn t trn xi :x = [x1, x2,, xn ]T, s l vc t ngu nhin biu din cc phn
t ngun si : s = [s1, s2,, sn ]T, A l ma trn trn cha cc h s trn aij :
A = [aij]. Cc vc t x, s c hiu l cc vc t ct.

Hnh A.4.3 - 1: Cc k hiu trong ICA

Nh trn gi s s ngun v s tn hiu trn l bng nhau th A l ma trn vung. M

hnh ICA tr thnh:
x = As.

(A.4.A_2)

Phng trnh (A.4.3_1) l phng trnh ICA c bn. Khi cn biu din dng ct ca ma
trn A, gi cc ct l ai ta c:
n

x = ai si

(A.4.3_3)

i =1

T (A.4.3_2) c th gii ra s = A-1x nu bit c A v A kh nghch, nhng thc t ta

cha bit A.

60

y cng l bi ton tch ngun m BSS (Blind Source Seperation), mt ngun y c

ngha l mt tn hiu ban u chng hn nh mt ngi ni (mt thnh phn c lp )
trong bi ton coctaikparty. M c ngha l ta cha bit thng tin g v ma trn trn A.
ICA c l l mt phng php c s dng rng ri nht cho bi ton BSS.
Vy ta c nh ngha cho ICA nh sau: nh ngha A.1
Cho s l cc ngun vo ban u, x l cc hn hp, A l ma trn trn ln vi cc h s a ij
(i = 1,n, j = 1,m). M hnh trn ln ca ICA c dng x = As + n, vi n l mt vct
nhiu ngu nhin. Cc thnh phn si c th c ti to li khi chng l c lp thng k
v c phn b phi Gauss.
Tuy nhin vic c lng m hnh trn khng n gin, v vy phn ln vic nghin cu
ICA c gi s l b qua vc t nhiu n, v A l mt ma trn vung.
A.4.4.Cc bt nh trong m hnh ICA:
Trong m hnh ICA c bn (A.4.3_2) v (A.4.3_3) chng ta d dng nhn thy cc bt
nh sau:
1. Khng th xc nh phng sai (nng lng ca cc thnh phn c lp)
L do l c A v s u cha bit v vy ta c th iu chnh ln ca cc thnh phn
c lp:
x = (

1
a i )( i s i )
i

vi i l mt i lng bt k. Do cc thnh phn c lp si l cc bin ngu nhin, cc

n gin nht l gi s chng c phng sai n v : E{si2} = 1, cn mt bt nh na l
v du, v ta c th nhn mi thnh phn c lp si vi -1 m khng nh hng ti m
hnh, may mn l im bt nh v du ny khng nh hng ln n hu ht cc ng
dng.
2. Khng th xc nh th t ca cc thnh phn c lp khi phn ly v c A v s
u cha bit nn ta c th thay i th t ca cc cc s hng trong (A.4.3_3) nn bt
c thnh phn c lp no cng c th l thnh phn u tin.

61

A.4.5. Cc gi s trong m hnh ICA:

Trong m hnh ICA c bn nu trn, phn ly c cc ngun ta phi gi s mt s
iu sau:
1. Cc ngun si l c lp thng k.
2. Khng c thnh phn c lp no (ngun) c phn b Gauss. (v ICA cn cc
thng k bc cao hn hai c lng cc thnh phn c lp)
3. Ma trn trn A l mt ma trn vung. Gi s ny c cp trong nh
ngha A.1. Gi s ny lm cho bi ton n gin hn rt nhiu. Khi , sau khi xp x
c ma trn A, ta c th xc nh cc thnh phn c lp bng cch:
s = A-1 x (gi s A kh nghch).
4. Cc thnh bin trn ln x v cc thnh phn c lp s c tr trung bnh khng.
A.4.6. Minh ha ICA:
Xt hai thnh phn c lp s1 v s2 c phn b ng nht sau:
1 / 2 3
if | si |<= 3
p( si ) =
otherwise
0

Min gi tr ca phn b ny c chn sao cho k vng bng khng v phng sai n
v. Khi mt kt hp ca s1 v s2 l ng nht trn mt hnh vung, c minh ha
trong hnh A.4.6 - 1

62

Hnh A.4.6 - 1: Phn b kt hp ca hai thnh phn c lp s1, s2 c phn b ng nht (trc ngang: s1,
trc ng s2)

By gi ta trn ln hai thnh phn c lp ny bng ma trn trn sau:

2
A0 =
2

Kt qu ta c hai bin trn ln l x1 v x2, v chng c phn b ng nht trn mt hnh

bnh hnh, minh ha trong hnh di

Hnh A.4.6 - 2: Phn b kt hp ca cc trn ln x1, x2 (trc ngang x1, trc ng x2)

thy rng ti sao cc bin ngu nhin c phn phi Gauss l khng th trong m hnh
ICA, gi s s1 v s2 c phn b Gauss v ma trn A trc giao, nh vy x1, x2 cng c phn
phi Gauss, v mt kt hp ca chng c dng:

p ( x1 , x 2 ) =

x 2 + x 22
1
exp( 1
)
2
2

63

S phn phi c minh ha trong hnh A.4.6 - 3

Hnh A.4.6 - 3: Phn phi kt hp ca hai bin Gauss.

Ta thy mt l i xng theo gc quay, do n khng cha bt k thng tin no v

hng cc ct ca m trn trn ln A, l l do ti sao ma trn trn ln khng th xp x
c.
A.4.7. Cc nguyn tc chnh ca c lng ICA:
A.4.7.1 Phi Gauss l c lp:
Nh ni trn, cha kha c lng m hnh ICA l tnh phi Gauss [10], theo
nh l gii hn trung tm mt kt qu kinh in trong l thuyt xc sut
cho bit phn b ca mt tng cc bin ngu nhin c lp c khuynh hng tin ti
phn b Gauss di mt s iu kin no , do tng ca hai bin ngu nhin c lp
thng c phn b gn phn b Gauss hn hai bin ngu nhin thnh phn.
By gi ta gi s rng vct d liu x c phn b theo m hnh d liu ICA nh trong
phng trnh (A.4.3_2), ngha l n l mt hn hp ca cc thnh phn c lp, n
gin ta gi s rng tt c cc thnh phn c lp c cng phn phi, c lng mt
thnh phn c lp ta xt mt t hp tuyn tnh ca xi, t
y = wTx =

wx
i

(wT = (w1, w2,, wn))

64

(A.4.7.1_1)

vi w l mt vc t c xc nh, nu wT l mt trong cc hng ca ma trn ngch

o ca A th t hp tuyn tnh ny s bng mt trong cc thnh phn c lp. Cu hi
t ra by gi l ta phi p dng nh l gii hn trung tm nh th no xc nh w T
sao cho n l mt hng ca ma trn nghch o ca A (hay i tm w), thc t chng ta
khng th xc nh w mt cch chnh xc v chng ta khng bit ma trn A, nhng chng
ta c th tm c lng ca A.
t z = ATw, ta c y = wTx = wTAs = zTs (v (AB)T = BTAT)
Do y l mt t hp tuyn tnh ca cc si vi cc trng s l zi. V tng ca hai bin
ngu nhin c lp c phn phi Gauss hn cc bin thnh phn, zTs th c phn phi
Gauss hn mi si , v tr thnh t gauss nht khi n bng chnh mt trong cc si, trong
trng hp , r rng ch c mt trong cc thnh phn zi ca z l khc khng.
V vy chng ta c th chn w nh mt vc t cc i tnh phi Gauss ca w Tx, nh
th mt vc t s tng ng vi z m ch c mt thnh phn khc khng, iu c
ngha l wTx = zTs bng mt trong cc thnh phn c lp. Cc i tnh phi Gauss ca
wTx cho ta mt trong cc thnh phn c lp.
A.4.7.2 Cc o tnh phi Gauss:
s dng tnh phi gauss trong c lng ICA, chng ta cn phi c mt o tnh phi
gauss ca bin ngu nhin y, n gin gi s rng y c trung bnh khng v c
phng sai n v.
A.4.7.2.1 Kurtosis:
Php o c in ca tnh phi Gauss l kurtosis hay tch ly bc bn, kurtosis ca y c
nh ngha:
Kurt(y) = E{y4} 3(E{y2})2
V ta gi s y c phng sai n v v trung bnh khng nn v phi tr thnh
E{y4}-3. iu ny cho thy kurtosis l mt phin bn ca moment bc bn E{y4.

65

Vi mt bin gauss y, moment bc bn bng 3(E{y2})2, v vy kurtosis ca mt bin

ngu nhin gauss th bng 0. Phn ln vi cc bin ngu nhin phi gauss kurtosis khc
khng.
Kurtosis c th m hoc dng, vi cc bin ngu nhin c kurtosis m c gi l di
gauss (subgaussian), v c kurtosis dng c gi l siu gauss (supergaussian). Phn
b siu Gauss khng cn dng hnh chung nh Gauss m tng nhanh trung tm tng
t nh phn b Laplace (Hnh A.4.7.2.1 - 1), cn phn b di Gauss khng nh ln
phn gia nh phn b Gauss m tin n phn b u v rt nh cc gi tr xa trung
tm.
Phn b Laplace: p ( y ) =

1
2

exp( 2 | y |)

Hnh A.4.7.2.1 - 1: Hm mt ca phn phi Laplace, mt in hnh ca phn phi siu Gauss, so vi
phn phi Gauss ng gch nt, c hai mt c chun ha phng sai n v.

Kurtosis hay gi tr tuyt i ca n c s dng rng ri nh mt o nh gi

tnh phi gauss trong ICA v cc lnh vc lin quan. L do l n n gin c v l thuyt
ln tnh ton. Trong tnh ton, kurtosis c th c lng mt cch n gin bng cch s
dng moment bc bn ca d liu mu. Phn tch l thuyt ca kurtosis th n gin bi
c tnh tuyn tnh sau, nu x1, x2 l hai bin ngu nhin c lp, ta c:

66

kurt (x1 + x2) = kurt (x1) + kurt (x2)

kurt ( x1) = 4kurt (x1).

Trong v hng.
minh ha cc thnh phn c lp c th c tm thy bng cch s dng cc i
hay cc tiu kurtosis nh th no ta xt v d n gin sau: Xt m hnh hai chiu:
x = As, gi s cc thnh phn c lp s1, s2 c gi tr kurtosis kurt (s1), kurt (s2) khc
khng. Chng ta cng gi s chng c phng sai n v. Chng ta tm kim mt
thnh phn c lp bi:
y = wTx.
t z = ATw, ta c y = wTx = wTAs = zTs = z1s1 + z2s2
Da trn tnh cht cng ca kurtosis ta c:
kurt (y) = kurt (z1s1) + kurt (z2s2) = z14kurt(s1) + z24kurt(s2)
Chng ta gi s y , s1, s2, c phng sai n v v trung bnh khng v vy ta c:
D(y) = D(z1s1 + z2s2)
<=> D{y} = z12 D{s1} + z22 D{s2}
<=> E{y2} = z12 + z22 = 1.
Theo hnh hc iu ny c ngha l vc t z b rng buc trn ng trn n v.
Vn ti u by gi l: vic cc i hm kurt (y) = |z14kurt(s1) + z24kurt(s2)| trn ng
trn n v l g?
Ngi ta ch ra rng (Delfosse and Loubaton, 1995) cc i ca hm l ti cc im
khi chnh xc mt thnh phn ca z khc khng v cc thnh phn cn li bng khng,
bi v z b rng buc trn ng trn n v nn thnh phn khc khng ca z bng +1
hoc -1, v do y bng si (tng ng thnh phn zi khc khng).
Trn thc t, chng ta s bt u t mt vi vc t trng s w, tnh ton theo hng m
kurtosis ca y = wTx l tng mnh nht (nu kurtosis l dng) hoc l gim mnh nht
(nu kurtosis l m), v dng phng php gradient hay mt phng php m rng khc
ca gradient tm mt vc t w mi.

67

Tuy nhin, vic o tnh phi Gauss bng kurtosis c mt vi bt li khi cc gi tr ca n

c tnh t cc mu quan st c, v kurtosis rt b nh hng bi cc gi tr bin
(outlier) quan st c hai ui ca phn b.
A.4.7.2.2 Negentropy:
Mt s o tnh phi Gauss quan trng hn l negentropy, Negentropy l i lng da
trn s sai bit ca l thuyt thng tin . Entorpy l mt nh ngha c bn ca l thuyt
thng tin, Entropy ca mt bin ngu nhin c th c xem nh mc thng tin m
cc bin mang li. Cc bin cng ngu nhin tc l cng khng c cu trc v cng
khng th d on th Entropy cng ln.
nh ngha Entropy ca mt bin ngu nhin ri rc Y:
H(Y) = - P (Y = a i ) log P (Y = ai )
i

Vi ai l cc gi tr c th ca Y.
Entropy H ca ca vc t ngu nhin Y vi hm mt f(y) c nh ngha:
H(Y) = - f ( y ) log f ( y )dy

Mt kt qu nn tng ca l thuyt thng tin l: mt bin Gauss c entropy l ln nht

trong tt c cc bin ngu nhin c cng phng sai. iu ny c ngha l entropy c
th c s dng nh mt o tnh phi Gauss.
c mt i lng o tnh phi gauss, vi cc bin gauss th i lng ny bng khng
v i lng ny lun khng m, ngi ta nh ngha negentropy ca vc t ngu nhin y
nh sau:
J(y) = H(ygauss) H (y)
Trong ygauss l mt bin ngu nhin Gauss c cng ma trn hip phng sai vi y.
Vi cc c tnh cp trn, negentropy th lun lun khng m v n bng khng khi
v ch khi y c phn phi Gauss. Mt thun li khi s dng negentropy nh mt nh gi
tnh phi Gauss l n s dng rt tt trong l thuyt thng k, tuy nhin vic tnh ton
negetropy li rt kh khn, do cc xp x n gin hn ca negetropy th rt cn thit.

68

A.4.7.2.2.1 Cc xp x ca negentropy:
Mt s tnh ton xp x c pht trin, mt trong s nhng xp x tt nht l :
J(y) [E{G(y)} E{G(ygauss)}] 2

(A.4.7.2.2.1_1)

Ta thy xp x ny lun dng v bng khng nu y c phn phi Gauss.

Hm phi tuyn G(.) c th chn mt trong hai biu thc sau:
G1(y) =

1
log cosh (a1y)
ai

G2(y) = -exp(-y2/2).
Trong 1<= a1<= 2 v thng c chn bng 1.
A.4.8. Tin x l cho ICA:
Trc khi p dng thut ton ICA, d liu thng c tin x l bi ton c lng
ICA n gin hn v c cc iu kin tt hn.
A.4.8.1 Qui tm:
Tinn x l c bn v cn thit nht l qui tm x, c ngha l tr i vc t trung bnh
E{x} x l bin trung bnh khng.
Gi x l cc hn hp ban u, gi d liu sau khi qui tm l x ta c:
x = x E{x}
do cc thnh phn c lp cng c trung bnh khng v:
x = As
=> E{x} = AE{s} = 0
=>

E{s} = 0.

Sau khi c lng c ma trn trn A v cc thnh phn c lp s vi d liu c

qui tm, chng ta c th khi phc li tr trung bnh b i ca d liu bng cch cng
thm c lng A-1E{x} vo cc thnh phn c lp (c tr trung bnh khng).

69

A.4.8.2 Trng ha:

Mt tin x l hu dng khc cho thut ton ICA l lm trng cc bin quan st (sau khi
qui tm), iu ny c ngha l trc khi p dng thut ton ICA (v sau khi qui tm),
~
~
chng ta bin i tuyn tnh vc t quan st x c mt vc t mi l x v x trng

ngha l cc thnh phn ca n bt tng quan v cc phng sai ca cc thnh phn l

~
n v. Hay ni cch khc ma trn hip phng sai ca ca x bng ma trn n v.
~~
E{ x x T} = I (ma trn n v)

S trng ha l lun lun c th, mt trong nhng phng php cho th tc trng ha l
s phn tch tr ring (eigen value decomposition EVD) ca ma trn hip phng sai:
E{xxT} = EDET ( x c tr trung bnh khng). Vi E l ma trn trc giao ca cc
vc t ring ca E{xxT} v D l ma trn cho ca cc gi tr
D = diag(d1,d2,,dn).
S trng ha c th c tnh ton nh sau:
~
x = ED-1/2ETx

(A.4.8.2_1)

Trong D-1/2 = diag(d1-1/2,d2-1/2,,dn-1/2).

~
S trng ha chuyn i ma trn trn A thnh mt ma trn mi A
, t (A.4.3_2) v

(A.4.8.2_1) ta c:
~
~
x = ED-1/2ETAs = A s

(A.4.8.2_2)
~

Tnh hu dng ca s trng ha l ma trn A l mt ma trn trc giao, ta c th d dng

thy iu ny:
~ ~
~
~ ~
~
~~
E{ x x T} = E{ A s ( A s)T} = A E{ssT} A T = A A T = I

(nn nh rng cc thnh phn si gi s l c phng sai n v v chng cng c

trung bnh khng do qui tm).
~

Vy ma trn lm trng A l mt ma trn trc giao. Tnh trc giao ca ma trn ny lm

gim s tnh ton, v mt ma trn nxn khng trc giao cha n2 t do, cn ma trn trc

70

giao ch cha n(n-1)/2 t do. Vi d liu hai chiu iu ny c ngha l t do ch l

mt cho mt bin i trc giao, vi mt d liu nhiu chiu hn, t do ca mt ma
trn trc giao ch l phn na ca ma trn khng trc giao. Do ngi ta ni lm trng
gii quyt phn na bi ton ICA [ICA thut ton v ng dng]. Bi v trng ha l
mt th tc n gin hn bt k thut ton ICA no nn l mt tin x l thng dng.
~

Sau khi c lng ma trn trn trng A , t (A.4.8.2_2) vic c lng cc thnh
phn c lp s tr thnh:
~

~
s = W x , vi W l ma trn ngch o ca A : W = A -1. Sau khi c c W

(W cng trc giao), vic c lng ma trn gc A cho bi:

Suy ra:

~
A = ED-1/2ETA
~
A = ED1/2ET A = ED1/2ETW-1 = ED1/2ETWT

Nh vy vic cn li l c tnh cc hng ca W. V vic c lng W s c thc

hin bng phng php cc i tnh phi Gauss nh trnh by ti mc A.4.7.
A.4.9. Thut ton FastICA:[16]
Trong phn trc chng ta gii thiu cc o khc nhau ca tnh phi gauss. Trong thc
hnh chng ta cn cc i ha mt hm i tng (objective function) hay cn gi l
hm tng phn (contrast function), v d hm (A.4.7.2.2.1_1), trong phn ny chng ta
s tip cn mt phng php ph bin cho tc v ny, y ta tha nhn l d liu
c lm tin x l l qui tm v trng ha.
A.4.9.1 FastICA cho mt n v (FastICA for one unit):
Gi l mt n v v chng ta tm mt thnh phn c lp. Tng t nh mng mt
nron: tip nhn vct u vo x, c mt vct trng w m neural c th cp nht trong
qu trnh hc v u ra l mt thnh phn c lp. Mc ch ca ta l cc i ha tnh
phi gauss ca wTx, tnh phi gauss y c o bng cch xp x negentropy J(wTx),
nh rng ta gi s wTx c phng sai n v.

71

FastICA da trn im c nh cc i tnh phi gauss ca wTx, vi o phi gauss

trong (A.4.7.2.2.1_1): J(y)

[E{G(y)} E{G(ygauss)}] 2

Vi cc hm G1, G2 cp trong mc A.4.7.2.2.1, ta c o hm ca G1, G2 l:

g1(u) = tanh (a1u)
g2(u) = u exp(-u2/2)
Vi 1<= a1 <= 2, thng chn a1 bng 1. Dng c bn ca thut ton FastICA nh sau:
1.Chn ngu nhin mt vct trng s ban u w.
2. t
3. t

w+ = E{xg(wTx)} E{g(wTx)}w.
w = w+/ || w+||

4. Nu cha hi t, quay li bc 2.
Hi t y c ngha l gi tr mi v gi tr c ca w cng hng, tc l dot-product
(tch v hng) ca chng gn nh bng mt (ch v w c chun ha).
Php ly o hm (derivation) ca FastICA nh sau:
Trc ht rng, t (8.2.2.1_1), xp x cc i negentropy ca wTx chnh l ti u
hm E{G(wTx)}. Theo Kuhn Tucker (Luenberger, 1969), ti u ca hm E{G(wTx)},
vi iu kin E{(wTx)2} = ||w||2 =1 thu c cc im trong :
E{xg(wTx)} - w = 0.
Trong

(A.4.9.1_1)

= E{w0Txg(w0Tx)}, vi w0 l gi tr ca w khi ti u. Chng ta gii

phng trnh (A.4.9.1_1) bng phng php lp Newton. t hm bn v tri l F, chng

ta c ma trn Jacobi ca n:
JF(w ) = E{xxTg(wTx)} - I.

(A.4.9.1_2)

n gin chng ta xp x s hng th nht trong (10_2), bi v x c lm trng,

mt xp x l: E{xxTg(wTx)}

E{xxT}E{g(wTx)} = E{g(wTx)}I. V vy ma trn Jacobi

l mt ma trn cho, v d dng nghch o. Nh vy ta c php lp Newton nh sau:

w+ = w [E{xg(wTx)} - w]/[ E{g(wTx)}- ]. (A.4.9.1_3)

72

Thut ton ny c th n gin hn bng cch nhn hai v ca phng trnh trn vi E{g(wTx)}, n cho php lp ICA.
A.4.9.2 FastICA cho nhiu n v (FastICA for several units):
Thut ton mt n v ch c lng mt trong cc thnh phn c lp. c lng
nhiu thnh phn c lp chng ta cn thut ton s dng nhiu n v (tng t nh cc
neural) vi cc vct trng s w1, w2,. wn. khng cho cc vct khc nhau hi t ti
cc i ging nhau, chng ta phi bt tng quan cc gi tr outputs w1Tx, w2Tx,, wnTx
sau mi vng lp.
Cch n gin thc hin bt tng quan l thc hin qu trnh trc giao ha
Gram-Schmidt, ngha l chng ta c lng cc thnh phn c lp tng thnh phn
mt. Khi chng ta c lng c p thnh phn c lp hay p vct w1,, wp, chng
ta thc hin thut ton mt n v cho wp+1, v sau mi bc lp, chng ta ly wp+1 tr i
cc wp+1Twjwj , j = 1,p, (chnh l ly wp+1 tr i hnh chiu ca n vo khng gian con
Wp), sau ta chun ha wp+1.
p

1. Chn wp+1 = wp+1 -

w
j =1

T
p +1

wjwj

2. Chn wp+1 = wp+1/ wTp +1 w p +1

Trong cc ng dng i khi yu cu gii tng quan i xng, trong khng c vct
no c quyn (privilege), iu ny c th thc hin c theo phng php:
t: W = (WWT)-1/2W
Trong W l ma trn (w1,,wn)T ca cc vect, v (WWT)-1/2 c c t khai trin tr
ring ca WWT = FDFT, vi (WWT)-1/2 = FD-1/2FT. Cch n gin hn l thut ton lp
(Hyv a rinen, 1999a).
1. Chn W = W/ || WW T ||

Lp li bc hai cho n khi hi t:

73

2.

3
2

1
2

Chn W = W WW T W

A.4.10. ICA cho ng dng rt trch c trng: [17][18] [19]

A.4.10.1 T chc d liu:
t X l ma trn d liu vi nr hng v nc ct. Chng ta xem cc ct ca ma trn X l
nh l cc kt qu quan st (cc th nghim c lp) ca cc thc nghim ngu nhin. Ta
xem hng th i ca X nh l cc gi tr quan st c ca bin ngu nhin X i thng qua
nc ln th nghim c lp. Ta c xc sut phn phi ca X1, , Xnr trong mi ct ca X l
1/nc. Ta ni hai hng i v j ca X l c lp nu khng th d on cc gi tr ca X j
thng qua cc ct t cc gi tr tng ng ca Xi.
P(Xi = u,Xj= v) = P(Xi = u)P(Xj = v) for all u,v thuc R.
Mc ch ca chng ta l tm mt tp nh c s tt i din cho tp nh khun mt.
Chng ta sp xp mi nh trong c s d liu nh l mt vect vi s chiu l s pixel
ca nh. Chng ta c hai cch tip cn (hai kin trc):
A.4.10.2 Kin trc 1:
Chng ta sp xp ma trn ma trn d liu X vi mi hng l l mt nh khc nhau, trong
cch tip cn ny cc nh l cc bin ngu nhin v cc pixel l cc quan st, y to
cho chng ta cm gic v s c lp ca cc nh. Hai nh i, j c gi l c lp nu
chng ta khng th d on gi tr ca pixel trn nh j da trn gi tr ca pixel tng
ng ct trn nh i.

74

Hnh A.4.10.2 - 1:Minh ha kin trc 1 trong m hnh ICA

Trong kin trc 1, cc nh u vo X c xem nh l mt s pha trn tuyn tnh ca tp

nh c s c lp thng k S kt hp bi mt ma trn trn A cha xc nh. Thut ton
ICA s xc nh cc trng s ca ma trn W c s dng phc hi li tp nh c
s c lp trong cc hng ca U. Trong kin trc ny s tch ngun c thc hin trn
khng gian cc khun mt. S chiu ca tp nh u vo xung b vect hc c W s
to ra cc nh c s c lp. S biu din nn ca mt nh khun mt (compressed
representation) hoc l s m khun mt (code image) l mt vct cc h s c s
dng cho s kt hp tuyn tnh ca tp cc nh c s c lp sinh ra nh (cc nh
trong X).
Hng gia trong hnh di ch ra 8 nh c s sinh ra bi kin trc 1, chng xc nh
khng gian khun mt khng ging nh PCA (hng trn) v ICA kin trc 2 (hng di).
Barlett [18,19] trc tin p dng PCA chiu d liu xung mt khng gian con m
chiu iu khin s thnh phn c lp xc nh bi ICA. Mt trong cc thut ton
ICA c p dng ln cc vect ring cc tiu ha s ph thuc thng k gia cc
nh c s kt qu. Nh vy PCA bt tng quan d liu u vo, cc s ph thuc bc
cao cn li c tch ra bi ICA.

75

Hnh A.4.10.2 - 2: vct c trng cho mi k thut. Hng u cha 8 vct ring vi 8 tr ring ln
nht trong PCA. Hng 2 cha cc vect c trng trong ICA vi kin trc I, hng 3 ch ra 8 vct c trng
trong ICA vi kin trc 2.

M t ton hc cho kin trc I:

t R l ma trn p*m cha m vct ring u tin ca tp n nh khun mt (trong PCA).
t p l s lng pixel trong mt nh hun luyn. Cc hng ca ma trn d liu u vo
ICA l cc bin ngu nhin v cc ct l cc s quan st v vy ICA c thc hin trn
RT ,m nh c s c lp trong cc hng ca U c tnh ton bi U = W*RT. Sau ma
trn B n*m cc h s ICA cho s kt hp tuyn tnh ca tp nh c s c lp trong U
c tnh ton nh sau:
t C = X * R (C kch thc n*m) nh vy X = C * RT (R trc giao).
T U = W*RT v tha nhn rng W kh nghch ta c
RT = W-1 * U. V vy X = C * RT = C * W-1 * U = B * U.
Mi hng ca B cha cc h s cho s kt hp tuyn tnh cc nh c s to nn cc
nh khun mt tng ng cc hng trong X. V vy X l s xy dng li ca d liu ban
u vi bnh phng li nh nht nh trong PCA
Minh ha:

76

Hnh A.4.10.2 - 3:M hnh tng hp nh cho kin trc 1 ca ICA

tm tp cc nh c lp, cc nh trong X c xem nh l s kt hp tuyn tnh ca

tp nh c s c lp thng k S, vi A l ma trn trn cha bit, tp nh c s c c
lng qua ICA l tp output U.

A.4.10.3 Kin trc 2:

t cc nh thnh cc ct ca ma trn d liu X. Trong cch sp xp ny, cc pixel xem
nh cc bin ngu nhin, s lng pixel ca mt nh cng chnh bng s micro, bng s
thnh phn c lp, cc nh l cc s quan st. y to cho chng ta cm gic ni v
s c lp ca cc pixel. V d pixel i va j c gi l c lp nu chng ta khng th d
on gi tr ca ca pixel i da trn gi tr ca pixel j tng ng trn cng mt nh. Cch
tip cn ny c thc hin bi Bell v Sejnowskis trn cc thnh phn c lp ca
nh t nhin.
Ci ch ca kin trc 1 l s dng ICA tm mt khng gian cc nh c s c lp.
Mc d cc nh c s tm thy trong kin trc 1 l xp x c lp, nhng cc h s m
cc nh th khng c lp (cc h s tng trng biu din cho cc nh input trong
khng gian con nh ngha bi cc nh c s th khng c lp). Kin trc 2 s dung

77

ICA tm mt s biu din m cc h s s dng m cc nh l c lp thng k

(tm s c lp thng k cc h s cho d liu input).
S tch ngun c thc hin trn cc pixel v mi hng ca ma trn trng s W l mt
nh. A, ma trn nghch o ca W, cha cc nh c s trong cc ct ca n. S c lp
thng k cc h s ngun trong S m cu thnh nn cc nh input th c khi phc
trong cc ct ca U = W * X, tc l mi ct ca U cha cc h s ca cc nh c s
trong A cho vic xy dng li mi nh trong X.
(Chng ta tm cc thnh phn c lp U, vi U = W * X. Mi ct ca U cha cc thnh
phn ca tp nh c s trong A cho vic xy dng li mi nh trong X. ICA c gng to
ra tp output U l c lp nht c th).

Hnh A.4.10.3 - 1:minh ha kin trc 2 trong m hnh ICA

Trong cch tip cn ny, s lng pixel ca mi nh l bng nhau v bng s thnh phn
c lp, nh vy nu kch thc nh qu ln, ta dng php phn tch thnh phn chnh
PCA lm gim chiu ca nh, sau a kt qu ny qua ICA tm cc thnh phn
c lp ca tp d liu.
Vi cch k hiu nh trong 11.2 cc h s c lp thng k c tnh ton:
U = W * CT v tp nh c s cha trong hnh A thu c t cc ct ca R*A
Minh ha ICA cho kin trc 2:

78

Hnh A.4.10.3 - 2:M hnh tng hp nh cho kin trc 2 ca ICA

Hnh A.4.10.3 - 3:M hnh tng hp nh cho kin trc 2 ca ICA trn pixel

S l cc ngun c lp cha bit. Ma trn A xem nh l tp nh c s. Mi nh trong tp

d liu X c xem nh l t hp tuyn tnh ca tp nh c s trong ma trn A. Tp nh

79

c s c kt hp vi mt tp cc ngun c lp S. A = WI-1 vi WI c c lng

thng qua ICA.

U cha cc h s c lp cho s kt hp tuyn tnh ca cc nh c s trong A cu

thnh nn mi nh trong X
A.4.11 Cc so snh v mt l thuyt gia ICA v PCA [18]:
A.4.11.1 So snh PCA v ICA:
-

Ci ch ca PCA l tm mt tp nh c s tt hn, trong tp nh c s mi

ny s phi hp cc nh l khng tng quan (cc thnh phn chnh l khng tng
quan), v tm tp d liu mi khng tng quan, PCA s dng thng k bc hai
(ma trn hip phng sai). V vy cc s ph thuc thng k bc cao vn cn tn ti
trong php phn tch PCA.
Trong cc cng vic nh nhn dng mt ngi, nhiu thng tin quan trng c th c
cha trong mi quan h thng k bc cao gia cc pixel ca nh, khng ch l thng k
bc hai, nh PCA. V vy chng ta cn tm mt phng php tng qut hn PCA, ICA
(independent component analysis) l mt phng php.
-

ICA c p dng thnh cng trong bi ton tch ngun m (cock tail party

problem), tch tn hiu in no (Electroencephalo gram EEG).

-

PCA c th c xem nh mt trng hp c bit ca ICA khi cc ngun c

phn phi Gauss, trong trng hp ny th ma trn trn khng xc nh c ( trnh

by trong nh ngha ICA).
Phng php PCA cha phi l mt phng php tt trong trng hp cc ngun c
phn phi phi Gauss. Theo kinh nghim quan st, ngi ta xc nh c rng nhiu
tn hiu t nhin nh m thanh, nh t nhin, v EEG l s t hp tuyn tnh ca cc

80

ngun vi phn phi siu Gauss (kurtosis dng), trong trng hp ny, ICA l mt
phng php tt hn PCA v:
1.

Cung cp mt m hnh xc sut tt hn ca d liu.

2.

N xc nh duy nht ma trn trn.

3.

N tm thy mt c s khng cn thit trc giao m c th xy dng li d liu tt

hn PCA.
4.

N p dng cc thng k bc cao trong d liu khng ch l ma trn hip phng

sai nh PCA.
5.

ICA tt hn PCA trong mi trng nhiu chng hn nh sng bin i, cc

bin i cm xc trn khun mt, trang im trn khun mt.

Hnh A.4.11.1 - 2 ch ra cc mu trong mt khng gian ba chiu, c xy dng bi s
kt hp tuyn tnh ca hai ngun c phn phi siu Gauss.
Cc vector c s ca PCA v ICA cng c xc nh, bi v ba vect c s ca ICA l
khng trc giao nn chng s thay i khong cch gia cc im d liu, v nu cc
vecto l trc giao th khi chiu d liu xun g khng gian trc giao mi ny th khong
cch gia cc im d liu khng thay i (PCA). iu ny c th hu dng cho cc
thut ton phn lp, ging nh ngi lng ging gn nht, lut quyt nh da trn
khong cch gia cc im d liu.

Hnh A.4.11.1- 2. S phn phi d liu trong khng gian 3 chiu v cc trc tng ng ca PCA v
ICA. Mi trc l mt ct ca ma trn nghch o ca ma trn trn W-1 tm thy bi PCA v ICA. Cc trc
ca PCA trc giao trong khi ICA th khng, do khong cch cc im d liu s thay i khi chiu xung
khng gian mi ny.

81

A.5 Support vetor Machine (SVM) [1][2][3][24]

Mc ch ca cc gii thut SVM l tm mt siu phng (hyperplane) ti u, cho php
chia tp cc im trong khng gian n chiu thnh 2 phn sao cho cc im cng mt lp
nm trong cng mt phn
A.5.1. SVM tuyn tnh
Ta bt u vi trng hp n gin nht: my hc c hun luyn trn b d
liu c th phn loi tuyn tnh.
B hc S = {xi, yi}, i = 1,...,n.

Trong y i { -1,1}, x i Rn.

Mc ch ca SVM l tm ra siu mt phn cch tha :

wTxi + b +1 vi yi = +1

(1)

wTxi + b -1

(2)

vi yi = -1

Vi w Rn l vector php tuyn ca siu mt phn cch.

Kt hp (1) v (2) thnh mt bt ng thc rng buc:
yi(wTxi + b) 1 , i = 1,2,..,n

(3)

Nu tt c cc thnh phn ca tp hc S tha bt phng trnh (3) th ta ni y l

trng hp phn cch tuyn tnh.

Hnh A.5.1 - 1 : Siu mt phn cch tuyn tnh cho trng hp phn cch c v
k hiu cc support vc tr chnh l cc im c bao bng vin trn

Vector h tr l cc vector xi tha phng trnh : yi(wTxi + b) 1 = 0.

82

Trong (Hnh1) vector h tr l cc im c khoanh trn, v ta cng thy c 2 siu

mt phng :
H1 : wTxi + b = 1 c khong cch n gc ta l |1-b| / ||w||
H2 : wTxi + b = -1 c khong cch n gc ta l |-1-b| / ||w||
(||w|| l ln Euclide ca w)
H1 v H2 u c vector php tuyn l w hay H1 v H2 song song vi nhau. Ta c
c khong cch b gia 2 lp : = 2/||w|| .
Nh vy trong vic hun luyn siu mt, SVM tm cch cc i . Do trong
trng hp tuyn tnh, vic tm siu mt phn cch cng l tm :
Cc tiu : (w) = ||w||2/2

(4)

Vi rng buc : yi(wTxi + b) 1 0 , i = 1,2,..,n

Nu bt phng trnh (3) khng ng vi mt s phn t trong tp hc ta gi y
khng phi l trng hp phn cch tuyn tnh v phn t l phn t li. gii
quyt trng hp ny, tp khng m { i }ni = 1 c thm vo, rng buc tr thnh :
yi(wTxi + b) 1 - i , i >0, i =1,2,,n

(5)

i gi l bin lng (slack). Trng hp i >1 l do im b li nm bn kia mt

phn cch, trng hp 0< i <1 l do im b li cha vt qua vch phn cch.

83

Do mc ch ca SVM by gi tm siu phng phn cch sao cho li phn loi

l thp nht v khong cch b l ln nht trong trng hp khng th phn cch tuyn
tnh bi ton tr thnh :
Cc tiu : (w,) = 21 ||w||2 + C

: yi(wTxi + b) 1 0 , i = 1,2,..,n

Vi iu kin
v

n
i=1 i

:0

(6)
(7)
(8)

Vi C l tham s do ngi dng chn la cn bng phc tp gia h v t l

li, C cng ln t l li cng thp.
Mt cch gii quyt bi ton l dng hm Largange. C hai l do cho iu
ny. Th nht l rng buc (3) s c th bng rng buc trn h s nhn Lagrange,
d lm vic hn. Th hai l d liu hun luyn s ch xut hin di dng php nhn v
hng gia cc vector. iu ny cho php tng qut ho cho trng hp phi tuyn. a
ra cc h s nhn Lagrange khng m i, v i , cho rng buc bt ng thc (3). Bi
ton i ngu tr thnh :
Cc i

Q() = ni=1i - 21 n i=1 nj=1i j yi yj xTi xj

Vi rng buc

(9)

ni=1i yi = 0

(10)
v

0 i C ,

i = 1,2,,n

(11)

Trong bi ton i ngu, hm lng gi Q() ch ph thuc vo d liu di dng

tch v hng {xTi xj }n(i,j)=1. Do cc h s nhn Lagrange c th c tnh da trn cc
rng buc (10-11).
iu kin Karush-Kuhn-Tucker (KKT) c vai tr quan trng i vi bi ton ti
u rng buc. Vi bi ton trn, iu kin KKT c th pht biu :
i [yi(wTxi + b) 1 + i ] = 0 , i=1,2,,n

(12)

i i = 0 , i=1,2,,n

(13)

trong i = C - i

84

C 2 dng tham s i, nu nh 0 < i C th mu tng ng l vector h tr.

Vector li gii cho bi ton chnh l vector xc nh bi :
N

w0 =

i=1i yi xi

(14)

Trong Ns chnh l tng s lng cc vector h tr.

Tuy nhin trong trng hp 0 < i < C, chng ta c i = 0 theo phng trnh (13)
ca iu kin KKT. Do , c th ly im hun luyn bt k tha 0 < i < C dng
cng thc (12) (vi i = 0) tnh b0. Cui cng c th ly gi tr trung bnh b0 trn ton
tp mu.
Mt khi (w0,b0) c xc nh ta c hm quyt nh chnh l :
(15)
ng thi siu mt phn cch chnh l g(x) = 0. Trng hp khc xy ra na l khi i =
C ( >0 c th c tnh bng (12)) v i = 0 (mu i c phn loi ng).

A.5.2. SVM phi tuyn

Trong trng hp khng th tm c mt phng siu phn cch trong trng hp
tuyn tnh, ta chuyn khng gian u vo Rn sang mt khng gian mi c kch thc ln
hn gi l khng gian c trng Rm vi m>n.
Hm (x) l hm nh x xi Rn sang yi = (xi), i = 1,2,..,n. Sau chng ta s tm
siu mt phng trong khng gian mi(khng gian c trng).
K ( x , xi ) = K ( xi, x ) = (x)T (x)

(16)

Do , bi ton i ngu tr thnh :

Cc i

Q() = ni=1i - 21 n i=1 nj=1i j yi yj K ( xi, xj )

85

(17)

Vi cng rng buc (10-11). iu kin duy nht chnh l hm x l chnh K ( xi,
xj ) phi tha iu kin Mercer. Bng cch s dng cc hm s l chnh, d liu c th
c phn loi nh sau :
x Lp dng nu g(x) > 0
x Lp m nu g(x) > 0

(18)

Trong g(x) l hm quyt nh sau :

(19)
A.5.3. SVM trong trng hp c nhiu phn lp
C 2 phng php gii quyt trng hp c nhiu hn 2 phn lp :

Phn loi mi lp vi mi lp cn li : to ra c b phn loi nh phn (c l s phn

lp). B phn loi th i c hm u ra l oi, b phn loi ny phn lp th i vi (c-1)
lp cn li. Khi a mu x vo phn loi, phng php ny s gn cho mu x lp
tng ng c u ra oi ln nht.

Phn loi theo tng cp : to ra mi b phn loi nh phn cho tng cp phn lp.
Nu c c phn lp th s c tt c c(c-1)/2 b phn loi nh phn. B phn loi nho
phn Cij s phn lp th i v th j. Khi a mu x vo phn loi nu b phn loi Cij
quyt nh x thuc v phn lp i th gi tr i din cho lp i tng ln 1, ngc li gi
tr i din cho phn lp j tng ln 1. Phng php ny s quyt nh mu x thuc v
phn lp c gi tr i din ln nht.

A.6 C s l thuyt mng nron [25]

A.6.1 M u:
Chng ta bit rng b no con ngi l mt cu trc rt c bit, n l mt mng
li gm nhiu Nron lin kt vi nhau, nh con ngi c th gii quyt nhng vn
phc tp nh nghe, nhn, phn bit c cc ging ni khc nhau, cc mu sc khc

86

nhau, cc i tng khc nhau trong th gii xung quanh. Kh nng c bit ny ca b
no l nh mng li nhng phn t nh gi l Nron, no phn b vic x l thng tin
a vo cho hng t Nron, iu khin lin lc gia cc Nron . Cc Nron khng
ngng nhn v truyn thng tin cho nhau. Mi Nron sau khi nhn tn hiu u vo th
n tch hp cc tn hiu vo thn Nron, ti y, khi tn hiu tng hp vt qu mt
ngng no th chng to ra tn hiu u ra, v gi tn hiu ny n cc Nron khc
thng qua dy thn kinh (axon). Cc Nron thn kinh lin kt vi nhau thnh mng, mc
bn vng ca cc lin kt ny xc nh mt h s gi l trng s lin kt (Weight). C
ch hot ng ca cc Nron c th l t lin kt, tng qut ho, v t t chc. Cc
Nron lin kt vi nhau to thnh mng Nron, mi mng c th c hng vn Nron,
mi Nron c th c hng vn lin kt vi hng vn Nron khc....Chnh kh nng c
bit ca mng Nron ca b no con ngi thc y cc nh khoa hc tm cch
nghin cu xy dng nhng h thng bt chc hot ng ca b no con ngi, to ra
tr thng minh nhn to. iu dn n s ra i ca mng Nron nhn to.

Hnh A.6.1-1: M hnh mt Nron thn kinh

87

A.6.2 Mng Nron nhn to (Artificial Neural network - ANN)

ANN ra i u tin nm 1949, Donald Hebb a ra l thuyt hc Nron v p
dng cho mng x l phn phi song song. Vo cui nm 1950, Frank Rosenblatt xy
dng m hnh mng Perceptron. Tuy nhin, m hnh ny cn nhiu hn ch khi ng dng
cho bi ton ln. Vo nhng nm 1980, cng ngh Nron c bit c quan tm v
xut hin nhiu kin trc mng a dng ng dng rng ri cho nhiu bi ton phc tp
trong thc t, nhng nh khoa hc v k thut mang li cc ng gp chnh trong giai
on ny l: Hopfiel, Grossberg, Widrow, Anderson v Kohonen. [1]
1

nh ngha mng Nron nhn to:

Mng Nron nhn to l s m phng sinh hc bng my tnh b no ca con

ngi. N c cu trc song song c cu thnh t nhiu phn t (Nron nhn to) lin
kt vi nhau thng qua cc trng s, tp hp cc trng s ny to thnh b trng ca
mng Nron nhn to. Mi Nron nhn to l mt h thng ng phi tuyn c kh nng
t hc. Cc Nron v b trng cu thnh cu trc mng. Do mng Nron c kh nng
hc t kinh nghim hay t tp mu. Mng Nron c gi l m hnh lin kt v vai tr
quan trng ca s kt ni gia chng. B trng lin kt chnh l b nh ca h thng.
[1]
A.6.2.1 M hnh tng qut ca mt Nron nhn to

88

C e ll b o d y

A xo n

D e n d r it e s

T he s h o ld

S u m m a ti o n
Hnh A.6.2.1 - 1: M hnh Nron nhn to

M hnh ton hc u tin ca mt Nron c ngh bi McCulloch v Pitt vo nm

1943. N l thit b nh phn dng u vo nh phn, u ra nh phn v mt ngng c nh.

A.6.2.1.1 nh ngha Nron nhn to

Mt Nron nhn to cng ging nh mt Nron sinh hc c cu thnh t hai
khi chnh: ni kt v thn vi nhn bn trong. Mi khi ny chu trch nhim cung cp
hai kiu bin i ton hc khc nhau hnh thnh ln c ch x l thng tin ca mt
Nron: Bin i ton hc ni kt v bin i ton hc bn trong thn Nron. M hnh
ton hc tng qut ca mt Nron c m t trong hnh sau:

89

Hnh A.6.2.1.1 - 1: M hnh ton hc tng qut ca mt Nron

A.6.2.1.2 Hot ng x l thng tin ca mt Nron [1]

1

T
n
Nhn vc tr nhp x(t ) = ( x1 (t ), x2 (t ),..., xn (t )) R

Xy dng ton t ni kt @ gia vc t nhp x(t) vi vc t trng ni kt

w(t ) = (w1 (t ), w 2 (t ),..., w n (t ))T R n

n
Xc nh tn hiu u ra ca khi kt ni u (t ) R , gi u(t) n khi thn bn

trong Nron.
4 Thc hin cc bin i ton hc bn trong thn Nron: Tnh tng, t ngng,
xy dng hm truyn phi tuyn.
5

Xc nh o y (t ) R ti u ra ca Nron.

A.6.2.1.3 Cc bin i ton hc trong Nron

1 Bin i ton hc ni kt:
ui (t ) = xi (t )@ w(t),

i = 1, n

2 Bin i ton hc bn trong thn Nron:

90

- Tnh tng:

v (t ) =
ui (t ), v(t ) R

- t ngng z (t ) = v (t ), z (t ) R
- Hm truyn phi tuyn y (t ) = f [z(t)], y (t ) R

Hnh A.6.2.1.3 - 1: Cc x l tng ng trong mt Nron

n
Hnh trn l m hnh tng qut ca mt Nron n gin: x(t ) R l cc u vo
n
n
ca Nron, w(t ) R l vc t trng s kt ni, (t ) R l ngng, z (t ) R l o

cc quan h qua li, y (t ) R l u ra ca Nron, @ l ton t ni kt,

l ton t

tnh tng, l ton t t ngng, l ton t hp nht, f[.] l hm truyn phi tuyn.
C nhiu loi hm truyn phi tuyn khc nhau, tuy nhin hm truyn thng s
dng nht l hm Sigmoid:
f ( x) =

1
1 + e x

Tm li, mc tiu chnh ca hot ng x l thng tin trong Nron l cung cp

n
mt nh x phi tuyn t vc t nhp x(t ) R n o u ra y (t ) R . Kh nng thch

nghi ca Nron c bo m bng vic hiu chnh b trng s w(t) nh mt thut gii
hc dnh ring. Hm truyn phi tuyn f[.] trong thn Nron tng cng tnh mm do v

91

cht ch ca tnh ton Nron trong nhng iu kin bt thng (c th bin i). vc t
trng s kt ni w(t) l mt i din cho nhng tri thc tch ly hc c np vo
trong khi ni kt ca mt Nron.
Hai bin i ton hc: Ni kt v Tnh tng c chc nng chnh l: thm d, o
lng mi quan h qua li gia vc t nhp x(t) vi b gi tr vc t trng ni kt w(t)
(biu din nhng kinh nghim tch ly trc ). o v(t) thu c t hai php ton
ni kt v tnh tng phn nh gi tr o lng tnh ng dng hay khc bit gia vc t
nhp x(t) v vc t trng w(t). Nu v(t) vt qua ngng (t ) th o z(t) phn nh mi
quan h qua li c chn lm gi tr kch hot Nron v u ra y(t) s c sn sinh
thng qua hm truyn phi tuyn f[.].
Hm ngng (t ) phn nh mc sai lch cho php i vi gi tr kch hot z(t).
Mc sai lch ny c thng k da trn nhng tri thc tch ly t trc (w(t)). Vi
mc tiu gii quyt nhng bi ton phc tp trong thc t, mi Nron trong mng Nron
cn c ngng sai lch cho php i vi gi tr kch hat sao cho m bo nhng yu cu
a dng ca cc hm tnh ton bn trong thn Nron. Hn na, ngng (t ) cng phi
thch nghi vi vc t trng ni kt w(t).
A.6.2.1.4 vc t tham s [25]
Vec-t tham s phn nh s kt hp gia ngng sai lch vi cc php ton tnh
tng v ni kt c xc nh nh sau:
x (t)=(x0(t) ,x1(t) ,x2(t),... ,xn(t))T Rn+1 , xo(t)=1.
w (t)=(w0(t),w1(t),w2(t),....,wn(t))T Rn+1 ,wo(t)=(t)

v
trong :

xo(t)=1 v wo(t)= (t) phn nh ngng sai s .

92

x(t) Rn

y(t) R

u ra ca Nron
( o)

u vo ca
Nron(Vector)

Nron-n v x l thng tin

nh x phi tuyn trong Nron

x(t) Rn

y(t) R
Khng gian u ra ca Nron

Khng gian u vo ca Nron

Hot ng tnh ton ca Nron : nh x phi tuyn t

x(t) Rn n y(t)R

Hnh A.6.2.1.4 - 1: Hot ng tnh ton ca Nron

A.6.2.1.5 nh x phi tuyn

Nhng bin i trong mt Nron t vc tr nhp x(t) Rn sang o y(t) R
c xem l nh x phi tuyn t nhiu u vo n mt u ra duy nht.
nh x phi tuyn chuyn i ca Nron c phn thnh hai php ton:
(i)

Php ton hp nht

93

(ii)

Hm truyn phi tuyn

Php ton hp nht: c nh ngha l php ton o lng cc quan h qua li gia
vect nhp x(t ) v vect trng ni kt w(t ) . Nh vy, n l mt bin i ton hc tch
hp gm: ni kt, tnh tng v t ngng, c xc nh nh sau:

z (t ) = [ w(t ), (t )]x(t )
trong l ton t hp nht v z (t ) l hm o lng cc quan h qua li.
Hm truyn phi tuyn f [.] : c chc nng xp hng o z (t ) sao cho m bo tnh
mm do v cht ch ca nh x Nron v tr v o y (t ) R ti u ra ca Nron.
A.6.2.2 Cc cu trc mng Nron nhn to:
Mng Nron nhn to bao gm tp hp cc lin kt qua li bn trong gia cc
Nron trn nguyn tc: u ra ca mi Nron c lin kt thng qua cc trng s n
cc Nron khc hoc ti chnh n. Nh vy, vic b tr cc Nron v s lin kt qua
li gia chng s hnh thnh mt kiu mng Nron nhn to.
w11
x1
w n1 w21

y1

w
x2 12

y2

w22

wn2
w1m w 2m
xn
w nm

yn
(a)

94

95

x1

y1

x2

y2

xm

ym
(e)

Hnh A.6.2.2 - 1: Nm s lin kt c bn ca mng Nron: (a) m hnh mngtruyn thng mt lp;
(b) m hnh mng truyn thng a lp; (c) m hnh: mt Nron n vi lin kt phn hi n chnh n; (d)
m hnh: mng lp mt lp; (e) m hnh: mng lp a lp.

Hnh (A.6.2.2 - 1a) i din cho mng Nron truyn thng mt lp, trong mi
Nron s kt hp vi cc Nron khc lm thnh mt lp cc Nron. vc tr u vo ni
kt vi cc Nron trong lp gn lin vi nhng gi tr trng s khc nhau, kt qu tr v
ca lp l tp cc u ra ng vi s Nron trong lp.
Hnh (A.6.2.2 - 1b) i din cho mng Nron truyn thng a lp, chng ta c th
to thm mt vi lp bn trong hnh thnh mng: Lp nhn cc vc tr u vo gi l
lp nhp, cc u ra ca mng hnh thnh t lp xut. Cc lp nm gia cc lp nhp v
lp xut gi l cc lp n v chng ch c cc lin kt bn trong mng m khng lin kt
trc tip vi mi trng bn ngoi. Mng Nron truyn thng a lp c gi l lin kt
y nu tt c cc u ra t lp trc c lin kt vi tt c cc Nron trong lp k
tip. Hnh (A.6.2.2 - 1b) l mt lin kt khng y .

96

Hnh A.6.2.2 - 2: Lin kt bn trong ca phn hi

C hai s trong hnh (A.6.2.2 - 1a) v hnh (A.6.2.2 - 1b) c gi l mng

truyn thng v s lin kt cc Nron trong mng lan truyn theo mt hng (khng
c u ra no ca Nron lp sau li l u vo ca Nron lp trc n ).
Khi u ra ca mt Nron li ng vai tr l u vo ca cc Nron trn cng lp
hoc trong cc lp trc th gi l mng phn hi. Hnh (A.6.2.2 - 1d) minh ho
mng phn hi mt lp: trong u ra ca Nron hng tr li n chnh n v n cc
Nron khc.
Phn hi trong u ra ca Nron c hng trc tip tr li lm du vo ca
cc Nron trong cng mt lp gi l phn hi nhnh.
Mng phn hi, trong : tt c cc chu trnh u kn gi l mng lp (Hnh
A.6.2.2 - 1c) ch ra mt mng lp n gin ( mi nt n phn hi n chnh n)). Trong
mng lp a nhnh (hnh1-6c), mi u ra ca Nron c hng tr li cc Nron
trc . ng thi, cng hng trc tip tr li chnh bn thn n v n cc Nron
khc trn cng mt lp. [1]
A.6.2.3 Cc lut hc
c im quan trng ca mng Nron l kh nng hc ca chng. Hc iu
chnh phn ng p li mi trng. Mt mng Nron c hun luyn t mt tp
vect nhp X to ra tp vect xut mong mun, hoc hc nhng c im bn trong v
cu trc ca d liu t tp X. Tp X dng hun luyn mng gi l tp hc. Mi phn

97

t ca tp X gi l mu hc. Qu trnh hc c phn nh qua s thay i ca b trng.

Trong qu hc, b trng s hi t dn v gi tr vect mong mun.
C 2 k thut hc trong mng Nron nhn to: hc tham s tp trung vo cp nht
li b trng lin kt trong mng Nron nhn to (ANN) v hc cu trc cp n s
thay i cu trc mng: gn lin vi s thay i s lng Nron v cc hnh thc lin kt
gia chng. Hai kiu hc ny c thc hin ng b hoc tch ri nhau.
nh ngha ma trn trng s ni kt:
Cho trc n Nron trong mng Nron nhn to v mi Nron c m trng s ni
kt ti n. Lc ny ma trn trng s ni kt W c nh ngha nh sau:

w1T
T
w2

W =

wn

w11 w12 w1m

w 21 w22 w2 m

w
w

n
1
n
2
nm

Hnh A.6.2.3

Trong

- 1: Ma trn trng s ni kt

w1 = ( wi1 , wi 2 ,...,wim ) T , i = 1, n l vect trng ca Nron th i v wij l

trng s lin kt Nron j n Nron th i.

Mt mng Nron nhn to mun hot ng hiu qu phi tm c b trng hay
ma trn trng thch hp nht. Vic hc trong mng Nron chnh l xc nh ma trn
trng, cn xy dng lut hc hiu qu hng ma trn trng s W v ma trn ti u sao
cho hiu qu hot ng ca mng nron nhn to l cao nht. Lut hc c th chia lm 3
loi :
Hc c gim st
Hc tng cng
Hc khng gim st

98

Hnh A.6.2.3 - 2 : Hc c gim st

Hnh A.6.2.3 - 3 : Hc tng cng

99

Hnh A.6.2.3 - 4: Hc khng gim st

Hc c gim st :
Trong hc c gim st, cho trc phc p mong mun d ca h thng tng ng

vi mu nhp X: u vo ca mng Nron nhn to (ANN). Lc ny, ANN c thng

bo chnh xc gi tr m n s phi tnh ton ti u ra. C th, vi phng php hc c
gim st, ANN tip nhn mt chui cc mu: (x(1), d(1)) , (x(2), d(2)), .. , (x(k), d(k)),...l
nhng cp u vo - u ra mong mun. Khi mu nhp x(k) c a vo ANN, th u
ra mong mun d(k) tng ng cng c np vo ANN. S khc nhau gia u ra thc
s y(k) v u ra mong mun d(k) c o lng bng b pht sinh tn hiu li, v b ny
s to ra nhng tn hiu li cho ANN iu chnh cc trng s ca n sao cho u ra
thc s c chuyn gn n u ra mong mun.
Trong hc c gim st, gi thit: cc gi tr u ra ch chnh xc c bit
trc ng vi tng mu nhp.Tuy nhin, trong nhiu tnh hung, ch rt t thng tin chi
tit c bit. V d, ANN ch c bo rng: gi tr u ra hin ti ca n l qu cao
hoc chnh xc 50%. Thm ch ch c mt gi tr phn hi bo hiu kt qu ca ANN
l ng hoc sai. Vic hc da trn c s thng tin nh ga ANN gi l hc tng
cng v thng tin phn hi c gi l tn hiu tng cng. Hc tng cng l mt hnh
thc hc c gim st v mng vn cn nhn mt vi phn hi t mi trng ca n. Tuy

100

nhin, phn hi ny mang ngha nh gi, nhng khng mang tnh ch dn. N ch nhn
xt u ra thc s l tt hay khng tt m khng a ra mt gi no cho ANN.
Tn hiu tng cng ny s c chuyn vo b pht sinh tn hiu nh gi to
ra nhng thng tin tn hiu nh gi truyn vo ANN. T , ANN s iu chnh b
trng ca n vi hi vng c c nhng nh gi phn hi tt hn trong tng lai.

Hc khng gim st:

Trong hc khng gim st, s khng c ngi thy no cung cp thng tin phn

hi cho ANN. Cng khng c phn hi t mi trng nh gi mc chnh xc u

ra ca ANN. Mng phi ch ng khai thc cc mu, cc c trng, cc qui tc, cc mi
lin h hoc cc chng loi ca d liu nhp v m ho chng trong u ra. Trong qu
trnh khai thc nhng c trng trn, cc tham s trong mng s c sa i: qu trnh
x l ny c gi l t t chc. Mt v d in hnh ca thut gii hc khng gim
st l: phn loi cc i tng m khng c cc thng tin v s lng lp cn phn loi.
Vic phn nhm chnh xc c hnh thnh t vic khai thc mc tng t v khc
bit gia cc i tng.
Ch : trong qu trnh hc, tham s ngng c gn lm trng s ca gi tr
u vo xm=-1, di ng vai tr lm tn hiu mong mun trong trng hp hc c gim st
hoc ng vai tr l tn hiu tng cng trong trng hp hc tng cng. Nh vy, vi
hai phng php hc trn, cc trng s ca Nron th i c sa i da theo tnh hiu
u vo m n nhn c, gi tr u ra ca n v cc phc p ch dn lin quan.
Trong phng php hc khng gim st, Nron sa i trng s ca n ch da
vo gi tr u vo v /hoc gi tr u ra t c.
Cng thc tng qut ca lut hc trng s trong ANN xc nh gia tng ca
vc tr trng wi ti bc lp r ng vi tnh hiu hun luyn r v mu nhp x(t) l:
wi(t) rx(t) hoc wi(t)= rx(t)
trong , l mt s nguyn dng gi l hng s hc: n xc nh tc hc v
r l tn hiu hun luyn c xc nh theo cng thc tng qut sau:

101

r= fi(wi,x,di) trong trng hp hc c gim st hoc hc tng cng v

r =fi(wi,x) trong trng hp hc khng gim st.

Hnh A.6.2.3 - 5: Lut hc pht sinh trng s (di khng c cung cp trong trng hp hc khng
gim st)

Theo 2 cng thc trn, vc tr trng s ti bc lp (t+1) c xc nh:

wi(t+1)=wi(t)+fr(wi(t),x(t),di(t))x(t)

(a)

Cng thc tp trung vo vic sa i trng s c min gi tr ri rc, v bn sao

ca n cho vic sa i cc trng s c min gi tr lin tc c xc nh nh sau:
dwi(t)/d(t) = r x(t)

(b)

Vi hai cng thc (a) v (b), cc trng s s c khi to (v d: khi to ngu

nhins) trc khi thc hin qu trnh hc.
Trn c s lut hc tng qut trong cng thc, nhiu lut hun luyn v hc cc
trng s c gim st v khng gim st c xy dng.

102

A.6.2.4 Hm truyn:
C nhiu loi hm truyn khc nhau, nhng chng ta thng s dng loi hm truyn
dng S , chng c im chung l th hm truyn c dng ch S. Mt hm S(u) l mt
hm truyn dng S nu n tho:

S (u) l hm b chn .Ngha l cc gi tr ca S (u) khng bao gi c vt qu chn

trn cng nh khng bao gii thp hn chn di, vi mi gi tr ca u.

S (u) l hm n iu tng. Gi tr ca S (u) lun tng khi gi tr ca u tng: ngha l

n phi tng u n. Do tnh cht th nht S (u) b chn -nn ta thy khi u tng, S
(u) cng ln dn nhng khng bao gi vt qu cn trn; v vy n tim cn ng
gii hn l chn trn .Tng t khi u nh dn, S (u) tim cn gii hn l chn di
ca hm.
S (u) l hm lin tc v trn.V hm S (u) lin tc nn n khng c khe v gc cnh.
Do tnh lin tc trn, hm c o hm v dc ca n r rng v phn bit ti tng
im.
Mi hm tho mn 3 tnh cht trn u c th c s dng l hm truyn trong
mng.Trong thc t hm logistic f(x) thng c s dng rng ri. N c min gi tr
nm trong khong [0, 1]. N c cng thc:
f ( x) =

1
1 + e x

trong e l c s ca logarit t nhin. Hng s e c gi tr khong 2.71828 .Tuy nhin

hng s e trong mu s khng phi bt buc; bt c hng s no ln hn 1 cng u c
th c. Hng cng ln hm s f(x) cng mau tip cn cc cn ca n; ngc li, hng
s cng nh, hm s cng chm tip cn cc cn.

103

Hnh A.6.2.4

- 1: Mt s dng hm dng trong nh x t u vo -> u ra

Mt s dng hm dng trong nh x t u vo u ra :

Tn hm
Hm ngng
gii hn

Cng thc

c im

Nu vect tnh tng v (t )

1 nu x>0
0 nu ngc li

Gii thch
ln hn ngng th Nron

Hm ri rc

trng thi hot ng (gi tr

1), ngc li n khng hot

Hm ngng

Ri rc

104

ng (gi tr 0).
Gi tr hot ng tng tuyn

tnh cng vi v(t ) , nhng sau

mt ngng no gi tr
tuyn tnh

u ra bo ho (tng ln 1).
Gi tr u ra ca Nron
Lin tc,

Hm Sigmoid
(logistic)

thuc mt khong.
Hm phi tuyn hnh ch S

dng, c gi

a = 1 /(1 + e u )

tr nm trong
khong [0,1]
u ra t mt Nron c th
l xung c th xy ra (nh

Hm Gaussian

du 1) hay khng xy ra
(nh du 0)
h(x)-(1-e-x)/(1+e-x)

Lin tc

Hm hyperpol c lin quan

dng, c gi mt thit vi hm

Hm hyperpol

tr nm trong
x

tanh(x)=(1-e )/(1+e )

logistic.Bi ta thy

khong [-1,1] h(x) =2f(x)-1

Lin tc,
Hm ny tin n cc gii

Hm Tang-

dng, nm

hn ca n nhanh hn hm

hyperpol

trong khong

hyperpol h(x).

[-1,1]
Bng A.6.2.4 1 : Mt s hm truyn thng dng trong mng Nron

Tt c cc hm truyn ny u phc v kh tt cho cc mc ch ca mng Nron

v chng ta c th thay th cho nhau. Do khc nhau v cc gii hn ca chng, ta c th
chn hm ny hoc hm khc ty theo khong cn thit ca gi tr kt xut (0 n 1 hay
-1 n 1), v kt xut ca mng phi ri vo gia cc gii hn ny. Tuy nhin cc gii
hn ny khng qui nh khong d liu nhp vo mng, d liu nhp c th nhn bt c

105

gi tr no, bt k hm truyn no c chn.V d liu nhp cng c th nhp trc tip

vo mng m khng cn phi bin i t l hoc chun ho, nhng cho mng c th
hc hiu qu ta phi chn cch biu din nhp xut hp l.
A.6.2.5 Cc loi lin kt trng nron:
Mng Nron bao gm cc Nron v cc b trng lin lin cc Nron. Hot ng
ca Nron ph thuc phn ln vo giao tip gia cc b trng. C 3 loi tng c bn:
tng nhp, tng n, tng xut. Hai tng Nron lin kt nhau thng qua b trng lin kt
ca mng. C 4 loi lin kt trng: feedforward, feedback, lateral, time-delay.

Hnh A.6.2.5 1 : Cc loi lin kt ca Nron

Feedforward: dng cho tt c m hnh Nron, d liu t Nron tng thp lan
truyn ti cc Nron tng cao hn thng qua lin kt feedforward.
Feedback: truyn d liu t cc Nron tng cao ngc v Nron tng thp hn.
Lateral: v d in hnh l kiu mng ngi thng ly tt c.
Time-delay: cc thnh phn c th nhp vo cc kt ni ca mng to thnh m
hnh ng theo thi gian. Loi ny thch hp cho nhn dng mu thi gian thc.

106

Kch thc mng Nron: trong mng feedforward nhiu tng, c nhiu tng n
gia tng nhp v tng xut. Kch thc mng ph thuc vo s tng v s nt n trn
mi tng. S nt n lin quan n kh nng ca mng.
A.6.3 Mng nron RBF: [26][27][28]
A.6.3.1. Cu trc mng RBF :
L mng neuron hai lp truyn tin (two-layer feed-forward network). Nhng
neuron ti lp th nht c kch hot bi mt hm radial-basic, trong hu ht cc
trng hp l mt hm Gaussian :
T -1
(X) = G(X) = e-1/2[X-] [X-]
Lp th hai (lp output) ca mng s dng dng ca mt lin kt tuyn tnh ca
nhng kt qu RBF. Do ton mng thi hnh vi hm :
F(X) =

M
i=1

wi (X) =

M
i=1

Hnh A.6.3.1

wi e-1/2[X-]

T -1
[X-]

- 1 : Cu hnh mng RBF tiu biu.

V phng din hnh hc, m hnh nhng kiu mu c m t t mt phn b

Gaussian theo kiu chia thnh mt min tp hp n trong khng gian mu. Nhng min

107

tp hp ny, c bit trong hnh dng hyper-ellpoids, phn chia khng gian mu thnh
mt s khng gian con. Thng thng mt khng gian con nh th tng xng vi min
phn b ca mt lp mu. Kch thc, v tr v nh hng ca nhng khng gian con
hyper-ellipsoidal c xc nh duy nht bi cc tham s v . vc tr miu t trung
tm khng gian con ca mt lp mu, v ma trn xc nh kch c v hng ca
khng gian con.
Khi nim RBF l phn chia mi noron RBF ng vi mt khng gian con tp hp
ca mu luyn.
Theo qui tc mng RBF c miu t bi l mt ma trn ng cho, nh l :

Nhng khng gian con miu t loi neuron RBF l hyper-balls trong khng gian
mu a chiu. Ta s miu t tng qut mng RBF l :

108

Trong mt khng gian mu tng qut, nhiu lp mu c l phi c miu t bi

nhng hm phi tuyn v khng lin tc, ph thuc vo phc tp ca s phn b trong
khng gian mu. trong nhung trng hp ny, nhng mu ca lp phi phn ra thnh
mt s tp hp ring bit trong khng gian mu.Do mt lp mu c th khng biu
din c bng mt khng gian con ring l v lin tc, m bi hai hay nhiu khng gian
con tch ri.
Hnh (A.6.3.1 - 2) l v d v biu din khng gian con v n c nh x cc
khng gian con ny ln cc neuron RBF trong hnh (A.6.3.1 - 3).

Hnh A.6.3.1 - 2 : Nhng tp phn lp trong khng gian 2 chiu.

109

HnhA.6.3.1 - 3 : nh x cc tp phn lp ln neuron RBF.

A.6.3.2. Hot ng :
Mng RBF dng hm truyn Gauss thay v dng S. Li im ca mng RBF l :
i vi bi ton c s bin nhp hn ch, mng hc nhanh hn so vi mng NN 3 lp lan
truyn ngc.
Hm nh x c xy dng t mng RBF khc vi nh x c xy dng mng
3 lp. Khc bit chnh l n v n . y ta c th coi nh 2 lp v gi tr ti lp n
c th c tnh t lp nhp m khng cn thay i b trng t lp nhp n lp xut.
Mi n v n c mt tham s cho mi u vo. Cc tham s ny khng phi l
cc trng, m chng l ta ca tm hm kt xut trong khng gian nhp. Hm kt xut
l mt hnh dng chung c chiu cao 1 tm v c chiu rng c iu khin bi 1
tham s b sung.
Gi s mng c I nt nhp, H nt n v O nt xut vi b trng W1 ni t lp
nhp n lp n. W2 ni t lp n n lp xut.
j = sqrt ((xi W1ij)) , i = 1,2,,I
Gi tr ti nt n th j :

110

yj = exp (-j2 / j2)

Vi j l b rng ca hm Gauss ti noron n th j (chn c nh).
Kt qu thu c ti nt xut th k :

Vi j chy t 1 n H trn lp n.
k chy t 1 n O trn lp xut.
* Qu trnh hc :
Mng RBF hc nhanh v chng tch mt bi ton kh thnh hai mnh d. Trc
tin chng sp xp cc n v n ph cc mu mt cch thch hp.Ri t tham s ca
cc n v xut cc tiu ha o sai s bc 2 n gin.
Ta c th dng phng php gim gradient hc.
* nh gi :
V mng RBF c th hc mt hm nh x bt k nn chng c th qu khp trn
d liu c nhiu. V th khi ng dng mng RBF c bn vo bi ton chng thc mt
ngi vi b hun luyn ln s khng tt v lng nh nhiu nhiu.
Mng RBF c th c ng dng vo cc bi ton c t tp mu v t c nhiu
nh nhn dng k s, nhn dng ch in hoa
A.6.4 Mng lan truyn thng ba lp v thut gii lan truyn ngc [25]:
A.6.4.1 Mng truyn thng ba lp:
Mng Nron truyn thng c xem l thit b nh x gia tp cc gi tr u vo
v tp cc gi tr u ra: N ng vai tr nh mt nh x f t tp u vo I sang tp u
ra O:
f : I->O

hoc

111

y=f(x) trong y O v x I.
Xt mt mng Nron ba lp vi T Nron trong lp nhp, H Nron trong lp n,
v C Nron trong lp xut, H l mt s c chn tng ng. Mng s lin kt y
cc lp k nhau.

Tng nhp

Tng n

Tng xut

Hnh A.6.4.1-1: Minh ha mng lan truyn thng ba lp

Hot ng ca mng c th hiu nh mt qu trnh x l phi tuyn: Nhp vo

mu X = ( x1 , x2 ,..., xT ) (cha bit thuc lp no) v tp cc lp = {1, 2, 3,, c},
mi Nron u ra s to ra yi thuc mt trong cc lp ny bi cng thc:
H

k =1

j =1

P (i | X ) = yi = f { wikom f ( wkjmi x j )}

wkjmi

om
l trng s gia Nron u vo th j v Nron n th k, wik l mt

trng s t Nron n th k n lp u ra th i, v f l hm phi tuyn Sigmoid c xc

nh theo cng thc: f ( x) =

1
. Cc gi tr i, m, o trong cc trng s c dng
1 + ex

xc nh lp u vo, s lp n, v s lp u ra ca mng Nron. Chn Nron c gi tr

112

ln nht vo lp tng ng. Mng ny c hun luyn bi thut gii lan truyn ngc
vi lut hc tng qut delta.
A.6.4.2 Thut gii lan truyn ngc vi lut hc tng qut delta: [25]
Cho khng gian cc mu hc (x, t) l gi tr cn hun luyn; t l gi tr kt qu
ch (u ra mong mun) ca qu trnh hun luyn. H s hc . Qui nh ch s lp l
tng dn t lp u vo n lp u ra. Thut gii lan truyn ngc c tm tt nh
sau:
(1). To mng truyn thng c nin Nron u vo, nhiden Nron trn mi lp n v h lp

n trong mng, vi nout u ra.

(2). Khi to b trng cho mng vi gi tr nh
(3). Trong khi <iu kin kt thc cha tho> lm :
Vi mi cp (x, t) trong khng gian mu hun luyn thc hin:

Xt lp nhp (*): truyn x qua mng, ti mi lp xc nh u ra ca mi

Nron. Qu trnh ny c thc hin n lp xut da theo cu trc mng
c th.

Xt lp xut: i vi u ra ok ca Nron k trong lp xut K, xc nh sai

s k = ok (1 ok )(tk ok ) .
Chuyn sang lp n L k n, t L=K-1

Xt cc lp n (**): vi mi Nron l trn lp n th L, xc nh sai s

l = ol (1 ol ) wil i
iL +1

Cp nht li trng s c trong mng, wji

wji = wji + wji vi wji = joji

Nu (L>1) th: chuyn sang lp n trn n : L=L-1 v quay li bc (**)

Ngc li: chn cp (x, t) mi trong khng gian mu hc, quay li bc
(*).

113

Tm li, gi tr sai s c tnh u tin cho cc nron lp xut, kt qu ny

c dng tnh sai s ca cc nron lp n cao nht, v lp n kt tip th tnh theo
kt qu ca lp n cao hn n.C th lan truyn cho n lp nhp.
Sau y l mt s ch trong vic hun luyn mng lan truyn thng 3 lp:
Vic khi to trng s: Bi v qu trnh hun luyn cho mng l mt qu trnh
lp li nhiu ln vic hc trn tp mu. Mi ln hc, ta cho mng thc hin vic nh x
cc mu u vo qua mt hm truyn, sau tnh ton sai s v cp nht li trng s vi
mc ch l lm sao cho sai s ngy mt gim, tc mng thc hin ngy mt tt hn.Tuy
nhin bi v lc u tin trng s cha c nn ta phi tin hnh khi to b trng.
H s hc: V nguyn tc, h s hc ch cn nh m bo s hi t ca mng, gi
tr ca n ch xc nh tc m mng t c mt cc tiu ca hm nh gi. Tuy
nhin, trong thc t, bi v mng him khi c hun luyn y t ti cc tiu li
hun luyn, nn h s hc c th nh hng kh nng ca mng. Thng thng ta chn
h s hc l mt s dng nh hn 1.

B. Mt s thut ton chnh, ci t v hng dn s dng

B.1 Mt s thut ton chnh:
B.1.1 Thut ton hin th Video ln khung hin th
Thut ton ny m nhim vai tr v ni dung ca Video ln khung hin th
Video ca mn hnh giao din.
u vo: Ni dung ca tp tin Video c gii m.
u ra: Hin th nh ca tng Frame ln mn hnh.
M phng thut ton:
DisplayVideo()
//Nu Frame hin hnh nh hn tng Frame ca on Video

114

While (nCurrentFrame < nTotalFrame)

//Ly Frame t Video c gii m
GetNextFrame();
//X l Frame nhn c c th hin th c trong khung hin
//th cho trc.
cvResize();
//Hin th Frame ln khung hin th
ShowImageEZ();
End while;
End DisplayVideo

B.1.2 Thut ton Gii m Video u vo

Thut ton ny m nhim vai tr gii m tp tin Video u vo. Trong lun vn
ny ch h tr cho tp tin AVI.
u vo: Tn ca tp tin Video (AVI)
u ra: Ton b ni dung ca tp tin Video c lu di dng tng BYTE.
M phng thut ton:
VideoDecoder ()
//Khi to b gii m
AVIFileInit();
//M tp tin Video vi ng dn cho trc
AVIFileOpen();
//Nu m thnh cng v nh dng ca tp tin Video u vo ng l
nh //dng c h tr th tin hnh c ni dung ca Video

115

AVIStreamGetFrameOpen();
//Cn nu khng c dc file hay nh dng khng c h tr th gii
phng b nh v thot.
AVIStreamRelease();
AVIFileExit();
End VideoDecoder
B.1.3 Thut gii lan truyn ngc:
Bc 1: To mng truyn thng c Nin nron u vo, Nhiden nron trn mi
lp n v h lp n trong mng, Nout nron u ra.
Bc 2: Khi to b trng cho mng vi gi tr nh
Bc 3 :Trong khi <iu kin kt thc cha tho> lm :
Vi mi cp (x, t) trong khng gian mu hun luyn thc hin:

Xt lp nhp : truyn x qua mng, ti mi lp xc nh u ra ca

mi Nron. Qu trnh ny c thc hin n lp xut da theo
cu trc mng c th.

For(int i=0;i<lsize[0];i++)
out[0][i]=in[i];
Gn d liu cho tng nhp :lsize[0] l kch thc tng nhp, in[i] l gi
tr nhp th i, out[0][i] l gi tr xut ca nron i ca tng nhp.
For (i=1;i< numl ;i++) {
//Xt tng tng vi numl l s tng
For(int j=0;j<lsize[i];j++){
//xt mi nron trong tng hin ti ang xt : lsize[i] l kch thc hay s
phn t trong tng th i
Sum=0.0;
// khi to sum
For(int k=0;k<lsize[i-1];k++){
//cho ng nhp t mi nron tng trc

116

sum+=out[i-1][k]*weight[i][j][k];
//weight[i][j][k] l trng s ni kt ca nron k tng (i-1) vi nron j
tng i
}
sum+=weight[i][j][lsize[i-1]];
out[i][j]=sigmoid(sum);
//gi hm sigmoid tnh v gn gi tr cho ng xut ca nron j trong tng i
(out[i][j])
}
}

Xt lp xut: i vi u ra ok ca Nron k trong lp xut k, xc

nh sai s k = ok (1 ok )(tk ok ) .

for(int i=1;i<lsize[numl-1];i++){
delta[numl-1][i]=out[numl-1][i]*(1-out[numl-1][i])*(tgt[i]-out[numl-1][i]);
}
Chuyn sang lp n L k n, t L=k-1

Xt cc lp n: vi mi Nron trn lp n th L, xc nh sai s

h o h (1 o h )koutput wkh k
For(i=numl-2;i>0;i--){
For(int j=0;j<lsize[i];j++){
Sum=0.0;
For(int k=0;k<lsize[i+1];k++){
Sum+=delta[i+1][k]*weight[i+1][k][j];
}
Delta[i][j]=out[i][j]*(1-out[i][j])*sum;
}
}
Cp nht li trng s c trong mng :

117

 ji ji + ji

vi

ji =
j x ji

for(i=1;i<numl;i++){
for(int j=0;j<lsize[i];j++){
for(int k=0;k<lsize[i-1];k++){
prevDwt[i][j][k]=beta*delta[i][j]*out[i-1][k];
weight[i][j][k]+=prevDwt[i][j][k];
}
prevDwt[i][j][lsize[i-1]]=beta*delta[i][j];
weight[i][j][lsize[i-1]]+=prevDwt[i][j][lsize[i-1]];
}
}
Nu (L>1) th: chuyn sang lp n trn n : L=L-1 v quay li bc 3.3
Ngc li: chn cp (x, t) mi trong khng gian mu hc, quay li bc 3.1

B.1.4 Thut ton SVM nhiu lp:

Bc 1: Khi to mng vote gm nr_class lp
int i;
int nr_class = model->nr_class;
double *dec_values = Malloc(double, nr_class*(nr_class-1)/2);
svm_predict_values(model, x, dec_values);
int *vote = Malloc(int,nr_class);
for(i=0;i<nr_class;i++)
vote[i] = 0;
Bc 2: Trong khi duyt trn cc lp t i=0 n nr_class-1
Trong khi duyt trn cc lp k j=i+1 n nr_class-1
Nu mu X thuc lp i th tng gi tr vote[i]
Ngc li X thuc lp j th tng gi tr vote[j]
int pos=0;
for(i=0;i<nr_class;i++)

118

for(int j=i+1;j<nr_class;j++)
{
if(dec_values[pos++] > 0)
++vote[i];
else
++vote[j];
}
Bc 3: Duyt trn mng vote, gi tr no ln nht th ch s ca mng s
l lp m mu X thuc v .
int vote_max_idx = 0;
for(i=1;i<nr_class;i++)
if(vote[i] > vote[vote_max_idx])
vote_max_idx = i;

B.2 Ci t:
Chng trnh th nghim c ci t bng ngn ng C++ trn h iu hnh
Microsoft Windows XP, s dng mi trng lp trnh Microsoft Visual C++ 6.0
IDE.
. c th nghim trn my PC P4 3GHz.

Cc lp chnh ca chng trnh:

CDetect: Dng pht hin khun mt, mt, ming.
CImageProcessing: Dng thc hin cc bin i x l nh.
CcreateData: Dng to d liu cho PCA v ICA, d liu cho ICA l d

liu c qua PCA.

CPCA: Dng bin i PCA.
CregICA: Dng nhn dng vi cch rt trch c trng ICA.
VideoDecoder: Dng gii m video.
CMPEGViewer: Dng hin th Video.
Cdetection: Dng pht hin i tng trong cc frame ca video.

119

Cc th vin c s dng trong lun vn:

Th vin x l nh OpenCV ca Intel: dng ci t PCA v d tm khun

mt.[31]
Th vin SVM svmlib2.8: dng phn lp SVM.
Th vin ton hc MKL (Math Kernel Library) ca intel dng cho thut

ton ICA [30]

Th vin itpp-3.10.9 c ci t sn ICA [29]

B.3 Hng dn s dng:

B.3.1 Mn hnh chnh:

120

Hnh B.3.1 1: Giao din chnh ca chng trnh

Mn hnh chnh gm c mt khung chy on Video u vo, mt khung hin

th Frame m ti frame h thng pht hin c i tng, mt vng hin
th thng tin ca on video u vo, mt khung hin th khun mt ca i tng
tng ng c lu trong c s d liu, mt text box Threshold, khi kt qu nhn
dng ln hn Threshold ny th mi c chp nhn, tc mi l mt i tng,
mt text box hin thng tin lu sn trong c s d liu v i tng ny, mt
text box hin th s lng khun mt i tng m h thng nhn dng c trong
cc frame. Nu frame no c cng lc hai khun mt i tng th ta to hai frame
copy ca frame ny, tuy nhin trong mi frame ta t bin mt khun mt ngi
s dng bit l ang nhn dng frame no.

121

B.3.2 Chc nng nhn dng trn video:

T menu File chn mc OpenAviFile m on Video cn test, chng trnh ch h tr
vi nh dng file AVI(uncompressed).

Hnh B.3.2 -1: Mn hnh minh ha chc nng test trn video

Sau khi chn file AVI test, nhn vo nt Play trn mn hnh chnh lc ny h thng s
hin th Video ln mn hnh ng thi tin hnh nhn dng cc i tng c trong on
video. Khi frame c chn a vo nhn dng c i tng, h thng s t ng nhn
dng v a ra thng tin v i tng tng ng trong c s d liu, ng thi t ng
cp nht s lng frame detect c i tng.

122

Hnh B.3.2 2 : Giao din ca chng trnh khi hot ng

B.3.3 Chc nng hun luyn cho mng Nron:

123

T menu Neural network, chn chc nng Train, mn hnh tham s s xut hin
nh bn di.

Hnh B.3.3 1 : Mn hnh tham s hun luyn cho mng nron.

Data in l ng dn th mc cha cc th mc nh hun

luyn ( nh ca khun mt sau khi qua detect bi adaboost), y
mi i tng s c lu trong mt th mc ring.

K face, K left eye, K right eye, K mouth l s chiu vc t c trng

sau khi gim bi ICA.

FNN Error l tng li ca mng Nron

Data out l ng dn ni lu b trng ca cc mng nron sau khi

hun luyn.

B.3.4 Chc nng Read Data (load data trained):

124

T menu Neural network, chn chc nng ReadData, mn hnh Readdata s xut
hin nh bn di.

Hnh B.3.4 -1 : Mn hnh c d liu hun luyn

Data for PCA: ng dn th mc cha cc file tnh ton c bi PCA

nh: avgVector, eigenVector.
Data for ICA: ng dn th mc cha cc file tnh ton c ca ICA:
in_mix_matrix (nghch o ma trn trn).

Data For FANN: ng dn th mc cha cc mng nron c hun

luyn.

B.3.5 Chc nng nhn dng cc nh trong th mc:

125

T menu TestOnImage chn chc nng TestFolder, mn hnh giao din s hin ra
nh hnh di:

Hnh B.3.5 1: Mn hnh test th mc

Folder In: ng dn th mc cn nhn dng, trong th mc ny cha cc

nh.
Folder Out: ng dn cha file kt qu, file kt qu l mt file text, gm
nhiu dng, mi dng l tn nh trong th mc nhn dng v th t ca
i tng c nhn dng tng ng.

B.3.6 Chc nng nhn dng trn nh tnh:

126

T menu TestOnImage chn chc nng TestImage.

Hnh B.3.6 1 : Mn hnh nhn dng trn nh tnh

Chn open m mt nh cn nhn dng.

Nhn Recognize nhn dng cc i tng c trong nh.

127

Ti liu tham kho:

[1] Lu Buon Vinh, Hong Phng Anh (2004), Nghin cu v xy dng h thng nhn
dng mt ngi da trn FSVM v Adaboost,

Lun vn c nhn tin hc Khoa cng

ngh thng tin, H KHTN TP HCM.

[2] Trn Phc Long, Nguyn Vn Lng (2003), Nhn dng mt ngi bng thng
tin khun mt xut hin trn nh, Lun vn c nhn tin hc Khoa cng ngh thng tin,
H KHTN TP HCM.
[3] L Minh Tr Nguyn Thy Hng (2006), xut phng php trch chn c
trng mi cho bi ton nhn dng mt ngi, Lun vn C nhn CNTT Trng i
Hc KHTN TPHCM.
[4] Markus Weber, Frontal Face Dataset, California Institute of Technology, 1999
http://www.vision.caltech.edu/html-files/archive.html
[5] CBCL Face Database, CMU and MIT
http://www.ai.mit.edu/projects/cbcl.old/software-datasets/FaceData2.html
[6] Nguyn Thnh Thi (12/2006) , Nhn dng mt ngi dng SVM v mng nron,
Lun vn Thcs CNTT Trng i hc KHTN, TPHCM
[7] Bi Xun Hi, Trn Nam Dng, i S Tuyn Tnh, NXB i Hc Quc Gia
TPHCM 2001.
[8] inh Vn Gng, L Thuyt Xc Sut V Thng K, NXB GioDc.
[9] T Vn Hng Nguyn Phi Kh (2000), i S Tuyn Tnh L Thuyt V Bi
Tp, NXB Thng k.
[10] Aapo Hyvarinen, Erkki Oja, (2000), Independent Component Analysis: Algorithm
and Applications, Neural Networks Research Centre Helsinki University of
Technology. Neural Networks, 13(4-5): 411 430
[11] L Hong Thi, L Minh Tr, Nguyn Thu Hng (6/2006), xut phng php
trch chn c trng mi cho bi ton nhn dng mt ngi, Hi tho Cng ngh
thng tin quc gia ln th IX, Lt, 15-17/6/2006.

128

[12] Trn Phc Tr (2006), Pht hin mt ngi bng Adaboost kt hp mng Nron,
Lun vn Thc s Cng Ngh Thng Tin, khoa cng ngh thng tin, H KHTN TPHCM
[13] Qing Chen (May 2, 2006), Hand Detection with a Cascade of Boosted Classifiers
Using Haar-like Features, Discover Lab, SITE, University of Ottawa,
www.discover.ottawa.ca
[14] Viola and Michael Jones(5/2004), Robust real-time face detection, International
Journal of Computer Vision, 57(2): 137-154.
[15] Pierre Comon (April,1994), Independent Component Analysis, Anew concept,
Vol. 36, no 3, Special issue on High order statistics, Appril 1994, pp 287 314
[16] Aapo Hyvarinen (1999), Fast and Robust Fixed point Algorithms for Independent
Component Analysis, Helsinki University of Technology Laboratory of Computer and
Information Science.
[17] Bruce A. Draper, Kyungim Baek, Marian Stewart Bartlett, J. Ross BeveRidge,
Recognizing Face with PCA and ICA Department of Computer Science Colorado State
University, Department of Biomedical Engineering Columbia University, University of
California of Sandiego Institute for Neural Computatation.
[18]

Marian Stewart Bartlet, Javier R.Movellan, Terrence J. Sejnowski, Face

Recognition by Independent Component Analysis,

IEEE Transactions on Neural

Networks, Vol. 13, No. 6, November 2002.

[19] Marian Stewart Bartlet, H. Martin Lades, Terrence J. Sejnowski, Independent
Component representation for face recognition, University of California Sandiego and
Salk Institute.
[20] V Minh Sn (2006), Tch nh dng bin i Wavelet v phn tch thnh phn
c lp, Lun vn Thc s khoa hc vt l, khoa in t trng i Hc KHTN
TPHCM

129

[21] Phan Duy Tun (2006), ng dng phn tch thnh phn c lp ICA trong lc
nhiu nh, Lun vn Thc s khoa hc vt l, khoa in t trng i Hc KHTN
TPHCM
[22] Trng Tn Quang, Nguyn Hu Phng (23/2/2006), Tch m dng phng
php phn tch thnh phn c lp, Tp ch pht trin KhoaHc v Cng Ngh, tp 9
s 2 nm 2006.
[23] Paul Cristea (2001) Independent Component Analysis (ICA) for Genetic signals,
University of Bucharest, Romania.
[24] C. J. C. Burges and A. J. Smola, editors (1999), Advances in Kernel Methods:
Support Vector Learning, The MIT Press, Cambridge, MA, 1999, pp 255-268.
[25] L Hong Thi (2004) Xy dng, pht trin, ng dng mt s m hnh kt hp
gia mng nron (NN), logic m (FL) v thut gii di truyn (GA), Lun n tin s ton
hc, H KHTN TPHCM. Ph lc C,D trang 1-17
[26] Hong Xun Hun, ng Th Thu Hin. An Efficient Algorithm for Training
Interpolation Radial Basis Function Networks, College of Technologies, Vietnam
National University.
[27] Hong Xun Hun, Nguyn Xun V, L Hong Thi. Developing An Interpolation
Radial Basis Function Networks for Verifying Human Face, College of Technologies,
Vietnam National University.
[28] Nguyn nh Thc[2000], Tr Tu Nhn To, Mng NRon Phng Php v
ng Dng, NXB Gio Dc 2000.
[29] Th vin itpp http://itpp.sourceforge.net/latest/
[30] Th vin intel MKL http://www.intel.com/cd/software/products/asmona/eng/307757.htm
[31] Intel Open Source Computer Vision Library OpenCV
http://www.intel.com/technology/computing/opencv/
[32] Fast Artificial Neural Network Library FANN

130

http://leenissen.dk/fann/
[33] www.codeproject.com

131