You are on page 1of 19

TRNG I HC QUC GIA H NI

I HC CNG NGH
-----o0o-----

BO CO
Mn: TR TU NHN TO NNG CAO

S DNG MNG NEURAL XY DNG NG


DNG NHN DNG CH VIT TAY

Ging vin hng dn : NNG TON


Hc vin thc hin

: NGUYN HU HUY
NGUYN XUN K
ON TH THY LINH

Lp

: K16T3

H ni, 4-2010

Mc lc
1. TNG QUAN V MNG NRON...............................................................4
1.1. Mng nron l g........................................................................................... 4
1.2. ng dng trong lnh vc g............................................................................4
2. CU TRC CA MT NRON..................................................................4

3. HC C GIM ST V HC KHNG C GIM ST.....................................5


3.1. Mng nron hc c gim st: .......................................................................5
3.2. Trong phng php hc khng c gim st:.................................................6
4. GII THUT LAN TRUYN NGC...........................................................6

5. LEARNING RATE....................................................................................7

6. HM ACTIVE.........................................................................................7
6.1. M hnh hm : y = 1 / (1 + Exp(-x))..............................................................8
6.2. M hnh hm : y = x......................................................................................8
6.3. Hnh 7.3: M hnh hm y = Log(1 + |x|)........................................................8
6.4. Hnh 7. 4: M hnh hm y = sin(x).................................................................9
6.5. Hnh 7.5: M hnh hm y=Tan(x)..................................................................9
7. BN T CHC KOHONEN SOM (KOHONEN SELF -ORGANIZING MAPS). .9

8. HM NEIGHBORHOOD..........................................................................11
8.1. Tng quan................................................................................................... 11
8.2. Hnh minh ha............................................................................................. 11
9. LI TOPOLOGY..................................................................................12

2/19

10. MT S HNH DNG CA LP KOHONEN ............................................12

11. THIT K MNG NRON.....................................................................13


11.1. S lng lp n (trong mng Backpropagation).......................................13
11.2. Mng neurol ban u...............................................................................14
11.3. S mu hun luyn...................................................................................14
12. CC LNH VC NG DNG MNG NRON............................................14
12.1. Gii quyt cc bi ton.............................................................................14
12.2. Phm vi ng dng.....................................................................................14
13. CI T MI TRNG PHT TRIN.....................................................15
13.1. Ngn ng lp trnh....................................................................................15
13.2. Cng c pht trin....................................................................................15
14. NEURONNETWORK CLASS .................................................................15
14.1. Neuron.Core namespace...........................................................................15
14.2. NeuronNetwork.Core.Initializers namespace.............................................16
14.3. NeuronNetwork.Core.LearningRateFunctions namespace.........................16
14.4. NeuronNetwork.Core.BackPropagation namespace..................................16
14.5. NeuronNetwork.Core.SOM namespace......................................................16
14.6. NeuronNetwork.Core.SOM.NeighborhoodFunctions namespace...............17
15. MT VI ON CODE HU DNG........................................................17
15.1. backpropagation network.........................................................................17
15.2. on m khi to mt Kohonen SOM........................................................18
16. CHNG TRNH NG DNG MNG NEURON ........................................18
16.1. Nhn dng k t bng cch v ch...........................................................19

3/19

1. Tng quan v mng nron


1.1.

Mng nron l g
Mng nron nhn to, Artificaial Neural Network (ANN) gi tt l mng nron
(neural network), l m hnh x l thng tin phng theo cch thc x l thng
tin ca cc h nron sinh hc. N c to nn t mt s lng ln cc phn t
(gi l phn t x l hay nron) kt ni vi nhau thng qua cc lin kt (gi l
trng s lin kt) lm vic nh mt th thng nht gii quyt mt vn c
th no .

1.2.

ng dng trong lnh vc g


Mt mng nron nhn to c cu hnh cho mt ng dng c th (nhn dng
mu, phn loi d liu,) thng qua mt qu trnh hc t cc mu hun luyn.
V bn cht hc chnh l qu trnh hiu chnh trng s lin kt gia cc nron

2. Cu trc ca mt nron
Cu trc tng qut

4/19

Gii thch k hiu:

Tp cc u vo: l cc tn hiu vo (input signal) ca nron, cc tn hiu


ny thng c a di dng vector N chiu

Tp cc lin kt: Mi lin kt c th hin bi mt trng s (gi l trng


s lin kt Synaptic weight). Thng thng cc trng s ny c khi
to mt cch ngu nhin thi im khi to mng v c cp nht
lin tc trong qu trnh hc mng.

B tng (Summing function): thng dng tnh tng ca cc tch u


vo vi trng s lin kt ca n

Ngng (cn gi l mt lch bias): Ngng ny thng c a vo


nh mt thnh phn ca hm truyn

Hm truyn (Transfer function): Hm ny c dng gii hn phm vi


u ra ca mi nron. N nhn u vo l kt qu ca hm tng v
ngng cho

u ra: L tn hiu u ra ca mi nron, vi mi nron s c ti a l


mt u ra

3. Hc c gim st v hc khng c gim st


Mt mng nron cn c o to trc khi n c th c a vo s dng.
Hun luyn lin quan n vic a vo mng nron mu hun luyn v cho
php n tm hiu bng cch hiu chnh trng s lin kt v cc thng s khc
nhau. mng n-ron c th c phn loi thnh hai loi da vo loi hc tp.

3.1.

Mng nron hc c gim st:


Trong phng php hc c gim st, mng nron hc t cc mu. Tp hun
luyn bao gm mt tp hp cc mu u vo v cc kt qu u ra mong
mun tng ng vi mu u vo. Cc mng nron iu chnh trng s lin kt
ca n tm hiu mi quan h gia cc cp u vo-u ra. Mng nron c
hun luyn thnh cng th c th h c s dng tm u ra ph hp nht
i vi bt k u vo hp l.
Mc tiu ca vic hc c gim st mt m hnh ton cc l tm ra mt hm f, khi
cho sn mt tp cc im c dng (x, f(x)).

5/19

3.2.

Trong phng php hc khng c gim st:


Mng nron ch nhn c mt tp hp cc u vo t mi trng bn ngoi. N
c v b n tng tng nhng g cc mng c th c th hc hi t thit lp
mt ch s u vo. Tuy nhin, c th chnh thc chng minh rng mt mng
li khng c gim st c th xy dng i din ca cc u vo c th c s
dng cho vic ra quyt nh.

4. Gii thut lan truyn ngc


hun luyn mng nron ta cung cp cho n mt b hun luyn v cho php
n hc bng cch iu chnh trng s ca cc lin kt mng. Mt tp hun luyn
l mt tp hp mu hun luyn.
Training Set = Set of training samples
Mt mu o to l mt cp ca mt vector u vo v mt mu vect u ra
mong mun. Trong trng hp o to khng c gim st, cc vector u ra
nn c null. Chiu di ca vector u vo nn c tng t nh s lng
cc nron trong lp u vo, v di vector u ra nn c bng s nron
trong lp u ra.
Training Sample = (input vector, desired vector)
Backpropagation thut ton l mt thut ton gim st thng c s dng
hun luyn cc mng feed-forward. N c gii thiu ln u tin bi Paul
Werbos trong cun sch 'The Roots ca Backpropagation'. tng c bn l xc
nh mng nron hot ng nh th no vi u vo mu, so snh khc nhau
nh th no gia cc hnh vi mong mun v sau iu chnh trng s ca cc
lin kt gim thiu s khc bit. Qu trnh ny lp i lp li cho tt c cc
mu o to trong nhiu ln thit lp m bo hun luyn ph hp.
Vic hun luyn mng MLP bi thut ton lan truyn ngc sai s bao gm hai
qu trnh: Qu trnh truyn thng v qu trnh truyn ngc. Trong qu trnh
truyn thng, cc vector u vo s c cung cp cho cc nron ca mng v
tn hiu s c lan truyn ln lt trn tng lp mng. Cui cng ta s tnh
c mt tp cc u ra thc s ca mng. Trong sut qu trnh truyn thng,
tt c cc trng s lin kt ca mng u c nh. Ngc li, trong qu trnh
truyn ngc, tt c cc trng s lin kt s c hiu chnh theo cc lut
hiu chnh trng s. Sai s ca mng s c o bng sai lch gia u ra
thu c vi cc gi tr mc tiu tng ng. Cc sai s ny sau s c lan
truyn ngc ln lt trn cc lp mng (t lp cui cng n lp u tin). Cc

6/19

trng s lin kt s c hiu chnh sao cho cc u ra thc s ca mng cng


gn vi cc gi tr mc tiu cng tt.

5. Learning Rate
Learning rate l mt trong nhng thng s m iu chnh vic lm th no
mt mng noron hc nhanh v lm th no vic hun luyn hiu qu.
Hy xem xt mt nron m ang tri qua qu trnh hc tp. Gi nh rng trng
s ca mt s lin kt trong mng mt phn c o to l 0,3. Khi mng c
gii thiu mt mu hun luyn mi, thut ton hun luyn yu cu cc lin kt
thay i trng s ca n n 0,7 n c th hc cc mu mi ph hp. Nu
chng ta cp nht trng s ngay lp tc, cc mng nron chc chn s hc cc
mu mi, nhng n c xu hng qun i tt c cc mu n hc trc .
iu ny l do trng s hin ti (0,3) l kt qu ca tt c vic hc m n tri
qua cho n nay.
V vy, chng ta khng trc tip thay i trng s ti 0,7. Thay vo , chng ta
tng n bi mt phn nh (chn 25%) ca s thay i cn thit. V vy, trng s
lin kt ca c thay i thnh 0,4 v chng ta chuyn sang mu o to tip
theo. Yu t ny (0,25 trong trng hp ny) c gi l Learning Rate. Cn c
theo cch ny, tt c cc mu hun luyn c hun luyn trong mt s th t
ngu nhin. Khi chu trnh o to lp i lp li nhiu ln, cui cng mng nron
hc tt c cc mu c hiu qu.
Learning rate l mt gi tr trong khong t 0 n 1. Chn mt gi tr rt gn
bng khng, i hi mt s lng ln cc chu trnh hun luyn. iu ny lm
cho qu trnh hun luyn rt chm. Mt khc, nu learning rate rt ln, trng s
khc nhau v lch hm mc tiu dao ng ln v mng t n mt trng
thi m vic hun luyn din ra v ch.

6. Hm Active
Hm active trong mng c xc nh l cch c c u ra ca neuron t
mt tp u vo da trn thut ton bakcpropagation.
Cc thut ton bakcpropagation yu cu mt hm active tha mn tnh lin
tc v kh vi. N yu cu cn c hm active d dng tnh ton.
Mt s hm active nh Sigmoid, Linear, Logarit, Tan, Sin

7/19

6.1.

M hnh hm : y = 1 / (1 + Exp(-x))

Hnh 7.1: M hnh hm : y = 1 / (1 + Exp(-x))

6.2.

M hnh hm : y = x

Hnh 7. 2: M hnh hm : y = x

6.3.

Hnh 7.3: M hnh hm y = Log(1 + |x|)

Hnh 7.3: M hnh hm y = Log(1 + |x|)

8/19

6.4.

Hnh 7. 4: M hnh hm y = sin(x)

Hnh 7. 4: M hnh hm y = sin(x)

6.5.

Hnh 7.5: M hnh hm y=Tan(x)

Hnh 7.5: M hnh hm y=Tan(x)

7. Bn t chc Kohonen SOM (Kohonen Self -Organizing


Maps)
Bn Kohonen SOM l phng php hc khng c gim st c s dng
rng ri gim tnh a chiu ca khng gian u vo nhng vn m bo
ng cu trc th ca n.
Mt kin trc SOM Kohonen in hnh c hin th di y. Bao gm mt lp
u vo kt ni vi mt lp u ra (Kohonen 2 chiu) thng qua mt Connector
Kohonen gm cc nrn Kohonen.
Mi t bo nrn trong mt lp Kohonen c lin kt vi mt tp cc nrn
khc trong khng gian hai chiu.

9/19

Lp t bo nron u vo vi n nron vi. Lp u ra t chc ring ca mnh


da vo u vo. V th gi l m hnh t t chc.

Trong giai on o to, mt SOM xy dng mt mu o to i din. Mng


li o to c th s dng bn vector u vo bt k l khng gian hai
chiu.
Mc tiu o to ca SOM l m bo rng cc phn khc nhau ca mng
phn ng tng t nh cc vector u vo.
V vy, vic o to ch yu lin quan n vic phn tch hnh vi ca cc mng
li cho mt mu o to v iu chnh trng lng ca mt phn t nron
m bo rng cc mng li tin hnh mt hnh vi tng t i vi mt u vo
ging nhau. Cc th tc lin quan n vic o to cc bc sau y:

Khi to trng nhng gi tr nh ngu nhin

Chn mt mu o to ngu nhin, giao cho vector u vo trn cc t


bo thn kinh v chy mng.

u ra ca mt neuron tng ng t l vi trng lng vector ca n v


cc vector u vo. Cc neuron u ra c gi tr cao nht s thng bo l
ngi chin thng cho cc neuron u vo hin hnh.

Tnh ton khong cch ca mi neuron u ra t nhng nron chin


thng bng cch s dng mt Hm Neighborhood

Cp nht cc thng s ca khp nron bng cch s dng cc cng thc


sau y.
a= Thng s gia ca nrol u vo
Trng s b= (Mc hc M) * (Gi tr so vi nrol hng xm) * a

Tng t nh vy, n s o to tt c cc mu theo mt th t ngu


nhin. iu ny hon tt thnh mt chu trnh o to.

10/19

Lp li cc bc hon thnh s quy nh ca cc th o to.

Cc SOM o to cc bn vector u vo bt k thnh mt neuron chin


thng, v c th c hiu nh l v tr ca vector trong khng gian hai chiu.

8. Hm Neighborhood
8.1.

Tng quan
Trong mt bn t t chc, hm neighborhood xc nh cc thng s ca mt
t bo nrol t t bo nrol ti nrol chin thng trong cng mt lp.
Gi tr Neighnorhood neuron b quyt nh bi vector trng s ca n vi
nhng thay i nrol chin thng.
N thay i t s khng (cho mt neuron khong cch xa v hn t ngi
chin thng) cui cng ngi chin thng l chnh n.

8.2.

Hnh minh ha
Mt s hm Neighborhood nh Gaussian v Mexican-Hat nh hnh v sau:

Hnh 9.1: Hm Gaussian Neighborhood

Hnh 9.2: Hm Mexican-Hat Neighborhood

11/19

9. Li Topology
Trong bn lp Kohonen Layer li topology quy nh c th s sp xp ca
nron li hai chiu to thnh lp. Hm Neighborhood c p dng trn
topology ny.
C 2 li Topology hay dng l hnh lc gic v hnh ch nht

Hnh 10.1: Rectangular Topology

Hnh 10.2: Hexagonal Topology

10.

Mt s hnh dng ca lp Kohonen


NeuronNetwork h tr cc hng v ct c th vo trn. Tnh nng ny c th
c dng to ra cc lp Kohonen vi hnh dng khc nhau.

12/19

Hnh 11.1: Dng phng

Hnh 11.2: Dng ng thng

Hnh

11.3:Dng hnh vng

Hnh 11.4: Dng hnh tr

11.

Hnh 11.5:Dng hnh xuyn

Thit k mng Nron


Thit k mt mng nron nhn to cho mt ng dng c th lin quan n vic
la chn ng loi mng, tm mt s thch hp ca cc lp n, phng php
thch hp khi to trng s, thut ton hc thch hp, cc th o to, t l
hc tp v s lng mu o to s dng. Hu ht cc thng s ny u ph
thuc vo ng dng m cc mng nron ang c thit k.
Di y l mt s hng dn chung v thit k mng nrron:

11.1.

S lng lp n (trong mng Backpropagation)

Mt mng li Backpropagation khng c lp n khng th thc hin phn loi


khng tuyn tnh. V vy, mt trong nhng lp n l phi cho mt mng li
backpropagation.
Hn na, n c ton hc chng minh rng mt mng li
backpropagation vi lp n duy nht khi o to ph hp, c th dng xp x
hm. V vy, lp n duy nht l s la chn tt nht trong hu ht trng hp.
C nhiu lp n tng tc qu trnh hc tp v mng c o to ph hp chnh
xc vi cc mu o to nhng khng thc hin tt trn cc d liu th nghim.
Hiu ng ny c gi l overtraining ni mng li hun luyn c xu hng ghi
nh cc mu hun luyn thay v hc tp chng.

13/19

11.2.

Mng neurol ban u

Khi to ng trng s lin kt nron c nh hng ln ti vic o to tc


cng nh xc nh hiu qu ca o to. Thng thng, trng s c khi to
vi gi tr ngu nhin t -0,5 n +0,5 (gi tr cao c xu hng kt qu trong
khu vc bo ha sau khi kch hot, cc gi tr ban u nh th ra cc gi tr gn
bng khng).
Mng nron thc hin qu trnh khi to nh l mt module pluggable. Tu
chnh cc thut ton khi to c th c cm vo bng cch thc hin
Initializer giao din.

11.3.

S mu hun luyn

Quyt nh nh th no cc mu i din cho cc chc nng hun luyn thc


t l tt nht. Thng thng, cc li hc hi tng so vi s gia tng kch thc
ca tp hun luyn, ng thi ta cng c th nhn thy rng gim li v mng
thc hin tt hn v d liu th nghim.
Mt mi quan h gia kch thc mng v s lng ti u ca mu o to c
th c tm thy.

12.

Cc lnh vc ng dng mng nron

12.1.

Gii quyt cc bi ton

Mng tr tu nhn to (Artificial neural networks) c ng dng gii quyt cc


bi ton trong cc lnh sau:

Bi ton v Function Modeling

Bi ton v phn loi (Classification Problems)

Ba ton x l v rt trch d liu (Data processing and feature


extraction)

12.2.

Phm vi ng dng

Phm vi ng dng ca tr tu nhn to rt ln: trong cc tp on ti chnh n


c dng phn tch v d on ti chnh, nhn dng ch k, nhn dng nh,
nhn dng ch vit, nhn dng sinh trc hc, m hnh ha cc h thng ng,
h thng t ng ha, h thng chuyn t dng ni ra ch vit, bc tch d liu,
tr tu nhn to dng trong tr chi my tnh, h thng xc nh ng i trong
giao thng vn ti, cc h thng my bay t li, etc.

14/19

13.

Ci t mi trng pht trin

13.1.

Ngn ng lp trnh

D n c vit trn ngn ng C#. Do vy bn cn phi s dng Microsoft .NET


framework.

13.2.

Cng c pht trin

S dng cng c pht trin l Visual Studio Express editions


(http://www.microsoft.com/Express/ )

14.

NeuronNetwork Class
Nhng thng tin m t v cc class c m t ti mc ny. V d nh: layers,
connectors, networks and TrainingSet. Tt c cc class ny c tha k t
class 'ISerializable' interface

14.1.

Neuron.Core namespace

INeuron : Interface m t mt nron thn kinh

ISynapse : Interface m t mt khp ni thn kinh trong mng

ILayer : Interface m t Layer trong mng nron

IConnector : Interface m t mt b kt ni (connector) (kt ni gia hai


layers)

INetwork : Interface m t mt mng nron

IInitializer : Interface dng khi to mt cch thc lm vic c th


ca cc lp v cc b kt ni trong mt mng c th.

ILearningRateFunction : Learning Rate Function interface.

Layer : Mt th hin mc tng qut ha ca interface 'ILayer' hay l


mt tp hp cc 'INeuron's

Connector : Mt th hin mc tng qut ha ca inter 'IConnector'


lin kt gia hai layers

Network : Mt lp c s th hin mt mng nron. N l trin khai


ca interface 'INetwork'.

TrainingSample : Lp ny l th hin mt cp cc vector: vector u

15/19

vo v vector u ra mong mun.

TrainingSet : L tp hp ca cc TrainingSample.

TrainingMethod : Th hin phng thc hc ca mng nron l hc c


gim st hay khng gim st

ConnectionMode : Th hin mt b k ni l mt-mt connector hay


nhiu-nhiu

14.2.

NeuronNetwork.Core.Initializers namespace

C rt nhiu cc phng thc khi to cho mng


nh: ConstantFunction, NGuyenWidrowFunction, ZeroFunction, Random
Function v NormalizedRandomFunction c m t trong namespace ny.

14.3.

NeuronNetwork.Core.LearningRateFunctions namespace

Namespace cha cc hm dng cho Learning rate


nh: HyperbolicFunction, ExponentialFunction v LinearFunction.

14.4.

NeuronNetwork.Core.BackPropagation namespace

ActivationNeuron : mt nron trong mng lan truyn ngc

BackpropagationSynapse : Cc b kt ni gia hai activation


neurons

ActivationLayer : mt lp tng qut ha ca activation neurons

BackpropagationConnector :

mt

tp

cc

BackpropagationSynapses kt ni gia hai activation layers

BackpropagationNetwork : Mt mng lan truyn ngc

LinearLayer, LogarithmLayer, SigmoidLayer, SineLayer v TanhLa


yer tha k lp ActivationLayer trin khai mt th hin tng ng
vi cc hm activation functions.

14.5.

NeuronNetwork.Core.SOM namespace

PositionNeuron : mt nron a Kohonen network

KohonenSynapse : Lin kt mt nron v mt PositionNeuron

KohonenLayer : mt layer ca PositionNeurons

KohonenConnector : mt tp hp cc KohonenSynapses lin kt cc

16/19

Kohonen Layers

KohonenNetwork : th hin mt Self-Organizing Map

LatticeTopology : ch nh dng li lattice topology l Hexagonal hay


Rectangular.

INeighborhoodFunction :

Interface

biu

din

mt

neighborhood

function

14.6.
NeuronNetwork.Core.SOM.NeighborhoodFunctions
namespace

Namespace

ny

bao

gm

cc

class

sau

GaussianFunction v MexicanHatFunction

15.

Mt vi on code hu dng

15.1.

backpropagation network

on m bn di to ra mt backpropagation network c mt lp u vo
l LinearLayer cha mi nron, mt lp n sigmoid cha nm nron v mt
lp u ra sigmoid cha by nron.

to mt mng nron, u tin cn khi to cc lp (layers), sau kt ni


cc lp bng cch to ra cc connectors v ri to ra mng nron bng cch
cung cp cc lp u vo v u ra. Khng c sa i cu trc ca mng sau
khi to ra n (Khng to ra bt k cc Layer hoc connector no khi to
mng thnh cng).
Cn ch lp u vo ca mt mng backpropagation lun lun l mt
linear layer bi v chng ta khng mun thay i d liu u vo thm lch
hay thc hin activation function.

17/19

15.2.

on m khi to mt Kohonen SOM.

Bt c lc no cng c th thay i thuc tnh ca cc layers v cc


connnectors.

Mt learning rate function c lin kt vi mi layer trong mng. Chng ta


c th thay i thuc tnh ny cho mi mt layer. Nu mun s dng duy nht
mt hm cho mi lp chng ta c th s dng hm SetLearningRate cho
network

dy mng ta cn cc mu v d (bao gm u vo v u ra). Chng ta cn


to ra mt tp hp cc mu training samples v chng ta c th dy mng
ny chng li cc training set.

16.

Chng trnh ng dng mng Neuron

18/19

16.1.

Nhn dng k t bng cch v ch.

y l mn hnh chng trnh nhn dng k t A bng cch v ch A.

Mn hnh dy chng trnh nhn dng k t A.

19/19

You might also like