Professional Documents
Culture Documents
Gi thiu v cc phng php hun luyn Tm hiu mt s gii thut thng dng hun luyn ANN. Phn ny tp trung vo gii thut Gradient descent v cc gii thut ci tin ca n Hm mc tiu Mt li v cc im cc tiu cc b Mt s v d v phng php hun luyn mng bng MATLAB Qui trnh thit k mt ANN Cc k thut ph tr Hin tng qu khp ca ANN
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gio trnh ny ch tp trung vo phng php hc c gim st. Hai phng php cn li, sinh vin s c hc trong chng trnh Cao hc.
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
C nhiu phng php xc nh gradient g(k), dn ti c nhiu gii thut truyn ngc ci tin.
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
ai(k)
+ -
ti(k)
ei(k)
pj(k)
wRj(k)
pR(k)
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Lp n Ng ra n-ron n l a1i
pi
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
10
lp ra: w 2 ij ( k + 1) = w 2 ij ( k ) + i ( k ) a1 j ( k )
11
p1
w111= -1 w121= 0
a21
p2
1 b21= 1
w222= 1
a22
b22= 1
12
p1=0
a21
p2=1
a22
Gi s ta c ng vo p=[0 1] v ng ra mong mun t=[1 0] Ta s xem xt tng bc qu trnh cp nht trng s ca mng vi tc hc =0.1
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
13
p1=0
a21
p2=1
a22
a12 = 2 a11 = f1(n11) = n11 =(-1*0 + 0*1) +1 = 1 a12 = f1(n12)=n12 = (0*0 + 1*1) +1 = 2
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
14
p1=0
a21=2
p2=1
a22=2
a12 = 2 a21 = f2(n21) = n21 =(1*1 + 0*2) +1 = 2 a12 = f2(n22)=n22 = (-1*1 + 1*2) +1 = 2 Ng ra a2 khc bit nhiu vi ng ra mong mun t=[1 0]
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
15
a11 = 1
w111= -1
21=
1= -1
p2=1
2= -2
a12 = 2 Vi ng ra mong mun t =[1, 0], Ta c cc error ng ra: 1 = (t1 - a21 )= 1 2 = -1 2 = (t2 - a22 )= 0 2 = -2
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
16
p1=0
p2=1
a1
=2
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
17
p2=1
a12 = 2 w211= 1 + 0.1*(-1) = 0.9 w221= -1 + 0.1*(-2) = -1.2 w212= 0 + 0.1*(-2) = -0.2 w222= 1 + 0.1*(-4) = 0.6
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
18
p1=0
1= -1
p2=1
2= -2
S dng li cc trng s trc khi cp nht cho lp ra, tnh gradient lp n i (k ) = f 1 ' n1i (k ) j (k ) w ji
j
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
19
p1=0
w1 w112= 0
w111= -1
21=
1= -1
p2=1
w122= 1
2= -2 2 = -2
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
20
p1=0
w1
11=
-1
1= -1
p2=1
2= -2
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
21
p1=0
w111= -1 w121= 0
p2=1
22
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
23
a11 = 1.2
w211= 0.9 w221= -1.2 w212= -0.2 w222= 0.6
p2=1
a12 = 1.6
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
24
p1=0
w111= -1 w121= 0
0.9
a21 = 1.66
p2=1
= 1.6 Gi tr ng ra by gi l a2 = [1.66 0.32]gn vi gi tr mong mun t=[1 0] hn. Bi tp: T kt qu ny, sinh vin hy thc hin thao tc truyn ngc v cp nht trng s ANN 1 ln na. a1
2
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
a22 = 0.32
25
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
26
1 MSE = N
N i =1
ti ai
c gi l hm mc tiu
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
27
a2 0 1
1 a1
2
.2
1
28
net.LW{2,1}=[2 1]; net.b{1}=[.5; .2]; net.b{2}=0; p=[1 2 3; 0 1 1]; t=[2 1 2]; a=sim(net,p) mse(t-a) ans = 51.1067
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
29
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
30
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
31
M hnh ANN
gradient descent
e=-
Cc bc cn thit:
Thu thp v x l d liu vo ra ca i tng Chn la cu trc v xy dng ANN Hun luyn ANN bng gii thut gradient descent Kim tra chnh xc ca m hnh bng cc tn hiu khc
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
32
Chun b d liu hun luyn: ANN_Dcmotor.m load data_DCmotor; % nap tap du lieu hoc P=[datain'; dataout(:, 2:3)']; % [V(k), V(k-1), V(k-2), (k-1), (k-2)] T=dataout(:,1)'; % theta(k)
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
33
Nhn xt:
Tc hi t ca gii thut Gradient descent qu chm. Sau 5000 Epochs, MSE ch t 6.10-4. Kt qu kim tra cho thy li ln. Cn gii thut ci tin.
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
34
35
(rad)
10
15
20
25 Time(s)
30
35
40
45
50
36
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Cc tiu cc b
nh hng ca tc hc
Qu trnh hun luyn mng, gii thut cn vt qua cc im cc tiu cc b (v d: c th thay i h s momentum), t c im global minimum.
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Local minimum
global minimum
37
Mt li
38
t hiu qu hun luyn cao, nhiu tc gi nghi gi tng gi tr ca moment v tc hc nn gn bng 1: [0.8 1]; [0 0.2]
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
39
Nhn xt: Sau 5000 Epochs, MSE t 4.10-4, nhanh hn gii thut gradient descent.
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
40
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
41
Nhn xt: Sau 5000 Epochs, MSE t 10-4, nhanh hn gii thut gradient descent with momentum
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
42
(1)
L gii thut gradient descent ci tin, m c tc hc v h s momentum c thay i thch nghi trong qu trnh hun luyn. Vic thay i thch nghi c thc hin tng th nh vic thch nghi tc hc . Th tc cp nht trng s ging nh gii thut GD with momentum:
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
(2)
Nhn xt: Sau 5000 Epochs, MSE t 3.10-5, nhanh hn gii thut gradient descent with thch nghi
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
44
wij(k+1) c tng thm 1 lng inc Nu Ek & Ek +1 khc du: wij(k+1) c gim i 1 lng dec w w ij ij
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
45
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
46
Nhn xt: Sau 5000 Epochs, MSE t 8.10-6, nhanh hn gii thut gradient descent with & thch nghi
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
47
Nhn xt: Sau 1100 Epochs, MSE t 2.10-7, nhanh hn gii thut Resilient
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
48
49
Nhn xt: Sau 20 Epochs, MSE t 1,7.10-7, nhanh hn gii thut Newton
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
50
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
51
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
A
53
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
54
min ps l vect d liu sau khi chun ha, th: ps = 2 p p 1 max min
p p
Nu ta a tp d liu c x l vo hun luyn mng, th cc trng s c iu chnh theo d liu ny. Nn gi tr ng ra ca mng cn c thao tc hu x l. Gi a l d liu ra ca mng, at gi tr hu x l, th: 1 at = (a + 1)( pmax pmin ) + pmin 2
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
55
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
56
ps =
p mean p std p
Gi a l vect d liu ra ca ANN, th vect d liu ng ra sau khi thc hin thao tc hu x l:
at = a * std p + mean p
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
57
Mean 0 Std 1
Mean = 0 Std = 1
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
58
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
59
Hm mc tiu c nh ngha li bng cch thm vo i lng tng bnh phng trung bnh ca cc trng s v ngng, MSW, khi :
Vi
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Phng php ny i hi chia tp d liu hc thnh 3 phn, gm d liu hun luyn, d liu kim tra v d liu gim st. P = [Ptrain, Ptest, Pvalidation] Sau li thi k hun luyn, tp d liu dm st Pvalidation c a vo mng kim tra li. Nu li thu c gim, qu trnh hun luyn c tip tc. Nu li thu c bt u tng (hin tng qu khp bt u xy ra), qu trnh hun luyn c dng li gi l ngng sm. Sinh vin c thm ti liu [1].
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
61
Vi i(t) [0, 4A] dng in ng vo y(t) l khong cch t nam chm vnh cu n nam chm in. cc tham s: =12, =15, g=9.8 v M=3.
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
62
Bi tp
1. Sinh vin thc hin li bi ton nhn dng m hnh motor DC v hun luyn mng bng tt c cc gii thut ca NN toolbox ca MATLAB. So snh tc hi t ca cc gii thut.
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
63
Chng 4
Mt s ng dng ca ANN
Gii thiu Nhn dng k t (OCR) Nhn dng ting ni Thit k cc b iu khin Kt lun Bi tp
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
64
Gii thiu
Gii thiu mt s hng ng dng ANN Pht trin thnh Lun vn tt nghip hay ti NCKH sinh vin
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
65
Nhn dng k t
Ma trn ha bitmap ca k t Gi lp cc hnh thc nhiu Tp hp d liu hun luyn, mi k t l 1 vect d liu ng vo Qui c ng ra, gi s l m ASCII tng ng ca k t. Xy dng cu v hun luyn ANN
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
66
Trch c trng tn hiu ting ni Tp hp d liu vo, qui c d liu ra Hun luyn v th nghim Phng php LPC & AMDF xc nh c trng ting ni
Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
67