Quatrinh Markov

Chng 1.
Qu trnh Markov

ng Hng Thng
Qu trnh ngu nhin v tnh ton ngu nhin.
NXB i hc quc gia H Ni 2007,
Tr 5 - 63.

T kho: Qu trnh ngu nhin, Qu trnh Markov, Xch Markov, Trng thi hu
han, Trng thi v hn m c.

Ti liu trong Th vin in t H Khoa hc T nhin c th s dng cho mc
ch hc tp v nghin cu c nhn. Nghim cm mi hnh thc sao chp, in n
phc v cc mc ch khc nu khng c s chp thun ca nh xut bn v
tc gi.

Chng 1
Qu trnh Markov
1.1 Xch Markov . . . . . . . . . . . . . . . . . . . . . 5
1.2 Phn loi trng thi xch Markov . . . . . . . . . 20
1.3 Qu trnh Markov . . . . . . . . . . . . . . . . . . 34
1.3.1 Trng hp khng gian trng thi hu hn . . . . 36
1.3.2 Trng hp khng gian trng thi v hn m c 42
1.3.3 Trng hp tng qut . . . . . . . . . . . . . . . . 54
1.4 Bi tp . . . . . . . . . . . . . . . . . . . . . . . . . 58
1.1 Xch Markov
Xt mt h no c quan st ti cc thi im ri rc 0, 1, 2, ... Gi s
cc quan st l X
0
, X
1
, ..., X
n
, ... Khi ta c mt dy cc i lng ngu
nhin (LNN) (X
n
) trong X
n
lg thi ca h ti thi im n. Gi thit
rng mi X
n
, n = 0, 1, ... l mt LNN ri rc. K hiu E l tp gi tr ca
cc (X
n
). Khi E l mt tp hu hn hay m c, cc phn t ca n
c k hiu l i, j, k... Ta gi E l khng gian trng thi ca dy.
6 Chng 1. Qu trnh Markov
nh ngha 1.1. Ta ni rng dy cc LNN (X
n
) l mt xch Markov nu
vi mi n
1
< ... < n
k
< n
k+1
v vi mi i
1
, i
2
, ...i
k+1
E
P{X
n
k+1
= i
k+1
|X
n
1
= i
1
.X
n
2
= i
2
..., X
n
k
= i
k
}
= P{X
n
k+1
= i
k+1
|X
n
k
= i
k
}.
Ta coi thi im n
k+1
l tng lai, n
k
l hin ti cn n
1
,...,n
k1
l qu
kh. Nh vy, xc sut c iu kin ca mt s kin B no trong tng
lai nu bit hin ti v qu kh ca h cng ging nh xc sut c iu kin
ca B nu ch bit trng thi hin ti ca h. chnh l tnh Markov ca
h. i khi tnh Markov ca h cn pht biu di dng: Nu bit trng thi
hin ti ca h th qu kh v tng lai c lp vi nhau.
Gi s P{X
m+n
= j|X
m
= i} l xc sut xch ti thi im m trng
thi i sau n bc, ti thi im m+n chuyn sang trng thi j. y l mt
con s ni chung ph thuc vo i, j, m, n. Nu i lng ny khng ph thuc
m ta ni xch l thun nht. Trong gio trnh ny ta ch xt xch Markov
thun nht.
K hiu
P
ij
= P{X
n+1
= j|X
n
= i}
P
ij
(n) = P{X
m+n
= j|X
m
= i}.
Ta gi (P
ij
, i, j E) l xc sut chuyn sau mt bc hay xc sut chuyn
cn (P
ij
(n), i, j E) l xc sut chuyn sau n bc. Ch rng
jE
P
ij
= 1
jE
P
ij
(n) = 1.
Phn b ca X
0
c gi l phn b ban u. Ta k hiu u
i
= P(X
0
= i).
nh l 1.1. Phn b ng thi ca (X
0
, X
1
, ..., X
n
) c hon ton xc
nh t phn b ban u v xc sut chuyn. C th ta c
P(X
0
= i
0
, X
1
= i
1
, ..., X
n
= i
n
) = u
i
0
P
i
0
i
1
...P
i
n1
in
.
1.1. Xch Markov 7
Tht vy theo cng thc nhn xc sut ta c
P(X
0
= i
0
, X
1
= i
1
, ..., X
n
= i
n
) =
= P(X
0
= i
0
)P(X
1
= i
1
|X
0
= i
0
) ...
P(X
k
= i
k
|X
0
= i
0
, ..., X
k1
= i
k1
) ...
P(X
n
= i
n
|X
0
= i
0
, ..., X
n1
= i
n1
).
S dng tnh Markov ta c
P(X
k
= i
k
|X
0
= i
0
, ..., X
k1
= i
k1
) = P(X
k
= i
k
|X
k1
= i
k1
)
= P
i
k1
i
k
.
Thnh th
P(X
0
= i
0
, X
1
= i
1
, ..., X
n
= i
n
) = u
i
0
P
i
0
i
1
...P
i
n1
in
nh l 1.2. (Phng trnh C - K (Chapman-Kolmogorov))
P
ij
(n +m) =
kE
P
ik
(n)P
kj
(m).
Chng minh. Theo cng thc xc sut y v tnh Markov ta c
P
ij
(n +m) = P(X
n+m
= j|X
0
= i)
=
kE
P(X
n
= k|X
0
= i)P(X
n+m
= j|X
n
= k, X
0
= i)
=
kE
P
ik
(n)P
kj
(m).
Trong trng hp E c d phn t, ta k hiu P = (P
ij
), P(n) = (P
ij
(n))
l cc ma trn vung cp dd. P c gi l ma trn xc sut chuyn, P(n)
c gi l ma trn xc sut chuyn sau n bc. Khi t phng trnh
Chapman-Kolmogorov tng ng vi
P(n +m) = P(n)P(m).
V P = P(1) nn bng quy np ta d thy
P(n) = P
n
.
Gi u
i
(n) = P(X
n
= i). K hiu vecto U(n) = (u
1
(n), ..., u
d
(n)) l vector
hng d - chiu m t phn b ca X
n
, U = U(0) = (u
1
, u
2
, ..., u
d
) l vector
hng d - chiu m t phn b ban u (phn b ca X
0
).
nh l 1.3. Ta c
U(m+n) = U(m)P
n
.
Ni ring
U(n) = UP
n
.
Chng minh. Tht vy, theo cng thc xc sut y ta c
u
j
(m+n) = P(X
n+m
= j) =
d
i=1
P(X
m
= i)P(X
n+m
= j|X
m
= i)
=
d
i=1
u
i
(m)P
ij
(n).
V d 1.1. Cho (
n
), n = 0, 1, 2, .. l dy cc LNN c lp, cng phn b.
Gi s P(
n
= i) = a
i
, i Z. t X
n
=
n
i=1
i
. Khi (X
n
) l mt xch
Markov vi khng gian trng thi Z.
P{X
n+1
= i
n+1
|X
0
= i
0
.X
1
= i
1
..., X
n
= i
n
}
= P{X
n
+
n+1
= i
n+1
|
0
= i
0
,
1
= i
1
i
0
, ...,
n
= i
n
i
n1
}
= P{
n+1
= i
n+1
i
n
} = P{X
n+1
= i
n+1
|X
n
= i
n
}.
Vy th (X
n
) l mt xch Markov vi khng gian trng thi Z. Xc sut
chuyn l P
ij
= a
ij
.
1.1. Xch Markov 9
V d 1.2. (M hnh Ehrenfest) Ta c hai bnh A, B v c d qu cu nh
s 1, 2, ...d. Ti thi im ban u c a qu cu trong A v da qu cu trong
B. Ti mi thi im n ta chn ngu nhin mt s trong tp {1, 2, ...d}. Khi
qu cu mang ch s c chn s c chuyn t bnh ang cha n sang
bnh kia. K hiu X
n
l s qu cu trong bnh A ti thi im n. Hin nhin
(X
n
) l xch Markov. Ta hy tnh xc sut chuyn P(X
n+1
= j|X
n
= i). V
A cha i qu cu nn vi xc sut i/d ta s chn c qu cu t A. Khi
qu cu ny c chuyn sang B. Vy P(X
n+1
= i 1|X
n
= i) = i/d. Tng
t vi xc sut 1 i/d s chn c qu cu ca B v qu cu ny s c
chuyn vo A. Vy P(X
n+1
= i + 1|X
n
= i) = 1 i/d. Thnh th
P
ij
=
_
_
i
d
nu j = i 1
d i
d
nu j = i + 1
0 nu j = i + 1, j = i 1.
(1.1)
M hnh ny c nh vt l ni ting Ehrenfest a ra nm 1907 nhm m
t s truyn nhit gia hai vt th.
V d 1.3. Ta nghin cu mt vn x hi no chng hn vn nghin
ht. Ta k hiu trng thi 0 l khng nghin v trng thi 1 l nghin. n
v thi gian l mt qu (3 thng). Thng k nhiu nm cho thy xc sut
mt ngi khng nghin sau mt qu vn khng nghin l 0,99 v xc
sut mt ngi nghin sau mt qu vn tip tc nghin l 0,88. Nh vy
trng thi ca mt ngi (nghin hay khng nghin) c m t bi mt xch
Markov vi hai trng thi E = {0, 1} vi ma trn xc sut chuyn nh sau
P =
_
0, 99 0, 01
0, 12 0, 88
_
Gi s lc u c 17% s ngi nghin. Nh vy phn b ban u l U(0) =
(0, 83, 0, 17). Sang qu hai, theo nh l 1.3 phn b s ngi nghin v khng
nghin s l
U(1) = U(0)P = (0.83, 0.17)
_
0, 99 0, 01
0, 12 0, 88
_
= (0, 845, 0, 155).
Sang qu ba na phn b s ngi khng nghin v nghin s l
U(2) = U(1)P = (0.845, 0.155)
_
0, 99 0, 01
0, 12 0, 88
_
= (0, 855, 0, 145)
tc l lc ny c 14,5% s ngi nghin.
V d 1.4. Gi s ta c d ca hng k hiu l 1, 2, ...d cng bn mt sn
phm no . Khch hng c th chn mua sn phm mt trong d ca hng
ny tu theo s thch ca h v trong tng thng h khng thay i ch mua
hng. Gi X
n
l ca hng m khch hng chn mua sn phm thng th
n. y l mt xch Markov c d trng thi, xc sut chuyn P
ij
c ngha l
xc sut khch hng, hin ti ang mua hng ti ca hng i sang thng
sau chuyn sang mua ca hng j. Xt d = 3 v ma trn xc sut chuyn
l
P =
_
_
_
0, 800 0, 100 0, 100
0, 070 0, 900 0, 030
0, 083 0, 067 0, 850
_
_
_
Gi s thng ging ca hng 1 chim 20% khch hng, ca hng 2 chim
50% khch hng v ca hng 3 chim 30% khch hng. Nh vy phn b ban
u l U(0) = (0, 2, 0, 5, 0, 3). Sang thng 2 phn b khch hng trong 3 ca
hng s l U(1) = U(0)P = (0, 22, 0, 49, 0, 29). Sang thng 3 phn b khch
hng trong 3 ca hng s l U(2) = U(1)P = (0, 234, 0, 483, 0, 283). Tip tc
qu trnh nh vy ta c th tnh thng 12 phn b khch hng trong 3 ca
hng s l U(11) = (0, 270, 0, 459, 0, 271) tc l trong thng 12 ca hng 1
chim 27% khch hng, ca hng 2 chim 45,9% khch hng v ca hng 3
chim 27,1% khch hng.
V d 1.5. Cho (X
n
) l mt xch Markov c 2 trng thi E = {0, 1} vi ma
trn xc sut chuyn l
P =
_
1 a a
b 1 b
_
1.1. Xch Markov 11
trong a, b > 0, 0 < a +b < 1. Ta hy tm biu thc ca ma trn xc sut
chuyn sau n bc P(n). Ta c
P
00
(n + 1) = P(X
n+1
= 0|X
0
= 0)
= P(X
n
= 0|X
0
= 0)P(X
n+1
= 0|X
n
= 0)+
+P(X
n
= 1|X
0
= 0)P(X
n+1
= 0|X
n
= 1)
= P
00
(n)(1 a) + (1 P
00
(n))b. (1.2)
Nu k hiu u
n
= P
00
(n), q = 1 a b t (1.2) ta c cng thc truy hi sau
u
n+1
= u
n
(1 a) + (1 u
n
)b = (1 a b)u
n
+b = qu
n
+b.
Ch rng u
0
= 1, t cng thc truy hi trn d thy
u
n
= q
n
+
b(1 q
n
)
1 q
= (1 a b)
n
+
b
a +b
(1 (1 a b)
n
)
hay
P
00
(n) =
b
a +b
+ (1 a b)
n
a
a +b
.
Suy ra
P
01
(n) = 1 P
00
(n) =
a
a +b
(1 a b)
n
a
a +b
.
Tng t ta c
P
10
(n + 1) = P(X
n+1
= 0|X
0
= 1)
= P(X
n
= 0|X
0
= 1)P(X
n+1
= 0|X
n
= 0)+
+P(X
n
= 1|X
0
= 1)P(X
n+1
= 0|X
n
= 1)
= P
10
(n)(1 a) + (1 P
10
(n))b. (1.3)
Nu k hiu v
n
= P
00
(n), q = 1 a b t (1.3) ta c cng thc truy hi sau
v
n+1
= v
n
(1 a) + (1 v
n
)b = (1 a b)v
n
+b = qv
n
+b.
Ch rng v
0
= 0, t cng thc trn ta thu c
v
n
=
b(1 q
n
)
(1 q)
=
b
a +b
(1 (1 a b)
n
).
Suy ra
P
10
(n) =
b
a +b
(1 a b)
n
b
a +b
P
11
(n) = 1 P
10
(n) =
a
a +b
+ (1 a b)
n
b
a +b
.
Vit di dng ma trn ta c
P(n) =
1
a +b
_
b a
b a
_
+
(1 a b)
n
a +b
_
a a
b b
_
.
nh ngha 1.2. Phn b ban u U = (u
i
), i E c gi l phn b dng
nu ta c U(n) = U vi mi n tc l u
i
(n) = u
i
i E, n. Khi dy
(X
n
) c cng phn b.
T nh l 3 ta suy ra U = (u
i
) l phn b dng nu v ch nu
1. u
i
0 v
iE
u
i
= 1
2. u
j
=
iE
u
i
P
ij
j E.
V d 1.6.
Cho (X
n
) l mt xch Markov c 3 trng thi E = {1, 2, 3} vi ma trn xc
sut chuyn l
P =
_
_
_
1/3 1/3 1/3
1/4 1/2 1/4
1/6 1/3 1/2
_
_
_
Hy tm tt c cc phn b dng.
t U = (x, y, z). Khi U l phn b dng khi v ch khi x, y, z l
nghim khng m ca h sau
_
_
x/3 +y/4 +z/6 = x
x/3 +y/2 +z/3 = y
x/3 +y/4 +z/2 = z
x +y +z = 1.
1.1. Xch Markov 13
T phng trnh th nht v th hai ca h kh z ta rt ra y = 5x/3. T
z = 3x/2. Th vo phng trnh (4) ta thu c x = 6/25, y = 10/25, z =
9/25.
V d 1.7.
Cho (X
n
) l mt xch Markov c 2 trng thi E = {0, 1} vi ma trn xc
sut chuyn l
P =
_
1 a a
b 1 b
_
trong a, b > 0, 0 < a +b < 1 (xem v d 1.5). Ta hy tm phn b dng.
t U = (x, y). Khi U l phn b dng khi v ch khi x, y l nghim
khng m ca h sau
_
_
(1 a)x +by = x
ax + (1 b)y = y
x +y = 1.
Phng trnh (1) v (2) ca h tng ng vi ax = by hay x =
by
a
. Th
vo phng trnh (3) ca h ta thu c
x =
b
a +b
, y =
a
a +b
.
Nh sau ny ta s thy phn b dng khng phi bao gi cng tn ti. Cu
hi t ra l: Vi iu kin no th tn ti phn b dng? Phn b dng nu
tn ti th c duy nht khng?
nh l 1.4. Gi s (X
n
) l xch Markov vi khng gian trng thi E =
{1, 2, ...} vi ma trn xc sut chuyn P = (P
ij
) v ma trn xc sut chuyn
sau n bc l P(n) = (P
ij
(n)). Gi s rng vi mi i, j E tn ti gii hn
lim
n
P
ij
(n) =
j
v gii hn ny khng ph thuc i. Khi
1.
jE
j
1 v
j
=
iE
i
P
ij
.
2. Hoc
j
= 0 vi mi j E, hoc
jE
j
= 1.
3. Nu
jE
j
= 1 th U = (
1
,
2
, ...) l phn b dng v phn b dng l
duy nht. Nu
j
= 0 vi mi j E th phn b dng khng tn ti.
Chng minh. 1. Theo b Fatou ta c
jE
j
=
jE
lim
n
P
ij
(n) liminf
n
jE
P
ij
(n) = 1.
S dng b Fatou v phng trnh C-K ta c
iE
i
P
ij
=
iE
lim
n
P
ki
(n)P
ij
liminf
n
iE
P
ki
(n)P
ij
= liminf
n
P
kj
(n + 1) =
j
.
t s
j
=
j
iE

i
P
ij
0 j E. Ta c
jE
s
j
=
jE
jE
iE
i
P
ij
=
jE
iE
jE
i
P
ij
=
jE
iE
jE
P
ij
=
jE
iE
i
= 0.
Vy s
j
= 0 j E hay
j
=
iE
i
P
ij
j E.
2. Ta c
j
=
iE
i
P
ij
=
iE
(
kE
k
P
ki
)P
ij
=
kE
k
(
iE
P
ki
P
ij
) =
kE
k
P
kj
(2).
1.1. Xch Markov 15
Bng quy np d thy vi mi n
j
=
kE
k
P
kj
(n).
V chui hi t u i vi n nn
j
= lim
n
kE
k
P
kj
(n) =
kE
k
lim
n
P
kj
(n) =
j
kE
k
Suy ra
j
_
1
kE
k
_
= 0 j E.
Vy nu
kE

k
< 1 th
j
= 0 j E.
3. Nu
kE
k
= 1 th t khng nh 1 ta suy ra = (
1
,
2
, ...) l phn
b dng. Ta chng minh y l phn b dng duy nht. Tht vy gi
s U = (u
i
) l phn b dng. Lp lun tng t nh trn ta c
u
j
=
kE
u
k
P
kj
(n).
V chui hi t u i vi n nn
u
j
=
kE
u
k
lim
n
P
kj
(n) =
kE
u
k
j
=
j
.
Do nu
jE
j
< 1 th phn b dng khng tn ti. Nu
jE
j
= 1 th
= (
1
,
2
, ...) l phn b dng duy nht.
nh ngha 1.3. Gi s (X
n
) l xch Markov vi khng gian trng thi
E = {1, 2, ...} vi ma trn xc sut chuyn P = (P
ij
) v ma trn xc sut
chuyn sau n bc l P(n) = P
ij
(n). Ta ni rng xch c phn b gii hn
nu vi mi i, j E tn ti gii hn
lim
n
P
ij
(n) =
j
.
Gii hn ny khng ph thuc i E v
jE
j
= 1. Ni cch khc, vecto gii
hn = (
1
,
2
, ...) lp thnh mt phn b xc sut trn E.
ngha ca phn b gii hn l nh sau: Gi u
i
(n) = P(X
n
= i). K
hiu vecto U(n) = (u
1
(n), u
2
(n), ...) l vector hng d-chiu m t phn b
ca X
n
. Ta c
P(X
n
= j) =
E
P(X
0
= i)P
ij
(n).
Do
lim
n
P{X
n
= j} =
E
P{X
0
= i} lim
n
P
ij
(n)
=
E
P{X
0
= i}
j
=
j
.
Vy phn b U(n) ca X
n
hi t ti phn b gii hn . Khi n kh ln ta
c P(X
n
= j)
j
.
Theo nh l 1.4 nu phn b gii hn tn ti th phn b dng cng
tn ti v duy nht. Hn na hai phn b ny trng nhau. Tuy nhin iu
ngc li khng ng tc l c nhng xch Markov c tn ti phn b dng
nhng khng tn ti phn b gii hn.
V d 1.8. Cho xch Markov (X
n
) c hai trng thi vi ma trn xc sut
chuyn l
P =
_
0 1
1 0
_
.
Ta c
P(2n) =
_
0 1
1 0
_
v
P(2n + 1) =
_
0 1
1 0
_
.
Do khng tn ti lim
n
P(n). Tuy nhin d dng kim tra c =
(1/2, 1/2) l phn b dng duy nht.
1.1. Xch Markov 17
Mt trong nhng bi ton quan trng trong nghin cu xch Markov l
tm nhng iu kin m bo s tn ti ca phn b gii hn v s tn
ti ca phn b dng. Di y l mt nh l nh vy.
nh l 1.5. Cho (X
n
) l xch Markov vi khng gian trng thi hu hn
E = {1, 2, ..., d} vi ma trn xc sut chuyn sau n bc l P(n) = (P
ij
(n)).
Khi c tn ti phn b gii hn = (
1
, ...,
d
) vi
j
> 0 j E khi v
ch khi xch l chnh quy theo ngha: Tn ti n
0
sao cho P
ij
(n
0
) > 0, i, j E.
Chng minh. Gi thit xch l chnh quy. Ta c nh j v t
m
j
(n) = min
iE
P
ij
(n)
M
j
(n) = max
iE
P
ij
(n).
Ta c
P
ij
(n + 1) =
k
P
ik
P
kj
(n)
k
P
ik
m
j
(n) = m
j
(n).
Suy ra m
j
(n + 1) m
j
(n). Vy dy (m
j
(n)), n = 1, 2, ... l dy tng v b
chn trn bi 1, do tn ti gii hn
lim
n
m
j
(n) = a
j
.
Lp lun tng t dy (M
j
(n)), n = 1, 2, ...) l dy gim b chn bi 0, do
tn ti gii hn
lim
n
M
j
(n) = A
j
.
Ta c m
j
(n) P
ij
(n) M
j
(n) do nh l c chng minh nu ta ch ra
a
j
= A
j
. K hiu r = min
i,j
P
ij
(n
0
) > 0. Ta c P
ik
(n
0
) r.1 P
jk
(n) nn
P
ik
(n
0
) rP
jk
(n) i, thnh th
P
ij
(n
0
+n) =
k
P
ik
(n
0
)P
kj
(n)
=
k
(P
ik
(n
0
) rP
jk
(n)) P
kj
(n) +r
k
P
jk
(n)P
kj
(n)
m
j
(n)
k
(P
ik
(n
0
) rP
jk
(n)) +rP
jj
(2n)
= m
j
(n)(1 r) +rP
jj
(2n).
V bt ng thc ny ng vi mi i nn ta c
m
j
(n
0
+n) m
j
(n)(1 r) +rP
jj
(2n).
Tng t ta c
M
j
(n
0
+n) M
j
(n)(1 r) +rP
jj
(2n).
Suy ra
M
j
(n
0
+n) m
j
(n
0
+n) (1 r) (M
j
(n) m
j
(n)) . (1.4)
Ta chng minh quy np rng vi mi k
M
j
(kn
0
+ 1) m
j
(kn
0
+ 1) (1 r)
k
(M
j
(1) m
j
(1)) . (1.5)
Tht vy vi k = 1 ng (Cho n = 1 (1.4)). Gi s ng vi k. Ta c
M
j
((k + 1)n
0
+ 1) m
j
((k + 1)n
0
+ 1)
= M
j
(kn
0
+ 1 +n
0
) m
j
(kn
0
+ 1 +n
0
)
(1 r) ((M
j
(kn
0
+ 1) m
j
(kn
0
+ 1))
(1 r)
k+1
(M
j
(1) m
j
(1)) .
Cho k trong (1.5) ta nhn c A
j
a
j
0. V A
j
a
j
0 nn ta kt
lun A
j
= a
j
.
o li gi s vi mi i, j E tn ti lim
n
P
ij
(n) =
j
> 0. Khi tn
ti n
0
(i, j) sao cho P
ij
(n) > 0 n > n
0
(i, j). t n
0
= max
i,j
n
0
(i, j) ta c
P
ij
(n) > 0 i, j En > n
0
V d 1.9. Mi ngi dn trong mt vng no c th trong ba tng lp:
giu, trung lu v ngho. Con ci ca h c th trong mt trong ba tng lp
ni trn vi cc xc sut khc nhau tu thuc vo vic h ang trong tng
lp no. Gi s bng thng k ngi ta xc nh c: Nu mt ngi giu
th vi xc sut 0,448 con h giu, vi xc sut 0,484 con h trung lu vi
xc sut 0,068 con h ngho. Tng t, vi mt ngi trung lu th xc sut
con h giu, trung lu hay ngho tng ng l 0,054. 0,699 v 0,247. Vi
1.1. Xch Markov 19
mt ngi ngho th xc sut con h giu, trung lu hay ngho tng ng
l 0,011, 0,503 v 0,486. Nh vy s thay i trng thi ca mt gia nh
trong x hi t th h ny qua th h khc c th m t bi mt xch Markov
ba trng thi : 1(giu), 2(trung lu), 3(ngho) vi xc sut chuyn nh sau
P =
_
_
_
0, 448 0, 484 0, 068
0, 054 0, 699 0, 247
0, 011 0, 503 0, 486
_
_
_
.
Xch Markov ny l chnh quy. Thnh th tn ti phn b gii hn =
(
1
,
2
,
3
). Phn b ny chnh l phn b dng duy nht v c tm bng
cch gii h phng trnh sau
(
1
,
2
,
3
)P = (
1
,
2
,
3
).
Gii ra ta tm c
1
= 0, 067;
2
= 0, 624;
3
= 0, 369. Nh vy qua nhiu
th h vng dn c ni trn s c 6,7% ngi giu, 62,4% trung lu v
36.9% ngi ngho.
V d 1.10. Xt xch Markov c d trng thi E = {1, 2, ..., d} v ma trn
xc sut chuyn ca n l chnh quy ng thi l mt ma trn kp ngha l
jE
P
ij
=
iE
P
ij
= 1.
Theo nh l trn, phn phi gii hn tn ti. Ta hy tm phn b gii hn
.Ta nhn xt rng phn b u = (1/d, 1/d, ..., 1/d) l phn b dng. Tht
vy t pi
j
= 1/d ta c
kE
pi
k
P
kj
= 1/d
kE
P
kj
= 1/d =
j
.
V phn b dng l duy nht v phn b gii hn chnh l phn b dng nn
ta kt lun rng phn b gii hn l phn b u = (1/d, 1/d, ..., 1/d) .
Chng hn ta tung con xc sc lin tip mt cch c lp. K hiu
n
l
s chm xut hin ln gieo th n, S
n
=
n
k=1
k
. S
n
l mt xch Markov
vi khng gian trng thi E = {1, 2, ...}. Gi X
n
l s d khi chia S
n
cho
7. Khi X
n
cng l mt l mt xch Markov vi khng gian trng thi
E = {0, 1, 2, ..., 6}. Ma trn xc sut chuyn ca X
n
l
P =
_
_
_
_
_
_
_
_
_
_
_
_
_
0 1/6 1/6 1/6 1/6 1/6 1/6
1/6 0 1/6 1/6 1/6 1/6 1/6
1/6 1/6 0 1/6 1/6 1/6 1/6
1/6 1/6 1/6 0 1/6 1/6 1/6
1/6 1/6 1/6 1/6 0 1/6 1/6
1/6 1/6 1/6 1/6 1/6 0 1/6
1/6 1/6 1/6 1/6 1/6 1/6 0
_
_
_
_
_
_
_
_
_
_
_
_
_
.
Xch Markov ny chnh quy v ma trn xc sut chuyn sau 2 bc
P(2) = P
2
=
_
_
_
_
_
_
_
_
_
_
_
_
_
1/6 5/36 5/36 5/36 5/36 5/36 5/36
5/36 1/6 5/36 5/36 5/36 5/36 5/36
5/36 5/36 1/6 5/36 5/36 5/36 5/36
5/36 5/36 5/36 1/6 5/36 5/36 5/36
5/36 5/36 5/36 5/36 1/6 5/36 5/36
5/36 5/36 5/36 5/36 5/36 1/6 5/36
5/36 5/36 5/36 5/36 5/36 5/6 1/6
_
_
_
_
_
_
_
_
_
_
_
_
_
c tt c cc phn t l s dng.
Vy phn b gii hn l
= (1/7, 1/7, 1/7, 1/7, 1/7, 1/7, 1/7, 1/7).
1.2 Phn loi trng thi xch Markov
gii quyt y hn bi ton v s tn ti ca phn b dng cng nh
bi ton v s tn ti ca phn b gii hn dn ta n vic phn loi cc
trng thi ca xch Markov nh sau:
1.2. Phn loi trng thi xch Markov 21
nh ngha 1.4. Ta ni rng trng thi i n c trng thi j v k hiu l
i j nu tn ti n o sao cho P
ij
(n) > 0. ( Ta quy c P
ii
(0) = 1, P
ij
(0) =
0 nu i = j)
Hai trng thi i v j c gi l lin lc c nu i j v j i. Trong
trng hp ta vit i j.
B 1.1 (Tnh cht bc cu). Nu i j, j k th i k.
Tht vy theo gi thit tn ti n, m sao cho
P
ij
(n) > 0, P
jk
(m) > 0.
Theo phng trnh Chapman - Kolmogorov ta c
P
ik
(n +m) =
jE
P
ij
(n)P
jk
(m) P
ij
(n)P
jk
(m) > 0.
T b , d kim tra rng quan h "lin lc c" l mt quan h tng
ng trn khng gian trng thi E. Theo quan h ny khng gian E c
phn hoch thnh cc lp ri nhau. Hai trng thi bt k cng thuc mt
lp th lin lc c vi nhau, hai trng thi khc lp khng th lin lc c
vi nhau.
nh ngha 1.5. Xch Markov c gi l ti gin nu hai trng thi bt k
l lin lc c. C ngha l theo cch phn lp trn th E khng th phn
hoch thnh cc lp con nh hn.
Nu xch khng ti gin th E c phn hoch thnh cc lp ri nhau
E = E
1
E
2
... E
k
. C th xem mi E
k
l khng gian trng thi ca xch
Markov ti gin. Nh vy vic nghin cu xch Markov c th quy v vic
nghin cu cc xch ti gin.
V d 1.11. Cho xch Markov vi 5 trng thi E = {1, 2, 3, 4, 5} vi ma trn
xc sut chuyn l
P =
_
P
1
0
0 P
2
_
trong
P
1
=
_
1/3 2/3
1/4 3/4
_
v
P
2
=
_
_
_
0 1 0
1/2 0 1/2
0 1 0
_
_
_
.
Khi
P(n) = P
n
=
_
P
n
1
0
0 P
n
2
_
.
Do E = E
1
E
2
vi E
1
= {1, 2}, E
2
= {3, 4, 5}.
V d 1.12. Cho xch Markov vi 4 trng thi E = {1, 2, 3, 4} vi ma trn
xc sut chuyn l
P
2
=
_
_
_
_
_
0 0 1/2 1/2
0 0 1/2 1/2
1/2 1/2 0 0
1/2 1/2 0 0
_
_
_
_
_
Xch ny l ti gin. Tht vy r rng 1 3, 1 4, 2 3, 2 4. Ta c
1 3, 3 2 suy ra 1 2. Li c 2 3, 3 1 suy ra 2 1. Vy 1 2 .
Tng t ta c 3 4. Vy hai trng thi bt k l lin lc c do y
l xch ti gin.
nh ngha 1.6. Chu k ca trng thi i k hiu l d(i) l c chung ln
nht ca tt c cc s nguyn dng n 1 m P
ii
(n) > 0. Nu P
ii
(n) = 0
vi mi n 1 th ta quy c t d(i) = 0.
nh l 1.6. Nu i j th d(i) = d(j). Vy cc trng thi cng mt lp c
cng mt chu k d v ta gi s d chung l chu k ca lp.
Chng minh. Do i j nn tn ti k, l sao cho P
ij
(k) > 0, P
ji
(l) > 0. Theo
phng trnh C-K ta c P
ii
(k + l) =
hE
P
ih
(k)P
hi
(l) P
ij
(k)P
ji
(l) > 0.
Vy d(i)|k +l. Gi s n 1 sao cho P
jj
(n) > 0. S dng phng trnh C-K
nh trn ta c P
ii
(k +l +n) P
ij
(k)P
jj
(n)P
ji
(l) > 0. Vy d(i)|k +l +n
d(i)|n. Vy d(i)|d(j). Tng t d(j)|d(i). Thnh th d(i) = d(j).
Gi s d l chu k ca mt xch ti gin vi khng gian trng thi E.
Nu d = 1 ta ni rng xch khng c chu k. Nu d > 1 th c th chng
minh rng E c phn hoch thnh d tp conE = C
0
C
1
... C
d1
sao
cho sau mt bc h s chuyn t mt trng thi thuc C
k
sang mt trng
thi thuc C
k+1
(quy c C
d
= C
0
). V vy mi tp con c th ly lm khng
gian trng thi ca mt xch Markov mi. Xch ny ti gin v khng c chu
k. Tm li chng ta c th quy vic nghin cu xch Markov tng qut v
vic nghin cu xch ti gin, khng c chu k.
nh ngha 1.7. K hiu f
ii
(n) l xc sut h xut pht t i ln u tin
quay li i thi dim n. Ngha l
f
ii
(n) = P (X
n
= i, X
n1
= i, ..., X
1
= i|X
0
= i)
v k hiu
f
ii
=
n=1
f
ii
(n)
l xc sut h xut pht t i quay tr li i sau mt s hu hn bc. Nu
f
ii
= 1 ta ni i l trng thi hi quy (quay li). Nu tri li f
ii
< 1 ta ni i
l trng thi khng hi quy.
nh l c bn sau y cho ta mt tiu chun xc nh tnh hi quy
ca mt trng thi.
nh l 1.7. Trng thi i l hi quy khi v ch khi
n=1
P
ii
(n) = . (1.6)
Chng minh. Chng minh s dng hai b sau
B 1.2. Ta c
P
ii
(n) =
n
k=0
f
ii
(k)P
ii
(n k)
trong f
ii
(0) = 0.
Chng minh b 1.2. Vi mi 0 k n gi E
k
l bin c :" X
n
= i v
h ln u tin quay li i bc th k " ta c
P
ii
(n) = P(X
n
= i|X
0
= i) =
n
k=0
P(E
k
)P(X
n
= i|E
k
, X
0
= 1)
=
n
k=0
f
ii
(k)P(X
n
= i|X
k
= i) =
n
k=0
f
ii
(k)P
ii
(n k).
B 1.3. (Aben)
(i) Nu chui
k=0
a
k
= a hi t th
lim
s1
k=0
a
k
s
k
=
k=0
a
k
= a.
(ii) Nu a
k
0 v lim
s1
k=0
a
k
s
k
= a < th
k=0
a
k
= lim
s1
k=0
a
k
s
k
= a.
Ta bt u chng minh nh l. Xt cc chui lu tha sau
P
ii
(s) =
n=0
P
ii
(n)s
n
, , |s| < 1
F
ii
(s) =
n=0
f
ii
(n)s
n
, , |s| < 1
Ta c
F
ii
(s)P
ii
(s) =
n=0
c
k
s
k
, |s| < 1
vi c
n
=
n
k=0
f
ii
(k)P
ii
(n k) = P
ii
(n) n 1 theo b 1. V c
0
=
0, P
ii
(0) = 1 nn ta suy ra F
ii
(s)P
ii
(s) = P
ii
(s) 1 hay
P
ii
(s) =
1
1 F
ii
(s)
. (1.7)
Gi s i hi quy tc l
n=0
f
ii
(n) = 1. Theo b Abel
lim
s1
F
ii
(s) = lim
s1
n=0
f
ii
(n)s
n
= 1.
T (1.7) suy ra
lim
s1
P
ii
(s) = lim
s1
n=0
P
ii
(n)s
n
= .
Vy li theo b Abel (ii) ta c
n=0
P
ii
(n) = .
o li gi s i khng hi quy tc l
n=0
f
ii
(n) < 1. S dng b Abel (i)
v h thc (1.7) ta rt ra lim
s1
P
ii
(s) < . Li p dng b Abel (ii) ta
thu c
n=0
P
ii
(n) < .
nh l 1.8. Nu i j v j hi quy th i hi quy.
Chng minh. Theo gi thit tn ti m, n sao cho P
ij
(n) > 0, P
ji
(m) > 0. Vi
mi s nguyn dng h t phng trnh C-P suy ra
P
ii
(n +h +m) P
ij
(n)P
jj
(h)P
ji
(m).
Vy
h=1
P
ii
(n +h +m) P
ij
(n)P
ji
(m)
h=1
P
jj
(h) = .
Thnh th i hi quy.
V d 1.13. Cho (r
n
) l dy cc LNN c lp c phn b xc sut nh sau
P(r
n
= 1) = p, P(r
n
= 1) = q, 0 < p < 1, p +q = 1.
(Dy ny c gi l dy Rademakher). Xt dy (X
n
) xc nh nh sau
X
0
= a, X
n+1
= X
n
+r
n+1
.
. Khi (X
n
) lp thnh xch Markov vi khng gian trng thi E = {0
1, 2}xc sut chuyn l P = (P
ij
)
P
ij
=
_
_
q nu j = i 1
p nu j = i + 1
0 nu j = i + 1, j = i 1.
Xch ny c gi l du ng ngu nhin 1 chiu m t s chuyn ng ngu
nhin ca mt ht trn ng thng: Sau mi n v thi gian ht dch sang
phi vi xc sut p v dch sang tri vi xc sut q. D thy y l mt xch
ti gin c chu k d = 2 v
P
ii
(2n) =
_
2n
n
_
p
n
q
n
. P
ii
(2n + 1) = 0.
S dng cng thc Stirling
n!
2ne
n
n
n
ta c
P
ii
(2n)
(4pq)
n
n
.
Nu du ng ngu nhin l i xng p = q = 1/2 th P
ii
(2n)
1
n
do
n
P
ii
(n) = .
Vy theo nh l 1.7 mi trng thi u hi quy. Nu p = q th 4pq = a < 1
cho nn
n
P
ii
(n) < v chui
a
n
n
hi t khi a < 1.
Do theo nh l 1.7 mi trng thi l khng hi quy. V mt trc gic ta
thy nu p > q th c mt xc sut dng ht xut pht t trng thi i s
i sang bn phi mi mi ( sang bn tri mi mi nu p < q) khng quay li
im xut pht.
V d 1.14. Xt du ng ngu nhin ca mt ht trn li im nguyn trn
mt phng. Gi s xc sut ht dch ln trn, dch xung di mt n v
(theo phng thng ng), dch sang phi,sang tri mt n v (theo phng
nm ngang) u bng nhau v bng 1/4. C th thy rng P
00
(2n + 1) = 0
v
P
00
(2n) =
i,ji+j=n
(2n)!
i!i!j!j!
(1/4)
2n
= (1/4)
2n
_
2n
n
_
n
i=0
_
n
i
__
n
n i
_
= (1/4)
2n
_
2n
n
_
2
.
Cng thc Stirling cho ta
P
00
(2n)
1
n
.
Vy
n
P
00
(n) = .
Vy trng thi 0 l hi quy. V xch l ti gin (d thy) nn mi trng thi
u hi quy.
Ngi ta chng minh c mt iu th v l vi du ng ngu nhin
i xng trong khng gian ba chiu, mi trng thi u khng hi quy.
nh l 1.9. K hiu Q
ii
l xc sut h xut pht t i quay li i v s
ln, Q
ij
l xc sut h xut pht t i i qua j v s ln. Khi
(i) Nu i hi quy th Q
ii
= 1, nu i khng hi quy th Q
ii
= 0.
(ii) Nu i hi quy i j th Q
ij
= 1. Ni ring, vi xc sut 1 h xut pht
t i sau mt s hu hn bc s i qua j.
Chng minh.
(i) Gi s Q
(m)
ii
l xc sut h quay li i t nht m ln. S dng cng
thc xc sut y v tnh Markov ca h ta thu c phng trnh
Q
(m)
ii
=
k=1
f
ii
(k)Q
(m1)
ii
= f
ii
Q
(m1)
ii
T
Q
m
ii
= (f
ii
)
m1
Q
1
ii
= (f
ii
)
m
v r rng Q
1
ii
= f
ii
. V rng Q
ii
= lim
m
Q
(m)
ii
ta rt ra Q
ii
= 0 hay 1
tu theo f
ii
< 1 hay bng 1.
(ii) Gi f
ji
=
k=1
f
ji
(k) l xc sut h xut pht t j s ving thm i
sau mt s hu hn bc. S dng cng thc xc sut y v tnh
Markov ca h ta thu c phng trnh
Q
jj

k=1
f
ji
(k)Q
ij
+ 1 f
ji
= Q
ij
f
ji
+ 1 f
ji
. (1.8)
(1.9)
V j hi quy theo (1) Q
jj
= 1. Vy t (??)
1 Q
ij
f
ji
+ 1 f
ji
f
ji
Q
ij
f
ji
.
V j i nn f
ji
> 0 Suy ra Q
ij
= 1.
V d 1.15. (S ph sn chc chn ca ngui chi c bc) Mt ngi vo
sng bc vi s tin trong ti l 1000 la v chi nh bc nh sau: Mi
vn chi anh ta tung mt ng tin cn i ng cht. Nu ng tin ra mt
nga anh ta c mt la, nu ra mt sp anh ta mt mt la. Gi X
n
l s tin anh ta c sau n vn chi. Khi X
0
= 1000 v X
0
, X
1
, ... l mt
du ng ngu nhin i xng vi trng thi ban u X
0
= 1000. Theo nh
l trn vi xc sut 1 X
n
s ving thm trng thi 0. Ni cch khc sm hay
mun anh ta s nhn ti.
nh l 1.10. Cho (X
n
) l xch ti gin khng hi quy. Khi vi mi i, j
n=1
P
ij
(n) < .
Ni ring
lim
n
P
ij
(n) = 0
v xch khng tn ti phn b dng.
Chng minh. Chng minh tng t nh b 2 ta c
P
ij
(n) =
n
k=1
f
ij
(k)P
jj
(n k). (1.10)
Vy
n=1
P
ij
(n) =
n=1
n
k=1
f
ij
(k)P
jj
(n k)
=
k=1
f
ij
(k)
n>k
P
jj
(n k)
=
k=1
f
ij
(k)
m=1
P
jj
(m)
= f
ij
m=1
P
jj
(m)
m=1
P
jj
(m) < .
nh l 1.11. Cho (X
n
) l xch ti gin hi quy khng c chu k . Khi
vi mi i, j
lim
n
P
ij
(n) =
1
j
=
k=1
kf
jj
(k).
Chng minh. Chng minh da c bn trn mt kt qu sau y ca gii tch
m ta s pht biu di dng mt b ( khng chng minh):
B 1.4. Cho (f
n
) l mt dy cc s khng m c tng bng 1 v c
chung ln nht ca tt c cc s j > 0 m f
j
> 0 bng 1. Cho (u
n
) l dy
xc nh truy hi theo cch sau
u
0
= 1, u
n
=
n
k=1
f
k
u
nk
.
Khi
lim
n
u
n
=
1
k=1
kf
k
.
C nh j. t u
n
= P
jj
(n), f
k
= f
jj
(k). B 4 cho php ta kt lun
lim
n
P
jj
(n) =
1
j
.
Tip theo vi quy c P
ij
(s) = 0 nu s < 0 ta vit li h thc (1.10) di
dng
P
ij
(n) =
k=1
f
ij
(k)P
jj
(n k).
Ch rng
lim
n
P
jj
(n k) =
1
j
v
k=1
f
ij
(k) = f
ij
= Q
ij
= 1 ( theo nh l 1.7) p dng nh l hi t b
chn ta thu c
lim
n
P
ij
(n) =
k=1
f
ij
(k) lim
n
P
jj
(n k)
= f
ij
1
j
=
1
j
.
Nu i l trng thi hi quy th
i
chnh l thi gian trung bnh ( hay s
bc trung bnh) h ln u tin quay li i. Thi gian trung bnh ny c
th hu hn hay v hn. Ta c nh ngha sau:
nh ngha 1.8. Trng thi hi quy i c gi l trng thi hi quy dng
nu
i
< v c gi l trng thi hi quy khng nu
i
= .
V d 1.16. Gi s (X
n
) l du ng ngu nhin i xng trn ng thng.
Trong v d ta thy mi trng thi l hi quy. Ta chng minh mi trng
thi l hi quy khng. Tht vy ta c
P
ii
(2n) =
_
2n
n
_
(1/2)
n
(1/2)
n
, P
ii
(2n + 1) = 0.
Thnh th
P
ii
(s) =
n
P
ii
(n)s
n
=
m
_
2m
m
_
(1/2)
2m
s
2m
=
m
_
2m
m
_
(s
2
/4)
m
= ((1 s
2
)
1/2
.
T phng trnh (1.7) ta suy ra
F
ii
(s) = 1 P
ii
(s)
1
= 1 (1 s
2
)
1/2
. (1.11)
Theo b Abel
i
=
k
kf
ii
(k) = lim
s1
k
kf
ii
(k)s
k1
= lim
s1
F
ii
(s) = lim
s1
s(1 s
2
)
1/2
= .
nh l 1.12. Gi s i j. Nu i hi quy dng th j hi quy dng. Nu
i hi quy khng th j hi quy khng.
Chng minh. Ta ch cn chng minh nu tn ti trng thi j l hi quy
dng th mi trng thi i m i j cng hi quy dng. Tht vy tn ti
m, n sao cho P
ij
(n) > 0, P
ji
(m) > 0. Vi mi k ta c
P
ii
(n +k +m) P
ij
(n)P
jj
(k)P
ji
(m).
T v t nh l 1.11 ta c
1
i
= lim
k
P
ii
(n +k +m)
lim
k
P
ij
(n)P
jj
(k)P
ji
(m) = P
ij
(n)P
ji
(m)
1
j
> 0.
Vy
i
< tc i l hi quy dng. Theo nh l 1.4, = (
1
,
2
, ...) l phn
b gii hn duy nht ca xch.
T nh l 1.11 v 1.12 suy ra
nh l 1.13. Gi s (X
n
) l xch ti gin khng c chu k vi khng gian
trng thi m c E. Khi s xy ra mt trong ba kh nng sau y:
1) Mi trng thi l khng hi quy. Khi vi mi i, j
lim
n
P
ij
(n) = 0.
Xch khng c phn b dng.
2) Mi trng thi l hi quy khng. Khi vi mi i, j
lim
n
P
ij
(n) = 0
Xch khng c phn b dng.
3) Mi trng thi l hi quy dng. Khi vi mi i, j
lim
n
P
ij
(n) =
j
> 0
v = (
1
,
2
, ...) l phn b gii hn (v cng l phn b dng) ca
xch.
nh l 1.14. Gi s (X
n
) l xch ti gin khng c chu k vi khng gian
trng thi hu hn E = {1, 2, ..., d}. Khi mi trng thi u hi quy dng
v xch c phn b gii hn = (
1
,
2
, ...,
d
). Phn b ny cng l phn b
dng duy nht ca xch.
Chng minh. Theo nh l 1.13 ta ch cn chng t kh nng 1) hoc 2)
khng xy ra. Tht vy nu tri li th vi mi i, j
lim
n
P
ij
(n) = 0.
Vy
1 = lim
n
d
j=1
P
ij
(n) =
d
j=1
lim
n
P
ij
(n) = 0.
Mu thun.
i vi xch ti gin (c th c chu k) ta c nh l sau y (khng
chng minh):
nh l 1.15. Gi s X
n
l xch ti gin vi khng gian trng thi E m
c. Khi
1. Vi mi i, j E
lim
n
n
1
n
k=1
P
ij
(k) =
1
j
.
Ni cch khc dy P
ij
(n) hi t theo trung bnh Cesaro ti
j
=
1
j
khng ph thuc i.
2. Dy = (
j
) tho mn
j=1
j
1

j
=
i=1
i
P
ij
.
Nh l mt h qu ca nh l trn ta c nh l sau (so snh vi 1.13):
nh l 1.16. Cho (X
n
) l xch Markov ti gin. Khi
1. Nu E hu hn c d phn t th (
1
, ...,
d
) l phn b dng duy nht.
2. Ch c cc kh nng sau:
Mi trng thi ca E l khng hi quy
Mi trng thi ca E l hi quy khng
Mi trng thi ca E l hi quy dng.
3. Nu E l v hn m c th xch c phn b dng khi v ch khi mi
trng tht ca E l hi quy dng. Trong trng hp ny phn b dng
l duy nht.
1.3 Qu trnh Markov
Xt h cc LNN ri rc (X
t
), t 0 vi tp ch s t l cc s thc khng
m t [0, ). K hiu E = X
t
() l tp gi tr ca X
t
. Khi E l mt
tp hu hn hay m c, cc phn t ca n c k hiu l i, j, k... .Ta
gi (X
t
) l mt qu trnh ngu nhin vi khng gian trng thi E .
nh ngha 1.9. Ta ni rng (X
t
) l mt qu trnh Markov nu vi mi
t
1
< ... < t
k
< t v vi mi i
1
, i
2
, ...i
n
, i E
P{X
t
= i|X
t
1
= i
1
, X
t
2
= i
2
..., X
t
k
= i
k
}
= P{X
t
= i|X
t
k
= i
k
}.
1.3. Qu trnh Markov 35
Nh vy, xc sut c iu kin ca mt s kin B no trong tng lai
nu bit hin ti v qu kh ca h cng ging nh xc sut c iu kin
ca B nu ch bit trng thi hin ti ca h. chnh l tnh Markov ca
h. i khi tnh Markov ca h cn pht biu di dng: Nu bit trng thi
hin ti X
t
ca h th qu kh X
u
, u < t v tong lai X
s
, s > t l c lp vi
nhau.
Gi s
P{X
t+s
= j|X
s
= i}
l xc sut xch ti thi im s trng thi i sau mt khong thi gian t
, ti thi im t +h chuyn sang trng thi j. y l mt con s ni chung
ph thuc vo i, j, t, s. Nu i lng ny khng ph thuc s ta ni xch l
thun nht. Trong gio trnh ny ta ch xt xch Markov thun nht.
K hiu
P
ij
(t) = P{X
t+s
= j|X
s
= i}.
Ta gi P
ij
(t) l xc sut chuyn ca h t trng thi i sang trng thi j sau
mt khong thi gian t . K hiu P(t) = (P
ij
(t), i, j E). P(t) l mt ma
trn hu hn hay v hn chiu. Ch rng
i)P
ij
(t) 0
ii)
jE
P
ij
(t) = 1.
Phn b ca X
0
c gi l phn b ban u. Ta k hiu u
i
= P(X
0
= i).
Chng minh tng t nh trng hp xch Markov ta c kt lun sau:
nh l 1.17. Phn b hu hn chiu ca qu trnh (X
t
) c hon ton
xc nh t phn b ban u v xc sut chuyn. C th vi t
1
< t
2
< ... < t
n
phn b ng thi ca (X
t
1
, ..., X
tn
) c tnh theo cng thc sau
P(X
t
1
= i
1
, ..., X
tn
= i
n
) =
=
iE
u
i
P
ii
1
(t
1
)P
i
1
i
2
(t
2
t
1
)...P
i
n1
in
(t
n
t
n1
).
nh l 1.18. (Phng trnh Chapman-Kolmogorov)
P
ij
(t +s) =
kE
P
ik
(t)P
kj
(s).
1.3.1 Trng hp khng gian trng thi hu hn
Gi s E = {1, 2, ..., d}. Khi t phng trnh C-K P(t), t > 0 l mt h
cc ma trn tho mn ng thc sau
P(t +s) = P(t)P(s).
Ni cch khc h (P(t), t > 0) lp thnh mt na nhm cc ma trn. T nay
v sau ta s lun gi thit thm rng
1. P
ij
(0) =
ij
2. lim
t0
P
ij
(t) =
ij
y
ij
l k hiu Kronecke
ij
=
_
_
_
1 nu i = j
0 nu i = j.
nh l 1.19. Hm ma trn P(t) l mt hm lin tc v tn ti
P
(0) = lim
h0+
P(t) I
h
.
Chng minh. Theo gi thit th P(0) = I v lim
t0
P(t) = I tc l P(t) lin
tc ti 0. Ta chng minh P(t) lin tc ti t > 0. Ta c do tnh cht na
nhm
lim
h0+
P(t +h) = lim
h0
P(t)P(h) = P(t)I = P(t).
Vy P(t) lin tc phi. Ta li c vi 0 < h < t th P(t) = P(h)P(t h) . Do
P(h) I khi h 0 nn tn ti P(h)
1
vi h nh. Vy
lim
h0+
P(t h) = lim
h0
P(t)P(h)
1
= P(t)I = P(t).
Vy P(t) cng lin tc tri do lin tc. (Thc ra c th chng minh c
rng P(t) lin tc u trn [0, ).
Tip theo s dng tnh cht na nhm ta c vi mi h > 0, n:
P(nh) I = (P(h) I) (I +P(h) +P(2h) +... +P((n 1)h)) . (1.12)
V P(t) lin tc nu h 0 sao cho nh t > 0 th
h
n1
k=0
_
t
0
P(u)du.
Tn ti t > 0 tch phn
_
t
0
P(u)du l ma trn khng suy bin. Ta li c
lim
n
P(nh) I = P(t) I. T (1.12) suy ra
lim
h0+
P(h) I
h
= (P(t) I)
__
t
0
P(u)du
_
1
K hiu
A = P
(0) = lim
h0+
P(h) I
h
.
A = (a
ij
) c gi l ma trn cc vi ca na nhm (P(t)). Ta c
lim
t0
P
ij
(t)
t
= a
ij
nu i = j
lim
t0
1 P
ii
(t)
t
= a
i
(1.13)
y a
i
= a
ii
. T (1.13) ta c
P
ij
(t) = a
ij
t +o(t)
1 P
ii
(t) = a
i
t +o(t).
Thnh th a
ij
c gi l cng chuyn t trng thi i sang trng thi j
v a
i
l cng thot khi trng thi i ca h.
T ng thc 0 =
j:j=i
P
ij
(t) + P
ii
1 chia hai v cho t v cho t 0 ta
thu c
j
a
ij
= 0
hay tng ng
a
i
=
j:j=i
a
ij
nh l 1.20. Cho qu trnh Markov vi na nhm P(t), t > 0 cc xc sut
chuyn. Gi A l ma trn cc vi ca na nhm. Khi ta c
P
(t) = P(t)A
P
ij
(t) =
k=j
P
ik
(t)a
kj
P
ij
(y)a
j
(1.14)
v
P
(t) = AP(t)
P
ij
(t) =
k=i
P
kj
a
ik
P
ij
(y)a
i
. (1.15)
Phng trnh ( (1.14)) c gi l phng trnh thun v phng trnh (1.15)
c gi l phng trnh ngc Kolmogorov.
Chng minh. Ta c do tnh cht na nhm
P(t +h) P(t)
h
=
P(t)(P(h) I)
h
=
(P(h) I)P(t)
h
.
Cho h 0 ta c iu cn chng minh.
Phng trnh thun v nghch l cc phng trnh vi phn vi iu kin
ban u P(0) = I c th gii c bng phng php quen thuc trong l
thuyt phng trnh vi phn. Ta c kt qu sau (khng chng minh):
nh l 1.21. Phong trnh (1.14) v phng trnh (1.15) c nghim l
P(t) = e
At
= I +
n=1
A
n
t
n
n!
.
Ngc li cho trc ma trn A = (a
ij
) cp d d tho mn a
ij
0 nu i = j
v
j
a
ij
= 0 . t
P(t) = e
At
.
Khi tn ti qu trnh Markov vi d trng thi nhn P(t) lm h ma trn
xc sut chuyn.
V d 1.17. (Qu trnh Markov hai trng thi) Xt qu trnh Markov vi
hai trng thi E = {0, 1}. (Chng hn ta xt s tin trin theo thi gian ca
mt h thng no trong trng thi 0 biu th trng thi tr tr cn trng
thi 1 biu th trng thi lm vic tch cc ca h thng). Gi s cng
chuyn t trng thi 0 sang trng thi 1 l v cng chuyn t trng
thi 1 sang trng thi 0 l . Ma trn cc vi l
A =
_

_
.
Ta i tm cng thc cho xc sut chuyn P
ij
(t) bng cch gii phng trnh
ngc. Phng trnh (1.15) cho ta
P
00
(t) = P
00
(t) +P
10
(t)
P
10
(t) = P
00
(t) P
10
(t).
Tr hai phng trnh trn v vi v ta c:
(P
00
(t) P
10
(t))
= ( +)(P
00
(t) P
10
(t))
P
00
(t) P
10
(t) = (P
00
(0) P
10
(0))e
(+)t
= e
(+)t
.
Vy
P
00
(t) = (P
00
(t) P
10
(t)) = e
(+)t
P
00
(t) = P
00
(0) +
_
t
0
P
00
(s)ds
= 1

+
(1 e
(+)t
)
=

+
+

+
e
(+)t
.
T
P
10
(t) = P
00
(t) e
(+)t
=

+

+
e
(+)t
.
Hon ton tng t t phng trnh (1.15) ta c
P
01
(t) = P
01
(t) +P
11
(t)
P
11
(t) = P
01
(t) P
11
(t).
ta tm c
P
01
(t) =

+

+
e
(+)t
P
11
(t) =

+
+

+
e
(+)t
.
By gi chng ta xt dng iu tim cn ca ma trn xc sut chuyn
P(t) khi t . Cho qu trnh Markov (X
t
) vi khng gian trng thi E
hu hn v ma trn xc sut chuyn P(t) = P
ij
(t). Ta ni rng qu trnh l
ti gin nu P
ij
(t) > 0 vi mi i, j E. (Ch rng ta khng c khi nim
"chu k ca mt trng thi" nh l i vi xch Markov).
nh l 1.22. Cho qu trnh Markov ti gin (X
t
)vi khng gian trng thi
E = {1, 2, ..., d} hu hn v ma trn xc sut chuyn P(t) = P
ij
(t). Khi
vi mi i, j E tn ti gii hn hu hn
lim
t
P
ij
(t) =
j
ch ph thuc j khng ph thuc i. Thm vo = (
1
,
2
, ...,
d
) l phn
b xc sut duy nht tho mn phng trnh
= P(t), t > 0.
Chng minh. Vi mi h > 0 c nh P(h) l mt ma trn xc sut chuyn
vi cc phn t u dong, vy theo nh l 1.5 ta c tn ti
lim
n
P(nh) = lim
n
P(h)
n
= (h)
trong (h) l ma trn vi cc dng nh nhau v bng (h) .Thm vo
(h) = (
1
(h),
2
(h), ...,
d
(h)) l phn b xc sut duy nht tho mn
phng trnh
(h) = (h)P(h), t > 0.
Mt khc ta bit rng P(t) lin tc u trn [0, ). Trong gii tch ta bit
rng nu mt hm P(t) lin tc u sao cho lim
n
P(nh) tn ti vi mi h > 0
th ko theo lim
t
P(t) tn ti. Vy ta kt lun tn ti
lim
t
P(t) =
vi = (h) vi mi h > 0. T suy ra kt lun ca nh l.
V d 1.18. Trong v d v qu trnh Markov hai trng thi t biu thc
ca P
ij
(t) ta c
lim
t
P
ij
(t) =
j
vi
0
=

+
,
1
=

+
.
Nu chn = (
0
,
1
) l phn b ban u ca qu trnh th
P(X
t
= 0) = P(X
0
= 0)P
00
(t) +P(X
0
= 1)P
10
(t)
=
0
P
00
(t) +
1
P
10
(t) =
0
=

+
.
Tng t
P(X
t
= 1) =
1
=

+
.
Nh vy phn b ca X
t
khng ph thuc vo t.
1.3.2 Trng hp khng gian trng thi v hn m
c
Trong trng hp khng gian trng thi E l v hn m c ta gp nhng
kh khn v ton hc khi mun m rng cc kt qu trong trng hp hu
hn. Ta c kt qu sau y (cng nhn khng chng minh):
nh l 1.23.
(1) Vi mi i = j gii hn
P
ij
(0) = lim
t0
P
ij
(t)
t
= a
ij
lun tn ti hu hn.
(2) Vi mi i gii hn
P
ii
(0) = lim
t0
P
ii
(t) 1
t
= a
ii
= a
i
tn ti nhng c th bng v cng.
i vi trng hp khng gian trng thi hu hn ta c
j
a
ij
= 0 hay
j=i
a
ij
= a
i
.
Trong trng hp E v hn ni chung ta ch c bt ng thc sau
j=i
a
ij
a
i
i E (1.16)
Tht vy ta c
j=i
P
ij
(t) = 1 P
ii
(t)
thnh th vi mi m
n
j=1,j=i
P
ij
(t) 1 P
ii
(t).
Chia hai v cho t v y t 0 ta thu c
m
j=1,j=i
a
ij
a
i
.
Cho m ta thu c (1.16).
T nay v sau ta ch xt cc qu trnh Markov tho mn iu kin
j=i
a
ij
= a
i
< . (1.17)
Ma trn v s chiu A = (a
ij
) cng c gi l ma trn cc vi ca qu trnh
ang xt.
nh l 1.24. Cho qu trnh Markov vi P(t) = (P
ij
(t)) l h cc ma trn
xc sut chuyn. Gi A l ma trn cc vi ca qu trnh. Khi ta c
P
(t) = P(t)A
P
ij
(t) =
k=j
P
ik
(t)a
kj
P
ij
(y)a
j
(1.18)
v
P
(t) = AP(t)
P
ij
(t) =
k=i
a
ik
P
kj
P
ij
(y)a
i
(1.19)
Phng trnh (1.18) c gi l phng trnh thun v phng trnh (1.19)
c gi l phng trnh ngc Kolmogorov.
Chng minh. Ta ch chng minh cho phng trnh ngc cn tha nhn s
ng n ca phng trnh thun v chng minh kh phc tp v ton hc.
Ta c:
P
ij
(s +t) P
ij
(t) =
k
P
ik
(s)P
kj
(t) P
ij
(t)
=
k=i
P
ik
(s)P
kj
(t) + (P
ii
(s) 1)P
ij
(t) (1.20)
Vi mi m c nh ta c
liminf
s0
s
1
k=i
P
ik
(s)P
kj
(t) liminf
s0
s
1
m
k=1,k=i
P
ik
(s)P
kj
(t)
=
m
k=1,k=i
a
ik
P
kj
(t).
Cho m ta c
liminf
s0
s
1
k=i
P
ik
(s)P
kj
(t)
k=i
a
ik
P
kj
(t). (1.21)
Tip theo vi m > i ta c
k=i
P
ik
(s)P
kj
(t)
m
k=1,k=i
P
ik
(s)P
kj
(t) +
m+1
P
ik
(s)
=
m
k=1,k=i
P
ik
(s)P
kj
(t) + 1 P
ii
(s)
m
k=1,k=i
P
ik
(s).
Chia hai v cho s v ly limsup ta thu c
limsup
s0
s
1
k=i
P
ik
(s)P
kj
(t)
m
k=1,k=i
a
ik
P
kj
(t) +a
i
k=1,k=i
a
ik
(s).
Cho m v ch n iu kin (1.17) ta c
limsup
s0
s
1
k=i
P
ik
(s)P
kj
(t)
k=i
a
ik
P
kj
(t).
So snh vi (1.21) ta nhn c
lim
s0
s
1
k=i
P
ik
(s)P
kj
(t) =
k=i
a
ik
P
kj
(t).
Trong 1.20 chia hai v cho s ri cho s 0 v p dng ng thc trn ta suy
ra (1.19).
By gi chng ta xt dng iu tim cn ca ma trn xc sut chuyn
P(t) khi t . Cho qu trnh Markov (X
t
) vi khng gian trng thi E v
hn m c v ma trn xc sut chuyn P(t) = P
ij
(t). Ta ni rng qu
trnh l ti gin nu P
ij
(t) > 0 vi mi i, j E. ( Ch rng ta khng c
khi nim "chu k ca mt trng thi" nh l i vi xch Markov).
nh l 1.25. Cho qu trnh Markov ti gin (X
t
)vi khng gian trng thi
E = {1, 2, ..., } m c v ma trn xc sut chuyn P(t) = P
ij
(t). Khi
vi mi i, j E tn ti gii hn hu hn
lim
t
P
ij
(t) =
j
ch ph thuc j khng ph thuc i. Thm vo gii hn = (
1
,
2
, ..., )
hoc l tt c bng khng
j
= 0 j E
hoc l tt c dng v lp thnh mt phn b xc sut. Phn b c gi
l phn b gii hn ca qu trnh
j
> 0 j E,
j
= 1.
Ta cng nhn v khng chng minh nh l ny.
nh l 1.26. Phn b gii hn = (
1
,
2
, ..., ) tho mn h phng trnh
tuyn tnh sau
j
a
j
=
k=j
k
a
kj
.
Chng minh. T phng trnh C-K
P
ij
(s +t) =
kE
P
ik
(s)P
kj
(t)
cho s ta thu c
j
=
k
P
kj
(t).
Suy ra
j
(1 P
jj
(t)) =
k=j
k
P
kj
(t).
Chia hai v cho t v cho t 0 ta c h thc phi chng minh.
V d 1.19. (Qu trnh sinh v cht.) Xt qu trnh Markov (X
t
) vi khng
gian trng thi E = {0, 1, 2, ...}. (X
t
) c gi l qu trnh sinh v cht nu
cc xc sut chuyn P
ij
(t) tho mn cc iu kin sau y
1. P
i.i+1
(t) =
i
t +o(t) i 0 khi t 0
2. P
i.i1
(t) =
i
t +o(t) i 1 khi t 0
3. P
ii
(t) = 1 (
i
+
i
)t +o(t), i 0 khi t 0
4. P
ij
(t) = o(t) vi |i j| > 1
5. P
ij
(0) =
ij
6.
i
= 0,
0
> 0,
i
,
i
> 0, i = 1, 2, ...
Qu trnh X
t
c s dng m t s pht trin ca mt qun th A no
. Mi c th ca qun th A ti mi thi im c th sinh ra mt c th
mi hoc b cht i. K hiu X
t
l s lng c th ca qun th ti thi im
t. Cc iu kin trn c ngha l nu ti thi im t qun th c i c th th
trong mt khong thi gian b (s, s + t) xc sut s lng qun th tng
thm mt c th l
i
t +o(t) v xc sut gim i mt c th l
i
t +o(t),
xc sut tng gim t nht hai c th l o(t). Cc tham s
i
, i = 0, 1, 2
c gi l cng sinh, cc tham s
i
, i = 1, 2, ... c gi l cng
cht.
Ma trn cc vi A = (a
ij
) c dng
a
i,i+1
=
i
a
i,i1
=
i
,
a
i
=
i
+
i
,
a
ij
= 0 nu |i j| > 1
tc l
A =
_
_
_
_
_
_
_
_
0

0
0 0 0...
1
(
1
+
1
)
1
0 0..
0
2
(
2
+
2
)
2
...
. . . . .
. . . .
_
_
_
_
_
_
_
_
Phng trnh thun (1.14) trong trng hp ny c dng
P
ij
(t) = P
i,j1
(t)
j1
+P
i,i+1
(t)
j+1
(
j
+
j
)P
ij
(t) (1.22)
P
i0
(t) =
0
P
i0
(t) +
1
P
i1
(t). (1.23)
Gi s qu trnh sinh v cht c phn b gii hn = (
i
). T nh l 1.26
ta c
0
=
1
1
(
k
+
k
)
k
=
k1
k1
+
k+1
k+1
t a
k
=
k
k

k1
k1
. T phng trnh trn suy ra a
k
= a
k+1
. V
a
1
=
1
0
= 0 nn suy ra a
k
= 0 k hay
k
=
k1
k1
k = 1, 2, ... (1.24)
k
=

k1
k1
(1.25)
k
=

k1
k2
...
0
k1
...
1
0
. (1.26)
V
k
= 1 nn t (1.24) suy ra
1 =
0
+
0
k=1
_
k1
k2
...
0
k1
...
1
_
.
Vy iu kin cn c phn b gii hn l chui
k=1
_
k1
k2
...
0
k1
...
1
_
(1.27)
hi t. Ngc li c th chng minh c iu kin chui (1.27) hi t cng
l iu kin qu trnh c phn b gii hn.
Ta xt mt s v d.
V d 1.20. (Mt m hnh n gin trong l thuyt xp hng) Gi s ca
hng dch v A ch c mt ngi phc v. Khch n xp hng i n lt
mnh c phc v v ca hng ch phc v tng khch mt. Khi ca hng
ang phc khch th cc khch mi c th n v xp hng ch. Khch c
phc v xong th lp tc ri khi ca hng. Gi s xc sut trong khong
thi gian (t, t + h) c mt khch mi vo hng l h + o(h) v cng gi s
rng nu ti thi im t khch ang c phc v th xc sut s phc v
hon tt trong khong thi gian (t, t + h) l h + o(h). Gi X
t
l s khch
c mt ti ca hng ti thi im t (tc l s khch ang xp hng ch phc
v cng vi khch ang c phc v ti thi im t). D thy y l mt
qu trnh sinh v cht vi cng sinh v cng cht u l hng s
i
= ,
i
= . Khi chui (1.27) tr thnh chui
k=1
(
)
k
.
Vy: Qu trnh c phn b gii hn (v y cng l phn b dng) khi v ch
khi < . Khi ta c
k
=
_
1

_
(
)
k
.
T s r =

gi l cng gii phng hng. Nu r < 1 th khi t kh ln, s

khch c mt ca hng X
t
l mt LNN c phn b hnh hc
P(X
t
= k) = (1 r)r
k
.
Suy ra s khch c mt trung bnh l EX
t
=
1
1r
.
Trong mt thi cc khc, ta gi s ca hng c rt nhiu nhn vin phc
v sao cho bt c ngi khch no n cng c phc v ngay. Gi X
t
l s
khch c mt ti ca hng ti thi im t (tc l s khch ang c phc
v ti thi im t). Ta c
P
i,i+1
(h) = P{X
t+h
= i + 1|X
t
= i}
= h +o(h),
P
i,i1
(h) = P(c ng mt khch trong i khch c phc v xong)
=
_
i
1
_
[h +o(h)][1 h +o(h)]
i1
= ih +o(h).
Vy X
t
l qu trnh sinh v cht vi
i
= ,
i
= i. Khi cng thc (1.24)
cho ta
k
=
1
k!
(
)
k
0
v chui (1.27) tr thnh chui
k=1
1
k!
(
)
k
= e
/
1.
T iu kin
1 =
k=0
k
=
0
e
/
rt ra
k
= e
r
r
k
k!
.
Nh vy khi t ln X
t
c phn b Poisson vi tham s r =

. Gi tr trung
bnh ca X
t
khi t ln l r, n chnh bng t s gia thi gian phc v
trung bnh 1/ vi khong thi gian trung bnh xut hin mt khch mi 1/.
Mt kt lun kh ph hp vi thc t!.
V d 1.21. (Qu trnh sinh thun tu) Nu qu trnh sinh v cht m khng
xy ra s cht th ta gi l qu trnh sinh thun tu. Nh vy i vi qu
trnh sinh thun tu ta c
j
= 0 j 1
v
P
ij
(t) = 0 nu j < i.
Phng trnh thun (1.22) tr thnh
P
ij
(t) =
j1
P
i,j1
j
P
ij
(t) (1.28)
P
ii
(t) =
i
P
ii
(t). (1.29)
V P
ii
(0) = 1 ta suy ra
P
ii
(t) = e
i
t
. (1.30)
gii phng trnh vi phn (1.28) ta c b d sau:
B 1.5. Phng trnh vi phn
f
(t) = f(t) +g(t) (1.31)

c nghim l
f(t) = f(0)e
t
+
_
t
0
e
(ts)
g(s)ds.
Tht vy nhn hai v ca (1.36) vi e
t
ta c
_
e
t
f(s)
_
= e
s
g(s).
Ly tch phn hai v t 0 ti t ta c
e
t
f(t) f(0) =
_
t
0
e
s
g(s)ds.
Nhn hai v vi e
t
ta c kt lun ca b .
V P
ij
(0) = 0 i = j nn t b v phng trnh (1.28) ta c
P
ij
(t) =
j1
_
t
0
e
j
(ts)
P
i,j1
(s)ds. (1.32)
Ta c th s dng (1.30) v (1.32) nh mt cng thc truy hi tm ln
lt P
ij
(t) vi j = i + 1, i + 2, .... Chng hn ta c
P
i,i+1
(t) =
i
_
t
0
e
i+1
(ts)
e
i
s
ds.
Suy ra
P
i,i+1
(t) =
_
_
_
i+1
i
_
(e
i
t
e
i+1
t
_
nu
i+1
=
i
i
te
i
t
nu
i+1
=
i
.
(1.33)
V d 1.22. (Qu trnh sinh tuyn tnh) Xt s tng trng c th ca mt
qun th no . Gi s rng mi c th ca qun th c lp vi nhau trong
khong thi gian (t, t +h) c xc sut sinh thm mt c th mi l h +o(h)
v xc sut khng sinh thm c th no trong khong thi gian (t, t +h) l
1 h +o(h). Go X
t
l s lng c th thi im t. X
t
l mt qu trnh
sinh thun tu v
P
ii
(h) = P{X
t+h
= i|X
t
= i}
= [1 h +o(h)]
i
= 1 ih +o(h),
P
i,i+1
(h) = P{X
t+h
= i + 1|X
t
= i}
=
_
i
1
_
[h +o(h)][1 h +o(h)]
i1
= ih +o(h).
Nh vy cng sinh l
i
= i.
T (1.33) ta c
P
i,i+1
(t) = ie
it
(1 e
t
).
tnh P
i,i+2
(t) ta t j = i + 2 trong (1.32) v thu c
P
i.i+2
(t) = (i + 1)i
_
t
0
e
(i+2)(ts)
e
is
(1 e
s
)ds
= (i + 1)ie
(i+2)t
_
t
0
e
2i
(1 e
s
)ds
= (i + 1)ie
(i+2)t
_
t
0
e
t
(e
s
1)ds
= (i + 1)ie
(i+2)t
(e
t
1)
2
2
_
i + 1
2
_
e
it
(1 e
t
)
2
.
Tip tc nh vy p dng cng thc truy hi (1.32) v bng quy np ta thu
c
P
i,i+k
(t) =
_
i +k 1
k
_
e
it
(1 e
t
)
k
.
Nh vy s gia tng dn s X
s+t
X
s
trong khong thi gian t c phn b
nh thc m vi cc tham s p = e
t
v r = i, i = X
s
. Thnh th
E[X
s+t
X
s
|X
s
= i] = ie
t
(1 e
t
).
Nu ti thi im ban u s = 0 qun th c s lng i c th th ti thi
im t s c th trung bnh s l
E[X
t
] = E[X
t
X
0
] +i = ie
t
.
Qu trnh sinh tuyn tnh ny i khi cn c gi l qu trnh Yule, do nh
ton sinh ngi Anh Yule a ra nm 1924.
V d 1.23. (Qu trnh Poisson.) Xt qu trnh sinh thun tu X
t
vi cng
sinh l hng s
i
= , i = 0, 1, ...
Khi cng thc (1.32) tr thnh
P
ij
(t) =
_
t
0
e
(ts)
P
i,j1
(s)ds. (1.34)
T cng thc (1.33) ta thu c
P
i,i+1
(t) = te
t
.
tnh P
i,i+2
(t) ta t j = i + 2 trong (1.34) v thu c
P
i,i+2
(t) =
_
t
0
e
(ts)
e
s
ds
=
2
e
t
_
t
0
sds =
()
2
2
e
t
.
Tip tc nh vy p dng cng thc truy hi (1.34) v bng quy np ta thu
c
P
i,i+k
(t) =
(t)
k
k!
e
t
.
Nh vy s gia tng dn s X
s+t
X
s
trong khong thi gian t c phn b
Poisson vi tham s t.
Mt cch tng qut ta s chng minh rng vi 0 < s < t th LNN
X
t
X
s
s c phn b Poisson vi tham s (t s). Tht vy ta c
P(X
t
X
s
= k) =
i=0
P(X
s
= i)P(X
t
= k +i|X
s
= i)
=
i=0
P(X
s
= i)P
i,i+k
(t s)
=
i=0
P(X
s
= i)
((t s))
k
k!
e
(ts)
=
((t s))
k
k!
e
(ts)
i=0
P(X
s
= i)
=
((t s)
k
)
k!
e
(ts)
.
Tip theo ta chng minh rng X
t
l qu trnh ngu nhin c gia s c lp.
Tht vy, vi 0 t
1
< t
2
< < t
n
ta c
P
_
X
t
2
X
t
1
= i
1
, ..., X
tn
X
t
n1
= i
n1
_
=
i=0
P(X
t
1
= i)P
0i
1
(t
2
t
1
)...P
0i
n1
(t
n
t
n1
)
= P
0i
1
(t
2
t
1
)...P
0i
n1
(t
n
t
n1
)
i=0
P(X
t
1
= i)
= P
0i
1
(t
2
t
1
)...P
0i
n1
(t
n
t
n1
)
= P(X
t
2
X
t
1
= i
1
)...P(X
tn
X
t
n1
= i
n1
).
Thnh th X
t
2
X
t
1
, ..., X
tn
X
t
n1
l cc LNN c lp.
Qu trnh X
t
, t 0 c gi l qu trnh Poisson vi cng > 0 nu
n tho mn cc iu kin sau
1. X
0
= 0.
2. Vi 0 s < t th LNN X
t
X
s
s c phn b Poisson vi tham s
(t s).
3. X
t
l qu trnh ngu nhin c gia s c lp.
Nh vy ta chng minh rng qu trnh sinh thun tu vi cng
sinh l hng s chnh l mt qu trnh Poisson vi cng > 0 . Qu
trnh Poisson c rt nhiu ng dng trong thc t. N dng m t s ln
xut hin ca mt s kin ngu nhin no trong khong thi gian t, chng
hn s ln gi n tng i, s khch hng n mt ca hng no , s ln
hng hc ca mt ng dy,...
1.3.3 Trng hp tng qut
Mt vi khi nim c bn v qu trnh ngu nhin. Xt qu trnh
Markov vi khng gian trng thi E bt k. Cho (E, A) l mt khng gian
o. Qu trnh ngu nhin X
t
c gi l qu trnh Markov nu
P(X
t+s
A|F
t
) = P(X
t+s
A|F
t
).
Ngha l : Nu ta bit trng thi ca h ti thi im hin ti t th mi thng
tin v hnh vi ca h trong qu kh khng c nh hng n s bin din
trong tng lai ca h. Ni cch khc: Qu kh v tng lai c lp vi nhau
khi bit hin ti. K hiu
P(s, x, t, A) = P(X
t
A|X
s
= x)
P(s, x, t, A) l xc sut h ti thi im s ang trng thi x sang thi
im t ri vo tp A. Ta gi P(s, x, t, A) l xc sut chuyn. H cc xc sut
chuyn c cc tnh cht sau:
nh l 1.27.
1. Vi mi s t, x E P(s, x, t, .) l mt o xc sut trn E
2. Vi mi s t, A A hm P(s, ., t, A) l mt hm o c trn E
3. ( Phng trnh C-K (Chapman- Kolmogorov) )
P(s, x, t, A) =
_
E
P(s, x, u, dy)P(u, y, t, A).
Qu trnh Markov X
t
c gi l thun nht nu xc sut chuyn
P(s, x, t, A) ch ph thuc vo hiu s t s ngha l
P(s +u, x, t +u, A) = P(s, x, t, A) u
Khi P(s, x, t, A) c dng
P(s, x, t, A) = P(t s, x, A)
P(t, x, A) = P(X
s+t
A|X
s
= x) l xc sut h ti thi im s
trng thi x sau mt khong thi gian t ( ti thi im t +s) ri vo A.
Phng trnh Chapman- Kolmogorov khi y tr thnh
P(t +s, x, A) =
_
E
P(t, x, dy)P(s, y, A).
Trong gio trnh ny ta ch xt qu trnh Markov thun nht.
Trong trng hp o P(t, x, .) c mt f(t, x, u) th phng trnh
C-K tng ng vi
f(t +s, x, z) =
_
E
f(t, x, y)f(s, y, z)dy.
Ngc li cho trc mt h cc hm P(t, x, A) tho mn cc iu kin nu
trong nh l (1.27). Vi mi o xc sut trn (E, A) ta xc nh h
cc phn b hu hn chiu (
t
1
,...,tn
) nh sau
t
1
,...,tn
(A
1
, ..., A
n
)
=
_
ynAn
...
_
y
1
A
1
_
xE
(dx)P(t
1
, x, dy
1
)P(t
2
t
1
, y
1
, dy
2
)...
P(t
n
t
n1
, y
n1
, dy
n
).
S dng cc tnh cht ca h xc sut chuyn ta chng minh c h cc
phn b xc sut ny tho mn iu kin tng thch Kolmogrov. Thnh
th tn ti mt qu trnh ngu nhin (X
t
) sao cho:
Phn b ca X
0
( phn b ban u) l .
Vi mi 0 t
1
< ... < t
n
phn b ng thi ca (X
t
1
, ..., X
tn
) l
t
1
,...,tn
.
Hn na c th chng minh rng X
t
l mt qu trnh Markov thun nht
nhn P(t, x, A) lm xc sut chuyn.
V d 1.24. (Chuyn ng Brown hay qu trnh Wiener.) Xt hm f(t, x, y)
cho bi cng thc
f(t, x, u) =
1
2t
e
(ux)
2
2t
v P(t, x, .) l o xc sut trn R nhn f(t, x, .) lm hm mt
P(t, x, A) =
_
A
f(t, x, du).
Khi ta c th chng minh rng h P(t, x, A) tho mn cc iu kin ca
nh l (1.27). Tht vy, gi
t
(u) l hm mt ca N(0, t) th nh ta bit
t+s
=
t

s
hay
t+s
(u) =
_

s
(v)
t
(u v)dv.
t u = x z, v = y z dv = dy suy ra
t+s
(x z) =
_

s
(y z)
t
(x y)dy
hay
f(t +s, x, z) =
_
f(t, x, y)f(s, y, z)dy.
Do tn ti qu trnh Markov vi h P(t, x, A) l h xc sut chuyn v
phn b ban u =
0
. Qu trnh Markov ny c gi l chuyn ng
Brown hay qu trnh Wiener v c k hiu l (W
t
).
Cc phn tch su sc hn na chng t (W
t
) c cc tnh cht sau
1. W
0
= 0 .
2. Vi mi 0 s < t th W
t
W
s
l LNN c phn b chun vi k vng
0 v phng sai l t s .
3. W(t) l mt qu trnh gia s c lp tc l vi mi 0 t
1
< t
2
<
... < t
n
cc LNN W
t
2
W
t
1
, W
t
3
W
t
2
, ..., W
tn
W
t
n1
l c lp.
4. W
t
c qu o lin tc.
By gi chng ta s trnh by phng php xy dng cc hm xc sut
chuyn. Nh bit mi o trn R ta cho ng vi phim hm T trn
C
b
(R) bi
T
=
_
R
(u)d(u).
Tng ng ny l mt-mt. Nu bit T vi mi th c th xc nh c
.Ta nh ngha
(t, x) =
_
R
(u)P(t, x, du). (1.35)
Nu vi mi ta tm c hm (t, x) tng ng th coi nh tm c
P(t, x, .). Ta c kt qu sau:
nh l 1.28. K hiu B(x, ) = [x , x + ], B
c
(x, ) = R \ B(x, ). Gi
s rng
lim
t0+
t
1
P(t.x, B
c
(x, )) = 0 x,
lim
t0+
t
1
_
B(x,)
(u x)P(t, x, du) = a(x)
lim
t0+
t
1
_
B(x,)
(u x)
2
P(t, x, du) = b(x)
Khi hm (t, x) xc nh bi (1.35) tho mn phng trnh vi phn sau
t
= a(x)
x
+
b(x)
2
x
2
(1.36)
vi iu kin ban u
lim
t0+
(t, x) = (x). (1.37)
Ngc li cho trc hai hm a(x), b(x). Khi vi mt s gi thit v hai
hm a(x), b(x) vi mi hm lin tc b chn phng trnh (1.36) vi iu
kin ban u (1.37) s c nghim duy nht (t, x) v do xc nh cho ta
o P(t, x, .). H o ny lp thnh mt h xc sut chuyn. Hn na
qu trnh Markov tng ng vi h xc sut chuyn ny c qu o lin tc.
1.4 Bi tp
1. Cho xch Markov X
n
, n = 0, 1, 2, ... vi khng gian trng thi E =
{0, 1, 2} v ma trn xc sut chuyn
P =
_
_
_
0.1 0, 2 0, 7
0.9 0, 0 0, 1
0, 1 0, 8 0.1
_
_
_
.
Bit phn b ban u l p
0
= 0, 3, p
1
= 0, 4, p
2
= 0, 3. Tnh
P(X
0
= 0, X
1
= 1, X
2
= 2).
1.4. Bi tp 59
2. Ngi ta truyn mt bc in gm cc tn hiu 0, 1 thng qua knh c
nhiu trm v mi trm nhn ng tn hiu vi xc sut a. K hiu X
0
l tn hiu truyn i v X
n
l tn hiu nhn c trm n. Bit rng
(X
n
) lp thnh xch Markov vi ma trn xc sut chuyn
P =
_
a 1 a
1 a a
_
.
Gi s tn hiu truyn i l tn hiu 0.
i) Tnh xc sut khng nhn sai tn hiu cho ti trm n = 2.
ii) Tnh xc sut nhn ng tn hiu ca trm n = 2.
ii) Tnh xc sut nhn ng tn hiu ca trm n = 5.
3. Cho xch Markov X
n
, n = 0, 1, 2, ... vi khng gian trng thi E = 0, 1, 2
v ma trn xc sut chuyn
P =
_
_
_
0.1 0, 2 0, 7
0.2 0, 2 0, 6
0, 6 0, 1 0.3
_
_
_
Tnh P
2
, P(X
3
= 1|X
1
= 0) v P(X
3
= 1|X
0
= 0).
4. Cho xch Markov X
n
P =
_
_
_
0.0 0, 5 0, 5
0.5 0, 0 0, 5
0, 5 0, 5 0.0
_
_
_
Tnh P(X
2
= 1|X
0
= 0) v P(X
3
= 0|X
0
= 0).
5. Gi s X
n
cht lng ca chi tit ti cng on th n ca dy chuyn
sn xut vi X
n
= 0 c ngha l tt cn X
n
= 1 c ngha l xu. Bit
rng (X
n
)l xch Markov vi ma trn xc sut chuyn
P =
_
0.99 0.01
0.12 0, 88
_
Tnh P(X
4
= 1|X
1
= 1).
6. Cho xch Markov X
n
P =
_
_
_
0.1 0, 5 0, 4
0.6 0, 2 0, 2
0, 3 0, 4 0.3
_
_
_
Phn b ban u l (0, 7; 0, 2; 0, 1).
i) Lp bng phn b xc sut ca X
2
.
ii) Tnh P(X
0
= 1, X
1
= 3, X
2
= 3, X
3
= 2).
iii) Tm phn b dng.
7. Cho xch Markov X
n
, n = 0, 1, 2, ... vi khng gian trng thi E =
P =
_
_
_
3/7 3/7 1/7
1/11 2/11 8/11
1/11 4/11 6/11
_
_
_
Gi s P(X
0
= 1) = 1. t
Y
n
=
_
_
_
1 nu X
n
= 1
2 nu X
n
= 1
Chng minh rng (Y
n
) l mt xch Markov. Tm ma trn xc sut
chuyn.
8. Cho (r
n
) l dy Rademakher P(r
n
= 1) = p, P(r
n
= 1) = q = 1 p.
Trong cc dy LNN sau y dy no lp thnh xch Markov
(a) X
n
= r
n
r
n+1
(b) X
n
= r
1
r
2
...r
n
1.4. Bi tp 61
(c) X
n
= (r
n
, r
n+1
) trong
(1, 1) = 1, (1, 1) = 2, (1, 1) = 3, (1, 1) = 4
i vi dy lp thnh xch Markov hy tm ma trn xc sut chuyn.
9. Mi ngi dn ca th trn N c mt trong ba ngh A, B, C. Con ci
ca h ni tip ngh ca cha mnh vi xc sut tng ng cho cc ngh
A, B, C l 3/5; 2/3; 1/4. Nu khng theo ngh ca cha th chng chn
mt trong hai ngh cn li vi xc sut nh nhau. Gi s th h hin
ti 20% theo ngh A 30% theo ngh B v 50% theo ngh C. Hy tm
(a) Tm phn b ngh nghip th h tip theo.
(b) Tm phn b gii hn theo ngh nghip ca dn c th trn trong
tng lai xa xi.
10. Cho xch Markov (X
n
), n = 0, 1, 2, ... vi khng gian trng thi E =
P =
_
_
_
0 0 1
1 0 0
1/2 1/2 0
_
_
_
(a) Chng minh rng xch ti gin.
(b) Tm chu k ca xch.
(c) Tm phn b dng.
11. Cho xch Markov (X
n
), n = 0, 1, 2, ... vi khng gian trng thi E =
{1, 2, 3, 4, 5} v ma trn xc sut chuyn
P =
_
_
_
_
_
_
_
_
0 1/3 2/3 0 0
0 0 0 1/4 3/4
0 0 0 1/4 3/4
1 0 0 0 0
1 0 0 0 0
_
_
_
_
_
_
_
_
(a) Chng minh rng xch ti gin.
(b) Tm chu k ca xch.
(c) Tm phn b dng.
12. Gi s ta theo di s xut hin ca bin c A theo thi gian. K hiu
X
t
l s ln xut hin A trong khong thi gian (0, t] v gi thit X
t
l qu trnh Poisson vi cng .
(a) Tm xc sut trong khong thi gian (0, s] bin c A xy ra m
ln nu bit rng trong khong thi gian (0, t] bin c A xy ra n
ln 0 < s < t, 0 < m < n.
(b) K hiu T
m
l thi im m bin c A xut hin ln th m. Tm
hm phn b xc sut ca T
m
.
(c) Tm hm mt ca T
m
.
(d) Tm P(T
1
s|X
t
= n) 0 < s < t.
13. Cho qu trnh cht thun tu X
t
vi khng gian trng thi E =
{0, 1, 2, ...}
(a) Vit phng trnh thun cho qu trnh X
t
.
(b) Tm P
ii
(t).
(c) Tm P
ij
(t) theo P
i,j+1
(t).
(d) Tnh P
i,i1
(t).
(e) Chng minh rng nu X
t
l qu trnh cht tuyn tnh tc l
i
= i
th vi i > j ta c
P
ij
(t) =
_
i
j
_
(e
t
)
j
(1 e
t
)
ij
.
14. Cho qu trnh sinh v cht X
t
vi
i
= i,
i
= i , l cc hng
s dng. C nh i. t
m
i
(t) = E
i
X(t) =
j=0
jP
ij
(t)
1.4. Bi tp 63
(a) Vit phng trnh thun cho qu trnh X
t
.
(b) S dng phng trnh thun chng minh rng m
i
(t) = (
)m
i
(t).
(c) Suy ra
m
i
(t) = ie
()t
.
15. Cho qu trnh sinh v cht X
t
vi
i
= i,
i
= i , l cc hng
s dng. C nh i. t
s
i
(t) = E
i
X
2
(t) =
j=0
j
2
P
ij
(t).
(a) S dng phng trnh thun chng minh rng
s
i
(t) = 2( )s
i
(t) + ( +)m
i
(t).
(b) Tm s
i
(t).
(c) Tm V arX
t
nu X
0
= i.
16. Chng minh rng h
f(t, x, y) =
t
(t
2
+ (x y)
2
)
lp thnh h mt chuyn tho mn phng trnh Chapman - Kol-
mogorov. Qu trnh Markov tng ng c tn l qu trnh Cauchy.

Quatrinh Markov

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Quatrinh Markov

Uploaded by

Copyright:

Available Formats

Chng 1.

gi l cng gii phng hng. Nu r < 1 th khi t kh ln, s

(t) = f(t) +g(t) (1.31)

You might also like