MM Lec6

H Ni - 2005/14
H MC V TM KIM NH
Bi 6
PGS.TS. ng Vn c
dvduc@ioit.ac.vn
dvduc-2005/14 Bi 6: Ch mc v tm kim nh
Ni dung
Gii thiu
Tm kim nh trn c s vn bn
Tm kim nh trn c s mu
Tm kim nh trn c s hnh dng
Tm kim nh trn c s hoa vn
Kt lun
2/40
1. Gii thiu
Nhc li kin trc CSDL a phng tin
User
Multimedia Query Engine
Document Index Image Index Audio Index Video Index
Query
Answer
3/40
Gii thiu
Nhc li loi d liu a phng tin
image
Once upon a time, there was a little...
text
video
4/40
Gii thiu
Khung lm vic ca h thng CSDL nh
dvduc-2005/14 Bi 6: Ch mc v tm kim nh 5/40
Gii thiu
C nhiu nghin cu v ch mc v tm kim nh hn
lnh vc m thanh v video.
Cc tim cn ch s ha v tm kim nh bao gm:
Trn c s thuc tnh c cu trc, trch chn th cng
Trn c s t ng nhn dng i tng
Trn c s vn bn m t nh
Trn c s cc c trng nh mc thp c trch chn t
ng.
V d cc c trng nh trn c s mu, hnh dng v hoa vn nh.
T hp cc c trng c ch s ha v tm kim hiu qu.
V d, nu ch s dng mu th khng phn bit c t v qu to
Vn
Ch mc t ng hay m t th cng
Cc c trng hay khi nim ng ngha.
6/40
2. Tm kim nh trn c s vn bn
nh c m t bng free text.
Cu truy vn di hnh thc cc t kha hay vn bn c
hoc khng c cc ton t Bool.
M t nh th cng, vic t ng hiu nh l khng th.
S dng c s tri thc v phn hi trong tm kim nh.
u im
C th thu thp cc khi nim v tru tng mc cao
V d: M t smile, happy c trong nh.
Nhc im
Mt vi c trng mc thp nh hoa vn (texture) v hnh dng
khng u,... rt kh m t bng vn bn v rt ch quan.
H thng tm kim nh trn c s text khng chp nhn cu truy
vn bng nh (v d Hy tm cc nh tng t nh nh ny).
7/40
M t nh bng vn bn
D dng m t mc cao vi mt s loi nh

Kh m t vi mt s loi nh
Who: 2 ngi, Chirac-Mendela, Tng thng.
What: Bt tay, ng .
Where: Thm , Ngoi tri, in Elyse.
When: Ban ngy, thng 7/1997.
8/40
M t nh bng vn bn
Kh m t mc cao vi mt s loi nh
Who?
What?
9/40
3. Tm kim nh trn c s im nh
y l k thut hay c s dng nht trong h thng
tm kim nh theo ni dung.
Khi nim nh a mc xm
L ma trn im nh, trong mi im nh c mt gi tr mc
xm t 0 n 255
Chuyn i nh mu RGB v nh a mc xm
nh mu C(i, j) vi ie[1, M], je[1, N]
nh a mc xm A(i, j) c gi tr mc xm ti i, j (ie[1, M],
je[1, N]) nh sau:
A(i, j) = w
1
.R(i,j)+w
2
.G(i,j)+w
3
.B(i,j), trong : w
1
+w
2
+w
3
=1
Cng thc hay c s dng:

) , (
3
1
) , (
3
1
) , (
3
1
) , ( j i B j i G j i R j i A + + =
10/40
Tm kim nh trn c s biu mu
Khi nim biu (histogram) a mc xm
Biu a mc xm ca nh kch thc M.N l vct
H=(h
0
, h
2
,..., h
j
, ..., h
255
),
vi , trong n
j
l tng s pixel nhn gi tr j.
N M
n
h
j
j
.
=
Lena Sderberg
11/40
Vi nh mu:
C th x l ring r tng mu
Phng php:
Mi nh biu din bng ba knh ca khng gian mu (v d: RGB
hay HSV).
Mi knh mu c s ha thnh m mc (v d: 16). Gi tng s
mu ri rc l K (gi l bin mu nhm mu) v c tnh bi:
K=m
3
=16x16x16=4096

Biu mu H(M) ca nh kch thc M.N l vct
H(M)=(h
1
, h
2
,..., h
j
, ..., h
K-1
),
vi , trong n
j
l tng s pixel nh ri vo bin th j.
Vect ny c lu tr lm ch s ca nh.
N M
n
h
j
j
.
=
12/40
V d biu mu
o khong cch hai nh
o khong cch L
p
(Minkowski)
Cho trc hai vct n chiu

Tnh khong cch gia hai vct x v y:

L
1
: Khong cch City block hay Manhattan,
L
2
: Khong cch Euclid
=
=
=
n
l
l l p
h i H I d
1
1
| | ) , (

=
=
=
n
l
l l p
h i H I d
0
2
2
] [ ) , (
p
p
n
i
i i p
y x L
/ 1
1
| |
|
|
.
|
\
|
=

=
T
n
i i i I ] ,..., , [
2 1
=
T
n
h h h H ] ,..., , [
2 1
=
14/40
3.1 Thut ton c s
Nhim v:
Tm kim nhng nh trong CSDL c mu cm nhn tng t
nh truy vn
Cc bc thc hin
Tnh biu mu ca nh truy vn (biu a mc xm/biu
cc knh mu)
Tnh khong cch gia biu mu ca cu truy vn vi tng
biu mu ca nh trong CSDL (s dng o khong cch
Mahattan hay Euclid)
Cc nh trong CSDL c khong cch biu mu n biu
mu nh truy vn nh hn ngng cho trc s l nh kt qu.

V d i snh nh trn c s biu mu
Gi s c ba nh kch thc 8x8, mi pixel ca nh c 1
trong 8 mu: C
1
n C
8
.
nh 1: mi mu t C1 n C8 u c 8 pixels
nh 2: T mu C
1
n C
4
c 7 pixels, t C
5
n C
8
c 9 pixels
nh 3: T mu C
1
n C
2
c 2 pixel, t mu C
3
n C
8
c 10
pixels.
Tnh biu mu v khong cch gia cc nh

Hai nh no tng t nhau?
H1=(8, 8, 8, 8, 8, 8, 8, 8)
H2=(7, 7, 7, 7, 9, 9, 9, 9)
H3=(2, 2, 10, 10, 10, 10, 10, 10)
h(H1, H2)= 1+ 1+1+1+1+1+1+1=8
h(H1, H3)= 6+ 6+2+2+2+2+2+2=24
h(H2, H3)= 5+ 5+3+3+3+1+1+1=23
16/40
Thut ton c s
Hn ch ca thut ton c s:
B qua tnh tng ng gia cc bins mu.
V d 1:
C hai bins biu din di mu 1-10 v 11-20,
Mu 10 s phn lp vo bin 1,
Mu 11 vo bin 2 v mu 20 vo bin 2.
Kt qu ca thut ton c s: Mu 11 cng mu vi mu 20
nhng khc hon ton mu 10 (?)
V d 2:

Cn quan tm n cc mu tng t trong cc bins
1
10
20
17/40
3.2 Biu mu tch ly (Cumulative Histogram)
Tnh biu mu thng thng H(M)
Biu mu tch ly CH(M)=(ch
1
, ch
2
, ..., ch
n
),
S dng khong cch L
1
hay L
2

so snh cc nh
Normal Histogram
Cumulative Histogram
s
=
i j
j i
h ch
18/40
3.3 Biu mu cc b
Biu mu ton cc khng tnh n phn b khng
gian ca mu trn ton b nh
Cp nh c biu mu ton cc ging nhau, nhng c
th hon ton khc nhau.

Gii php:
Chia nh thnh cc vng vi s im nh c nh, tnh biu
mu cc b cho tng vng.
19/40
3.4 Tch biu mu nn v i tng
Nhiu nh c hai vng phn bit: nn v cn cnh
S lng pixel ca nn ln hn nhiu v u hn.
Biu ca nh s c xu hng n mu ca bloc ln hn
Nu hai nh c cng mu nn, phng php c s cho kt qu
hai nh tng t nhau mc d chng c cn cnh rt khc nhau
V d vi 3 nh:
t trn ng (mu nn en)
t trn thm nu showroom
t xanh trn thm nu.
K thut c s s cho kt qu l nh th 2 gn ging nh th 3
Tch biu mu nn v i tng
Lu v Phillips xut vic tnh tch bit biu mu ca
nn v ca cn cnh
Vic tch nn khi cn cnh khng nht thit phi tht
chnh xc
Cn cnh c th l ch nht nh nht cha cc i tng nh
C th tch t ng
Mi nh trong CSDL c ch mc bi biu mu ca
nn v ca cn cnh.
4. Tm kim nh trn c s hnh dng
22/40
tm kim c nh trn c s hnh dng, cn
Phn on nh thnh cc i tng c lp bng k thut no
. C th s dng k thut bn th cng.
Biu din hnh dng
o tng t gia cc biu din hnh dng.
Thuc tnh ca biu din hnh dng v o tng t
Biu din hnh dng duy nht, bt bin vi dch chuyn, xoay v
co dn
Cho kh nng nhn bit i tng c kch thc, v tr v hng khc nhau.
Cc hnh dng tng t phi biu din tng t c th o
c mc tng t.
Biu din hnh dng
Contour
Vng
23/40
Tch bin i tng: Hai phng php c bn
Pht hin bin trc tip
p dng k thut ly o hm: Bc 1 (Gradient), bc 2 (Laplace)
Trong nh s: Ly o hm c ngha l m phng v xp x o
hm bng k thut nhn chp.
Pht hin bin gin tip
Thng qua phn vng nh, ranh gii gia cc vng l bin ca
i tng.

24/40
o hm bc nht v o hm bc 2 ca tn hiu nh

25/40
Mt s khi nim lin quan
Trc chnh: on thng ni hai im xa nht trn bin i tng
Trc ph: on thng vung gc vi trc chnh. Cng vi trc
chnh, n hnh thnh ch nht bao i tng
Ch nht c s: Hnh thnh bi trc chnh v trc ph
lch tm: T l gia trc chnh v trc ph.
Cc o hnh dng trn y hnh thnh biu din th
hnh dng
C th c s dng lm ch mc v tm kim hnh dng
Thng thng n c s dng cng vi cc c trng khc
ca hnh dng.
QBIC (IBM) s dng vng hnh dng, hng trc chnh, v cc
mmen bt bin ch mc v tm kim nh trn c s hnh
dng.
26/40
4.1 Moment hnh dng
Moment c s dng nhn bit nh v c s
dng trong nhiu h thng tm kim nh.
nh ngha:
Cho trc nh f(x,y) vi kch thc [1,n]x[1,l]
Moment bc (p+q) c nh ngha bi

Moment trung tm c nh ngha nh sau:

00
01
00
10

m
m
y v
m
m
x = =
= =
=
n
x
l
y
q p
pq
y x f y x m
1 1
) , (
=
x y
q p
pq
y x f y y x x ) , ( ) ( ) (
(x,y) - ta im nh
f(x,y) cng im nh
Trong ,
27/40
Moment hnh dng
Moment trung tm chun ha c tnh nh sau

Ngi ta chng minh 7 moments sau y l bt bin
vi dch chuyn, co dn v xoay

2 / ) 2 (
,
,
0 , 0
+ +
=
q p
q p
q p

q
| |
| |
| | ) )( ( 4 ) ( ) ( ) (
) ( ) ( 3 ) )( 3 (
) ( 3 ) 3 ( ) ( ) 3 (
) ( ) (
) 3 ( ) 3 (
4 ) (
03 21 12 30 11
2
03 21
2
12 30 02 20 6
2
03 21
2
12 30 03 21 03 21
2
03 21
2
12 30 12 30 12 30 5
2
03 21
2
12 30 4
2
03 21
2
12 30 3
2
11
2
02 20 2
02 20 1
q q q q q q q q q q q |
q q q q q q q q
q q q q q q q q |
q q q q |
q q q q |
q q q |
q q |
+ + + =
+
+ + =
+ =
+ =
+ =
+ =
28/40
Moment hnh dng

S dng 7 moment |
1
... |
7
m t hnh dng
Khong cch Euclidean gia cc m t hnh dng c
s dng nh khong cch gia hai hnh dng
Trong mt s trng hp, cc moments tng t khng
m bo cho cc hnh dng tng t.
Hiu nng ca ch mc v tm kim trn c s moment
hnh dng l khng cao.
| |
| |
2
03 21
2
12 30 03 21 30 12
2
03 21
2
12 30 12 30 30 21 7
) ( ) ( 3 ) )( 3 (
) ( 3 ) ( ) ( ) 3 3 (
q q q q q q q q
q q q q q q q q |
+ + + +
+ + + + =
29/40
4.2 B m t Fourier (Fourier Descriptor - FD)
Trong phng php trn c s b m t Fourier (FD),
hnh dng c biu din bi cc hm c trng f(t) (gi
l shape signature)
p dng bin i Fourier ri rc trn cc hm c trng
f(t) c FD hnh dng
Cc FDs c s dng ch mc v tnh ton hnh
dng

=
(
=
1
0
2
exp ). (
1
N
t
u
N
ut j
t f
N
F
t
trong , u=0, 1,..., N-1;
N l tng s mu ca f(t).
30/40
B m t Fourier (Fourier Descriptor - FD)
C nhiu loi hm c trng, v d
Trn c s ng bao
Trn c s bn knh
Trn c s ta bin.
Hiu nng h thng s dng ba loi hm c trng trn
khng khc nhau nhiu. Tuy nhin, hm c trng trn
c s bn knh c ci t n gin nht.
Hm c trng bn knh bao gm cc khong cch c
th t t tm hnh dng n cc im bin (gi l bn
knh)

trong , (x
c
, y
c
) l ta tm hnh dng. (x
i
, y
i
) vi i=0..63 l ta
64 im mu dc theo bin hnh dng.

2 2
) ( ) (
i c i c i
y y x x r + =
31/40
B m t Fourier
Sau khi bin i Fourier hm c trng bn knh, ta tnh
cc vct c trng nh sau ch mc hnh dng.

Vct c trng x (FD) bt bin vi dch chuyn, co dn
v xoay.
o khong cch gia hai vct c trng: Euclidean
Khng s dng trc tip bn knh ch mc hnh dng v
kh chun ha xoay v co dn.
(
(
=
0
63
0
1
,...,
F
F
F
F
x
32/40
B m t Fourier
V d cc hm c trng hnh dng khc
Gc tch ly

Hm din tch

| | ) 2 mod( . ) 0 ( ) ( ) ( t u u = t t
x
y
u(t)
u(0)
z(t)
z(0)
(t)
| ) ( ) ( ) ( ) ( |
2
1
) (
1 2 2 1
t y t x t y t x t A =
4.3 Tm kim nh trn c s vng
tng:
Ph li hnh vung trn vng hnh dng
Gn 1 cho nhng t bo b li ph t nht 25%, gn 0 cho
nhng t bo cn li.
Hnh dng s c biu din bi dy nh phn.
Nhn xt:
Kch thc t bo nh hn s chnh xc hn nhng i hi tnh
ton nhiu hn.
y
x
34/40
Tm kim nh trn c s vng
Chun ha xoay
Sp t hnh dng vo hng chung duy nht
Xoay hnh sao cho trc chnh song song vi trc x hoc trc y
(c th theo 1 trong 2 cch, ty thuc vo ci t chng trnh)

Gii php gim dung lng nh trong CSDL: Biu din hnh
dng trong cu truy vn bi hai dy nh phn.
Hnh 6.5
y
x
A
B
y
x
A
B
35/40
Tm kim nh trn c s vng
Chun ha co dn
Co dn mi hnh dng sao cho cc trc chnh ca chng c
cng di c nh (192 pixels).
o tng t
R v R l cc vct hng ca hai nh
C v C l cc vct ct ca hai nh
Tnh ton tng t
Tnh ton cc vct hng v ct ca mi vng trong nh truy vn
Tm khc bit hng v ct gia cc vng trong hai nh

Trong , R
d
v C
d
l hiu s hng v ct gia hai nh
R
i
v C
i
l bit th i ca cc vct nh th nht
R
i
v C
i
l bit th i ca cc vct nh th hai.
Nu (R
d
+C
d
) < T (ngng) th hai nh tng t.
R
d
= E
i
(|R
i
R
i
|)
C
d
= E
i
(|C
i
C
i
|)
36/40
5. Tm kim nh trn c s hoa vn
Hoa vn (texture) l c trng kh m t v cm nhn l
rt ch quan.
Su c trng (Tamura et al):
tnh th (coarseness),
tnh tng phn (contrast),
tnh theo hng (directionality),
tnh tng t on thng (line likeness),
tnh u n (regularity)
tnh g gh (roughness).
Tnh ton nh lng cc c trng hoa vn l rt kh
khn.
37/40
7. Kt lun
Ch mc v tm kim nh l ch c nghin cu
nhiu nht v c nhiu kt qu nht.
Cc tim cn ch mc v tm kim nh
Tm kim nh trn c s vn bn m t
Biu mu v tm kim nh trn c s biu mu
Tm kim nh trn c s hoa vn
Nhu cu nghin cu nng cao hiu nng ch mc v tm
kim nh.
38/40
Cu hi n tp
M t bn tim cn chnh tm kim nh. u im v
nhc im ca chng l g?
M t nguyn l ca k thut tm kim nh trn c s
mu c s.
Cc hn ch ca k thut tm kim nh trn c s mu
c s? M t cc phng php vt qua cc hn
ch ny.
Trnh by u im v nhc im ca phng php tm
kim nh trn c s biu mu tch ly.
Nhng tiu ch biu din hnh dng no ph hp vi tm
kim v ch mc nh?
M t phng php biu din hnh dng trn c s b
m t Fourier vi hm c trng bn knh.
Cu hi n tp
Hy xy dng dy nh phn cho cc hnh dng di y
v tnh ton khong cch gia hai hnh dng ny. Hy
to lp cc gi thit ph hp cho kch thc li v
di chun ca trc chnh.

Lit k cc c trng c bn ca nh p dng trong
MMDBMS m anh/ch bit.

MM Lec6

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

MM Lec6

Uploaded by

Copyright:

Available Formats

H Ni - 2005/14

You might also like