RIM.

Reconocimiento de Imágenes
Image Recognition
Subspace Methods and Appearance-based
Image Recognition

Francesc J. Ferri

Dept. d’Informàtica.
Universitat de València

November 24, 2009

F.J. Ferri (Univ. València) RIM November 24, 2009 1 / 21

Introduction

Image Representation

Some image recognition problems can be decomposed into several

subproblems: segmentation, detection, registration.

Sooner or later, image or image parts conveniently represented using

appropriate descriptors need to be “recognized” by using a pattern
recognition system.

Segmentation, detection and understanding can be very interleaved in

some cases.

F.J. Ferri (Univ. València) RIM November 24, 2009 2 / 21

 Introduction

We will assume that images or image parts have been processed and are to
be used (as a whole) as inputs for a given pattern recognition system.

These images can be used in its raw form of after a convenient change of
representation (global transform, normalization, global descriptors of color,
texture, etc.

Examples (possibly after detection, alignment and normalization):

Face recognition.
(particular) object recognition.
Fingerprint recognition
OCR
Texture classification
window-based image segmentation
etc.

F.J. Ferri (Univ. València) RIM November 24, 2009 3 / 21

Introduction

Appearance-based image recognition

If images to be considered can be considered as reasonably aligned, the

same image values (pixels) can be directly considered as features.

If the recognition system uses or is based on prototypes or feature vectors,

these can also be seen as images allowing for a complementary viewpoint
of the recognition process.

F.J. Ferri (Univ. València) RIM November 24, 2009 4 / 21

 Introduction

Representation spaces

One of the most important facts is that each image object is a point in a
very high dimensional space.

In the case of images of aligned and normalized gray level images of faces

Objects are n × m-dimensional L-valued feature vectors.

F.J. Ferri (Univ. València) RIM November 24, 2009 5 / 21

Introduction

Subspace-based recognition

Very high dimensional feature spaces

Linear (but kernelizable) projection based methods
Small sample problem (more dimensions than objects!)
Classical approach to some image recognition problems.

objects / dimensionality / classifier /

reduction subspace

F.J. Ferri (Univ. València) RIM November 24, 2009 6 / 21

 Introduction

Generative vs discriminative and taxonomy

Generative methods
are based on preserving variability or correspondingly removing redundancy
or noninformative parts of the objects.

Discriminative methods
are based on increasing the dissimilarity or the sepparation among different
categories of the considered objects.

Linear methods
consist of applying a linear transformation on the data followed by a trivial
selection of dimensions (linear projection).

Nonlinear and kernel methods

More general. Kernel methods are linear methods in a nonlinearly
transformed space which is implicitly defined.

F.J. Ferri (Univ. València) RIM November 24, 2009 7 / 21

Introduction

Taxonomy

 

 
 unsupervised

 


  (PCA,ICA,NMF)



linear

 
 supervised (LDA, Null

 

  Space)
Feature
Extraction 
 

 

 
 nonparametric (NDA,MDS, Sammon)



nonlinear kernel-based (KPCA, KLDA, KICA)

 

 

 manifold-based (Isomap, LLE)

F.J. Ferri (Univ. València) RIM November 24, 2009 8 / 21

 Introduction

The classics: Eigenpictures

Basically consists of applying PCA in this particular high dimensional

context. This method is also known as eigenfaces, eigenimages or
eigenwhatever.

Let x be a (vector) image which is linearly transformed as

y = AT x

x ∈ IRD and y ∈ IRd and A is an D × d matrix.

Let X = {xi }ni=1 be a given (training) set of images. D >> n.

P
Let Sx = k (xk − µ)(xk − µ)T be the scatter matrix of X .

F.J. Ferri (Univ. València) RIM November 24, 2009 9 / 21

Introduction

Principal Component Analysis

Define A as the (D × D) eigenvector matrix such that

AT Sx A = diag (λ1 , . . . , λD )

If eigenvalues, λi are decreasingly ordered, the D × d matrix given by

A = (u1 , . . . , ud ) is the linear transformation into a d-dimensional
subspace that maximizes variability or minimizes reconstruction error.

X
T 2
E [||x − AA x|| ] = λk
discarded

A set of n column vectors X can be transformed as

(y1 · · · yn ) = Y = AT X = AT (x1 · · · xn )

F.J. Ferri (Univ. València) RIM November 24, 2009 10 / 21

 Introduction

Example. Face Recognition

Let X be

This constitutes a set of 12 objects in a 10304-dimensional space.

F.J. Ferri (Univ. València) RIM November 24, 2009 11 / 21

Introduction

Best 2D projection
1

2000
0.9

0.8 1000
variance

0.7 0

0.6
−1000

0.5
−2000

0.4
1 2 3 4 5 6 7 8 9 10 11 −3000 −2000 −1000 0 1000 2000
dimensions

F.J. Ferri (Univ. València) RIM November 24, 2009 12 / 21

 Introduction

F.J. Ferri (Univ. València) RIM November 24, 2009 13 / 21

Introduction

Classifying with eigenfaces

A transformation matrix into an appropriate low dimensional space is

computed.
Any classifier can be applied. Usually the nearest mean or nearest
neighbor classifier.
Alternatively, a transformation matrix per class is obtained.
A measure of similarity to each class (in different subspaces) is
computed and a distance-based criterion is used.

F.J. Ferri (Univ. València) RIM November 24, 2009 14 / 21

 Introduction

Discriminant subspaces: Fisherfaces

As in standard dimensionality reduction, linear discriminant analysis can be

applied to find a low dimensional subspace where classes are optimally
discriminated under certain circumstances.

Plain LDA cannot be directly applied because of numerical problems.

Usually, PCA followed by LDA is used.

F.J. Ferri (Univ. València) RIM November 24, 2009 15 / 21

Introduction

Fisherfaces
P
Let SW = ci Si be the within-class scatter matrix (Si is the i-th class
scatter matrix).
P
And let SB = ci ni (µi − µ)(µi − µ)T the between-class scatter matrix
(Sx = SW + SB )

The generalized Fisher criterion

|AT SB A|
J(A) = T
|A SW A|

Optimization of J(A) gives rise to a generalized eigenproblem

SB w = λSW w.
−1
The solution is given by the eigenvectors of SW SB (if SW is invertible)
(c − 1 at most).

F.J. Ferri (Univ. València) RIM November 24, 2009 16 / 21

 Introduction

Nonlinear extensions

Subspace methods based on eigenanalysis can be extended by using

the kernel trick.
Basically, the data is implicitly mapped to a higher dimensional space
where a linear projection is constructed.
The final projected space is the composition of the kernel mapping
with the linear projection

F.J. Ferri (Univ. València) RIM November 24, 2009 17 / 21

Introduction

Kernel Principal Component Analysis

PCA
Given X and its covariance matrix, Sx , solve λv = Sx v (eigenanalysis).
Then construct A = (v1 , . . . , vd ) and project data as AT x.

A linearization mapping
Let φ be a mapping into a high dimensional space. Let Sφ be the
covariance matrix corresponding to φ(X ). Then we need to solve
λw = Sφ w .

As w ∈ span(φ(X )):
λφ(xi ) · w P
= φ(xi ) · (Sφ w ) ∀i
∃αi : w = i αi φ(xi )

F.J. Ferri (Univ. València) RIM November 24, 2009 18 / 21

 Introduction

The vector α that defines eigenvectors in the linearization space can

be obtained from eigenanalysis of:
(Mλ)α = K α
where K is the kernel matrix, Kij = φ(xi )φ(xj ).
(α must be renormalized to have vectors w normalized.)

How can we construct now the corresonding A without explicitly having

vectors w ?

F.J. Ferri (Univ. València) RIM November 24, 2009 19 / 21

Introduction

x −→ φ(x) −→ AT φ(x) = (w1 · φ(x), . . . , wd · φ(x))

and

X
wk · φ(x) = αi φ(xi ) · φ(x)
i

This means we can compute the nonlinear components manipulating only

dot products of mapped vectors which can be substituted by kernels,
k(x, y ) = φ(x) · φ(y ).

F.J. Ferri (Univ. València) RIM November 24, 2009 20 / 21

 Introduction

KPCA (for centered data)

Construct the kernel matrix, K , from training data, X .

Diagonalize K and normalize its eigenvectors, αk , k = 1, . . . , d.
To project x, compute first its kernel vector,
Kx = (φ(x1 ) · φ(x), . . . , φ(xn ) · φ(x) and then compute each
component as αk · Kx .

F.J. Ferri (Univ. València) RIM November 24, 2009 21 / 21