CONTENT BASED IMAGE

RETRIEVAL

SUYOG DUTT JAIN

Dept. of CSE
Manipal Institute of Technology
Presentation Outline

 Objective of the seminar

 Basics of Content Based Image Retrieval
 Existing Systems
 Algorithms based on the feature – color
 Applications
 Conclusion
 References
Objective of the seminar

 To throw light upon research in Image

Processing
 Discussion of Content Based Image Retrieval
[CBIR]
 Detailed discussion of some basic but very
efficient algorithms for Image Retrieval
Basics of Content Based
Image Retrieval
 Definition
Retrieval of images based on visual features
such as color, texture and shape.
 Steps Involved:
(i) Feature Extraction (ii) Feature Comparison
 Challenging Tasks:
(i) Maintaining and Searching through database
(ii) Formulation of exact query
(iii) Evaluation of close results
Basics of Content Based
Image Retrieval : Retrieval
Methods
 Color
Feature Extraction : Color Intensities
Feature Comparison: Color Histograms
 Texture
Feature Extraction : Relative Brightness etc.
Feature Comparison: Degree of Contrast etc.
 Shape
Feature Extraction : Aspect Ratio, Local
Features
Feature Comparison: Directional Histograms
Basics of Content Based
Image Retrieval: Architecture
Basics of Content Based
Image Retrieval : Concepts
 Digital Image
 Pixel
 Image Quantization
 Color Histogram
 Grayscale Images
 Conversion of Color Image to Grayscale Image
Basics of Content Based
Image Retrieval : Query
Format

 Query by example
 Using a test image

 Query by low level features

 Using a sketch drawn by user
Existing Systems

 QBIC

 VIRAGE

 EXCALIBUR
Algorithm Color Histogram
Matching : Basics
The histogram of a digital image with gray levels in the
range 0…L-1 is given by a discrete function
Hist (rk) = nk
Where,
rk is the kth gray level
k=0 …L-1,
L is the number of intensity levels.
nk = number of pixels at gray level rk.
Algorithm Color Histogram
Matching : Feature
Extraction
For height of bitmap to 0
For 0 to width of bitmap
Read pixel
if pixel value=ith gray level
increment (histogram at ith gray level)
else
continue
Algorithm Color Histogram
Matching : Feature Matching
255

histdist [dataset] = Σ |hist_database[j]-hist_query[j]|

j=0

where,
j denotes the various gray levels
hist_query is the histogram of query image,
hist_database is histogram of the database image.
histdist is error difference or distance metric.
The nearest matching database images with the
query image has the least distance metric. The exact
match is the one with the zero distance metric.
Algorithm Bit Plane
Histogram Matching : Basics
 Image is composed of eight one bit planes,
 bit plane 0 for the Least Significant Bit
.
.
 bit plane 7 for Most Significant Bit.
 Visually significant information is contributed by
higher order bits and least significant bits
contribute less.
 When total number of pixels over the entire
image is counted in which a particular ith bit is
set to 1, it gives the ith bit plane Histogram
 Algorithm Bit Plane
Histogram Matching :
Feature Extraction
Pixel

yes
7 bit
th
7th bit hist
set
no
yes
6th
6th bit hist
bit
set
no

yes
0th 0th bit hist
bit
set
Algorithm Bit Plane
Histogram Matching :
Feature Extraction
For height of bitmap to 0
For 0 to width of bitmap
Read pixel
if pixel value at ith bit plane is 1
increment (histogram [ith bit plane])
else
continue
Algorithm Bit Plane
Histogram Matching :
Feature Matching
 Bit plane histogram is computed for the query image
 Distance between histograms of database images
and query image is as shown:

hist (|i|, database) =|hist_database[i]-hist_query[i] |

i=7, 6...0

 hist_database[i] and hist_query[i] are the bit plane

histograms i.e. number of pixels in the image having
ith bit as 1.
Algorithm Hierarchical Bit
Plane Histogram Matching

tep1: Compute bit plane histogram error difference at

7th bit plane between query feature and n number of
database features. Apply threshold and reduce.

tep2: Computing 6th bit plane histogram error

difference on images obtained in step 1.

………..

ontinue till 0th bit plane.

Algorithm Spatial Histogram
Matching : Basics
 It is common that the major object in an image is
located in the central position
 Image is divided in to sub-regions that are of equal-
size in terms of percentile area
 The color histograms for each sub-region are
computed for all the database images and query
image Sub Region 4
Sub Region 3
Sub Region 2

Sub Region 1
Algorithm Spatial Histogram
Matching : Defining Regions
class Point Region()
{ int x,y; {
Point() tl.x=0;tl.y=0;
{x=0;y=0;}} tr.x=0;tr.y=0;
bl.x=0;bl.y=0;
class Region br.x=0;br.y=0;
{ Point tl=new Point();
Point tr=new Point(); }
Point bl=new Point();
Point br=new Point(); };
Algorithm Spatial Histogram
Matching : Defining Regions
public void defineRegion() r2.bl.x=w1*2;r2.bl.y=h1*6;
{ int w1=iw/8; r2.br.x=w1*6;r2.br.y=h1*6;
int h1=ih/8; //Region 3
//Region 1 r3.tl.x=w1*1;r3.tl.y=h1*1;
r1.tl.x=w1*3;r1.tl.y=h1*3; r3.tr.x=w1*7;r3.tr.y=h1*1;
r1.tr.x=w1*5;r1.tr.y=h1*3; r3.bl.x=w1*1;r3.bl.y=h1*7;
r1.bl.x=w1*3;r1.bl.y=h1*5; r3.br.x=w1*7;r3.br.y=h1*7;
r1.br.x=w1*5;r1.br.y=h1*5; //Region 4
//Region 2 r4.tl.x=0;r4.tl.y=0;
r2.tl.x=w1*2;r2.tl.y=h1*2; r4.tr.x=iw;r4.tr.y=0;
r2.tr.x=w1*6;r2.tr.y=h1*2; r4.bl.x=0;r4.bl.y=ih;
r4.br.x=iw;r4.br.y=ih; }
Algorithm Spatial Histogram
Matching : Finding Region of
Pixel(x,y)

ublic String findRegion(int x,int y)

String region="";

if((x>=r1.tl.x&&x<=r1.tr.x)&&(y>=r1.tl.y&&y<=r1.bl.y))

region="Region1";
Algorithm Spatial Histogram
Matching : Finding Region of
Pixel(x,y)
else if((x>=r3.tl.x&&x<=r3.tr.x)&&(y>=r3.tl.y&&y<=r3.bl.y))
{
region="Region3";
}
else
{
region="Region4";
}
return region;
}
Algorithm Spatial Histogram
Matching : How it Works

alculate the grayscale histogram features for all the four

sub regions. Then compare the Histogram features
starting from innermost sub region to outermost sub
region, hierarchically.

tep1: Compute gray scale histogram error difference at

sub-region1 between query feature and n number of
databases features. Apply threshold and reduce.

tep2: Compute gray scale histogram error difference at

sub-region2 between query feature and features of
Algorithm Spatial Histogram
Matching : How it Works

tep3: Compute gray scale histogram error difference at

sub-region3 between query feature and features of
reduced set of images obtained in step2. Apply
threshold and reduce.

tep4: Compute gray scale histogram error difference at

sub-region4 between query feature and features of
reduced set of images obtained in step3. Apply
threshold and reduce.

inally, least distance metrics will represent the similar

Applications of Content
Based Image Retrieval
 Search Engines
 Object Recognition and tracking
 Crime Investigation
 Art Collections
 Medical Records
Conclusion
 This research area is growing very rapidly
 Current systems are still in prototype stage and lack
reliability
 Current techniques are based on low level features
and there is a huge semantic gap existing
 Much more research work is needed for coming out
with a reliable and semantically competent system
 References
[1]. Manjunath KN, Renuka A, “Bit plane histogram matching for CBIR”, National Level
Technical paper presentation, Kadi, Gujarat.
[2]. Manjunath KN, Renuka A, Harischandra Hebbar N, “Hierarchical Bit plane histogram
matching for CBIR”, IEEE’s Signal Processing Society, EMBS, TIFAC-CORE sponsored
National Conference on Image Processing, MSRSAS, Bangalore.
[3]. Manjunath KN, Renuka A, Harischandra Hebbar N, “Spatial Bit plane histogram
matching for CBIR”, AICTE, ISTE New Delhi sponsored National Conference on
Graphics, Vision and Image Processing, J.N.N College of Engineering, Shimoga.
[4]. Kato, T., Database architecture for content-based image retrieval in: Jambardino, A. A.,
and Niblack, W. R., (Eds.), Image Storage 439 and Retrieval Systems. Proc SPIE 1662,
112–123, 1992.
[5]. Swain, M. J., and Ballard, D. H., Color indexing. Int. J. Comput. Vis. 7(1):11–32, 1991.
[6]. Stricker, M., and Orengo, M., Similarity of color images. In: Niblack, W. R., and Jain, R.
C., (Eds.), Storage and Retrieval for Image and Video Databases III. Proc SPIE 2420, pp
381–392, 1995.
[7]. Stricker, M., and Dimai, A., Color indexing with weak spatial constraints. In: Storage and
Retrieval for Image and Video Databases IV. Proc SPIE 2670, 29–40, 1996.
[8]. Gonzalez, R. C., and Woods, R. E., Digital image processing, 2004 2nd Edition, pp 94–103.

[9]. Flickner, M et al “Query by image and video content: the QBIC system” IEEE Computer
28(9), 23-92