MTF for color Bayer pattern detector

Elor Yotam, Pinsky Ephi and Yaacobi Ami.

Rafael Ltd., P.O.B. 2250, Haifa 31021, Israel
yotame@rafael.co.il
ABSTRACT
We present a model for calculating the Spatial Frequency Response (SFR) for Bayer pattern color detectors. The model
is based on the color detector response to B/W scenes. When a Bayer color detector is compared to a B/W detector, SFR
difference results from the interpolation process. This process exists only in the Bayer pattern detectors. In this work
we ascribe the MTF and the spurious response to the interpolation process.
The model may be applied to any linear interpolation. Although the interpolation is linear, it is not Shift Invariant
(SI). Therefore, calculating the interpolation MTF is not a trivial task. Furthermore, the interpolation creates a spurious
response. In order to calculate the interpolation SFR, we introduce a separable constraint (for x and y directions) by
using a scene that varies only on one axis and is fixed on the other. We further assume integration in the direction of the
fixed axis. By using these two assumptions, we have been able to separate the response into two axes and calculate the
SFR.
For distant scenes, colors saturation decreases, the colors are less visible and mostly grey colors are sensed. In these
cases the Johnson Criterion can be roughly applied. In order to apply the Johnson Criterion it is required to know the
MTF of the sensing system. The sensing system MTF includes the interpolation MTF. We show that the interpolation
process degrades the system performance compared to B/W sensor. Another application of the model is in comparing
different interpolation algorithms.
Keywords: Bayer Pattern, MTF, OTF, Resolution, Spurious response.

BAYER BACKGROUND
The Bayer Pattern is made of a repeating array of red, green and blue filters (figure 1) applied on each detector element.
Adding a Bayer filter enables B/W detector to create color images(1,2). On the other hand, creating color images using
Bayer pattern causes under-sampling of the available data. Furthermore, each pixel contains only one color channel out
of the three. Therefore, an interpolation process must be applied in order to create three complete color image planes.

B
G
B
G
B

G
R
G
R
G

B
G
B
G
B

G
R
G
R
G

B
G
B
G
B

Figure 1. Bayer Pattern Filter: Each square represents a pixel on the detector. Each pixel receives data for only one color (Red Green
or Blue).

THE MODEL
Consider two imaging systems. The first system is B/W imaging system and the second is Color (figure 2). The optics
and the detector are identical in both systems. Both systems are viewing a B/W scene.

B/W imaging system

B/W display

B/W
Sensor

"Perfect" optics
Color imaging system

B/W
Sensor

White
balance

Interpolation

Color display

Bayer
Filter
Array

"Perfect" optics
Fig 2. Two imaging systems. the top is B/W imaging system and the bottom is Bayer color imaging system.

The two systems are identical except for four components:

1.

2.
3.

Bayer pattern + White balance system: The Bayer filter attenuation for white light is different for each color
channel. The white balance system is built to compensate this effect. Therefore the White balance system will
amplify each color channel accordingly. In the case of viewing a B/W scene, the effects of these two
components cancel each other.
Interpolation system: This system exists only in the color imaging system. The interpolation process degrades
the system SFR performances. This system will be discussed later.
Display: Since both systems are viewing a B/W scene, we will consider the color display to be presenting a
B/W image where the gray levels are the Luminosity values. Then, considering the pixel size in both displays
to be equal, the two displays behave in a similar way.

By comparing the two systems above, we conclude that the SFR difference between the two systems results only
from the interpolation process.

LINEAR INTERPOLATION PROCESS

This paper analyzes the performance of Bayer pattern color detectors with a linear interpolation process(3,4). A linear
process means that the color value of each pixel after the interpolation is a linear function of its surroundings before the
interpolation. Recently, intensive work has been done in the area of nonlinear interpolation algorithms(5-7). Although
Non linear advanced algorithms perform better than linear ones, linear interpolation is faster and easier to implement.
Therefore, linear interpolation processes are still being used.
Ignoring edge effects, the linear interpolation process for one color channel can be described by:

(1) siint ( x, y ) = si ( x' , y ') f i ( x, y, x' , y ')

x'

y'

si is the color channel i signal input into the interpolation system, siint is the result of the interpolation for
channel i and f i is the interpolation weight function.
where

In Bayer pattern, the color channel

i input signal is a sample of the signal introduced into the interpolation system.

s ( x, y ) pixel ( x, y ) color channel i

(2) si ( x, y ) =
else
0

where s (x, y )

is the signal input into the interpolation process. In the linear interpolation system

f i ( x, y, x' , y ') = 0 for every pixel (x,y) that does not belong to the i color channel, therefore the i index may be
omitted from the signal si in equation (1).
There are four kinds of pixels (figure 3) each related to a specific weight functions for each Color channel. So

siint

can be written as:

s * * f i1 ( x , y )

2
s * * f i ( x, y )
int
(3) si ( x, y ) =
3
s * * f i ( x, y )
s * * f i 4 ( x, y )
Where the bottom index of

x Odd , y Odd
x Odd , y Even
x Even, y Odd
x Even, y Even

f i j refers to the color channel, the top index refers to the kind of pixel and ** stands for

two dimensional convolution. Equation (3) can be understood as four image planes for each color channel, each image
plane is the convolution of the input signal with one of the weight functions. Hence, the image plane for each color
channel is an alternate sampling of the four image planes.

G
R
R

R
1

G
G

G
1

G
3G

2G 4

B
B

G
G

Fig 3. Decomposition of Bayer filter pattern into its three color channels. The four kinds of pixels are marked 1 to 4. Each pixel on
the detector grid belongs to one of the four kinds.

THE INTERPOLATION PROCESS MTF AND SPURIOUS RESPONSE

The interpolation process, as described above, is a discrete process that is neither shift invariant (SI) nor separable. In
order to ascribe a one dimensional MTF to the system, we must impose it to be separable. A scene that varies at one
axis and is invariant on the other will be used, i.e. a scene composed of some spatial frequencies along one direction and
no spatial frequencies other than zero, in the other direction. Let the modulated axis be the x axis and the constant axis y. However, the system is not SI, thus the system output might be composed of spatial frequencies not included in the
system input. As a consequence, the output signal may include spurious spatial frequencies in the y direction. This
response is an unwanted effect. In order to assign a one dimensional MTF to the system, the signal in the y direction is
averaged, hence all spatial frequencies other than zero are ignored. More accurate analysis may include that spurious
response. However due to the system symmetry, the spurious response in the y direction is proportional to the spurious
response in the x direction which is analyzed later on.
Using equation 3 and a constant scene in the y direction, the interpolation process output averaged in the y direction
can be described by:

s * * f i1

2
1 N int
1 N s * * f i
int
(4) si ( x ) si ( x, y ) =
N y =0
N y =0 s * * f i 3
s * * f i 4
int
Now, si ( x ) can be written as

s * hi1
(5) siint ( x ) =
2
s * hi

x Odd
x Even

x Odd , y Odd

f i1 + f i 2
s
*
x Odd , y Even
2
y
=
3
f + fi4
x Even, y Odd
s* i
2
x Even, y Even
y

x Odd
x Even

where,

hi1 =

1
f i1 (x, y ) + f i 2 ( x, y )

2 y

hi2 =

1
f i 3 ( x, y ) + f i 4 ( x , y )
2 y

After reducing the problem to one dimension, the three color channels are then combined back into one B/W
channel by using a simple luminosity equation(8):

(7) L = [K1
In this case

K2

s1int

int
K 3 ] s 2int = K s
s3int

s1int is red, s2int is green and s3int is blue. Therefore K = [0.3 0.6 0.1] will be used. The

equivalent for the B/W signal can be presented by:

s * h1
s int B / W = L = K s int =
2
s * h
h1j

h j = K h2j
j = 1,2
h3j

x Odd
x Even

So far the interpolation system output B/W signal equivalent was calculated. The Fourier transform of that signal is
presented by

H 2 ( ) + H 1 ( )
H 2 ( ) H 1 ( )
+ S ( )
2
2
f
Where S ( ) is the Fourier transform of s and =
.
f Nyquist

(9) S int B / W ( ) = S ( )

In order to write the Fourier transform of the B/W image on the display, the system frequency components are
multiplied.

H 2 ( ) + H 1 ( )
MTF display ( ) +
2
H 2 ( ) H 1 ( )
S ( ) MTF optics ( ) MTF det ector ( )
MTF display ( )
2

S out = S ( ) MTF optics ( ) MTF det ector ( )

(10)

The MTF is properly defined only for linear SI systems. Bayer color imaging system is not an SI system. However,
for practical uses it is desired to ascribe an MTF to the system. It is suggested to extend the MTF definition to include
some non SI systems. Accordingly, the MTF for a single specific frequency is defined as the output signal in that
frequency divided by the input signal in that frequency, where only that frequency is input into the system. Note that for
an SI system, a single input frequency will generate an output response of only that frequency.
The first term in equation (10) is the system attenuation in the frequency domain. Using the MTF extended
definition above this term will be referred as the system MTF.

(11) MTF system =

S out ( )
H 2 ( ) + H 1 ( )
optics
det ector
(
)
(
)
=
MTF

MTF

MTF display ( )
in
S ( )
2

where the MTF of the interpolation process is obtained as:

(12) MTF int erpolation =

H 2 ( ) + H 1 ( )
2

H i ( ) functions are bounded by one. When designing an imaging system, high MTF is usually required.
i
Therefore, while designing an interpolation process, the mean of the two H ( ) is one of the parameters to optimize.
The

The second term in equation (10) is the system response at frequencies other then the input frequency. We call this
term: Spurious Response. These are undesirable effects originated from the lack of system SI nature. These affects are
hard to quantify since they are input signal's dependant. Yet, the spurious response is roughly proportional and is
attributed to the term:

(13) Spurious int erpolation =

H 2 ( ) H 1 ( )
2

When designing a system, it is desirable to minimize the spurious effects as much as possible by minimizing the

H i ( ) functions. The H i ( ) functions are the Fourier transforms of the h i ( x ) functions.

i
The system designer controls the h ( x ) functions by selecting the interpolation algorithm. Different interpolation
i
algorithms create different h ( x ) functions and different spurious responses. By choosing an interpolation algorithm
i
that render h ( x ) as similar as possible, the system designer can minimize the spurious effects.
difference between the two

EXAMPLE FOR SIMPLE INTERPOLATION PROCESS

The MTF and spurious response effects added by interpolation process, are well demonstrated by calculating the MTF
and spurious response of a specific interpolation process - the Bilinear interpolation(3). The Bilinear interpolation
algorithm interpolates the missing colors from the close vicinity surrounding eight pixels.
The interpolation process is fully described by the

1
f1 =

(14)

0.25
0
0.25

0 0.25
0
0
0 0.25

0 0.25 0
1
f 2 = 0.25
0
0.25
0 0.25 0
0 0 0
1
f 3 = 0 1 0
0 0 0

f i j functions:

0 0 0
2
f1 = 0.5 0 0.5
0 0 0

0 0.5 0
3
f1 = 0 0 0
0 0.5 0

0 0.5 0
2
f 3 = 0 0 0
0 0.5 0

0 0 0
3
f 3 = 0.5 0 0.5
0 0 0

0 0 0
2
f 2 = 0 1 0
0 0 0

0 0 0
3
f 2 = 0 1 0
0 0 0

0 0 0
4
f1 = 0 1 0
0 0 0

0 0.25 0
4
f 2 = 0.25
0
0.25
0 0.25 0
4
f3 =

0.25
0
0.25

0 0.25
0
0
0 0.25

After derivation of the h functions and performing Fourier transform one receives

(15)

MTF int erpolation = 0.65 + 0.35 cos(2f )

Spurious int erpolation = 0.1 0.1cos(2f )

Where

(f

< fN )

1 cycle
f stands for spatial frequency [cy/pxl] and f N = [
].
2 pixel

In order to derive the whole imaging system MTF, it is required to assume MTF functions for the optics, detector
and display. Furthermore, in order to evaluate the spurious response, the signal introduced into the system is required.
Let it be designated by ( f ) . Considering "perfect" optics with cutoff at Nyquist frequency, square detector and
"perfect" display one receives:

1
MTF optics ( f ) = MTF display ( f ) =
0
(16) MTF det ector ( f ) = sin c( f )

f < fN
else

( f ) = sin c( f )

Using equation (10), the signal may be separated on the display into two parts: The desired signal and the spurious
response. Both responses are presented in figure 4. The B/W imaging system response is also presented for comparison.
1
B/W system
output

Amplitude

Color system
output

Color system
spurious
response

0
-1

-0.5

0
Frequency [cycle/pixel]

0.5

Fig 4. Frequency domain presentation of the Bayer color imaging systems, the B/W imaging system and the input scene.

It is clear from figure 4 that when viewing a B/W scene, the Bayer color imaging system is inferior to the B/W
system. The performances are degraded in two ways. First, the desired signal is further attenuated in a color system due
to the lower MTF. Second, the color system adds significant undesired effects to the scene (the spurious response).
1

0.8

Amplitude

B/W detector
MTF
0.6

0.4

0.2
Color detector +
interpolation MTF
0

0.5
Frequency [cycle/pixel]

Fig 5. MTF comparison between B/W and color imaging systems. For the B/W system, the MTF includes the detector MTF only. For
the Color system the MTF includes the detector and interpolation process MTFs.

Figure 5 demonstrates clearly the widely known superior resolution of a B/W system MTF with same pixel size over
Bayer configuration color system. In the vicinity of Nyquist frequency, the signal of the color system is about one third
of the B/W system.

CONCLUSIONS
In this paper the spatial frequency response of a Bayer configuration color detector for a B/W scene is analyzed. It is
shown that Bayer detector is inferior to B/W detector in two ways. First, a Bayer configuration detector attenuates the
signal significantly compared to the B/W detector. Second, the Bayer detector creates a spurious response. Both effects
result from the interpolation process.

An MTF is ascribed to the interpolation process for B/W scenes. This MTF can be used for applying Johnson
criterion on a color detector as a practical method to assess effective detection, recognition and identification ranges of
objects in the scene. This is essential for designing surveillance systems based on Bayer configuration color sensors.
The model proposed here, allows introducing Bayer effect into the system ranges evaluation, which otherwise tends to
be ignored. Furthermore, this approximation is indeed adequate to long ranges scenes, since at long range scenes, colors
saturation decreases to a level that mostly grey colors are sensed.
Evaluation of a color imaging system performance is a complex task. The model described in this work ignores the
hue data and evaluates the imaging system performance for B/W scenes, meaning that the scenes are composed of only
brightness differences. However, in many natural scenes, most of the scene's spatial data is brightness differences.
Therefore the model provides a reasonable estimation for the SFR properties of the imaging system.

REFERENCES
1. B. E. Bayer, "Color imaging array", U.S. Patent No 3,971,065 (1976).
2. Spatial Frequency Response of Color Image Sensors: Bayer Color Filters and Foveon X3.
3. Rajeev Ramanath, North Carolina State University, "Demosaicking methods for Bayer color arrays", journal of
Electronic Imaging 11(3), p. 306-315 (July 2002).
4. Malvar, H.S, Li-wei He Cutler, R. "High-quality linear interpolation for demosaicing of Bayer-patterned color
images". Acoustics, Speech, and Signal Processing, ICASSP Proceedings. Volume 3, p. 485-488 (May 2004).
5. D. Darian Muresan, Steve Luke and Thomas W. Parks. "Reconstruction of color images from CCD arrays".
Cornell University, Ithaca NY.
6. Hanno Scharr, "Optimal Separable Interpolation of Color Images with Bayer array format" Inter disciplinary
Center for scientific Computing. Ruprecht-Karls-Universitat Heidelberg.
7. A. Lukin and D. Kubasov "High-Quality Algorithm for Bayer Pattern Interpolation". Moscow State University.
Programming and Computer Software, Vol. 30, No. 6, 2004, p. 347-358.
8. Keith Jack. 1955 - Video Demystified: a Handbook for the digital engineer. 3rd ed.
9. Analysis of Sampled Imaging Systems. Richard H. Vollmerhausen and Ronald G. Driggers. Tutorial Texts in
Optical Engineering Volume TT39. SPIE PRESS.