Professional Documents
Culture Documents
1, 2003
ABSTRACT
A wide class of nonlinear filters for image processing is outlined and described in a
unified way that may serve as a base for the design of their implementations in
optoelectronic programmable parallel image processors. The filters are treated in
terms of a finite set of certain estimation and neighborhood building operations. A set
of such operations is suggested on the base of the analysis of a wide variety of
nonlinear filters described in the literature.
1. Introduction
Since J.W. Tukey introduced median filters in signal processing ([1]), a vast
variety of nonlinear filters and families of nonlinear filters for image processing has
been suggested. The remarkable feature of these filters is their inherent parallelism.
This motivates attempts to develop unification and structurization approaches to
nonlinear filters to facilitate filter analysis, usage and design. In this paper, a
structurization and unification approach to image processing filters based on
fundamental notions of signal sample neighborhood and estimation operations over
the neighborhood aimed at filter implementation in parallel computing networks is
1
2. Main definitions
Main assumptions that constitute the suggested unified and structurized treatment
of nonlinear filters are:
window samples with their attributes form a primary window neighborhood Wnbh.
On the next level, this first level neighborhood NBH1 is used to form, through a
number of neighborhood building operations such as shown in Fig. 1 grouping and
intermediate estimation operations, a second level neighborhood NBH2 which, in this
illustrative example, is used for generating, by means of an estimation operation, filter
output pixel for this particular position of the window. It is only natural to associate
the filter output in each position of the window with the central pixel of the window.
We will outline typical attributes of pixels and and principles of neighborhood
formation in Sect. 3 and 5, respectively.
filters known from the literature. As one can see from the table, these secondary
attributes reflect features of pixels as members of their neighborhood.
Attributes Rank and Cardinality reflect statistical properties of pixels in
neighborhoods. They are interrelated and can actually be regarded as two faces of the
same quality. While Rank is associated with variational row, i.e ordered in ascending
order sequence of neighborhood pixel values, Cardinality is associated with the
histogram over the neighborhood.
Geometrical attributes describe properties of images as surfaces in 3-D
spaces, respectively. Membership in neighborhood and Spatial connectedness are
binary 0/1 attributes that classify topological relationship between a given signal
sample and a neighborhood. Neighborhood elements are regarded spatially connected
if one can connect them by a line that passes through the samples that all belong to the
neighborhood.
This list of signal attributes does not pretend to be complete. Rather, it reflects
the state of the art and may suggest directions for further extensions.
4. Estimation operations
Typical estimation operations used in known nonlinear filters are listed in
Table 2. In the filter design, selection of estimation operation is, in general, governed
by requirements of statistical or other optimality of the estimate. For instance, MEAN
is an optimal MAP- (Maximal A Posteriori Probability) estimation of a location
parameter of data in the assumption that data are observations of a single value
distorted by an additive noncorrelated Gaussian random values (noise). It is also an
estimate that minimizes mean squared deviation of the estimate from the data. PROD
is an operation homomorphic to the addition involved in MEAN operation: sum of
and those (vectorial) that are used in multi stage process of forming neighborhood.
The latter generate, for neighborhood elements, a new set of elements with their
attributes that form a neighborhood of the next stage. Scalar operations are basically
the same as those listed in Table 2. Typical vectorial operations are listed in Table 3.
One can distinguish three groups of vectorial neighborhood building
operations: functional element-wise transformations, linear combinations and
grouping/selection operations. Functional transformations are nonlinear functions
such as, for instance, logarithmic one, applied, element-wise, to all neighborhood
elements. MULT-operations assume multiplying neighborhood elements by scalar
weight coefficients that are selected according certain attributes (co-ordinates,
value, rank, cardinality), or combination of attributes. Replication REPL-operations
can be regarded as a version of weighting with integer weights and are used in data
sorting. A special case of replication are SELECT_A operations that select from the
neighborhood some elements (replication factor 1) and neglect others (replication
factor 0). In particular, shape-neighborhoods are formed by selection from the filter
window pixels those that form a certain special spatial shape, such as, for instance,
cross, diagonal, etc. Other examples of sub-neighborhoods formed by feature
controlled selection of neighborhood elements are shown in the table EV-, KNV-, ER, KNR-,Q-, CL-, FLAT-neighborhoods.
Linear combination operations multiply neighborhood elements by a matrix
and/or add/subtract a constant. Although the matrix can, in principle, be arbitrary,
orthogonal transform matrices are used in known filters.
number of stages in building neighborhood they use for generating final estimation of
the filter output. The tables do not pretend to contain all filters that have been
published by now. They are mostly based on data collected in Ref.[9] as well as in
Refs. [3, 4, 8, 9, 10] to which readers can refer for detailed information regarding
properties and applications of the filters.
Table 4 lists the simplest nonlinear filters that use one-stage NBH1
neighborhood, the primary window Wnbh-neighborhood. In particular, one can find
in the table such popular in signal and image processing filters as moving average,
median and local histogram equalization filters.
It appears that the majority of known nonlinear filters belong to the family of
two-stage NBH2 neighborhood filters listed in table 5. According to the type of the
NBH2-neighborhood used, the filters form four groups: MULT_A-, REPL_A-, V-,
and R-neighborhood filters. Some of them such as root mean square error (RMSE)
optimal linear, L- and C-filters are, in fact, families of filters.
Among three-stage neighborhood filters listed in Table 6 one can find two
large families of filters: transform domain filters and stack filters. Transform domain
filters nonlinearly modify transform coefficients of filter window samples to generate
filter output by means of applying to them operation MEAN which is an
implementation of the inverse transform for the window central sample. Two the most
advanced modifications of these filters are sliding window DCT ([8,11]) and wavelet
shrinkage filters ([12]). A popular in signal/image processing community Local
Linear Minimum Mean Square Error filter is a special case of transform domain filters
in which signal squared transform coefficients
Stack filters are yet another large family of filters. They originate from the
7. Iterative filtering.
An important common feature of the nonlinear filters is their local adaptivity:
the way how filter output is computed depends, in each filter window position, on
window sample attributes. In order to understand in what sense filters provide an
optimal estimate of signals one can assume that signal processing quality is evaluated
locally as well. Mathematical formulation of this assumption is provided by local
criteria of processing quality ([8]):
Here AVLOSS(k) is, for a signal sample with coordinate k, averaged value of losses
LOSS (a m , a m ) due to replacement, by filtering, of signal true values {a m }in
coordinates {m} within the window by their estimates {a m }. The averaging is two
fold. Spatial averaging is, in general, a weighted summation carried out over a
subset of signal samples associated with central sample k of the window (its
neighborhood
W (m ; a k ) 0 , if {m} NBH (k )
LOC (m ; a k )=
,
0,
otherwise
9. Some new filters that emerge from the structurization and unification
approach
The presented approach to the nonlinear filter unification eases analysis of
structure of nonlinear filters by reducing it to the analysis of what type of
neighborhood, neighborhood building and estimation operations they use. This
analysis may also lead to new filters that fill in logical niches in the classification.
Several examples of such filters are given In Table 10.
10
of the sigma-filter (Table 5). Original Sigma filter tends to leave untouched
isolated noisy pixels (Fig. 2c). EV-neighborhood of these pixels is very small in size
since they substantially deviate from their true values. In
SizeEV-controlled
11
those that are useful for subsequent estimation operation by evaluation of their
nearness to the selected sample in terms of their gray levels and ranks. It certainly
may happen that the resulting neighborhood will contain samples that are not spatially
connected to the center of the neighborhood. One can refine such a selection by
involving an additional check-up for spatial connectivity of neighborhood elements
which is of especial importance in image filtering applications. An improved image
denoising capability of such filters was recently reported in ([17]).
12
RANK(NBH).
11. Conclusion
It is shown that concepts of signal sample neighborhoods, estimation and
neighborhood building operations provide a unified framework for structurization and
classification of image processing nonlinear filters oriented on their implementation in
parallel multi-layer neuro-morphic networks. Many of the introduced concepts are
applicable for multi component signals such as color or multi spectral images as well,
although exhaustive extension of the approach to multi component signals requires
additional efforts.
12. Acknowledgement
The work was partly carried out at Tampere International Center for Signal
Processing, Tampere University of Technology, Tampere, Finland
13
13. References
1. J. W. Tukey, Exploratory Data Analysis, Addison Wesley, 1971
2. J. Serra, Image Analysis and Mathematical Morphology, Academic Press, 1983,
1988
3. V. Kim, L. Yaroslavsky, Rank algorithms for picture processing, Computer
Vision, Graphics and Image Processing, v. 35, 1986, p. 234-258
4. I. Pitas, A. N. Venetsanopoulos, Nonlinear Digital Filters. Principles and
Applications. Kluwer, 1990
5. E. R. Dougherty, An Introduction to Morphological Image Processing, SPIE
Press, 1992
6. H. Heijmans, Morphological Operators, Academic Press, 1994
7. E. R. Dougherty, J. Astola, An Introduction to Nonlinear Image Processing, SPIE
Optical Engineering Press, 1994
8. L. Yaroslavsky, M. Eden, Fundamentals of Digital Optics, Birkhauser, Boston,
1996
9. J. Astola, P. Kuosmanen, Fundamentals of Nonlinear Digital Filtrering, CRC
Press, Roca Baton, New York, 1997
10. E., Dougherty, J. Astola, Nonlinear Filters for Image Processing, Eds., IEEE
publ., 1999
11. L.P. Yaroslavsky, K.O. Egiazarian, J.T. Astola, Transform domain image
restoration methods: review, comparison and interpretation, Photonics West,
Conference 4304, Nonlinear Processing and Pattern Analysis, 22-23 January,
2001, Proceedings of SPIE, v. 4304.
12. D. L. Donoho, I.M. Johnstone, Ideal Spatial Adaptation by Wavelet Shrinkage,
Biometrica, 81(3), pp. 425-455, 1994
14
13. P. D. Wendt, E. J. Coyle, and N. C. Gallagher, Jr., Stack Filters, IEEE Trans.
On Acoust., Speech and Signal Processing, vol. ASSP-34, pp. 898-911, Aug.
1986.
14. http://www.eng.tau.ac.il/~yaro/RecentPublications/index.html
15. T. Szoplik, Selected Papers On Morphological Image Processing: Principles and
Optoelectronic Implementations, SPIE, Bellingham, Wa., 1996 (MS 127).
16. V. Kober, J. Alvarez-Borrego, Rank-Order Filters With Spatially-Connected
Neighborhoods, NCIP2001, June 3-6, 2001, Baltimore, MD, USA
15
Value
Co-ordinate
Cardinality
Primary attributes
ak
k(a)
Secondary attributes
Number of neighborhood elements with
H(a)=HIST(NBH,a) the same value as that of element a
(defined
for
quantized
signals): H (a ) = (a a k )
k NBH
Rank
Ra=RANK(NBH,a)
3. Ra = H (v )
v =0
Geometrical
attributes
Membership in
the neighborhood
Spatial
connectedness
COORD(NBH,R )
GRDNT(NBH,k )
CURV(NBH,ar )
MEMB(NBH,a).
CONCTD(NBH,a)
16
Definition
MEAN(NBH)
PROD(NBH)
the
MAX(NBH)
MODE(NBH)
arg max (H (a ))
a
RAND(NBH)
17
MULT_H(NBH)
KNVnbh( NBH ; a k , K )
nearest
by
value""K
neighborhood of element a k
RNGnbh(NBH,Vmn,Vmx)Range-neighborhood
18
H-neighborhoods:
pixel cardinalities as attributes
Neighborhood elements that belong to the
CLnbh( NBH ; a k ) "Cluster" neighborhood of element a k . same cluster of the histogram over the
neighborhood as that of element a k .
G-neighborhoods
Geometrical attributes
FLAT(NBH)
Neighborhood elements with values of
Flat-neighborhood
Laplacian (or module of gradient) lower
than a certain threshold
LINEAR COMBINATION OF ELEMENTS OF NEIGHBORHOOD
Orthogonal transform T of neighborhood elements
T(NBH)
Differences between elements of the neighborhood and
DEV(NBH,a)
certain value a
SELECTION OF SUB-NEIGHBORHOOD
FROM A SET OF SUB-NEIGBORHOODS
MIN_Std(SubWnbh1, SubWnbh2, ,
Neighborhood standard deviation as the
SubWnbhn )
attribute
MIN_RNG(SubWnbh1, SubWnbh2, ,
SubWnbhn )
19
a k = MODE( Wnbh )
Signal enhancement filters
Local histogram
equalization
Quasi-range filter
Local variance
filter
a k = RANK (Wnbh)
a k = QSRNG( Wnbh ) = R_ROS( Wnbh ) - L_ROS( Wnbh )
a k = STDEV( Wnbh )
20
Table 5. Two-stage (NBH2 -based) filters classified according to the type of the
neighborhood used
FUNC(NBH)
a k = MEAN(FUNC( NBH ))
General
a k = MEAN(MULT _ R( Wnbh ))
ak = MEAN(MULT _ RC( Wnbh ))
Erosion filter
Soft Morph. filters
V-neighborhoods
ak = MEAN(KNV (Wnbh; ak ; K ))
ak = MEAN(EVnbh(Wnbh; ak ; Vpl ; Vmn ))
[1 MEMB( Qnbh(Wnbh, R , R ), a )]
SMTH (Qnbh(Wnbh , R , R ))
left
left
right
right
[1 MEMB( Qnbh(Wnbh, R , R ), a )]
MEDN (Qnbh(Wnbh , R , R ))
left
left
21
right
right
a
=
1
a
+
MEAN( Wnbh ) ,
Minimum Mean
k
2 k
2
(
(
)
)
(
(
)
)
STD
Wnbh
STD
Wnbh
Square Error
filter
where 2 is a filter parameter
Double Window
Modified
ak = MEAN (EVnbh(Wnbh ; MEDN( SHnbh); Vpl ; Vmn )) .
Trimmed Mean
filter
ak =
Stack filters
MAX(MIN (SubWnbh1 ), MIN( SubWnbh2 ),..., MIN(SubWnbhn ))
.
[ (
)]
Weighted
Majority of m
Values with
Minimum Range
-filters, Shorthfilters
ak =
MEAN(MULT_C(PROD(SubWnbh1 ),..., PROD(SubWnbhn )))
({
(m )
where SubRnbh i
elements.
}))) ,
22
P
a k = MEAN Wnbh (RANK (Wnbh))
))
23
T able 10.
Size controlled
Sigma-filter
))
( ))
STEP (Thr SIZE(EVnbh(Wnbh ; a
)))
; Vpl ; Vmn ,
where
Size_EV filter
0 , x 0
.
STEP ( x ) =
1 , x > 0
ak = SIZE(EVnbh(Wnbh ; Vpl ; Vmmn ))
a
a max
v =0
v =0
P
a k = (H (v )) /
P- histogram
equalization
Cardnl-filter
(H (v ))
a k = HIST( Wnbh , a k )
EVnbh-histogram
equalization
KNVnbh-histogram
equalization
SHnbh-histogram
equalization
NBH2-histogram equalization
ak = RANK (EVnbh(Wnbh ; ak ; Vpl ; Vmn ))
ak = RANK (KNV (Wnbh ; ak ; K ))
a k = RANK(SHnbh )
24
Fig. 1 Illustrative flow diagram of signal filtering by a nonlinear filter with 2-stage
neighborhood building procedure
25
It
c) Iterative sigma filter, Evpl=Evmn=20,
Wnbh=5x5; 5 iterations
a) Original image
b) Size-EV-filtered image
c) Cardnl-filtered image
Fig. 3. Size-EV and Cardnl-filtering
27
RANK(Wnbh25x25)
RANK(EV(Wnbh25x25;7,7))
28
feedback
Fig. 5. Schematic diagram of a multilayer parallel network with feedback
29
Input layer
Layer of
summation units
Output layer
Layer of lookup-tables
Layer of lookup-tables
Layer of
summation units
30
Input layer
Layer of look-up-tables
IOutput
31