Critical Analysis of Adaptive Biometric Systems

Norman Poha , Ajita Rattani and Fabio Roli Department of Computing, FEPS, University of Surrey, Guildford, UKa Department of Electrical and Electronic Engineering, University of Cagliari Piazza dArmi, Cagliari, Italy
normanpoh@ieee.org,{ajita.rattani,roli}@diee.unica.it

December 18, 2012

Abstract Biometric based person recognition poses a challenging problem because of large variability in biometric sample quality encountered during testing and a restricted number of enrollment samples for training. Solutions in the form of adaptive biometrics have been introduced to address this issue. These adaptive biometric systems aim to adapt enrolled templates to variations in samples observed during operations. However, despite numerous advantages, few commercial vendors have adopted auto-update procedures in their products. This is due in part to the limited understanding and limitations associated with existing adaptation schemes. In view that the topic of adaptive biometrics has not been systematically investigated, this paper works toward lling this gap by surveying the topic from a growing body of the recent literature and by providing a coherent view (critical analysis) of the limitations of the existing systems. In addition, we have also identied novel research directions and proposed a novel framework. The overall aim is to advance the state-of-the-art and improve the quality of discourse in this eld.

1 Introduction
While the biometric technology continues to improve, an intrinsic characteristic of this technology is that a systems error rate, e.g., the false accept rate (FAR), false reject rate (FRR) and equal error rate (EER) (the rate at which FAR is equal to FRR), cannot attain the absolute zero. A major cause of these errors is the compound effect of the scarcity of training samples during the enrollment phase as well as the presence of substantial sample variations due to human-sensor interaction and the acquisition environment during operations [1]. Apart from this, being biological tissues in nature, biometric traits can be altered either temporarily or permanently, due to ageing [2], diseases or treatment to diseases. An important consequence of these factors is that a biometric reference 1 (obtained during
1 A template refers to the biometric sample used for enrolment. The term model refers to a statistical representation derived from one or more biometric samples. In order for our discussion to cover both types of method, we shall adapt the standard vocabulary that is biometric reference or

enrollment) cannot be expected to fully represent a persons identity. Solutions in the form of adaptive biometrics have been introduced to address this issue of reference representativeness [3, 4]. These adaptive biometric systems attempt to update reference galleries by integrating information captured in input operational samples. The two-fold aim is to continuously adapt the biometric system to the intraclass variation of the input data as a result of (1) changing acquisition conditions that may have adverse impact on the system, e.g., pose and illumination changes for face biometrics, and (2) age and life-style related changes that can cause permanent changes to the biometric trait. Most of the existing automated adaptive biometric systems have adopted semi-supervised learning [11, 4] for the purpose of adaptation. Semi-supervised learning is a machine learning scheme based on the joint use of labeled and unlabeled samples. In other words, input samples are assigned identity labels using enrolled references and the positively classied samples are used to adapt the references. A commonly adopted adaptation procedure is to augment the reference set with the newly classied input samples. The efcacy of the system can be gauged by comparing the obtained performance gain with a traditional biometric system which does not have any adaptation mechanism. The expected performance gain is dependent on the effective labeling (classication) of the input samples. This is because misclassication errors will introduce impostor samples into the updated reference set, the result of which can be counterproductive. An adaptive biometric system may also operate in supervised mode in which biometric samples are manually labeled [3]. The supervised method represents the best case performance as all the available positive (genuine) samples are used for adaptation. However, manual intervention may be time consuming and costly. Therefore, it is generally infeasible to manually update references regularly. In contrast, an adaptive biometric system has numerous advantages. First, with this system, one no longer needs to collect a large number of biometric samples during enrollment. Second, it is no longer necessary to re-enrol or re-train the system (classier) from scratch in order to cope up with the changing environment [3]. This convenience can signicantly reduce the cost of maintaining a biometric system. Third, the actual observed variations can be incorporated into the references. Despite these advantages, to our knowledge, few biometric vendors such as BIOsingle (ngerprint) and Recogsys (hand geometry) have incorporated automated adaptation mechanism into their technologies at the time of this writing. This is due in part to the limited understanding and limitations associated with existing adaptive biometric systems. The goal of this manuscript is to advance the state-of-the-art in adaptive biometrics by improving the understanding and drawing on the limitations of the existing adaptive biometric systems. To this aim, critical analysis of the existing literature is conducted. Based on the ndings of the critical analysis, we propose a novel framework that aims to mitigate some of the limitations and investigate possible future research avenues. Specic contributions of this manuscript are as follows:
simply reference. A reference is subsequently used for comparing a biometric test/query sample to obtain a similarity score.

1. a taxonomy of adaptive biometric systems through a number of key attributes, 2. use of a meta-analysis technique to objectively compare the effectiveness of key attributes across various systems reported in the literature, and 3. identication of novel research directions based on the ndings of the above meta-analysis. A preliminary version of this manuscript appeared in [3] in the form of critical survey. The current manuscript substantially differs from [3] in the following ways. First, novel attributes that distinguish an adaptive system from one another are introduced. Second, meta-analysis is utilized to aid analysis of various state-of-the-art adaptive systems. Last but not least, a novel framework, as well as research directions, is proposed. The paper is organized as follows: section 2 formulates the key attributes and conducts the meta-analysis. Section 3 provides the novel framework and research directions. Conclusions are drawn in section 4.

2
2.1

Attributes and Critical analysis

Attributes of the existing adaptive biometric systems

In an attempt to categorize adaptive biometric systems, the most logical way to proceed is to dene a number of key attributes. On surveying the current state-of-the-art, we nd that the following attributes are relevant to distinguish one adaptive biometric system from another: 1. Supervised against Semi-supervised: The foremost attribute in classifying adaptive systems is indisputably on the basis of whether the data labeling process is supervised [3, 4, 5, 9] or unsupervised [12, 13, 4]. While in supervised adaptation, samples are manually labeled, in the unsupervised case, they are inferred by the system. The latter approach is generally referred to as semi-supervised learning because the enrolment biometric reference (template) is effectively labeled but the potential operational biometric samples that are used for adaptation are unlabeled. As mentioned before, supervised adaptation represents the best case scenario, i.e., resulting in the best possible performance because all available genuine samples are used for the process of adaptation. Therefore, it is generally useful to report both strategies when comparing different adaptive methods. 2. Self- against Co-train: For an automated adaptive systems based on semi-supervised learning, self- [12, 13, 14] and co-training [17, 4] are the commonly adopted schemes for adaptation. In self-training, the system updates itself by adding only highly condently classied input samples as additional data for training. A sample is said to be highly condently classied if its matching score on comparison with the enrolled templates is above a stringent operating threshold. The reason to adopt highly condently classied samples for adaptation is to avoid impostor intrusion into the updated template set.

On the other hand, a co-training based scheme utilizes the mutual and complementary help of the two biometrics to update the references. Intuitively, one system is expected to assign correct labels to biometric query samples that are difcult for another system. Consider an example of face and ngerprint co-training system. While on its own, the face sub-system may have difculty in labeling a query sample in difcult conditions, the ngerprint sub-system may classify the associated ngerprint sample with very high condence. In this case, the face system can benet from the high condence of the ngerprint by incorporating the additional face samples for training. Therefore, two systems operating at high threshold can still help each other to identify difcult samples exhibiting large intra-class variations. 3. Verication against Identication: These adaptive biometric system can also be differentiated on the basis of the systems basic mode of operation i.e., verication (input sample is compared to the references of the claimed identity) or identication (input samples are matched to the references of all the users in the database and then the correct identity is determined among the top most retrievals) [1]. Accordingly, the performance gain will be measured using EER or rank-one performance metrics, respectively. 4. Level of adaptation: In addition to the adaptation at the reference level, the process of adaptation can also take place at score or decision level where the matching score or decision functions are adapted to the variations of the input biometric samples. For instance, Reference [5] uses biometric sample quality to adapt the matching score so as to render the nal accept/reject decision independent of the input sample quality. 5. Online against ofine: Online adaptive systems [12, 13] adapt themselves as soon as the input data is available after the recognition process. On the other hand, ofine methods [15, 16, 4] adapt themselves after a batch of input samples have been accumulated over a period of time. Another ne distinction between the two is that while an online method follows the chronological ordering of the availability of the samples during adaptation, the ofine one may not adhere to such an ordering. 6. Quality against non-quality based: Recent advancement in the biometric community
2

shows that biometric

sample quality has considerable impact on the system performance for various traits like ngerprint, iris, face, and etc, as well as for fusion [7]. Quality measures quantify the degree of excellence or conformance of biometric samples to some predened criteria known to inuence the system performance. However, it is only recently that biometric sample quality has been considered for adaptive biometric systems [5, 6]. Quality based adaptation requires maintaining a different set of updated references for each type of condition. Since a query sample is always acquired under a particular condition, the inference (matching task) requires identication of its quality condition and matching with the set of references of the same quality type [6]. In this manuscript, the resultant system is termed condition-adaptive system.
2 http://www.itl.nist.gov/iad/894.03/quality/workshop/

7. Impostor against Non-impostor attack: Adaptive biometric systems deployed in a real operational environment are vulnerable to impostor attack where an unauthorized user attempts to gain access to the system. However, early studies did not assume impostor attack during the systems operation [9, 12, 14, 15]. This is evident by the fact that the data used for adaptation (called the adaptation set) contained only genuine samples for adaptation. Later on, this limitation was identied and impostor samples were introduced in the adaptation set to simulate update process in a real operational environment [16, 21, 23, 28, 4]. In the next section, we will quantify existing adaptive biometric systems based on the mentioned attributes using meta-analysis.

2.2

Meta Analysis: a tool for critical analysis

Meta-analysis is a quantitative method for analyzing results from multiple papers on the same subject [8]. It has the property [8] of synthesizing (summarizing) results from multiple independent experiments in a quantitative manner. Therefore, we adopted meta-analysis as a tool to perform critical analysis and to compare existing adaptive biometric systems based on the identied key attributes (in section 2.1). To this end, we divided the existing adaptive systems based on the key attributes i.e., supervised vs. semisupervised, self- vs. co-train, online vs. ofine, and quality vs. non-quality based. Furthermore, we distinguished these systems on the basis of inclusion or exclusion of impostor attacks during the adaptation process i.e., impostor vs. non-impostor attack. The effect of a systems different mode of operation (i.e., verication or identication) is mitigated by standardizing the performance metrics (explained in detail in the following subsection). Then we use meta-analysis to validate the following hypotheses: 1. Is supervised adaptation better than semi-supervised? 2. Can co-training outperform self-training? 3. Is ofine adaptation better than its online counter-part? 4. Can quality-based adaptation outperform its non-quality based counterpart? 5. Is there any performance bias if one excludes non-match (impostor) samples in the adaptation set (i.e., no impostor attack)? As it turns out, there are already 23% of existing papers addressing the second hypothesis above. Nevertheless, it would still be interesting to infer the expected difference in the performance gain of co-training over self-training based on the existing studies. However, if we would like to infer whether or not the use of biometric sample quality can outperform a non-quality based adaptive system (hypothesis 4), none of the selected papers directly tested this hypothesis. Only meta-analysis 5

can offer the possibility of utilizing the diverse experiments performed by independent researchers to test hypothesis 4 without recourse to direct experimentation. In summary, irrespective of the different protocols and data sets, meta-analysis offers a means to objectively quantify and compare different adaptive biometrics in our survey, as the rst recourse. The resultant set of hypotheses may then be subject to direct testing, i.e., explicit comparison of two adaptive systems on a common data set, if clear inferences cannot be drawn. For the purpose of meta-analysis, we selected a total of 22 research papers based on the criteria that these papers provided sufcient details regarding the obtained performance and clearly stated different attributes of the experiments (as listed in section 2.1); they are: [9, 12, 13, 14, 15, 17, 16, 20, 21, 5, 22, 23, 18, 24, 19, 25, 10, 26, 27, 28, 29, 30]. Since each paper reported several experiments (with a median of 4), a total of 103 experiments are available for meta-analysis. We characterize and summarize the outcome of these 103 experiments 3 , based on pre-identied attributes, using meta-analysis. A generalized linear model (GLM) with linear output is trained with a data table containing one experiment per line. This model takes a set of attributes (as binary variables) in order to predict the performance gain as its output. In the following sections, we rst explain how a generalized linear model (GLM) can be used to characterize the outcome of one of the 103 experiments and how the attributes are encoded. We then present the experimental protocols. The nal subsection presents the ndings of the meta-analysis.

2.2.1

Standardizing the Performance Statistics

Proceeding to meta-analysis is not straightforward since the performance quoted by each paper is not consistent. In particular, there are two types of performance metric that are systematically quoted: Equal Error Rate (EER) and rank-one recognition performance. EER quanties the probability of error at an operating threshold where the rate of false acceptance is equal to that of false rejection. It is often quoted in a biometric verication scenario. The rankone recognition performance, on the other hand, is quoted in a biometric identication scenario. It is dened as the probability of a target user is indeed ranked the top from a gallery of registered users. In order to handle the different metrics used, we opted to derive a secondary metric called performance gain. It is dened as the amount of improvement with respect to the baseline system as well as the primary target metric one would like to achieve. For EER, it is dened as: Perf. gain = EERb EERa EERb 0 (1)

where EERa is the EER of the adaptive system, EERb is the EER of the baseline system, and the zero value in the denominator is the target EER value which one would like to achieve. For the rank-one recognition performance, we
3 The

data used for meta-analysis is available in the following link: https://sites.google.com/site/ajitarattaniitaly/resources

used the following performance gain denition, instead: Perfa Perfb 1 Perfb

Perf. gain =

(2)

where Perfa denotes the performance of the adaptive system whereas Perfb is the performance of the baseline system, and the unit value in the denominator is the target rank-one recognition performance. Despite the differences in denition of the EER and the rank-one recognition metrics, the performance gain metric has the following properties in both cases. First, a positive performance gain implies improvement over the baseline system. Second, the maximum performance gain can be almost equal to one. This can be simply veried by the fact that Perfa 1 and EERa >= 0. Therefore, given that an adaptive biometric system is always reported to be better than its baseline counterpart, the performance gain will be bounded between 0 and 1. If a value of 1 is registered, then the target metric is achieved (that is zero for EER and one for rank-one recognition performance). Therefore, the performance gain we introduced is a viable means to handle the differences in the two primary metrics used (due to the different mode of operation i.e., verication or identication) by the researchers, allowing the performance gain of different systems to be compared on equal ground.

2.2.2

Encoding the Attributes of an Experimental Outcome

An experiment is assigned a code of 4 bits in order to represent the following binary attributes, namely: 1. Presence of quality (quality): 1 means yes; and 0, otherwise 2. Use of co-training (co-train): 1 means yes; and 0, otherwise, which implies either self-training or supervised adaptation 3. Use of supervised adaptation (supervised): 1 means yes; and 0, otherwise, which implies semi-supervised adaptation (co-training or self-training) 4. Presence of non-match samples in the data set reserved for adaptation (impostor attack): 1 means presence; and 0, otherwise. For instance, an experiment coded as 1101 implies that the experiment involves an adaptive biometric system that uses biometric sample quality, relies on co-training hence, cannot be supervised and contains non-match samples (impostor attacks) in the adaptation data set. In order to compare different attributes, two types of meta-analysis experiments are performed, namely singlefactor and multi-factor analysis. In the former, only one of the four attributes (as explained earlier) is considered, whereas in the latter, all four attributes are considered at the same time. In order to carry out the two types of experiments, we used a generalized linear model with linear output that estimates performance gain using eq.(1) or (2). 7

Apart from these binary attributes, we also collected other contextual variables that may impact on the generalization performance of our analysis. These variables are the database size, number of samples used for adaptation, modality involved etc. Collected data for the contextual variables
4

demonstrate that face is the commonly adopted modality

followed by ngerprint in the existing studies. The adopted adaptation procedure is the same irrespective of the modality involved. Existing adaptive studies have handled short-to-medium term temporal variations without explicitly considering the ageing effect over long time span. This is evident by the fact that the adopted databases are collected over 14-15 weeks in most of the studies. However, since these contextual variables are not used in inference, they are not considered in tting the generalized linear model (GLM). By inference, we mean that the model will be used to predict a novel but valid combination of attributes not necessarily represented by the data table. Therefore, our primary goal is to study the attributes that are likely to dictate the performance gain of an adaptive biometric system. The inuence of contextual variables are not of interest here but this can be a subject of future investigation. Next, we explain how the generalized linear model is trained.

2.2.3

Training with Generalized Linear Model (GLM)

The generalized linear model [31] takes a set of four attributes as independent variables in order to predict the performance gain as its output. If a [a1 , . . . aN ] is a vector of binary attributes encoded as a binary string, and w [w1 , . . . wN ] is the weight vector of real numbers whose elements are associated with those in a, then, GLM produces y = waT + w0 as output. The training process involves estimating the vector of coefcients w including a bias term, w0 R. After training, the weights {w0 , . . . , wN } are obtained. The GLM is inferred by enumerating a subset of valid attributes {a}. In the single-factor analysis, eq. (3) is then invoked to consider only an attribute (N = 1) which can take either a 1 or a 0. The performance gain inferred by both cases, along with their respective upper and lower condence intervals, are then compared. In the multi-factor analysis scenario, the multi-dimensional attribute a is enumerated but invalid combinations are excluded. The performance gain of each valid attributes in {a} is then compared. Next, we shall report the ndings of single-factor meta-analysis followed by that of multi-factor one. (3)

2.2.4

Findings of Single-factor Meta-analysis

The result for single-factor analysis is shown in Figure 1.

4 Collected

data is available in the tabulated form (excel le) in the following link: https://sites.google.com/site/ajitarattaniitaly/resources

 Supervised adaptation is likely to outperform (about 22.2% more performance gain) semi-supervised method to adaptation such as self-training and co-training. As mentioned before, performance of the supervised adaptation can be considered as best case as the references are adapted to all the available genuine samples [4]. This is in contrary to methods based on semi-supervised learning in which only selective (mostly highly condently classied) samples are used for adaptation. Our meta-analysis ndings show large variance in the performance gain of the supervised method such that it overlaps signicantly with that of semi-supervised one, indicating also the effectiveness of the latter. However, for the real time deployment of automated methods (based on semi-supervised learning), their performance should be equal to their supervised (manual) counterparts. Thus, further indicating the need of effective adaptation schemes for automated systems. Co-training is likely to boost the performance gain by about 25.3 % in comparison to its self-training counterpart. The use of biometric sample quality appears to be much better than not using this information. According to our ndings, adaptive biometric systems considering quality measurements resulted in about 47% more performance gain. Including impostor samples in the adaptation set can result in lesser performance gain (16% lesser in our experiments) than if the samples were not present. Since an automated adaptive system deployed in operational environment is vulnerable to impostor attack, it is unrealistic not to include impostor samples in the adaptation set. As a consequence, our exercise here shows that not including impostor samples in adaptation set can over-estimate the performance gain. 2.2.5 Findings of Joint-factor Meta-analysis

Figure 2 summarizes performance gain for the joint-factor meta-analysis scenario spanned by four binary attributes: quality, co-train, supervised, and impostor attack. For instance, 0001 implies that an adaptive system that does not use biometric sample quality, that is based on self-training (hence, not supervised), and the system has been tested with non-match samples (impostor attack) in the adaptation data set. The attribute impostor attack is always true as this strategy reports a less biased performance gain, as explained before. The rst three attributes are then enumerated, excluding invalid combinations. For instance, it is not possible that co-training and supervised adaptation to be present at the same time, as co-training is a semi-supervised learning strategy; hence, cannot be supervised. Note that the adaptive systems considering supervised adaptation and quality at the same time (quoted as 1011) are managing the updated references on the basis of quality type. The query biometric samples are matched to the references of the same quality type [5]. Adaptive systems based on co-training exploit mutual and complementary information of the bi-modal system for template adaptation as well as testing. On the other hand, existing studies on supervised adaptation have been reported only for single biometric modality. This explains the superiority of co-training over supervised adaptation. 9

impostor

supervised

cotrain

quality

20

25

30

35 40 45 50 Performance gain (%)

55

60

65

Figure 1: The performance gain for a given attribute obtained by the trained generalized linear model. A blue (red) bar denotes a 95% condence interval around the expected performance gain, denoted as circle (square), when a given attribute is present (absent). These ndings suggest that there is a natural increase in performance as one exploits co-training, supervised adaptation and biometric sample quality systematically. To sum-up, our meta-analysis ndings (both single and joint factor analysis), support the conjecture that quality and co-train are important attributes for the design of automated adaptive biometric systems.

Novel Framework and Research Directions

In this section, we propose a novel framework and set some future research directions that are motivated by the

ndings of the meta-analysis. The results of both the single and joint factor analysis indicate that quality and co-training are important ingredients when designing an adaptive biometric system. Furthermore, when analyzing the contextual variables of the reported systems, such as the adopted database size and the number of samples, we found that these systems did not consider the notion of time in order to account for the ageing effects which may induce temporal performance variation over a long time span. Motivated by this, we shall propose a novel system that can make use of quality and further include the notion of time in a single framework. This proposed novel framework is termed as condition-and age-adaptive system.

10

1101

1011

1001

0101

0011

0001 0 10 20 30 40 50 Performance gain (%) 60 70 80

Figure 2: The performance gain along with the condence intervals of various congurations spanned by four binary attributes: quality, co-train, supervised, and impostor (see text).

3.1

Framework for condition and age adaptive system

Existing adaptive biometric systems have not considered the ageing effect explicitly. A possible reason for this is that, the effect of ageing is often considered to be very different from that caused by biometric sample quality. As a consequence, methods that aim to address ageing often assume that the biometric sample is free from noise, that is, images are often well aligned and acquired in controlled conditions. In practice, however, an adaptive biometric system has to deal with both the aspects (i.e., adaptation to ageing and quality conditions) for the life-long learning and coping under non-stationary conditions caused by changes in biometric sample quality. Two separate strategies are needed in order to handle variations caused by biometric sample quality and those caused by ageing because while the former can cause dramatic changes to the captured biometric features almost instantaneously, age-related changes are, in comparison, a much more slower and irreversible process. However, beyond a certain limit of time, the variation due to age-related factors will dominate over that due to the quality-related ones. This is illustrated in Figure 3, where one image is taken under a somewhat controlled condition, another with a signicantly different quality (head pose) but taken at the same time, and then another taken after two years. Thus in order to cope with the changes in the quality as well as temporal variations in the input sample, we propose a possible framework called a condition and age adaptive system. We have adopted a Bayesian approach for the formulation of the system. This choice is appropriate because biometric features are generated by a stochastic process. As a result, no two consecutive samples obtained from a biometric trait are exactly the same. Thus the uncertainty at the feature space can be characterized using a distribution

11

Figure 3: Illustrating an example of face images taken at different quality conditions (left versus middle) and over time (left versus right). dened over the feature space. Indeed, a number of state-of-the-art face and speaker recognition classiers are based on Bayesian formulation, e.g., [32] and [33]. Furthermore, the state-of-the-art online template update method used in the ngerprint literature [12, 13, 14] can be interpreted using a Bayesian framework [6]. Next, we introduce the bayesian framework and explain the proposed system.

3.1.1

Bayesian Framework and notations

The recursive formulation of Bayesian estimation allows one to update the parameters of an old or initial model with a new ones given only the latest sample. Thus, given a sequence of observations collected over time, (x1 , . . . , xT ) or (x1 : xT ), one can estimate a statistical model parameterized by , p(x|), in the following way (ignoring the normalizing factor in each step since we are only interested in maximizing the function with respect to ):
T i=1

p(|x1 : xT )

p(xi |)p() p(xi |)p(|x1 ) p(xi |)p(|x1 , x2 )

T i=2 T i=3

. . . p(xT |)p(|x1 : xT 1 ) (4)

This recursive formulation implies that in order to calculate the optimal value of given all previously observed T samples, one only needs to use the parameter calculated up to T 1 to do so. The above recursive formulation shows the benet of learning for density-based classiers as an example, leading to nding the optimal value of . This recursive formulation is known as true recursive Bayesian learning. The right- hand term, p(|x1 : xT ), is a reproducing density and the term p() is a conjugate prior [34]. Although the above adaptation is well established and appears to be sound, it does not consider biometric sample quality nor the ageing effect. A theoretical framework for model (reference) adaptation using biometric sample quality has been proposed in

12

[6] but did not consider the time effect. The model is henceforth referred to a condition adaptive system. In the subsequent section, we shall propose a theoretical formulation of condition and age adaptive system that considers both aspects, hence will be capable of life-long learning (age adaptive) and learning under non-stationary environment causing concept drift (condition adaptive). Let x be a biometric feature vector; j N, the users identity; Qu {Q0 , Q1 , . . . , QQ }. The condition in which a biometric sample is captured, with Q0 being the enrolment (controlled) condition and Qu |u = 0 being other uncontrolled conditions. Each condition Qu is due to a number of factors. Let these factors be enumerated by (f (1) , . . . , f (F) ). For instance, for the face biometrics, f (1) is lighting; f (2) corresponds to facial expression types; f (3) indicates the presence of glasses, f (4) estimates the head pose, and etc. Then, each Qu is a compound effect of these factors, i.e., Qu = (f (1) , . . . , f (F) ). It is arguable that, in practice, the condition Q is countable but the total number of conditions, i.e., Q+1 (including the enrolment condition Q0 ), cannot be determined exactly. This number, however, is not impossible to estimate. For instance, it can be estimated by clustering quality measures, or by manual annotation [6]. Let t N, the time at which a sample is captured. This notion of time is discrete; it is loosely dened such that two samples that are close in time (say a few seconds apart) will have the same t value. The rationale for using this denition of t is that the appearance of each biometric trait does not change, as a result of ageing, at the same rate. Using the above notation, the feature distribution of person j can be completely specied by p(x|j, Q, t). Let x(j, Q, t) be a sample drawn from p(x|j, Q, t). We shall refer to x(j, Q0 , t0 ) as a reference or model where t0 is the time at which this sample is obtained; and p (x|(j, Q0 , t0 )), a model with parameter (j, Q0 , t0 ) that approximates the true density p(x|j, Q0 , t0 ). Q0 implies that the sample is taken under controlled conditions, that is one in which all the quality-related factors have been carefully controlled, i.e., F (1) = F0 , . . . , F (F) = F0 . The notation also allows us to describe non-ideal samples, for instance, non-frontal head poses, presence of glasses, as may be captured during enrollment i.e, {x(j, Qu , t0 )} for u = 0. The distribution dened over these samples is written as p(x|j, Qu , t0 ) for each u; and their corresponding approximated model, as p (x|(j, Qu , t0 )). Let y R be a matching score. Furthermore, let j be the claimed identity and x x(j, Q, t ) be a query sample taken at time t from an unknown person j under an unknown condition state Q. For simplicity and without loss of generality, we also write x x(j, Q, t) but write in full in order to emphasize a particular state, e.g., x(j, Q, t ) to emphasize a given time t and x(j, Q , t) to emphasize a given state Q .
(1) (F)

13

We can then dene the following modes of operation:

enrol matchtq matchbayes adaptcond adapttime

: biometric trait x(j, Q0 , t0 ) : x, x(j , Q0 , t0 ) y : x, p(x|, Q, t), p(x|j , Q, t), y : p (x|(j, Q, t)), x(j, Q , t) p (x|(j, Q , t)) : p (x|(j, Q, t)), x(j, Q, t ) p (x|(j, Q, t )),

where Q is a short hand for Q = Q to emphasize that Q assumes a particular quality state. The same convention is adopted for t . The operation matchtq produces a similarity or a dissimilarity measure between a reference, denoted as x(j , Q0 , t0 ), and a query sample, x(j, Q, t), with the unknown identity j . On the other hand, the operation matchbayes typically takes a query sample and a pair of densities (one representing the universal background or world model and another representing the client-specic model) as input and produces a likelihood ratio or a posterior probability as output. For instance, the state-of-the-art speaker verication system computes: p (x|(j , Q, t)) ) p (x|(, Q, t))

matchbayes (x) = log(

(5)

as output, where p (x|(, Q, t)) is the density of the general population, also known as the universal background or world model: p (x|(, Q, t) =
j =j

p (x|(j, Q, t)P (j )

where P (j ) weighs the contribution of client-specic density p (x|(j, Q, t) to the nal general-population density.This approach can be traced back to Neyman-Pearson theorem. An equivalent, alternative formulation is to invoke the Bayes rule, which computes matchbayes (x) = P (j | (j , Q, t)) as output, noting that the parameter (j , Q, t) is not the same as the s used before if the posterior probability is approximated using a discriminative classier such as a multi-layer perceptrons. Accordingly, the condition adaptation (adaptcond ) operation adapts an existing model to a new quality condition, Q , represented by a given input sample taken in a different condition. On the other hand, temporal adaptation (adapttime ) updates an existing model to a new one given the most current sample taken at time t . A non-adaptive biometric system is neither adaptive to the age-related factors nor the quality related ones. These systems can be dened as p(x|(j , Q0 , t0 )) p (x|(, Q0 , t0 )) 14

match(x) = log

Based on the mentioned notation, next we will describe the proposed method.

3.1.2

Towards a Condition and Age Adaptive System

In order to describe the proposed condition and age-adaptive system, rst, we will present a condition-adaptive system and an age-adaptive one separately. The condition adaptive system has been proposed in [6]. However, different from [6] which considers only condition-adaptive systems, we further introduce the age-adaptive system and the proposed mixture of both condition- and age-adaptive systems here. Accordingly, a condition-adaptive system operates in two modes: adaptation mode and matching (comparison) mode. In the adaptation mode, the system adapts its model using one or more samples taken from an unseen condition Q. Adaptation:

p (x|j , Q, ) p (x|, Q, )

= adaptcond ( p(x|j , Q0 , t0 ), x) for all Q = adaptcond ( p(x|, Q0 , t0 ), x) for all Q

The result is a new model that will operate optimally on the novel condition. We assume for now that the query sample, x, has already been identied to belong to the claimed identity j for now. The matching mode of a condition-adaptive system will be slightly more complicated, since there are several models each of which can only operate optimally under a given condition Q. Intuitively, the model with the same condition as the query sample will be chosen or weighed more heavily than the rest. Formally, this weight is called the posterior probability of a condition Q given the observation a set of quality measures, q-assess(x). The posterior probability is written as P (Q|q-assess(x)). The inference using the log-likelihood ratio is computed as follows: Matching:
Q match(x) = log

P (Q|q-assess(x)) p(x|j , Q, t0 ) P (Q|q-assess(x)) p(x|, Q, t0 )

(6) (7)

= log

p (x|j , t0 , q-assess) . p (x|, t0 , q-assess)

The sum over Q implies that this variable is marginalized because it is not observable. This summation in eq. (6) shows that if one were to estimate the density p (x|j , t0 , q-assess) correctly, one will need sufcient number of quality states Q + 1. It is recommended that Q be determined by clustering the quality measures, q-assess [6]. The above formulation of condition-adaptive systems can be found in [6]. An experiment is realized in [5] using samples that are manually labeled, providing the most favorable scenario for adaptation. Nevertheless, the performance

15

gain of 30% is signicant. An age-adaptive system does not have any mechanism to handle variation in biometric sample quality. Almost all reported literature assumes that a query sample is always captured in the controlled condition, Q0 . The goal of an ageadaptive system is to retain a number of time-dependent models, i.e., p (x, Q0 , tu ) at various point in time {tu |u U }. The set U reects the time window in such a way that models that are too old will be eventually abandoned[12, 14]. During inference, a more recent model is given more important consideration than the rest of the models. The two modes of operation are computed as follow: Adaptation: p (x|j, Q0 , tu ) = adapttime ( p(x|j, Q0 , t0 ), x(j, Q, tu ) for u U

Matching: f (t tu ) p(x|j, Q0 , t ) log uU p(x|, Q0 , t ) uU f (t tu )

match(x)

where the function f () is a decreasing non-negative function. The argument for function f cannot be negative since t > tu . Although the age-adaptive model can be easily replaced with a synthesized model using image-based regression the synthesized model does not take into account of the person-specic variation. These include facial surgery, the use of make-up products such as Botulinum Toxin Type A injections that can make a person look a lot younger than they actual are, and other life-style related changes such as diet regime and weight-lifting exercise. In contrast, by adapting the model to the actually observed samples, an age-adaptive system could possibly be a better solution. A major weakness of the age-adaptive system is that it does not consider variation in biometric sample quality due to changing acquisition conditions. This can be remedied by either restoring a query sample to the same enrollment condition or by allowing the system to adapt to different quality as well as age-related conditions. These systems will be described next. A condition- and age-adaptive system contains a set of references that vary in time as well as in conditions. During inference, the more recently adapted models (references) with the matching conditions are given higher weights than the rest of the models during inference. The system operates in two modes, as follow:

16

Adaptation:

p (x|j, Q, tu )

= adapttime ( p(x|j, Q, t), x(j, Q, tu ) for all Q and u U

p (x|, Q, tu ) = adapttime ( p(x|, Q, t), x(j, Q, tu ) for all Q and u U

Matching: match(x) =
u Q log u Q

f (t tu )P (Q|q-assess(x)) p(x|j, Q, t ) f (t tu )P (Q|q-assess(x)) p(x|, Q, t )

where the function f is a non-negative decreasing function so that more important weights are given to more recent samples. In the proposed condition and age adaptive system, the input samples may be labeled and added to the reference set using either co-training or self-training. However, the updated reference set will be managed and inferences will be drawn (for input samples) considering the quality and the notion of time using the proposed framework. However, one of the challenges at the moment is the lack of large-scale, longitudinal and multi-modal biometric database, capturing quality as well as long term temporal variations. The available databases capturing long term ageing effect (for instance MORPH face database 5 ) do not provide adequate number of samples with changes in quality conditions (like pose or illumination changes for face) and vice versa. Thus the evaluation of the proposed framework remains a part of future work. Nevertheless, the proposed framework is a step ahead to the eld of adaptive biometric systems and will provide important incentives, ideas and future directions to the research community.

3.2

New Research Directions

Our ndings related to impostor against non-impostor attack suggests that classication errors in the labeling process can result in sub-optimal performance of an adaptive biometric system. This is because classication error (false acceptance) cause adapting user references with the impostor samples. Thus increasing the vulnerability to template security and undermining the integrity of adaptive biometric systems. To this front, modeling and early stoppage of impostor attack is an important research direction to be pursued. Developed solutions will allow vendors to adopt auto-update procedures in their commercial biometric products. Furthermore, there is a need for incorporating robust labeling scheme in the adaptive biometric systems. This is supported by our ndings related to supervised against semi-supervised methods to adaptation where supervised scheme generalizes better than semi-supervised one. These results indicate that the use of condently classied (la5 http://www.faceaginggroup.com/projects-morph.html

17

beled) input samples (as used by most of the existing automated systems) may not be an efcient strategy for adaptation. Thus emphasizing the need for more robust labeling schemes that are capable of correctly classifying genuine (with substantial variations) as well as impostor samples. Furthermore, our ndings related to supervised against semi-supervised and online against ofine mode of adaptation need direct testing via single experimental framework. This is because of large variance and overlap in the obtained performance gain on comparing adaptive systems based on these mentioned attributes. As a consequence, our results do not allow us to state the conjecture rmly.

Conclusion
This manuscript has worked towards advancing the state-of-the-art related to adaptive biometric systems. This

has been achieved by identifying key attributes related to adaptive biometric systems followed by the comparison and critical analysis. Meta-analysis has been used as a tool to perform comparison, critical analysis and to draw on the limitations of the existing systems. Specically, meta-analysis has been used to gauge the relative impact of single and joint attributes on the generalization behavior (performance gain) of adaptive biometric systems. Our meta-analysis ndings has generally supported our conjecture that biometric sample quality and co-training are important ingredients for an adaptive biometric system. Furthermore, a novel framework termed condition-and-age adaptive system has been proposed and future avenues have been set. Collection of a large scale, longitudinal multibiometric database capturing both quality and age related variations will be useful for future work in this eld.

Acknowledgement
This work has been partially supported by Regione Autonoma della Sardegna ref. no. CRP2-442 through the

Regional Law n.7 for Fundamental and Applied Research, in the context of the funded project Adaptive biometric systems: models, methods and algorithms. Rattani was partly supported by a grant awarded to Regione Autonoma della Sardegna, PO Sardegna FSE 2007-2013, L.R. 7/2007 Promotion of the scientic research and technological innovation in Sardinia. Poh was partially supported by Biometrics Evaluation and Testing (BEAT), an EU FP7 project with grant no. 284989.

References
[1] Ross, A., Nandakumar, K., Jain, A. K.: Handbook of Multibiometrics (Springer Verlag, 2006) [2] Lantinis, A.: A Survey of the effects of aging on biometric identity verication, Int. Journal of Biometrics, 2010, 2, (1) , pp. 34-52 18

[3] Rattani, A., Freni, B., Marcialis, G.L., Roli, F.: Template Update Methods in Adaptive Biometric Systems: A Critical Review. Proc. 3rd Int. Conf. on Biometrics, Sardinia, Alghero, 2009, pp. 847-856 [4] Rattani, A.: Adaptive Biometric System based on Template Update Procedures. PhD thesis, University of Cagliari, Italy, 2010 [5] Poh, N., Kittler, J., Marcel, S., Matrouf, D., Bonastre, J-F.: Model and Score Adaptation for Biometric Systems: Coping With Device Interoperability and Changing Acquisition Conditions. Proc. Int. Conf. on Pattern Recognition, Istambul,Turkey, 2010, pp. 1229-1232 [6] Poh, N., Wong, R., Kittler, J., Roli, F.: Challenges and Research Directions for Adaptive Biometric Recognition Systems. Proc. 3rd Int. Conf. on Biometrics, Sardinia, Alghero, 2009, pp. 753-764 [7] Poh, N., Bourlai, T., Kittler, J.: Quality-based Score Normalisation with Device Qualitative Information for Multimodal Biometric Fusion, IEEE Trans. on Systems, Man, Cybernatics Part B : Systems and Humans, 2010, 40, pp. 539-554 [8] Hedges, L.V., Olkin, I.: Statistical Methods for Meta-Analysis (Academic, New York, 1985) [9] Uludag, U., Ross, A., Jain, A.: Biometric template selection and update: a case study in ngerprints, Pattern Recognition, 2004, 37, (7), pp. 1533-1542 [10] Ozawa, S., Toh, S. L., Abe, S., Shaoning, P., Kasabov, N.: Incremental learning for online face recognition. Proc. 5th IEEE Int. Joint Conf. on Neural Networks, Montreal, Canada, 2005, pp. 3174-3179 [11] Zhu, X.: Semi-supervised learning literature survey, Computer Science TR 1530,2008 [12] Jiang, X., Ser, W.: Online Fingerprint Template Improvement, IEEE Trans. on Patter Analysis and Machine Intelligence, 2002, 24, (8), pp. 1121-1126 [13] Ryu, C., Hakil, K., Jain, A. K.: Template adaptation based ngerprint verication. Proc. 18th Int. Conf. on Pattern Recognition, HongKong, 2006, pp. 582-585 [14] Liu, X., Chen, T., Thornton, S. M.: Eigenspace updating for non-stationary process and its application to face recognition, Pattern Recognition, 2003, 36, (9), pp. 1945-1959 [15] Roli, F., Marcialis, G. L. : Semi-supervised PCA-based face recognition using self training. Proc. joint IAPR Int workshop on S+SSPR06, HongKong, China, 2006, pp. 560-568 [16] Rattani, A., Marcialis, G.L., Roli, F. : Biometric template update using the graph mincut: a case study in face verication. Proc. 6th IEEE Biometric Symposium, Tampa, USA, 2008, pp. 23-28

19

[17] Roli, F., Didaci, L., Marcialis, G.L. : Template co-update in multimodal biometric systems. Proc. IEEE/IAPR Int. Conf. on Biometrics, Seoul, Korea, 2007, pp. 1194-1202 [18] Gayar, N. E., Shaban, S. A., Hamdy, S. : Face recognition with semi-supervised learning and multiple classiers. Proc. 5th WSEAS Int. Conf. on Computational Intelligence, Man-Machine Systems and Cybernetics, Venice, Italy, 2006, pp. 296-301 [19] Rattani, A., Marcialis, G.L., Roli, F.: Capturing large intra-class variations of biometric data by template coupdate. Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops, Anchorage, Alaska, USA, 2008, pp. 1-6 [20] Amayeh, G., Bebis, G., Nicolescu, M.: Improving Hand-Based Verication Through Online Finger Template Update Based on Fused Condences. Proc. 3rd IEEE Int. Conf. on Biometrics: Theory, applications and systems, Washington, DC, USA, 2009, pp. 352-357 [21] Pavani, S.K., Sukno, F.M., Butakoff, C., Planes, X., Frangi, A.F.: A Condence Based Update Rule for Selfupdating Human Face Recognition Systems. Proc. Int. Conf. on Biometrics, Alghero, Italy, 2009, pp. 151-160 [22] Garcia, R., Perales, J.F.: Adaptive Templates in Biometrical Authentication. Proc. 16th Int. Conf. in Central Europe on Computer Graphics, Visualization and Computer Vision, Czech Republic, 2008 [23] Rattani, A., Marcialis, G. L., Roli, F.: Self Adaptive Systems: An Experimental Analysis of the Performance Over Time. Proc. IEEE Workshop on Computational Intelligence in Biometrics and Identity Management CIBIM, Paris, France, 2011, pp. 36-43 [24] Martinez, C., Fuentes, O.:Face recognition using unlabeled data, Computacin y Sistemas, Iberoamerican Journal of Computer Science Research, 2003, 7, (2), pp. 123-129 [25] Rattani, A., Marcialis, G.L., Roli, F.: Temporal analysis of biometric template update procedures in uncontrolled environment. Proc. 16th Int. Conf. on Image analysis and processing, Ravenna, Italy, 2011, pp. 595-604 [26] Freni, B., Marcialis, G.L., Roli, F.:Replacement algorithms for ngerprint template update. Proc. 8th Int. Conf. on Image Analysis and Recognition, Portugal, 2008, pp. 884-893 [27] Giot, R., Dorizzi, B., Rosenberger, C.:Analysis of Template Update Strategies for Keystroke Dynamics. Proc. IEEE Workshop on Computational Intelligence in Biometrics and Identity Management, Paris,France, 2011, pp. 21-28 [28] Marcialis, G. L., Rattani, A., Roli, F.:Biometric template update: an experimental investigation on the relationship between update errors and performance degradation in face verication. Proc. in Joint IAPR Intl Workshop on SSPR+SPR08, Orlando,Florida, USA, 2008, pp. 684-693 20

[29] Zuo, J., Nicolo, F., Schmid, N.:Adaptive Biometric Authentication Using Non-linear Mappings on Quality Measures and Verication Scores. Proc. in IEEE Int. Conf. on Image Processing, Hong Kong, 2010, pp. 40774080 [30] Guerra-Casanova, J., Sanchez-Avila, C., de Santos Sierra, A., del Pozo, G. B.: Score optimization and template updating in a biometric technique for authentication in mobiles based on gestures, Journal of Systems and Software, 2011, 84, (11), pp. 2013-2021 [31] Collett, D.:Modelling Binary Data (Chapman and Hall /CRC Press, 2nd edition, 2002) [32] Cardinaux, F., Sanderson, C., Bengio, S.:User Authentication via Adapted Statistical Models of Face Images, IEEE Trans. on Signal Processing, 2006, 54, (1), pp. 361-373 [33] Reynolds, D. A., Quatieri, T., Dunn, R.:Speaker Verication Using Adapted Gaussian Mixture Models, Digital Signal Processing, 2000, 10, pp. 19-41 [34] Duda, R. O., Hart, P. E., Stork, D. G.:Pattern Classication and Scene Analysis (Digital Signal John Wiley and Sons, New York, 2001)

21