An Introduction
to Multivariate
Calibration and Analysis
Kenneth R. Beebe
Bruce R. Kowalski
Department of Chemistry, BG-10
Laboratory for Chemometrics
University of Washington
Seattle, Wash. 98195
The advent of the laboratory computer
has allowed analytical chemists to col-
lect enormous amounts of data on a
wide variety of problems of interest.
With this ability, however, has come
the realization that more data does not
necessarily mean more information.
One can collect reams of computer out-
put without knowing anything more
about the system under investigation.
Only when the data are interpreted and
put to use do they become valuable to
the chemist and to society in general;
then data become information. For this
reason, data analysis methods that can
0003-2700/87/A359-1007/$01.50/0
© 1987 American Chemical Society
handle large quantities of data are be-
coming a necessity for the practicing
analytical chemist. The number of
books in this area (1-3) is increasing as
the awareness of the need for more so-
phisticated techniques grows. Howev-
er, widespread use of the full set of ana-
lytical tools will be realized only when
the analyst is familiar with the general
goals and advantages of the methods.
This REPORT serves as an introduc-
tion to the area of multivariate anal-
yses known as multivariate calibration.
Our goal is to give the chemist insight
into the workings of a collection of sta-
tistical techniques and to enable him or
her to judge the appropriateness of the
techniques for an application of inter-
est. The list of methods described is by
no means a complete representation of
those that can be applied to multivari-
ate data. Also, we do not compare the
results obtained using one method over
another; instead, we describe the meth-
ods in the line of a tutorial. For this
reason, the examples chosen illustrate
how the methods work rather than
compare among them. The three multi-
variate methods that will be discussed
are multiple linear regression (MLR),
principal component regression (PCR),
and partial least squares (PLS). MLR,
an extension of ordinary least
squares, is the easiest to understand
and also the most commonly used;
PCR and PLS have yet to achieve
widespread acceptance in chemistry.
Calibration and prediction
Multivariate statistics is a collection of
powerful mathematical tools that can
be applied to chemical analysis when
more than one measurement is ac-
quired for each sample. For the sake of
consistency, near-infrared (near-IR)
reflectance analysis will be used as an
example of such an analytical method
throughout this REPORT. However, the
techniques described can be used in
ANALYTICAL CHEMISTRY, VOL. 59, NO. 17, SEPTEMBER 1, 1987 · 1007 A
REPORT
any situation where multiple measure-
ments are acquired.
One example of an analytical prob-
lem solved by near-IR reflectance anal-
ysis is the estimation of the protein and
moisture content of wheat samples. As
with many analytical methods, this
procedure consists of two phases: cali-
bration and prediction. The chemist
begins by constructing a data matrix
(R) from the instrument responses at
selected wavelengths for a given set of
calibration samples. In the case of
near-IR reflectance analysis, the R ma-
trix can be constructed from the loga-
rithmic reflectances (log(l/ref)) or
from some other combination of reflec-
tances obtained at the various wave-
lengths (4). A matrix of concentration
values (C) is then formed using inde-
pendent or referee methods such as
Kjeldahl nitrogen analysis for protein
determination and oven-drying for the
determination of moisture content.
The goal of the calibration phase is to
produce a model that relates the near-
IR spectra to the values obtained by
the independent or referee methods.
Figure 1 illustrates the resulting ma-
trices for this example. R is a 3 X 8
matrix with three rows of near-IR spec-
tra from the analysis of three samples
and eight columns corresponding to
the eight near-IR wavelengths chosen
for the analysis. (Throughout this RE-
PORT, these columns will be referred to
as the original variables for R.) The C
matrix is 3 X 2 dimensional for this
example. Again, the three samples oc-
cupy the three rows and the two col-
umns represent the protein and mois-
ture content of the samples as deter-
mined by the referee methods.
According to this notation, the terms
Cji and c^ represent the protein content

ΙχJ
Γ *
2 12 1 1 5 3 1
12 2 3 7 4 71
12 1 3 6 2 6 2
3 4
=
ΙχΚ 2 7
8 6
Figure 1. Configuration of R and C
matrices.
and moisture content, respectively, for
the ith wheat sample found in the ith
row of C.
The next step in the calibration
phase is the foundation of the entire
analysis. The analyst must choose an
appropriate mathematical method
that will best reproduce C given the R
matrix. How the analyst defines best
will ultimately determine the method
that is chosen.
MLR assumes that the best ap­
proach to estimating C from R is to
find the linear combination of the vari­
ables in R that minimizes the errors in
reproducing C. It proposes a relation­
ship between C and R such that C =
RS + E, where S is a matrix of regres­
sion coefficients and Ε is a matrix of
errors associated with the MLR model.
S is estimated by linear regression, and
the term
Err=|;^(c i k -c i k )2=f;5\
k= 1i=1 k=1i=1
is minimized. In this expression, Cjk is
the actual concentration element in the
ith row and kth column of C, C;k is the
MLR estimate of the same element us­
ing S, and «ii; is the corresponding ele­
ment of the matrix E.
This approach appears to be reason­
able, and in fact it is the best method
when the analyst is dealing with well-
behaved systems (linear responses, no
interfering signals, no analyte-analyte
interactions, low noise, and no collin-
1008 A · ANALYTICAL CHEMISTRY, VOL. 59, NO. 17, SEPTEMBER 1, 1987
earities). However, problems can be en­ sion of the same C as in the first exam­
countered with the use of real data. A ple onto this new matrix R 2 is given in
model chosen solely according to this S 2 . For this model, Err = 0.07, where
criterion attempts to use all of the vari­ the smaller value implies that this sec­
ance in the R matrix, including any ir­ ond model is more effective at model­
relevant information, to model C. ing C. To further evaluate these re­
When the resulting model is then ap­ sults, note that when R is multiplied by
plied to a new sample, the model will S, the jth column of R is always multi­
assume that the correlation found be­ plied by the jth row of S. The impor­
tween the calibration R and C matrices tance of variable j to the model can
also exists in that sample. Because the therefore be determined by examining
Ί model was built using irrelevant infor­ the jth row of S. The nonzero entries in
mation in the R matrix, this assump­ the fourth row of S 2 reveal that a vari­
tion will not be true. Unfortunately, able consisting of random numbers
even noise has a very high probability (column number four of R 2 ) was chosen
of being used to build the model. The as a significant contributor to the cali­
following example illustrates this bration model. In an ideal analysis, this
point. Consider the matrices random variable would have been ig­
nored in the model-building phase of
75 152 102 the analysis. Furthermore, the inclu­
63 132 82 sion of this column in R 2 has changed
R
96 218 176 the estimated model coefficients so
that they no longer represent the true
69 157 124_
model. The upper 3 X 3 portion of S 2 ,
which represents the model for the first
2 7 5
three variables in R 2 , is not equal to S,
4 3 3 which represents the true model for the
9 12 3 same variables. The addition of a col­
6 8 2 umn of random numbers has resulted
in a model that appears to be better, in
-0.71 0.55 0.48 that it is more effective at reproducing
0.42 - 0 . 4 1 -0.24 C, and yet it does not describe the true
s= relationship. This is because MLR uses
-0.08 0.28 0.05
all of the matrix R to build the model,
MLR was used to determine the S regardless of whether or not it is rele­
matrix, as described earlier. A standard vant in describing the true model.
measure of the effectiveness of the Therefore, an erroneous model can be
model is the value of Err as presented derived and subsequently used to pre­
above. For this example Err = 0.49, and dict the characteristics (e.g., protein
it will be assumed that the matrix S is content) of new samples. Thus MLR
an accurate estimate of the true model. alone often will generate misleading
The coefficients in S, therefore, will models with subsequent errors in pre­
closely approximate the true relation­ diction.
ship between the variables in R and C. The remainder of this R E P O R T in­
To illustrate how a calibration meth­ vestigates the advantages of PCR and
od can be inappropriate, a column of PLS over MLR. To understand these
random numbers ranging from zero to advantages, it is necessary to under­
100 was added to the R matrix. The stand how the methods work. Graphi­
addition of this column is analogous to cal representations of many of the con­
the inclusion of a wavelength in near- cepts have been included so that the
IR analysis that has no useful informa­ reader not well versed in linear algebra
tion for describing the protein or mois­ will be able to follow the discussions.
ture content of the samples. The result­
ing matrix R 2 and MLR model are Notation
75 152 102 9l" Standard linear algebra notation will
be employed throughout this article.
63 132 82 36
i k 2 R2 - Bold upper case letters refer to matri­
96 218 176 74 ces; plain upper case letters are used for
_69 157 124 51. their row and column dimensions. For
(1)
example, the letter R refers to the ma­
"2 7 5~ trix of responses and is I X J dimen­
4 3 3 sional. Bold lower case letters signify
C = column vectors such as r 2 , which repre­
9 12 3
sents the second column of matrix R.
_6 8 2_ By this convention r 2 T represents the
second column of R written as a row
0.71 0.18 0.42 vector. Plain lower case letters are sca-
-0.42 - 0 . 1 9 -0.20 lars, representing single elements of
>2 -
0.24 0.20 0.03 vectors (a;), matrices (ry), or other con­
.-0.12 0.03 0.01 stants such as regression coefficients
(b). The transpose of a matrix or vector
The model resulting from the regres­ is represented by a superscript T. (See
References 5 and 6 for books on linear
algebra.)
Matrix representations. Matrices
of low dimensionality (i.e., the num­
bers of rows and columns are small in­
teger values) can be represented graph­
ically. Consider the following 2 X 3 ma­
trix
—ESS
R can be represented graphically as R
in row space or as R in column space.
Row space is the space formed with the
rows of R as the axes. It is, therefore,
two-dimensional. Because there are
three columns, graphing R in row space
consists of plotting three points (each
point corresponding to a column) in
two-dimensional space. The coordi­
nates of the points representing the
columns in each dimension are simply
the matrix entries in the corresponding
row. The resulting plot for this matrix

Row2

(4,6)
6-
/ ~*— Column 3

/ , Column 1

/ S s Column 2
(2,1) /

ι
1
ι
1
ι
1 1 1 1
' ι Sim
ι I I I I -7-6-
-7-6-5- t-3-2-1 1
2 3 4 5 6 7 Column 1
Row 1

-6-
Column 2

Figure 2. The matrix R in row space. Figure 3. The matrix R in column space.

1.2 equiv PPh 3

CH 2 C1 2 , 15 min. 25°C

H H
Dithiatopazine: The First Stable 1,2-Ditbietane
K. C. Nicolaou, et. ai, J. Amer. Chem. Soc. 1987, 109, 3801

Two Easy-To-Use Software Tools

Designed By Chemists For Chemists

ChemDrawL & Chem3D

Isn't it time you discovered a more effective way to convey your ideas?
ChemDrawr the desktop publishing software for chemists, and Chem3D™
the molecular modeling package, will do just that-clearly and effectively!
With ChemDraw, you can easily create top-quality drawings of your (
chemical structures, using chemical drawing tools: bonds, arrows, rings, text,
circles, and more. Transfer drawings between ChemDraw and other software to
combine pictures and text for scientific articles. C/
CherrûD provides a wide array of features for manipulating 3-dimensional
molecular models. Define your own atom types and substructures. Create molec-
ular "movies". Import and export structures to ChemDraw, MM2, and other
programs, or build models with CherrûD's powerful building tools.
ChemDraw and CherrûD are designed for Apple " Macintosh™ computers
(including the new Macintosh II), and output to any compatible printer, including
high-quality laser printers and phototypesetters. ChemDraw structures have For more information, please call or write:
appeared in prestigious journals such as the Journal of the American Chemical Cambridge Scientific Computing inc.
Society and Tetrahedron Letters. P.O. Box 2123, Cambridge, HA 02238
617-491-6862
>m3l) are truderm

CIRCLE 28 ON READER SERVICE CARD

ANALYTICAL CHEMISTRY, VOL. 59, NO. 17, SEPTEMBER 1, 1987 · 1009 A

is shown in Figure 2.
Analogous to row space, the column
space has two points and is three-di-
mensional. Figure 3 is a two-dimen-
sional representation of this three-di-
mensional space. This example can be
used to understand higher dimensional
matrices, so that a 10 X 12 matrix
would have 12 points in 10-dimensional
row space and 10 points in 12-dimen-
sional column space. Examples in high-
er dimensions than three can be under-
stood if one refers back to two or three
dimensions and considers situations of
higher dimensionality as expansions of
these simpler cases.
Projections. Another concept that
is important to understand is that of
projecting either a point or a vector
onto a vector or plane. Each of these
can be viewed as being the perpendicu-
lar shadow of one object onto another.
Figure 4 illustrates the result of pro-
jecting the vector a onto a plane in
three-dimensional space.
Factors. The last concept that is
very basic to understanding both PCR
and PLS is that of a factor. This is
because both PCR and PLS are factor-
based modeling procedures. For our
purposes, a factor is defined as any lin-
ear combination of the original vari-
ables in R or C. It can be shown that
given J factors for an I X J matrix R,
one can also represent the variables in
R as a linear combination of these same
Figure 4. The projection of a vector onto a plane.
1010 A · ANALYTICAL CHEMISTRY, VOL. 59, NO. 17, SEPTEMBER 1, 1987
J factors. It is important to determine
factors for greater freedom in repre-
senting the useful information in the R
matrix. For example, the information
in an I X J matrix R can be expressed as
an I X J matrix R', where the columns
of R' are linear combinations of the
original columns in R. The advantage
of this is that if a particular column in
R is not useful (as in the MLR example
cited earlier), one can attempt to find a
set of factors that gives that column
small weights when forming R'.
This is only one of the advantages of
a factor-based calibration method over
the methods that simply use the raw
data. The other possible advantages
can be understood by examining one of
the factor methods mentioned in the
introduction: PCR (3, 7,8). The follow-
ing discussion concerns the prelimi-
nary step of PCR: principal component
analysis (PCA). An intuitive feel for
how PCA works can be gained by con-
sidering the result of PCA performed
on the 5 X 2 matrix R shown below.
2 4
1 2
R = 0 0
1 -2
2 In real analyses, the columns of R are
-4_
In column space, the matrix is five
points in two dimensions as shown in
Figure 5.
Column 2
4.
3-
2-
7 Eigenvector #1
1 •
Column 1
I '
-2 -1 / 1 2
/
7-2-
f -3-
-4-
Figure 5. The matrix R plotted in col-
umn space with the first eigenvector.
often mean-centered and scaled. Mean
centering simply involves subtracting
the average of a column from each of
the entries in that column. Scaling,
which gives equal weights to each vari-
able, involves dividing each entry by
the variance of the column. PCA is then
performed on the covariance matrix
R T R formed from the mean-centered
and scaled matrix. The first eigenvec-
tor corresponding to the largest eigen-
value is, by definition, the direction in
the space defined by the columns of R
that describes the maximum amount of
variation or spread in the samples. Fig-
ure 5 shows the data and the direction
of the first eigenvector where the space
defined by R is a plane.
In this example, all of the variation
in the data can be described using one
eigenvector. The samples all fall on a
line in column space, and therefore all
of the variation lies in one direction.
When all of the variation in the sam-
ples cannot be accounted for using only
one eigenvector, a second eigenvector
can be found that is perpendicular or
orthogonal to the first and describes
the maximum amount of residual vari-
ation (not described by the first eigen-
vector) in the data set. Figure 6 is the
plot of a 30 X 2 matrix and the associat-
ed first two eigenvectors. The direction
of the first eigenvector describes the
maximum amount of variation or
spread in the data. In this particular
case, the samples in column space hap-
pen to fall within a two-dimensional
ellipse, and the first eigenvector corre-
sponds to the major axis of the ellipse.
 To illustrate how eigenvectors can
define factors for a matrix R, the re­
sults of the eigenvector analysis shown
in Figure 5 are presented. The eigen­
vector in Figure 5 is equal to the vector
v T = [0.447, 0.894]. In vector notation,
the first factor or principal component
can be shown to equal a linear combi­
nation of the original variables as fol­
lows:
Rv = u (2)
where R is the original matrix of re­
sponses, ν is the first eigenvector of
R T R, and u is the so-called score vec­
tor, which is the projection of R onto
the first eigenvector. The vector u is a
factor for the columns of R because it
can be written as a linear combination
of the columns. Likewise, when J eigen­
vectors are calculated and used to form
an I X J matrix U of scores, the original
variables in R can be expressed as the
linear combination of the scores:
RVV T = UV T (3)
T
R = UV (VV ) T _1
(4)
where V T (VV T ) _ 1 , obtained from lin­
ear algebra, is called the generalized
inverse (GI) of V. In PCA notation, the
elements in the GI of V are called the
principal component "loadings."
These loadings range from —1 to +1
and are the cosines of the angles be­
tween the eigenvector and the variable
axes. High loadings correspond to high
correlations (large coefficients) where
the angle between the eigenvector and
a variable is very small [cos(0°) = 1].
Small loadings correspond to low corre­
lations (small coefficients) where the
eigenvector is orthogonal or nearly or­
Figure 6. A 30 X 2 matrix plotted in column space with
the first two eigenvectors.
1012 A · ANALYTICAL CHEMISTRY, VOL. 59, NO. 17, SEPTEMBER 1, 1987
thogonal to the variable [cos(90°) = 0].
In the example given in Figure 6, the
second eigenvector corresponds to the
minor axis of the ellipse. Because the
eigenvectors lie in the same space as
the original variables, they can be used
as a new set of axes for the matrix R.
Each point or sample has a new set of
coordinates from their projections onto
the new axes, and these axes may be
more useful for describing the variation
or spread of the samples. The new axes
also can be viewed as a rotation of the
original axes. In Figure 6 the transfor­
mation from the old to the new axes can
be achieved by rotating the original
axes by approximately 45 degrees.
In this example, the object (ellipse)
formed by the sample points in column
space was two-dimensional. In more
complicated situations the object can
be of higher dimensionality than two,
and the process of determining eigen­
vectors continues until all of the varia­
tion in the samples is explained. The
dimensionality of the space containing
the samples equals the number of col­
umns in R; thus the maximum possible
dimensionality of the samples equals
the number of columns in R. The col­
lection of points, however, often can be
presented by fewer dimensions. This
occurs when there is collinearity be­
tween the variables in the matrix under
investigation. The presence of collin­
earity means that some of the variables
in the original matrix are correlated to
the other variables and therefore con­
tain redundant information. In this sit­
uation, PCA can be used to describe the
variation in R using fewer dimensions
than the number of columns in R. The
eigenvectors can be used to determine a
subspace in the space defined by the
Figure 7. The plane formed by the two columns of
matrix R plotted in row space.
columns of R that includes all of the
original variation. One can visualize
this by considering the column space
spanned by R as a room and the sam­
ples lying on a table in the room. The
column space is three-dimensional, but
only two dimensions are necessary to
describe the position of each of the
sample points relative to the others.
The two eigenvectors necessary to de­
scribe the samples would define a plane
that includes the top of the table. This
characteristic of the factor-based
methods is exploited to yield more in­
formative models, as is discussed in the
following sections.
Multiple linear regression
The first method considered, MLR,
does not attempt to model underlying
structure or factors that may exist in R
or C. The goal of MLR is to find the
linear combination of the variables η
such that the model's estimates of the
values of the c's in the calibration set
given by
j
j=i
are as close to the known values of C as
possible. As stated earlier, the criterion
of closeness for MLR is defined as min­
imizing the sum of the squares of the
deviations of the predicted values from
the true values (see Equation 1). In sta­
tistical language, MLR is the regres­
sion of the columns of C onto the space
defined by the columns of R. The
mathematical procedure for this can be
found in most books on multivariate
analysis (3).
Graphically, MLR finds the projec­
tion of the columns of the matrix C

Figure 8. Geometric representation of the regression of c onto R.
onto the variables or columns of R in its
row space. To illustrate this, consider
the following matrix and vector
"39 29" "30"
R = 18 15 c = 20
.11 6_ -10,
where R might represent a near-IR
data set with two columns of absor-
bance measurements at two wave­
lengths and three rows corresponding
to three spectra from different samples
of wheat, and vector c represents the
protein content of the three wheat sam­
ples from Kjeldahl nitrogen analysis.
Figure 7 is a graph of the column
vectors r t and r 2 plotted in row space.
The two columns of R define a sub-
space (a tilted plane) within the row
space, and regressing c onto R is equiv­
alent to finding the projection of c onto
the subspace defined by R. To perform
MLR, one could plot the c vector in
this same row space and project c onto
the plane formed by Γ] and r 2 . Figure 8
is an expanded view of the projection of
c onto the plane, where proj c is the
estimate of c from matrix R. The re­
sulting regression coefficients for the
above example are given as s
0.20'
0.85
From s an estimate of c can be obtained
as c = proj c = Siri + β2Γ2· Or in matrix
notation,
"39 29" "32.5"
"0.20"
proj c 18 15 — 16.4
_0.85_
.11 6. - V.3_
Another point illustrated by this ex­
ample is that the ability to estimate c
depends on how close c lies to the sub-
space defined by R. In this example, c
is near that subspace, and R's estimate
of c, proj c, is the vector in the plane
defined by ri and r 2 that lies closest to
c. The model error can be described
geometrically as the length of the seg­
ment joining the heads of vector c and
proj c.
To predict the concentrations of an
unknown sample with a given response
1014 A · ANALYTICAL CHEMISTRY, VOL. 59, NO. 17, SEPTEMBER 1, 1987
vector (e.g., r u n k T = [10 5]), one multi­
plies the vector r un k by s as follows:
0.20
[10 5] = 6.25 = c pred
0.85
where cpred is the predicted concentra­
tion in the new sample.
The method of MLR is indeed an
adequate procedure in ideal situations.
However, it is based on the problematic
step of inverting a matrix, and it can
incorporate significant amounts of ir­
relevant variance (information) into
the model.
Principal component regression
This model-building procedure has
two steps. The first step is the determi­
nation of the eigenvectors or factors for
a matrix R, as was described in the
section on factors. This information is
used to convert R into a score matrix U
by projecting the R matrix onto the
space defined by the eigenvectors (see
Equation 2). Until now, no attempt has
been made to describe the rationale be­
hind the use of eigenvectors as the new
axes. We will show that there is a logi­
cal interpretation of the significance of
the eigenvectors. If the analyst is to
redefine the variables using a smaller
number of factors, one reasonable ap­
proach is to find the factors that best
describe the original variables. The
first principal component is simply the
linear combination of the original vari­
ables that points in a direction that is
most correlated to all of the columns in
row space. In mathematical terms, it is
the linear combination that has the
smallest squared errors when used to
estimate the original variables. It is
also the direction in column space that
best describes the variation in the sam­
ples. To illustrate these characteristics,
consider the following matrix R:
R -3-- V
Figure 9 is a representation of the col­
umns of R and the first principal com­
ponent plotted in row space. No other
vector in this space will yield a smaller
sum of errors when used to estimate the
two columns of R. In the general case of
an I X J matrix R, the first principal
component will be a vector in I-dimen-
sional space that has the smallest sum
of squared errors when used to esti­
mate the J column vectors in R.
The difference between MLR and
PCR lies in the re-expression of R as a
score matrix U. This matrix results
from projecting R onto the eigenvec­
tors V, as RV = U. The score matrix U
is composed of the original data points
in a new coordinate system described
by the eigenvectors. The second step of
PCR uses MLR to regress the C matrix
onto the score matrix as U S = C. The
eigenvectors, V, and the matrix of re­
gression coefficients, S, together form
the PCR model.
The prediction step uses the V and S
matrices derived in the calibration set
as follows. Multiply the new sample re­
sponse vector or matrix R un k by V to
obtain Uunk> then multiply Uunk by S to
yield Cunk, where Cunk contains the es­
timates of the concentrations of the an-
alytes in the unknown samples.
If for an I X J matrix R all of the J
eigenvectors are used to form V, PCR
will yield results identical to MLR. The
advantages of PCR are based on the
concentration of the information in R
into fewer factors than J. Because the
eigenvectors optimally describe the
maximum variation in the original
variables, they can be used to deter­
mine the rotation that best describes
the information in the matrix R using
the minimum number of dimensions.
The first factor is the most descriptive;
each successive factor describes less of
the information contained in R. Delet­
ing some of the latter factors that con­
tain mostly noise therefore does not
significantly reduce the amount of use­
ful information present. This reduction
of dimensionality is often of utmost im­
portance for practical and mathemati­
cal stability. Another important char­
acteristic of the matrix U is that if the
eigenvectors are calculated from a sym­
metric matrix such as the correlation or
Row 2
3--
-4-3-2-1 1 2 3 4
H—I—I—r- •^ ι I—|—|— Row 1
' V ^ " * * * " Column 1
- \ \ ^ Principal
component #1
Column 2
Figure 9. The matrix R in row space
with the first principal component.

Figure 10. Geometric representation of the workings of PLS.
covariance matrix, each of the columns
in U will be mutually orthogonal.
One way of viewing this procedure is
in terms of the space defined by R.
MLR uses all of the space described by
the columns of R, whereas PCA deter­
mines a subspace by possibly ignoring
some of the eigenvectors. When one of
the J columns in R is a linear combina­
tion of the other J-l columns, R can be
represented as a matrix U with J-l col­
umns without a loss of information.
This new matrix U defines a subspace
of R. The first example in this article
illustrated how a noisy variable can ad­
versely affect the true model relating R
to C. It would be advantageous to ig­
nore this variable by finding a portion
of the space (a subspace) defined by R
that would be more effective at model­
ing C.
The advantages of PCR make it the
obvious choice over MLR in many situ­
ations, but there is room for improve­
ment. At the crucial factor-building
step, PCR ignores the information con­
tained in the C matrix. The C matrix is
not used until the second step of the
procedure, when the factors in R have
already been determined. The correla­
tion between these factors and C is not
taken into account when the factor
model is built. The rotation that is de­
termined has very good descriptive
qualities concerning the R matrix, but
there is not much reason to assume it
will optimize the modeling of C. The
ultimate goal is to model the informa­
tion in the C matrix, hence it seems
reasonable that C should be used to aid
in determining the factors for R.
Partial least squares
The method of PLS {9,10) is a model­
ing procedure that simultaneously esti­
mates underlying factors in both R and
C. These factors are used to define a
subspace in R that is better able to
ANALYTICAL CHEMISTRY, VOL. 59, NO. 17, SEPTEMBER 1, 1987 · 1015 A
model C. With PCR, the rotation de­
fined by the eigenvectors was used to
find a subspace in R that subsequently
was used to model C. The approach
taken by PLS is very similar to that of
PCA, except that factors are chosen to
describe the variables in C as well as in
R. This is accomplished by using the
columns of the C matrix to estimate the
factors for R. At the same time, the
columns of R are used to estimate the
factors for C. The resulting models are
R = TP + Ε (5)
and
C = UQ + F (6)
where the elements of Τ and U are
called the scores of R and C, respec­
tively, and the elements of Ρ and Q are
called the loadings. The matrices Ε and
F are the errors associated with model­
ing R and C with the PLS model.
The Τ factors are not optimal for es­
timating the columns of R as was the
case with PCA, but are rotated so as to
simultaneously describe the C matrix.
In the ideal situation, the sources of
variation in R are exactly equal to the
sources of variation in C, and the fac­
tors for R and C are identical. In real
applications, R varies in ways not cor­
related to the variation in C, and there­
fore t ^ u . However, when both matri­
ces are used to estimate factors, the
factors for the R and C matrices have
the following relationship
u=bt + ( (7)
where the b is termed the inner rela­
tionship between u and t and is used to
calculate subsequent factors if the in­
trinsic dimensionality of R is greater
than one.
Geometrically, this states that the
vectors u and t are approximately
equal except for their lengths. To get a
feel for where these factors lie in rela­
tionship to the original R and C matri­
ces, one can try to picture both of these
matrices and their factors plotted in
row space. R will have J vectors, one for
each column, whereas C will have Κ
vectors corresponding to its columns.
The factor t will lie close to the J vec­
tors for R and will be a good estimator
for these vectors. Likewise, u will lie
close to the Κ vectors for C. If there is a
good underlying model that relates R
to C, this will be evident by the similar­
ity of the vectors u and t in this space,
and e in Equation 7 will have a small
value. If the first principal component
were plotted in this same space, one
would find that the t vector would dif­
fer from it in that t would be rotated
toward the C vectors.
In a similar manner, one can attempt
to view PLS working in the column
spaces of R and C. Figure 10 represents
a hypothetical example of a 6 X 3 re­
sponse matrix and corresponding 6 X 3
concentration matrix plotted in their
 factor model that simultaneously de­
scribes R and C.
For an example of the PLS algo­
rithm, the same R and C matrices used
in the MLR example will be employed.
The data were mean-centered before
the analysis, and the following two fac­
tors were determined.
20.5" 10"
-4.7 u = 0 a score vector (tunk) is calculated. Using
t =
.-15.8. .-ίο.
Plotting t versus u (Figure 11) reveals
the nearly linear relationship that ex­
ists between them. The calculated val­
ue of b as defined in Equation 7 is 0.53.
If the intrinsic dimensionality of R is
greater than one, that is, more than one
Figure 11. The R matrix factor (t) plot­
ted versus the C matrix factor (u).
factor is necessary to describe the vari­
ation, additional factors are deter­
mined. The relationship given in Equa­
tion 7 is then used to describe the mod­
column spaces. The directions of the
el for C. Instead of using C = UQ + F,
first PLS factors (t and u) would ex­
one can substitute the equality stated
plain much of the variation of the sam­
in Equation 7 to yield C = bTQ + G. In
ples in the respective spaces. If the PLS
other words, one can estimate the
model is valid, plotting t versus u will
scores of C from the scores of R.
also yield a linear relationship. In fact,
PLS will compromise the ability of the An analysis of data using PLS can
factors to describe the samples in the then be summarized as the determina­
individual spaces (Figure 10, top) to tion of factors in R and C using all of
increase the correlation of t to u (Fig­ the information available. The final
ure 10, bottom). It is this compromise model consists of score matrices Τ and
that allows for the determination of a U (called scores for R and C) that are
APPLIED MATHEMATICIAN
Boehringer Mannheim is a rapidly growing leader in the field of med­
ical diagnostics and medical research. This leadership has been at­
tained through emphasis on quality and team effort and Boehringer
Mannheim employees are responsible for the growth and leadership.
Due to this growth, we are currently seeking a qualified individual for
the following opportunity in our INDIANAPOLIS corporate head­
quarters.
Will implement packaged and tailored algorithms for support of re­
search and commercial systems. Requires B.S. degree in Computer
Science, Chemistry, Engineering, Physics or Mathematics with 5
years experience to include convolutions techniques, database work,
curve fitting and statistical techniques.
At Boehringer Mannheim we offer an exciting and challenging work
environment as well as excellent compensation and benefits pro­
grams. If you meet the above requirements and would like to join an
organization that is a leader in its field, please send your resume with
salary history for confidential consideration. To inquire about other
opportunities not listed above, please call our job line at (317)
845-7035.
Dept. MZAC
Human Resources
Boehringer Mannheim
P.O. Box 50100
Indianapolis, IN 46250-0100
BOEHRINGER
MANNHEIM
Λ'-Ι.Ιίι.ΜΤΙ
DIAGNOSTICS
An Equal Opportunity Employer
1016 A · ANALYTICAL CHEMISTRY, VOL. 59, NO. 17, SEPTEMBER 1, 1987
linearly related with a coefficient, b;,
describing the relationship for each of
the L factors.
To perform prediction on an un­
known sample, the model used to de­
rive the scores of R and C and the rela­
tionship bf are used in the following
manner. The vector of responses runit
for the unknown sample is taken
through the calibration model (P), and
b; and Equation 7, the score tun^ yields
an estimate of the scores for the pre­
dicted matrix of concentrations (uunk)·
The vector uunk is then transformed
into concentration estimates using the
calibration model for the C matrix (Q).
For more information concerning the
mechanics of calibration and predic­
tion using the PLS algorithm, see Ref­
erence 11. For an even more rigorous
mathematical treatment of PLS in the
context of singular value decomposi­
tion and the power method, see Refer­
ence 12. The main point of this section
is that PLS estimates factors for R and
C using all of the information available.
In PLS these factors u and t are called
latent variables and are similar to the
principal components in PCR.
Conclusion
It is instructive to summarize briefly
the mechanics of each of the methods
discussed in this REPORT. MLR mod­
els C from R using a least-squares cri­
terion. All of R is available to model C,
and the method is dependent on the
inversion of a matrix. PCR first models
R by a score matrix U and then uses
MLR to estimate the relationship be­
tween U and C. The possibility of de­
leting factors and thus reducing dimen­
sionality was discussed as its major ad­
vantage. Finally, PLS builds factors for
R and C using both matrices. The ad­
vantage of PLS over PCR is that it in­
corporates more information in the
model-building phase.
The advantages gained by employing
the factor-based methods are realized
because of the characteristics of the
factors. The first advantage of factor
models comes from a computational
limitation of MLR. MLR cannot be
used in cases where the columns or
measurement variables in R are linear
combinations of each other. This is be­
cause MLR is based on the inversion of
the matrix R T R, which is not possible if
the matrix R has columns that are ex­
act linear combinations of the remain­
ing columns. Furthermore, the inver­
sion process is unstable when some col­
umns are nearly linear combinations of
others. A model built on such an in­
verse will give large errors in concentra­
tion estimates when noise is present in
the R matrix. Much work has been
done in statistics in the area of variable
selection to eliminate or adjust for col-
linearities among the variables of R
(e.g., stepwise regression, Mallow's sta­
tistics [3]). These nonfactor-based
methods are disadvantageous in that
they are based on the ultimate deletion
of variables, and in doing so they may
eliminate useful information. Factor-
based models can eliminate collineari-
ties without removing useful informa­
tion.
The second main advantage of fac­
tor-based models is the possibility of
removing noise from the data matrix.
Again, using PCA as an example one
can demonstrate that the inclusion of
all of the eigenvectors can actually de­
crease the predictive ability of the
model. To demonstrate when this is oc­
curring, the analyst can use cross vali­
dation (73) and build a factor-based
calibration model using a set of sam­
ples termed the training set. This cali­
bration model is then applied to the R
matrix of an independent set of sam­
ples (termed the test set), and an esti­
mate of the C matrix for the test set is
obtained. (For the test set, the C ma­
trix is also known.) The sum of squared
deviations of the predicted values for
the Cij's and the true c.j's are calculated
for the test set. This value is termed the
PRESS value (predictive residual error
sum of squares). Factors are then add­
ed one at a time, and the PRESS value
is calculated. What usually occurs is
that the PRESS value levels off or
reaches a minimum before all of the
factors have been included. Inclusion
of additional factors beyond this point
actually results in an increased PRESS
value, which means poorer prediction.
This is because the random variation or
noise in the R matrix is used to fit the C
matrix (this is termed overfitting). By
not including the factors that describe
mainly noise, it is possible to increase
the predictive ability of the model.
Both PCR and PLS offer the capability
of such noise reduction.
Data contain relevant information as
well as noise and irrelevant informa­
tion. Chemometricians and statisti­
cians are working to develop the best
possible tools that can extract the full
complement of useful information
from all of the data chemists acquire in
their investigations.
References
(1) Sharaf, Μ. Α.; Illman, D. L.; Kowalski,
B. R. Chemometrics; Wiley: New York,
1986.
(2) Malinowski, E. R.; Howery, D. G. Fac­
tor Analysis in Chemistry; Wiley: New
York, 1980.
(3) Draper, N.; Smith, H. Applied Regres­
sion Analysis, 2nd éd.; Wiley: New York,
1981.
(4) Wetzel, D. L. Anal. Chem. 1983, 55,
1165A.
(5) Strang, G. Linear Algebra and Its Ap-
plications; Academic Press: New York,
1980.
(6) Wesson, J. R. Lessons in Linear Alge-
bra; Charles E. Merrill Publishing Co.:
Columbus, 1974.
(7) Rummel, R. J. Applied Factor Analy-
sis; Northwestern University Press: Ev-
anston, 1970.
(8) Jackson, J. E. Journal of Quality Tech-
nology, 1980,12, 201-13.
(9) Wold, H. In Research Papers in Statis-
tics; David, F., Ed.; Wiley: New York,
1966, pp. 411-44.
NEW
(10) Wold, H. In Systems Under Indirect
Observation, vol. 2; Joreskog, H.; Wold,
H., Eds.; North-Holland: Amsterdam,
1982, pp. 1-54.
(11) Geladi, P.; Kowalski, B. R. Anal.
Chim. Acta, 1986,185,1-17.
(12) Lorber, Α.; Wangen, L.; Kowalski,
B. R. Journal of Chemometrics, 1986, /,
1.
(13) Wold, S. Technometrics, 1978,20,397.
GROTON
TECHNOLOGY
introduces
HPLClike
never before.
VALUE
Best value today in a
D.A.D. system.
PERFORMANCE
Bruce R. Kowalski received his Ph.D. Unparalleled resolu­
in chemistry from the University of
Washington in 1969. In 1972, he joined
tion. Incomparable
the chemistry faculty at Colorado State sensitivity.
University and then returned to the
University of Washington, where he is
a member of the chemistry faculty and
VERSATILITY
codirector of the Center for Process True IBM
Analytical Chemistry. His research in­ compatibility-
terests in analytical chemometrics in­ options for your
clude developing new methods of mul­ application.
tivariate analysis for the resolution
and calibration steps in analytical
instrumentation, extending various FUNCTIONALITY
chromatography-spectrometry meth­ Full function software
ods, and using chemical sensors in pro­ overlays, ratios and
cess analysis and control. more.
Kenneth R. Beebe is a research assis­
tant at the University of Washington. For information call
He received his B.S. in chemistry in (617)647-9400.
1981, and his M.S. in 1985 from the
University of Washington. Currently
GROTON
he is enrolled in the Ph.D. program in
the analytical division of the chemis­ TECHNOLOGY
try department. His research interest
is in the application of various chemo­
INCORPORATED
Waltham, MA 02154
metrics techniques to optimize multi­
variate calibration procedures.
CIRCLE 61 ON READER SERVICE CARD
ANALYTICAL CHEMISTRY, VOL. 59, NO. 17, SEPTEMBER 1, 1987 · 1017 A