A Flexible New Technique for

Camera Calibration
Zhengyou Zhang

Sung Huh
CSPS 643 Individual Presentation 1
February 25, 2009

1
Outline
 Introduction
 Equations and Constraints
 Calibration and Procedure
 Experimental Results
 Conclusion

2
Outline
 Introduction
 Equations and Constraints
 Calibration and Procedure
 Experimental Results
 Conclusion

3
Introduction
 Extract metric information from 2D
images
 Much work has been done by
photogrammetry and computer vision
community
◦ Photogrammetric calibration
◦ Self-calibration

4
Photogrammetric Calibration
(Three-dimensional reference object-based calibration)

 Observing a calibration object with

known geometry in 3D space
 Can be done very efficiently
 Calibration object usually consists of two
or three planes orthogonal to each other
◦ A plane undergoing a precisely known translation is
also used
 Expensive calibration apparatus and
elaborate setup required

5
Self-Calibration
 Do not use any calibration object
 Moving camera in static scene
 The rigidity of the scene provides
constraints on camera’s internal parameters
 Correspondences b/w images are sufficient
to recover both internal and external
parameters
◦ Allow to reconstruct 3D structure up to a similarity
 Very flexible, but not mature
◦ Cannot always obtain reliable results due to many
parameters to estimate

6
Other Techniques
 Vanishing points for orthogonal directions
 Calibration from pure rotation

7
New Technique from Author
 Focused on a desktop vision system (DVS)
 Considered flexibility, robustness, and low
cost
 Only require the camera to observe a planar
pattern shown at a few (minimum 2)
different orientations
◦ Pattern can be printed and attached on planer surface
◦ Either camera or planar pattern can be moved by hand
 More flexible and robust than traditional
techniques
◦ Easy setup
◦ Anyone can make calibration pattern

8
Outline
 Introduction
 Equations and Constraints
 Calibration and Procedure
 Experimental Results
 Conclusion

9
Notation
2D point, m  u, v 
T

 3D point,  
T
M  X , Y , Z

 Augmented Vector, m  u, v,1 , M   X , Y , Z ,1

T T

 Relationship b/w 3D point M and image

projection m
  u0 
sm  A  R t  M  
A   0  v0  (1)

 0 0 1 

10
Notation
 s: extrinsic parameters that relates the
world coord. system to the camera coord.
System
 A: Camera intrinsic matrix
 (u0,v0): coordinates of the principal point
 α,β: scale factors in image u and v axes
 γ: parameter describing the skew of the
two image

11
Homography b/w the Model Plane
and Its Image
 Assume the model plane is on Z = 0
 Denote ith column of the rotation matrix R by
ri X 
u  Y  X 
s  v   A r1 r2 r3 t     A r1 r2 t   Y 
0
 1     1 
1
 Relation b/w model point M and image m
sm  HM H  A r1 r2 t  (2)

 H is homography and defined up to a scale

factor

12
Constraints on Intrinsic Parameters
 Let H be H = [h1 h2 h3]
h1 h2 h3    A r1 r2 t 
 Homography has 8 degrees of freedom &
6 extrinsic parameters
 Two basic constraints on intrinsic
parameter
h1T A T A 1h 2  0 (3)

h1T A T A 1h1  hT2 A T A 1h 2 (4)

13
Geometric Interpretation
 Model plane described in camera
coordinate system
x
T  
 3   y
r w  0
rT t   z   0,  w  1
3  
 
 w
 Model plane intersects the plane at
infinity at a line
r1  r2 
0, 0 
   
14
Geometric Interpretation
r1  r2 
x  a    b  
0 0
 x∞ is circular point and satisfy xT x   0 , or
a2 + b2 = 0
 Two intersection points
r1  ir2 
x  a  
 0 
 This point is invariant to Euclidean
transformation
15
Geometric Interpretation
 Projection of x∞ in the image plane
m  A  r1  ir2   h1  ih2

 Point m is on the image of the absolute

conic, described by A-TA-1

 1 2 T 1
 h1  ih 2   0
T
h  ih A A

 Setting zero on both real and imaginary

parts yield two intrinsic parameter
constraints
16
Outline
 Introduction
 Equations and Constraints
 Calibration and Procedure
 Experimental Results
 Conclusion

17
Calibration
 Analytical solution
 Nonlinear optimization technique based
on the maximum-likelihood criterion

18
Closed-Form Solution
 B11 B21 B31 
 Define B = A-TA-1 ≡ B B22 B32 
 12
 B13 B23 B33 
(5)
 1  v0  u0  
  
  2
 2  2 
  2 1   v0  u0   v0 2 
  2    2
    2 2  2    
2 2

 
  v0  u0   v02  0 0   v0  1
 
2
 v0  u0  v  u 2

  2   2
  
2 2
  2 2  2 

 B is defined by 6D vector b
b   B11 B33 
T
B12 B22 B13 B23 (6)

19
 Closed-Form Solution
 ith column of H = hi
hi   hi1 hi 3 
T
hi 2

 Following relation hold

hTi Bh j  vTij b (7)
vij 
T
 hi1h j1 hi1h j 2  hi 2 h j1 hi 2 h j 2 hi 3h j1  hi1h j 3 hi 3h j 2  hi 2 h j 3 hi 3h j 3 

20
Closed-Form Solution
 Two fundamental constraints, from homography,
become
 T
v12  (8)
 T
b0
 v11  v 22  
 If observed n images of model plane
Vb  0 (9)

 V is 2n x 6 matrix
 Solution of Vb = 0 is the eigenvector of VTV
associated w/ smallest eigenvalue
 Therefore, we can estimate b

21
Closed-Form Solution
 If n ≥ 3, unique solution b defined up to a
scale factor
 If n = 2, impose skewless constraint γ = 0
 If n = 1, can only solve two camera
intrinsic parameters, α and β, assuming u0
and v0 are known and γ = 0

22
Closed-Form Solution
 Estimate B up to scale factor, B = λATA-1
 B is symmetric matrix defined by b
 B in terms of intrinsic parameter is known
 Intrinsic parameters are then


v0   B12 B13  B11 B23  B11B22  B122 
  B33   B132  v0  B12 B13  B11B23   B11
   B11
   B11  B11 B22  B122 
   B12 2  
u0   v0   B13 2 

23
Closed-Form Solution
 Calculating extrinsic parameter from
Homography H = [h1 h2 h3] = λA[r1 r2 t]
r1   A 1h1 r2   A 1h 2 r3  r1  r2 t   A 1h 3
  1 A 1h1  1 A 1h 2
 R = [r1 r2 r3] does not, in general, satisfy
properties of a rotation matrix because of noise
in data
 R can be obtained through singular value
decomposition

24
Maximum-Likelihood Estimation
 Given n images of model plane with m points
on model plane
 Assumption
◦ Corrupted Image points by independent and
identically distributed noise
 Minimizing following function yield maximum
likelihood estimate

mij  mˆ  A, Ri , t i , M j 
n m


2
(10)
i 1 j 1

25
Maximum-Likelihood Estimation

ˆ A, R i , t i , M j
m  is the projection of point Mj in
image i
 R is parameterized by a vector of three
parameters
◦ Parallel to the rotation axis and magnitude is equal to
the rotation angle
 R and r are related by the Rodrigues formula
 Nonlinear minimization problem solved with
Levenberg-Marquardt Algorithm
 Require initial guess A, Ri , t i | i  1..n

26
Calibration Procedure
1. Print a pattern and attach to a planar surface
2. Take few images of the model plane under
different orientations
3. Detect feature points in the images
4. Estimate five intrinsic parameters and all the
extrinsic parameters using the closed-form
solution
5. Refine all parameters by obtaining maximum-
likelihood estimate

27
Outline
 Introduction
 Equations and Constraints
 Calibration and Procedure
 Experimental Results
 Conclusion

28
Experimental Results
 Off-the-shelf PULNiX CCD camera w/ 6mm lense
 640 x 480 image resolution
 5 images at close range (set A)
 5 images at larger distance (set B)
 Applied calibration algorithm on set A, set B and Set
A+B

29
Experimental Result

Angle b/w image axes

30
Experimental Result
http://research.microsoft.com/en-us/um/people/zhang/calib/

31
Outline
 Introduction
 Equations and Constraints
 Calibration and Procedure
 Experimental Results
 Conclusion

32
Conclusion
 Technique only requires the camera to
observe a planar pattern from different
orientation
 Pattern could be anything, as long as the
metric on the plane is known
 Good test result obtained from both
computer simulation and real data
 Proposed technique gains considerable
flexibility
33
Appendix
Estimating Homography b/w the Model Plane and its Image

 Method based on a maximum-likelihood

criterion (Other option available)
 Let Mi and mi be the model and image
point, respectively
 Assume mi is corrupted by Gaussian
noise with mean 0 and covariance matrix
Λmi

34
Appendix
 Minimizing following function yield
maximum-likelihood estimation of H
  mi  mˆ i  m1i  mi  mˆ i 
T

1  h
T
1 Mi

 where ˆi  T
m   with hi = ith row of H
h3 M i h 2 M i 
T

35
Appendix
 Assume Λ   I for all i
mi
2

 Problem become nonlinear least-squares

one, i.e. min  m  mˆ
H i i i
2

 Nonlinear minimization is conducted with

Levenberg-Marquardt Algorithm that
requires an initial guess with following
procedure to obtain

36
Appendix
T
x=  h h 
T T T
 Let 
1 h 2 3
 Then (2) become

 n above equation with given n point and can be

written in matrix equation as Lx = 0
 L is 2n x 9 matrix
 x is define dup to a scale factor
 Solution of x LTL associated with the smallest
eigenvalue

37
Appendix
 Elements of L
◦ Constant 1
◦ Pixels
◦ World coordinates
◦ Multiplication of both

38
Possible Future Work
 Improving distortion parameter caused by
lens distortion

39
Question?

40