Professional Documents
Culture Documents
Abstract
This paper presents a reverse engineering system for rapid modeling and manufacturing of products with complex surfaces. The
system consists of three main components: a 3D optical digitizing system, a surface reconstruction software and a rapid prototyping
machine. The unique features of the 3D optical digitizing system include the use of white-light source, and cost-effective and quick
image acquisition. The surface reconstruction process consists of three major steps: (1) range view registration by an iterative closed-
form solution, (2) range surface integration by reconstructing an implicit function to update the volumetric grid, and (3) iso-surface
extraction by the Marching Cubes algorithm. The modeling software exports models in STL format, which are used as input to an
FDM 2000 machine to manufacture products. The examples are included to illustrate the systems and the methods. r 2002 Elsevier
Science Ltd. All rights reserved.
0736-5845/02/$ - see front matter r 2002 Elsevier Science Ltd. All rights reserved.
PII: S 0 7 3 6 - 5 8 4 5 ( 0 1 ) 0 0 0 2 6 - 6
54 L. Li et al. / Robotics and Computer Integrated Manufacturing 18 (2002) 53–67
scans. Detailed scans can be generated quickly, and the image into alignment with the overlapping portion of
wide scanning volume reduces the importance of the another range image. Integration is the process of
angle at which the tracking light hits the target object creating a single surface obtained from the multiple
[2]. range images [6].
Post-scanning data processing is an important step for For a real world 3D object, no single range image
any successful application of these scanning systems. suffices to describe the entire object surface. Multiple
The objective of post-scanning data processing is to views must be digitized in order to obtain the complete
reconstruct surface models from 3D point clouds. geometric information of the object. In order to extract
Strategies for surface reconstruction have proceeded in a 3D model from multi-view range images, the range
two main directions: reconstruction from unorganized images must be aligned in one global coordinate system.
points and reconstruction that exploits the underlying One of such methods that can achieve this is to capture
structure of the acquired data. The latter is called the range images using a digitizer mounted on a
surface reconstruction using structured data. These two precision motion device such as a coordinate measuring
strategies can be further divided according to whether machine (CMM). This method is costly and incon-
they operate by reconstructing parametric surfaces or by venient for many real world applications.
reconstructing implicit surfaces. The algorithm em- An alternative solution is to use a registration
ployed here falls into structured data reconstruction algorithm to align the range images. Two main
using implicit function methods that use samples of a approaches are used for the registration: (1) match
continuous function to combine structured data. Pre- ‘‘created’’ features of the images, such as the method
vious relevant work includes [3–6]. These methods that was presented in [7]; or (2) minimize the distance
mainly apply to laser scanning technologies. In this between all points on the overlapped surface represented
paper, the existing algorithms were adapted for building by the two images. Examples of the latter approach
of 3D models from point clouds generated by the 3D include [6,8–10]. The minimal distance method was used
optical digitizing system. in this work.
To create a single model from multiple range images, After registration is completed, the final surface can
two fundamental issues must be addressed: registration be extracted directly from range surfaces by the
and integration. Registration refers to computing a rigid following two methods: (1) geometric details of all scans
transformation that brings the points of one range are averaged and the vertex positions of range surface
L. Li et al. / Robotics and Computer Integrated Manufacturing 18 (2002) 53–67 55
meshes are refined; or (2) volumetric grid points are automatic fringe pattern analysis, and it has higher
updated according to each range surface and iso- accuracy when a fringe analysis algorithm is utilized
surfaces are extracted from the volumetric grid. The because there are no higher harmonic spatial frequencies
latter method was preferred in this research because except for plus and minus fundamental frequency
it allows the extraction of the iso-surface at any stage components in the spatial frequency domain. Second,
of the registration without considering point-to-point a spatial and parallel phase-shift algorithm was used,
correspondence of two meshes that share a common rather than using temporal phase-shift algorithm to
surface of the object. calculate the phase distribution. Such a parallel property
The final step in the reverse engineering process is to in phase calculation would be more preferable to the
manufacture the object based on the geometric model case where the digitalization of live subjects such as
generated from the previous procedures. In this re- human or animal is required. Finally, a robust phase
search, the manufacturing process is carried out using unwrapping algorithm based on constructing a hybrid
an FDM rapid prototyping machine. mask was applied to this system for reconstructing the
This paper is organized as follows: the next section 3D range image of object surface.
introduces the 3D optical digitizing system; the third The photograph of the system is shown in Fig. 2. This
section describes 3D data pre-processing; an algorithm optical digitizing technology does not require any
for registration of a set of range views is described in the
fourth section; the volumetric method and its imple-
mentation are presented in the following section; for
demonstration and analysis, a few experiments and
associated analysis are also provided. Conclusions are
given in the final section of the paper.
scanning mechanisms as do conventional laser scanners. Whenever the system configuration is fixed, K becomes
Also, a single-view object surface can be captured in a a constant. The fringe sensitivity K can be determined
parallel mode. To capture a complete aðx; yÞ image of an under the first order approximation and is given by
object, several approaches can be used. For example, the 2pL cos y
object is mounted on a turntable. By rotating the table K¼ ; ð4Þ
P0 L0
with different angles, multi-view surface acquisition was
carried out. In the recording step, structured light with where P0 ¼ ðL0 PÞ=ðdcos yÞ stands for the spatial
sinusoidal fringes is projected onto the object surface. period of grating projected onto the reference plane; d is
Due to the depth variation of the surface the structured the distance between the exit pupil of projection optics
light is phase-modulated, leading to a deformed spatial and the center of the grating; P represents the spatial
carrier fringe pattern in which the topographic informa- period of projection fringes; y depicts the angle between
tion of the object surface has been encoded. Assuming the imaging-optics axis and the projection-optics axis;
that a 2D gray-scale image of the object surface is and L0 delineates the distance from entrance pupil of
represented by a function, the intensity distribution of both imaging and projection optics to the reference
deformed spatial carrier fringe pattern can be described plane. Therefore, the phase distribution of an object
as surface can be linearly mapped to a range image, i.e., a
3D range image of a test surface being reconstructed.
Iðx; yÞ ¼ aðx; yÞ þ bðx; yÞ cos½2pf0 x þ fðx; yÞ; ð1Þ
The concavity or convexity of the surface is deter-
where aðx; yÞ provides the background intensity dis- mined directly from the unwrapped phase function. The
tribution (the texture image of the object surface) of the data acquisition, phase function calculation and phase
fringe pattern; bðx; yÞ is a local contrast variation; f0 is a unwrapping have been conveniently automated and
spatial carrier frequency generated by projected grating; computerized, and the whole process is carried out by a
and fðx; yÞ is a phase distribution to be determined. software package dedicated to this digitizing system.
Both geometric shape and the texture information When geometric parameters of an optical configuration
are recorded in Eq. (1), where aðx; yÞ represents the are determined, the fringe sensitivity is determined
texture information and fðx; yÞ involves topographic accordingly. Thus, the unwrapped phase function, f
information. ðx; yÞ; is proportional to the surface shape function z
The combination of imaging optics and CCD photo ðx; yÞ: An example given in Section 8 illustrates the
sensor array is used to capture the deformed grating that process of the data acquisition of the 3D optical
is then digitized and stored in frame memory for later digitizing system.
processing. The process of information decoding for the
geometric shape of object surface extracting fðx; yÞ from
Eq. (1) is quite similar to that we reported previously 3. 3D data pre-processing
[12]. However, in this approach the laser interferometer-
based projector was replaced by a white-light (halogen The pre-processing of range images begins by
lamp) projector. The retrieved phase function can be converting the range images into range surfaces by
expressed as connecting the nearest neighbor points with triangles of
Pp the range images. This process results in piecewise linear
Ik ðx; yÞsin dk
fðx; yÞ ¼ tan1 Ppk¼1 ; ð2Þ range surfaces, as shown in Fig. 3. Each point in the
k¼1 Ik ðx; yÞcos dk range image is a potential vertex in the triangle mesh. To
where dk ¼ 2pk=p is the constant phase shift and p the remove background data from the data set, a filter-value
spatial phase-shifted interferograms that are obtained (threshold) is set and data from the object being scanned
digitally from only one recorded fringe pattern. This are included in the range surfaces. Also, triangles with
spatial parallelism is an essential feature of the spatial an edge longer than the pre-set distance threshold are
carrier techniques. discarded, thus to avoid inclusion of questionable or
The relationship between the phase distribution and noise data. The object shown in Fig. 3 was processed
the range image of a 3D object surface is determined by and the result is shown in Fig. 4.
optical sensitivity that depends on the geometry of After the piecewise linear range surfaces are created,
system design and system parameters. Such a relation the normals of the triangles on the range surfaces are
can be simply expressed as calculated. By averaging the normals of triangles that
are incident to a vertex, normals to vertices on the range
fðx; yÞ ¼ Kzðx; yÞ; ð3Þ
surface are assigned.
where K represents the fringe sensitivity depending on Next, a weight is defined for each vertex on the range
the optical geometry; fðx; yÞ is the unwrapped phase surface. Weights are intended to represent variations in
distribution; zðx; yÞ represents the height variation of certainty across the range surface. The definition of
object surface, that is, a 3D image or a range image. weights should be specific to the range scanning
L. Li et al. / Robotics and Computer Integrated Manufacturing 18 (2002) 53–67 57
technology. For this scanner, the position of the sensor Horn [9] proves a closed-form solution for finding the
is fixed for every range view. For a sample vertex on the rigid transformation that minimizes the least-squared
range surface, the weight is defined as distance between point pairs. Besl and McKay [8] used a
nij ~
~ ns method that they called as the ‘‘iterated closest-point
wij ¼ ; ð5Þ algorithm’’. This method finds the nearest points on one
ds
surface to a collection of points on another surface and
where nij is the unit normal of surface at the point, as then transforms one surface so as to minimize the
shown in Fig. 4; ns is the unit normal from the vertex to collective distance through an iterative process. This
the sensor; and ds is the distance between the vertex and method cannot be used directly to register range images
sensor. The dot product between each vertex normal because it requires that every point on one surface have
and the unit vector from the vertex to sensor reflects the a corresponding point on the other surface. Turk and
influence of viewing direction on certainty. The smaller Levoy [6] adapt the method to register the range images
the angle between these two vectors the higher the cer- from a laser scanner.
tainty. ds reflects the influence of distance on certainty; Based on these algorithms and methods, an algorithm
the bigger the distance the lower the certainty. was devised to register range images from the scanning
Range data at the boundaries typically have greater system. The algorithm consists of two steps: (1) find the
uncertainty and therefore have less weight. In practice, it overlapping portion on two range surfaces; and (2)
can be determined whether a vertex is on boundary by transform one range surface to align with the other one.
counting the number of triangles that are incident to the For example, with two different range views meshes A
vertex. The weight of a point other than the vertices of and B are formed, as shown in Fig. 5. To find the
a triangle is calculated by linearly interpolating the overlapping portion of two range surfaces, it is first to
weights of the intersecting triangles’ vertices. With the find the nearest point of mesh A to each vertex of mesh
above information, the range images can be registered B: This is done by casting a perpendicular ray from a
with one another and integrated into a single ‘‘iso- point Bi in mesh B to triangles on mesh A; until the
surface’’. intersection is found, which is located inside a triangle of
mesh A; then Bi and the intersection Ai ; constitute a
pair; otherwise skip the triangle. The number of point
4. Registration algorithm pairs is denoted by Np : By defining a distance threshold,
it defines the distance between the vertex of mesh B and
After a range image is converted to a triangle mesh each vertex of a triangle of mesh A: If any of them is
with a weight at each vertex, the next step is to align the greater than the distance threshold, the triangle is
range surfaces from different views for integration. skipped. This operation discards the pairs of points
58 L. Li et al. / Robotics and Computer Integrated Manufacturing 18 (2002) 53–67
that are too far apart. Finally, the overlapping portion is and Mckay [8]. The calculation is carried out by the
represented by the point pairs. The triangles in one mesh following steps. P
are stored in a BSP tree to speed up this operation. First the 3 3 cross-covariance matrix, , is calcu-
Once all point pairs are found, the weighed least- lated as
square distance between pairs of points can be defined as N
X 1 X p
follows: ¼ wi ½ðBi Bc ÞðAi Ac ÞT
Np Np i¼1
X 2 3
E¼ wi jjAi RðBi Bc Þ T Bc jj2 ; ð6Þ Sxx Sxy Sxz
i¼1 6 7
¼ 4 Syx Syy Syz 5: ð9Þ
where Bi and Ai are pairs of 3D points and Bc is the
Szx Szy Szz
centroid of Bi : The centroid is defined by
N N From the matrix in the above equation, the following
1 X p X p
2 3
Sxx þ Syy þ Szz Syz Szy Szx Sxz Sxy Syx
6 Syz Szy Sxx Syy Szz Sxy þ Syx Szx þ Sxz 7
6 7
Q¼6 7: ð10Þ
4 Szx Sxz Sxy þ Syx Sxx þ Syy Szz Syz Szy 5
Sxy Syx Szx þ Sxz Syz þ Szy Sxx Syy Szz
zero crossing. But it should also be as narrow as possible resolve ambiguous cases [14] was used. There are two
to avoid influencing surfaces on the other side. Similarly, main steps in the Marching Cubes algorithm. The first
the signed distance and weight functions should not be step is to decide how to define the section or sections of
extended too far in front of the surface. Restricting the surface that divide up an individual cube. The second
functions to the vicinity of the surface yields a more step involves forming the correct facets from the
compact representation and reduces the computational positions where the iso-surface intersects the edges of
expense of updating the volume. When employing the the grid cell.
distance cut-off idea for updating the volumetric grid, it Each corner of a unit cube is classified into either
is found that the distance is dependant on the resolution being below or above the iso-value. This gives 256
of the range image. possible configurations of corner classifications to a unit
cube. Two of these are trivial. When all the values of the
5.3. Iso-surface extraction using marching cubes 8 corners are above or below the iso-value, it does not
algorithm contribute to the iso-surface. For all other configura-
tions it is necessary to determine where, along each cube
To extract the surface from the volumetric grid, a edge, the iso-surface crosses, and use these edge
Marching Cubes algorithm [13] with a lookup-table to intersection points to create one or more triangular
L. Li et al. / Robotics and Computer Integrated Manufacturing 18 (2002) 53–67 61
patches for the iso-surface, and facets from adjacent grid Fig. 13 shows the 2D grid to illustrate that some of
cells connect together correctly. the boundary grids would be updated whereas others
Two tables are used to look-up, which triangles need would not. Thus, the surface extraction procedure is
to be created for each cube. First, an 8-bit cube index is restricted to skip boundary voxels. This is reasonable
calculated based on the vertex values. This cube index is because range surfaces come from multi-views and when
used as a key for the edge table to find which edges they have sufficient areas of overlapping with the cube
intersect the iso-surface. The edge table returns a 12-bit size being small enough.
number with each bit corresponds to one edge and a 1
indicating the edge has an intersection. The intersection
point on each intersected edge is calculated by linearly 6. Rapid prototyping of models
interpolating the vertex values. These intersection points
and the triangle table are used to create facets for the After the 3D surface representation of an object is
iso-surface inside the cube. The Marching Cubes generated, the next step is to create a model using rapid
algorithm is shown in Fig. 12. Surface patches can be prototyping. This process comprises two steps: (1)
created for a single voxel. This process can be applied to convert the model into STL format (the standard model
the entire volume. Then all these triangle patches format for rapid prototyping machines); and (2) slice the
constitute an iso-surface. model for rapid prototyping operations. This system has
L. Li et al. / Robotics and Computer Integrated Manufacturing 18 (2002) 53–67 63
method are summarized: (1) There was no vertex or range surfaces, overlaps between range surfaces would
triangle correspondence between the two surfaces. The help to eliminate this effect. (3) The maximum relative
vertices of the iso-surface were interpolation of original error was 3.6%, which appeared around the boundary
data according to volumetric implicit function. The shown in Fig. 16(c). Bigger overlaps could help to
density of the triangles on the iso-surface varied compensate the error of extraction.
according to the slope of the surface at the given unit The last example shows the process of building a
cube size. These features were very useful when model from the range surfaces with different views via
extracting 3D surface models from range surfaces of registration and integration. The object vase was first
different range views. (2) Details at the surface edge digitized using the 3D optical digitizing system in five
might be lost after iso-surface extraction. For multiple different views. The basic mechanical registration
L. Li et al. / Robotics and Computer Integrated Manufacturing 18 (2002) 53–67 65
Fig. 16. Comparison of initial range surface and iso-surface. (Contours appearing on the iso-surface correspond to small triangles created when the
iso-surface clips the edges and corners of voxel cubes.)
indicated that the rotational angles between the first 8. Concluding remarks
four views were 901 about x-axis; and the rotation
between the first and the fifth views was about 1001 This paper presented a reverse engineering system for
around y-axis. This information was used as an input to rapid modeling and manufacturing of objects with
the integration software, shown in Figs. 17(a) and (b). complex surfaces. The system consists of three major
Fig. 17(c) shows the result of the registration. It is clear parts: the 3D optical digitizing system, the surface
from the figure that the range surfaces of different views reconstruction software and an FDM rapid prototyping
are in alignment with each other. Then the range machine. The optical digitizer utilized a white-light
surfaces were cited to update the volumetric grid, from source for image acquisition that makes this technology
which the final iso-surface model of vase was extracted, cost-effective, fast in image acquisition and portable for
as is shown in Fig. 17(d). This model contains some various applications. The specially designed software
holes around the shoulder of the vase because the range system processed the image data cloud acquired by the
surfaces did not overlap sufficiently in all locations. If digitizer which includes the 3D data pre-processing by
more range surfaces are used, the holes would be creating triangle meshes from range images. The vertices
eliminated. For example, the integrated surface shown of newly created range surfaces were also associated
in Fig. 17(e) is more complete because 25 range surfaces with weights. The range surfaces were then registered by
were integrated rather than five range surfaces of minimizing the weighed least-square distance between
Fig. 17(d). the points on the overlapping portion of two surfaces.
66 L. Li et al. / Robotics and Computer Integrated Manufacturing 18 (2002) 53–67
The volumetric method aimed to find the volume that [4] Grosso E, Sanini G, Frigato C. Extraction of 3D information and
contained all the range surfaces, and divide the volume volumetric uncertainty from multiple stereo images. Proceedings
into voxel grid according to the size of unit cubes. Then of the Eighth European Conference on Artificial Intelligence,
August, 1988. p. 683–688.
the iso-surfaces were extracted from the volumetric grid, [5] Hilton A, Toddart AJ, Illingwroth J, Windeatt T. Reliable surface
which were a collection of points inside the volumes to reconstruction from multiple range images. Proceedings of the
represent the iso-surface. The STL was created from the Forth European Conference on Computer Vision, vol. I, April
iso-surface models and exported to an FDM 2000 1996. p. 117–126.
machine, which produced prototypes. Currently, the [6] Turk G, Levoy M. Zippered Polygon Meshes from Range Images.
Proceedings of SIGGRAPH’94, Orlando, FL, 24–29 July.
entire system is being tested for various applications. New York: ACM Press, 1994. p. 311–318.
[7] Nobuhio W, Toriyama H, Tanaka HT, Kishio F. Reconstruction
of an Object Shape from Multiple Incomplete Range Data Sets
Acknowledgements Using Convex Hulls. Computer Graphics International’93,
Lausanne, Switzerland, June 21–25, 1993. p. 193–203.
[8] Besl PJ, Mckay ND. A Method for registration of 3-D shapes.
The authors wish to acknowledge the Natural IEEE Trans Pattern Anal Machine Intelligence 1992;14(2):
Sciences and Engineering Research Council of Canada 239–56.
(NSERC) for providing the support to this research [9] Horn BKP. Closed-form solution of absolute orientation using
through a Strategic Grant STR192769. unit quaternion. Opt Soc Am 1987;4(4):629–42.
[10] Huang X, Gu P. CAD-model based inspection of sculptured
surfaces with Datums. Int J Prod Res 1998;36(5):1351–76.
[11] Peng X, Zhu SM, Su CJ, Mithchell Tseng. Model-based digital
References Moir!e topography. Optik, 1996;110(4):184–90.
[12] Peng X, Gao Z, Zhu SM. Surface contouring by a new type of
[1] Yan X, Gu P. A review of rapid prototyping technologies and digital Moir!e technique. Optik 1995;100(2):63–7.
systems. Comput Aided Des 1996;28(4):307–18. [13] Lorensen WE, Cline HE. Marching cubes: a high resolution
[2] Aronson RB. Forward thinkers take to reverse engineering. 3D surface construction algorithm. Comput Graphics (SIG-
Manufacturing Engineering Dearborn 1996;117:34 –5. GRAPH’87 Proc) 1987;21(7):163–9.
[3] Curless B, Levoy M. A Volumetric method for building complex [14] Montani C, Scateni R, Scopigno R. A modified look-up table for
models from range images. Proceedings of SIGGRAPH 1996, implicit disambiguation of marching cubes. Visual Comput
New Orleans, LA, 5–9 August. New York: ACM Press, 1996. 1994;10(6):353–5.
p. 303–312. [15] Stratasys. FDM 2000 manual. Release 2.0, 1998.