Proceedings of the 30th Chinese Control Conference July 22-24, 2011, Yantai, China

Model Optimization and Image Compensation in 3D Printing

CHEN Xinwei, LIU Jingtai, LIN Sen, SHI Tao, CHEN Kuo, WEI Qing
Institute of Robotics and Information Automatic System, Nankai University, Tianjin 300071, P. R. China E-mail: chenxw@robot.nankai.edu.cn Abstract: A novel 7-axis robot for industrial large 3D printing applications is presented in this paper, which is developed by Instiute of Robotics and Automatic Information System, Nankai University. Base on the prototype, the mechanism on

three-dimensional printing is deeply dissected. By investigating the reason of color distortions in 3D printing, a novel multi-step compensation algorithm based on model optimization and image compensation is developed. Finally, the experimental results show the performance of the 3D printing robot platform and the proposed algorithms.
Key Words: 3D Printing Robot, Model Optimization, Image Compensation

Introduction

Since the 1980's, the large-scale inkjet printer was widely used in a variety of flat printing tasks with full color, and established its unshakable status in the printing industry. With the improvement of the printing technology and the development of the computer 3D simulation, more and more 3D printing requirement become burst. However, the existing ink-jet printing equipment could only print on the plane material, the printing technology for the three-dimensional model is currently only at the exploratory stage, which is far behind 3D technology in computer. Furthermore, there are few reports for the theoretical issues in research three-dimensional printing. Flat printer can print on different flat materials, such as PVC, marble, glass. Also Gazeau Jean-Pierre[1] and his partner invent a 5-axis robot for printing high resolution pictures from media on vertical wide surfaces. In 3d printing, the mechanical design and printing control will face many complex challenges because the printhead must be very close to the material. At present, there are very few devices commercially available, capable to printing on 3D surfaces of objects.

When sculpturing, we use a main spindle motor and a graver at its end. The max speed of the motor reaches 24000r/min. During printing, a mechanical device is installed. It consists of four axes, which parallel to the z-axis. The end-effector is composed by at least one printing board and several inkjet printheads, e.g. four heads spraying yellow, cyan, magenta and black inks. To insure the spatial displacement of the pringheads, the mechanical structure is designed to be as compact as possible. The figure 2 represents a CAD view of the mechanical structure with 4-axes which drived by stepped motors.

Fig. 1: 3D printing robot

3D Printing Robot

In overall design, we use a Cartesian coordinate robot as the base platform of this robot. A Cartesian coordinate robot has three principal prismatic axes (X, Y and Z) that are at right angles to each other. Because of the highly rigidity, the Cartesian Robots are very accurate and repeatable. Due to their mechanical structure, these robots are very easy to program and visualise. The Cartesian Coordinate Robot's relatively simple design and straightforward operation make it highly desirable in manufacturing. Its work space can reach 1.2m*2.4m*0.6m, which meets the large surface demand in 3d printing. 3d printing includes two processes: sculpture and printing. The robot accomplishes these operations by changing its end-effectors.
Fig.2 a CAD view of the mechanical structure
This work is supported by National High Technology Research and Development Program 863 under Grant 2009AA04Z222 and Tianjin Nature Science Foundation under Grant 09JCZDJC23700

The 3D printing robot integrates the necessary components to fulfil its 3D printing job. These components concern actuation and instrumentation, ink supply, inkjet printing

4029

and robot control, and power supply. The figure 3 presents the whole robot system and its components.

Fig.4 The color distortion in 3D Printing

3.2. Modeling The distortion dose not appear in flat printing, because the distance between printhead and material to be printed is an fixed value, less than 5mm. 3D model has a irregular surface, with different curvature at every point. Due to the special physical structure of the printhead, every point has its special print distance H, shown in Figure 5.

Fig.3 The control system and components of 3D printing robot

The actuation of the robot is insured by three servomotors and four stepped motors respectively associated with the seven axes of this robot. Furthermore, seven encoders of the motor axes are used to determine the movement of the servomotors, to build a position feedback system(PFS). The role of the real-time control device is to ensure the simultaneous control of the severn axes of the robot. The printing assembly is based on a FPGA board. The printhead utilises two custom ICs in a serial arrangement.The FPGA program should simulate the data load sequence and timing for all models of the printhead. The trigger signal, nFIRE, is controlled by the position feedback of the motion control system. The printhead acts like a pump and therefore is capable of pulling ink through it. A negative ink supply pressure must be used with the printhead to stop ink flowing in an uncontrolled manner through the printhead. An ink supply system should consist of several components. A small ink reservoir, stores a small volume of filtered ink supplying it to the printhead, as it is required at the recommended pressure. As the ink in the small reservoir is consumed this must be replaced. Most commonly a small liquid pump supplies this ink, through a system filter to the small reservoir, from a main ink tank. The pump is controlled by a feedback system and sensor in the small reservoir. Finally, compatible ink tubing provides the ink flow path between the components.

Fig.5 3D printing mechanism and restricted condition

The Distortion and Modeling in 3d Printing

3.1. Distortion In 3D printing experiment, different from flat printing, there will be color fades phenomenon. Color distortion seems more serious in the slope region. The figure 4 presents the distortion. In the flat lower surface area, the image shows more details and clear. However, in the upper half of the model, because of the steep surface, the colors appear more obvious distortion.

To compensate the distortion, the mathematical model should be constructed. Printhead has a recommended printing distance. Print distance specified is from nozzle plate. Print quality beyond that distane recommended, will vary. The reason could be temperature, humidity, air turbulence and also the printhead s physical construction.First three are irresistible factors, so that they are diffcuclt to modeling. In our model, they are controlled in a certain range to reduce the impact on distortion. Printheads technology document shows that the nozzles of printhead have deviation. The ink dropped from the nozzless deviation will reach 1.8 degrees. The distribution of the deviation is stochastic, so that we use a Normal Distribution to model the probability distribution of ink droppoint. By optimizing the parameters, N(0,0.46) is more accurate, shown in (1).

f ( x)

2S

e(xP )

( 2V

0 .5 9 e  1 .0 9 x

(1)

The ink drop deviated from the accurate place associates

4030

with the degree deviation of the nozzle and the print distance H. The error increase in direct ratio. The average diameter of the ink drop is 100 P m . Thus, we can calculate the probability P of the drop deviation from the exact place less than 100 P m , with different H, as shown in Table 1.
Table 1 The relation between printing distance and probability

the terrain model shown in Fig.6. We quantitatively compare the algorithms that use the indicators as follow:

z The average print distance of the model H avrg . z The maximum print distance of the mode H max . z The average error between models e avrg .
Table 2 shows that the Butterworth Filter is able to reduce the average print distance effectively by use a proper cutoff frequency with a tolerable error. It represents a significant improvement over the other algorithms.
Table 2 Optimization algorithms and results Algorithm H avrg H max ( (mm) mm) Original model Wavelet Transform(order=1) Wavelet Transform(order=2) Wavelet Transform(order=3) median filter arithmetic mean Butterworth filter(cutoff frequency=20) Butterworth filter(cutoff frequency=15) Butterworth filter(cutoff frequency=5) 14.1 12.3 11.3 9.6 12.4 12.2 8.3 7.5 6.4 84 62 60 52 77 77 41 33 17

H(mm) P(%) 1 99.9 2 99.4 3 93.3 5 72.9 8 50.9 10 41.8 15 28.6 From Table 1, if the H greater than 8mm, P will less than 50%. Under normal light conditions, the resolution of the human eyes is close to 100 P m . Thus, the deviation can easily be detected by eye if H>8mm. In paper[2], we also require the gradient D  70q by modeling the curvature parameter. Through the above, we establish the prerequisite to judge whether a 3D model is suitable for 3D printing: z The average print distance H avrg z The max gradient D max

e avrg
(mm) 0 7.1 11.9 16.7 1.1 0.8 3.1 4.0 5.8

 8mm .

 70q .

Model Optimization and Image Compensation

4.1. Model Optimization 3D printing can play an important role in many fields, such as military, construction, design and so on. But not all the models match conditions in section 3. Figure 6 shows the terrain model of Taiwan. The model is full of geographic features, such as ravines, ridges, valleys and rivers, etc. The printhead cannot be very close to the material.

The Butterworth filter is a type of signal processing filter designed to have as flat a frequency response as possible in the passband so that it is also termed a maximally flat magnitude filter. The magnitude of the transfer function for this filter is:

H (u , v)
where

1 1  [ D(u , v) / D0 ]2 n

(2)

D0 is the cutoff frequency, n is the order of the

u 2  v2 .

filter and D (u , v)

Fig. 6 Taiwan terrain model

So we want to let more models to comply with the prerequisite in 3D printing through optimization of the model data. Optimization objective is to reduce the average print distance as well as minimize the error between the original model and optimized model. As shown in Fig.5, the curve on the right looks more smoothly. We have tried a variety of optimization methods, including multi-order Wavelet Transform [3], median filter, arithmetic mean and Butterworth filter. For these experiments, we use

To use Butterworth filter as the method of model optimization, we should follow the five steps, as shown in Fig.7. (1) Get model data. Give an example of terrain model, we can download the digital elevation model (DEM) data from Google Server by using the Google Earths API function with specified the latitude and longitude. (2) Pre-processing. Model data has some different from image. Image data should in the range, such as 0~256 in an 8-bit image. But model data may have invalid data. So before optimization, we should pre-process to the model data. (3) Discrete Fourier Transform(DFT), because the Butterworth filter works on frequency-domain. (4) Butterworth lowpass filter. We select the proper cutoff frequency by calculating the average print distance and error values. As shown in Table 2, cutoff frequency=15 would be better than others. (5) Inverse DFT and post-processing, than we will get an output model, which can be use for 3D sculpture and 3D printing.

4031

Fig.7 The work of Butterworth filter

Figure 8 shows the original and optimized Taiwan models. The right part of the picture is contour lines of the models. By compare with the two models, we can obviously find that the optimized model has more gentle mountains, and also the contour lines have become sparse.

Fig. 9 The error between models shown in Fig.8

When we look at a satellite photo of a mountainous area, we found that valleys always have lower color intensity than their neighboring areas while peaks always have higher intensity than their neighboring areas. This is because that peaks can reflect sun light very well but valleys cannot. Our 2D image compensating algorithm is based on this fact. In area where D(i, j)>0, we lost a peak by substitute G for F. To this area, we should enhance the intensity of the image corresponding to this area. In area where D(i, j)<0, we lost a valley by substitute G for F. To this area, we should reduce the intensity of the image corresponding to this area.

Fig.8 The original and optimized models with contour lines

4.2. Image Compensation Assume that the original DEM data is F. Due to the limit of machining precision, engraving and printing a model based on F is sometimes impossible and not necessary. We now let F pass a low pass filter first, and get new DEM data G, as shown in Figure 8. It is much easier to engrave a model of G than F. However, this process leads to loss of some geographical information. For example, at some peaks F(i, j) is higher than the corresponding point G(i, j); and at some valleys F(i, j) is lower than G(i, j). Suppose the difference between F and G is D. D=F-G or D(i, j)=F(i, j)-G(i, j). D represents the lost geographical information, shown in Figure 9, where green means D=0, blue means D<0, and yellow means D>0. Our work in the following part is to compensate this loss by processing the image printed on the model. This compensation aims at: 1. Using 2D image to represent 3D details that cannot be engraved due to the limit of machining precision. 2. Avoiding large areas being covered by monotonous color, for example green, which lead to visual fatigue.

Fig. 10 The principle of image compensation algorithm The algorithm can be represented as follow: 1. D=D/max of(|D|); so that for every (i , j), |D(i,j)|<=1; 2. r_dem is the number of rows of D, c_dem is the number of columns of D. r_image is the number of rows of the image, c_image is the number of columns of the image. r_rate=r_dem/r_image, c_rate=c_dem/c_image; for i=1:r_image for j=1:c_image R(i,j)=R(i,j)*(1+x*D(i*r_rate, j*c_rate)); G(i,j)=G(i,j)*(1+x*D(i*r_rate,j*c_rate)); B(i,j)=B(i,j)*(1+x*D(i*r_rate,j*c_rate)); end end Where R(i, j) G(i, j) B(i, j) are RGB value of the original image. R(i, j) G(i, j) B(i, j) are RGB value of the compensated image. x is a constant number and is pre-assigned, 0<x<1.

Fig. 11The image compare with compensation

4032

(5) 3D Printing.

Fig. 12 3D simulation view of images

Figure 11 and 12 show the different of the images. The right part of them is the compensated image. The algorithm is able to make up the loss by model optimization, with more particulars and high contrast ratio. Explanations: 1. In the former analysis, we want to change the color intensity of the image while keep hue and saturation unchanged. It seems that we should transform the representation of the 2D image from RGB color model to HSI color model (Hue, Saturation, Intensity). However, according to the relationship between RGB and HSI, multiplying R, G and B value respectively by the same rate does not change hue and saturation but only affects intensity. 2. The principle of this algorithm is to change the color intensity of 2D image printed on the 3D model. This algorithm satisfies our requirement that wherever D(i, j )>0, its intensity should be enhanced. Wherever D(i, j)<0, its intensity should be reduced. In our algorithm, color intensity multiplies a linear function of D(i , j), which is 1+x*D(i, j). 3. Thresholds may be set. For point (i, j) only when D(i, j)>threshold1 or D(i, j)<-threshold2, point (i, j)s corresponding pixels intensity will be changed.

Fig. 13 Image data and DEM data of Shenzhen terrain model

Fig. 14 3D sculpture and printing model

From the experimental results, as shown in Figure 14, we can see that the full color geographic model can demonstrate the color and landscape of the terrain.

Conclusion

3D Experiment and Results

From the sections above, the 3D printing robot has been introduced, including the mechanism, the motion control system and print system. To compensate for color distortion, the model optimization and image compensation algorithm are described. Based on them, we use the robot to print on a Shenzhen terrain model. (1) Preparation Work: a 600mm*600mm*300mm cube material, made of polyurethane (PU). Download image and DEM data from Google server by using Google Earth API, shown in Figure13. (2) 3D Sculpture: model optimization and sculpture. Optimization algorithm is Butterworth filter. Sculpture path planning is based on Extracted NURBS [4]. The model drop is 120mm. (3) Surface Preparation: The printing is in quad tone (cyan, magenta, yellow and black). To obtain printed colors that are identical to the colors of the model, a white primer layer may be previously applied on the surface. Also the ink-jet printing coating is requisite for ink absorption, related to SiO2, PVA and solvent. (4) Image Processing: including image compensation, colour-separation and image halftone processing. The halftone method is three dimensional halftoning algorithm based on surface features [5].

A novel 3D printing robot has been presented in this paper. The robot is an integrated innovation of the robotics and ink-jet printing. Using this technology may lead to lots of new applications like printing on fixed surfaces like terrain model or embossment. Furthermore the robot architecture could be easily extended for printing on larger surfaces.

References
[1] Jean-Pierre Gazeau, Zeghloul Said, Gabriel Ramirez-Torres. A novel 5-axis robot for printing high resolution pictures from media on 3D wide surfaces[C]. Proceedings of the 2009 IEEE International Conference on Industrial Technology, 2009:1-6. [2] Chen Xinwei, Liu Jingtai, Sun Lei, etc. Ink Drop and Color Distortion Modeling in Three Deimensional Printing[C].World Congress on Intelligent Control and Automation, 2010:335-340. [3] Niu Xuejuan,Liu Jingtai,Sun Lei. Multi-resolution analysis of grid point clouds based on wavelet transform [C].Control Conference, 2008.:341-346 [4] Niu Xuejuan,Liu Jingtai,Sun Lei.Robot 3D Sculpturing Based on Extracted NURBS[C],Robotics and Biomimetics, 2007.:936-1940 [5] Liu Jingtai, Chen Xinwei, etc. Three Dimensional Halftoning Algorithm Based on Surface Features [J]//. High Technology Letters, 2011.

4033