You are on page 1of 12

Automation in Construction 31 (2013) 128139

Contents lists available at SciVerse ScienceDirect

Automation in Construction
journal homepage: www.elsevier.com/locate/autcon

Bucket trajectory classication of mining excavators


Rashi Tiwari , 1, Jeremy Knowles, George Danko
Mackay School of Earth Sciences and Engineering, University of Nevada, Reno, NV 89557, USA

a r t i c l e i n f o a b s t r a c t

Article history: Large mining excavators impose numerous challenges to the operator during a dig. A semi-automated or fully
Accepted 6 November 2012 automated machine that could assist the operator during excavation would positively affect efciency, ore se-
Available online 30 December 2012 lectivity, and reduce machine wear at the same time. Besides, the dependence on operator skills and atten-
tiveness will be reduced. In order to realize any control on an excavator, it is prudent to know the dig
Keywords:
patterns followed by the operator during a real time dig. This paper aims to classify the dig by analyzing
Mining
Excavator
the actual task performed by the operator. The trajectories are classied based on the location and angle of
Trajectory the bucket. Both 2-dimensional and 3-dimensional analyses of the bucket trajectory were performed on a
Bucket EX 3500 mining excavator at the Newmont open pit mine using Arial Performance Analysis System (APAS).
2012 Elsevier B.V. All rights reserved.

1. Introduction Maciejewski et al. [4] discussed the importance of teeth spacing in


the excavator bucket. Several experiments with 1) horizontal bucket
Automation of the large mining excavators has been gaining popularity movement at different cutting angles and, 2) vertical bucket move-
due to the intricacy of the tasks performed by the operator. A typical dig ment with 45 bucket angle, were conducted to study the soil cutting
cycle, shown in Fig. 1, consists of lling the machine's bucket from the problem using vertical rigid walls of various widths as the tool. The
muck pile (that extends vertically above the oor), hoisting and swinging research concluded that the bucket without teeth was more efcient
the load over a haul truck, dumping the load, and returning to the digging than that with teeth, with respect to energy consumption. Similarly,
front. Despite the straightforwardness of the steps, they require coordina- Plonecki et al. [5] observed that when the end of the tool was moved
tion and attentiveness of the operator. In addition, the working environ- close to the slip lines in the pile, the specic energy of the earth lling
ment in a mine changes continuously with the production requiring for process was less due to reduction in the material cohesion. Hall and
continuous adjustment of the control. The control equipment is required McAree [6] compared two digging styles and their implications on the
to be tolerant in terms of precision and climate. machine. The two styles recognized in the paper based on the 4 differ-
The energy consumed during each dig cycle is dependent on the ent operators are altus, where the bucket tip moves through a trajectory
force and the trajectory. Due to the repetitive nature of the digging well above the oor and brevis where the bucket tip enters at the toe of
cycle, determining an optimal trajectory can lead to signicant sav- the much pile and progresses along the oor before lifting into the ma-
ings in energy. Singh et al. [1] developed a simulation kit to plan the terial. They concluded that the altus style results in lower loading on the
excavation task. The accuracy of the simulation was restricted by machine. Bisse et al. [7] described four loading strategies dened by tool
the fact that the actual mine environment is not static and is also differ- trajectory and bucket motion. The results presented were based on
ent among different mines. Himami [2] proposed a general approach for computer simulation.
determining a trajectory, associated with minimum energy consump- From most of the previous research it is evident that the trajectory
tion, based on a path experiencing minimum resistive forces. The planning may need to be made by the operator on the y, making a
presented method was tested for automating the scooping and the built-in trajectory planner meaningless. A solution to the problem
loading operation of a loader type machine at mines. Likewise, Lever was proposed by Danko [8], applying a differential control architec-
et al. [3] described a few bucket actions with one set of cylinders con- ture that allowed for continuous adjustments of a family of desired
trolling bucket translation in a vertical plane (lift), and a second cylinder trajectories by the operator.
set rotating the bucket in the vertical plane (tilt) for a particular orien-
tation. When the muck pile was difcult to dig, the operator then lifted 2. Machine automation
the bucket to penetrate into the soil. He mentioned that for a very dif-
cult pile, the bucket gains very little in load with each forward action. Machine automation has been widely used due to energy benets.
Better control improves machine efciency while reducing stress on
Corresponding author. Tel.: +1 775 247 8688.
the operator. This leads to energy savings and a safer working envi-
E-mail address: rashi.tiwari@asme.org (R. Tiwari). ronment. A review of the fundamental control processes in excava-
1
Currently at Dow Chemical, Core R&D, Midland MI. tion was presented by Hemami [9].

0926-5805/$ see front matter 2012 Elsevier B.V. All rights reserved.
http://dx.doi.org/10.1016/j.autcon.2012.11.006
R. Tiwari et al. / Automation in Construction 31 (2013) 128139 129

The automation of an excavator can be divided into different cat- Cartesian coordinate system. A time delay control with switching action
egories. The rst category consists of tele-operation, in which the op- with integral sliding surface was discussed by Lee and Chang [15]. The
erator is physically removed from the machine, but is still in charge of controller was designed for a straight-line motion of a 21-ton heavy-
controlling the machine to follow the required trajectory. Thus the duty excavator. Swing velocity tracking control was achieved through
operator controls the machine from a remote place via hand control- adaptive robust control by Yao et al. [16]. Non-linearity in the system
lers (joysticks, or steering wheel). However, if certain phases of the was considered in the design and analysis of the controller through
operation are automated then it is also possible that the operator simulation.
may control more than one machine [10]. In order to facilitate the development of a set of useful software
Supervisory control [11] is the second category of machine automa- kinematics for mining equipment, it was necessary to evaluate and
tion. Here the operator performs main decision making tasks while the classify actual machine trajectories used during mine excavation.
robot is used for repetitive operations. Plonecki et al. [5] discussed auto- The following questions are raised in order to derive suitable param-
mation of heavy machines to assist the machine operator. They separat- eters for the software-dened kinematics for machine automation:
ed the tasks between robot and operator with the operator having to
play the main decision and control role. Pre-programmed, predictable 1. What are the motion characteristics of the bucket during digging
operations were utilized under standard working conditions for realiza- and loading under different conditions?
tion of tool motions that otherwise were difcult to control manually or 2. What is the similarity between consecutive digging and loading
to correct the trajectory dened by the operator for optimal control. On cycles?
similar lines, Alami et al. [12] proposed control architecture for mobile 3. What types of adjustments, to the machine motion, are needed
robots, which reportedly integrated human supervisory direction from one cycle to another?
into an automated robotic machine. According to their architecture, 4. What internal relationships may exist between motion parameters
a human supervisor observed the motion and position of an end ef- such as bucket position, bucket angle, and movement velocities?
fector, and provided mission guidance to a trajectory planner. The
trajectory planner selected a software control protocol correspond- For answers to these questions, it was necessary to evaluate and
ing with the mission guidance instruction and provided power and classify the semi-repetitive machine motion patterns observed during
motion control to the machine actuators (e.g. hydraulic pistons and typical mining operations. Of particular interest is the trajectory of the
cylinders), to execute the mission guidance instruction. The software bucket and its relationship to the coordination of multiple machine
control protocol iterated every step needed, including every actuator links. Trajectory evaluation was accomplished using a multi-camera
adjustment, for the end effector to execute the desired trajectory. computer vision method aligned with a model based image analysis
Complete automation is the third category of automation. Here system [17].
the operator is eliminated from the control loop altogether. An adap-
tive control strategy with real-time self-tuning ability was studied by 3. Mine data collection
Chiang et al. [13]. A new hydraulic valve system was mounted on a
large excavator, and a PID controller was utilized for valve control. Mine data were collected in cooperation with the Lone Tree Mine,
State space control along with MRAS (model reference adaptive sys- Nevada. In order to develop effective video data collection methods, a
tem) was employed for the desired path tracking. Lee et al. [14] doc- number of video samples were collected and transferred to University
umented the control of a heavy-duty excavator under working speed of Nevada Reno (UNR) for processing. As illustrated in Fig. 2, three
conditions. A time delay controller was designed based on a dynamic video cameras were used to record the motion of several pieces of
model of the machine to control the movement of the bucket tip in a mining equipment, during normal operation at an open pit mine.

a) b)

d) c)

Fig. 1. Loading cycle consisting of (a) bucket placement at the digging front, (b) lling the bucket, (c) hoisting the load over the loading truck and (d) dumping the load.
130 R. Tiwari et al. / Automation in Construction 31 (2013) 128139

3.1. Marker tracking using Ariel performance analysis system

The Ariel Performance Analysis System (APAS) [19] is capable of


performing marker tracking. The system analyzed machine move-
ment characteristics providing a means to quantify motion using
multiple cameras. 2D analysis was performed with a single camera
and required 4 known reference points for accurate results. For 3D
analysis, the APAS system required that the motion be simultaneous-
ly recorded from at least 2 camera angles. In order to provide accurate
3D tracking of markers, the 3D coordinates of at least 6 known reference
points must be entered for each camera view, prior to analysis.
Multiple camera views (see Fig. 3) must be synchronized prior to 3D
analysis. Accurate camera synchronization was performed using a syn-
Fig. 2. Exemplary camera arrangement around a mining shovel in operation.
chronization sound, which was recorded by all video cameras at the be-
ginning of data collection. The APAS software compared the audio
High-contrast markers (approx. 30 cm black circles on white back- tracks of all camera recordings and aligned the video data based on
grounds) were painted on several pieces of equipment at an open-pit this synchronization sound.
mine including two Hitachi EX3500 front shovels, as shown in Fig. 3.
Equipment marking was performed during scheduled equipment main- 3.1.1. Preliminary marker tracking test
tenance periods, in cooperation with the Maintenance Department at A simple 2D marker-tracking test was performed in the laboratory
the Lone Tree Mine. The three-dimensional location of each marker using a single marker consisting of a 3/4-inch diameter black dot on a
was measured with respect to a reference coordinate system assigned white paper (see Fig. 4(a)). This marker was moved by hand in view
to each machine link. Appendix A provides a detailed marker coordinate of a single camera, and the resulting video sequence was processed
gures for EX 3500. The measured marker locations were used along with APAS in 2D. The auto marker tracking function of the APAS system
with data obtained from the manufacturer's data sheet for the EX3500 provided good results in this test (see Fig. 4(b)). The units in Fig. 4(b)
machine [18], in order to generate location of markers in the machine are arbitrary, as the software was not calibrated for the real dimensions.
coordinate for trajectory evaluation. Additional marker tracking tests were performed with the small
In order to facilitate the use of the marker tracking software Bobcat 225 excavator in the laboratory. The purpose of these tests
discussed in the next section, it was required to synchronize the was to determine the best method of marking the machines, as well
videos collected from the different cameras. Accurate recording of as the best size for the markers. Several marking methods were test-
the synchronization sound for multiple camera views was assisted ed, including using double-coated adhesive tape to attach paper or
by the use of small hand-held radios placed near each video camera. plastic markers to the machine, and painting markers directly on
At the beginning of each data set, a loud synchronization sound was the machine surface. Variables such as marker durability, contrast,
broadcast simultaneously to these radios from another radio used by and marker application time were weighed. In the majority of cases,
the camera operator. This eliminated potential synchronization er- the most convenient and effective method was found to be directly
rors due to differences in sound travel time from the source to each painting the markers on the equipment surface.
camera, and helped to ensure that the sound level was similar at all Initially, 11.5 inch markers were used for marker tracking tests on
camera locations, regardless of their distance from the sound source. the Bobcat 225 excavator in the laboratory. However, the marker size

Fig. 3. Mine data video from three cameras around shovel EX 3500.
R. Tiwari et al. / Automation in Construction 31 (2013) 128139 131

Fig. 4. (a) Video clip frame of hand-held marker during laboratory experiment and (b) 2D trajectory of hand-moved marker-tracking point (tracked using APAS in image
coordinate).

Fig. 5. APAS marker tracking test on Bobcat 225 machine showing initial and nal machine positions with marker trajectory overlay in image coordinate (inset: for close-up of
boxed region).

was subsequently increased to about 4 in. to improve tracking. Fig. 5 marker trajectories (red) are shown in the gure. These trajectories
shows the preliminary results of one marker-tracking test using 4-inch closely followed the motion of the two markers on the excavator arm,
markers on the Bobcat 225 excavator. The last frame of the video is as the arm was lifted up.
shown superimposed over the rst frame, so that the starting and end- Fig. 5 inset shows a close-up view of marker B along with the prelim-
ing positions of the machine are visible on the same plot. Two selected inary tracking data for this marker point. The error between the plotted

Fig. 6. Control points (left) and tracking point (right) along with the xed point (inside square).
132 R. Tiwari et al. / Automation in Construction 31 (2013) 128139

Fig. 7. Orthographic view of the tracking point (left) and nal tracked trajectory of the markers (right).

a) b)
0 0

100 100

200 200

300 300

400 400

0 100 200 300 400 500 600 700 0 100 200 300 400 500 600 700

Fig. 8. Marker point trajectories plotted in camera coordinates on initial video frame; (a) Type 2-excavation example, (b) Type 4-excavation example. The axes are in image coor-
dinate and can be converted to actual trajectory measurement using the measurements in Appendix A.

marker trajectory and the actual center of the marker is attributed to points were selected so that they cover a large part of the frame in
the fact that pixels near the edge of the black marker appear in the order to maximize the accuracy of the APAS results. A variety of
video to be darker than those in its center. This preliminary analysis video clips were selected for initial processing, spanning various dig-
did not account for errors due to slight variations in apparent marker ging conditions and digging patterns. This representative sample
color. However, additional image processing techniques are expected was evaluated using the APAS software. The tracked markers were
to eliminate this problem. viewed in one of the APAS module (see Fig. 7).
Trajectories were processed by using APAS for marker tracking and
then feeding the output 2D marker trajectories (in image coordinates),
3.1.2. 2D marker tracking into a newly developed multi-frame version of the model-based tting
The APAS software was used to track the markers on the shovels. Ini- algorithm [17]. The output of the multi-frame tting algorithm was a 3D
tial analysis was carried out using one camera view for each video clip bucket trajectory, with its associated bucket angle curve. The trajectory
with ve control points and four motion tracking marker points. Fig. 6 was plotted over the initial frame of the video for visual conrmation of
shows the 4 points used for tracking and one xed point in the back- the t results (Fig. 8).
ground. MATLAB was used to provide the coordinates of the initial
control points (from the rst frame of the video) for APAS in image
coordinate. Since the y-axis of MATLAB and APAS are opposite to 3.1.2.1. Trajectory classication. Analysis of the bucket digging trajec-
each other, it was required to change the sign of the y coordinates be- tories observed in the videos for the Hitachi EX3500 front shovels
fore feeding it to APAS. showed four major classes of bucket trajectories. These classications
The initial set of marker points was selected to facilitate a focused were based on the initial bucket angle, and on whether the bucket
evaluation of the bucket trajectories, since bucket trajectory data are was initially placed at the base of the pile, or midway up the pile. Ex-
likely the most important for development of software-based ma- amples of each type of trajectory, along with the associated bucket
chine kinematics which improves digging efciency. The control angle (see denition in Fig. 9) plotted as a function of horizontal
bucket displacement, are shown in Figs. 1013. The initial position
of the bucket corresponds to zero displacement of the bucket in the
plots. Table 1 summarizes the four main trajectory types observed
in the videos.
3.1.2.1.1. Type 1 trajectory. This trajectory starts from the base of
the pile with a negative bucket angle, as shown in Fig. 10. The initial
position of the bucket is close to the machine body. The operator
rst rolls the bucket to a larger (but still negative) bucket angle, ei-
ther immediately, or with a slight forward bucket motion. After
x
rolling the bucket, the operator follows a trajectory similar to the
Type 2 classication below. The distinctive feature of the Type 1
class is a sharp increase in bucket angle near the beginning of the tra-
Fig. 9. Denition of bucket angle with respect to the horizontal. jectory, whereas the Type 2 bucket angle increases smoothly.
R. Tiwari et al. / Automation in Construction 31 (2013) 128139 133

Fig. 10. Examples of Type 1 trajectory shapes (top) and corresponding bucket angles Fig. 12. Examples of Type 3 trajectory shapes (top) and corresponding bucket angles
(bottom). (bottom).

Fig. 11. Examples of Type 2 trajectory shapes (top) and corresponding bucket angles Fig. 13. Examples of Type 4 trajectory shapes (top) and corresponding bucket angles
(bottom). (bottom).
134 R. Tiwari et al. / Automation in Construction 31 (2013) 128139

Fig. 14. Three-camera view showing control points for 3D analysis of a loading cycle using APAS.

3.1.2.1.3. Type 3 trajectory. The bucket starts at the base of the pile
and is initially moved nearly horizontally as in Type 2. However, the
Type 3 trajectory rapidly curves upward as the bucket is quickly lled,
as shown in Fig. 12. The main difference from Type 2 is that the initial
bucket angle is nearly zero, and increases dramatically as the maxi-
mum bucket capacity is approached.

Fig. 15. Example of locating invisible control point in a camera view.

3.1.2.1.2. Type 2 trajectory. The bucket starts at the base of the pile
and is rst pushed forward nearly horizontally (effectively smoothing
the previously excavated area just in front of the pile). The bucket
edge trajectory then gradually curves upward, and becomes approxi-
mately parallel to the slope face. The bucket is then lifted out of the
pile and (almost simultaneously) begins to swing toward a nearby
haul truck where it is to be emptied. Type 2 trajectories are character-
ized by an initial bucket angle, which is signicantly negative. The
bucket angle increases approximately linearly, but remains negative
for up to 50% of the arc length of the trajectory (Fig. 11).

Table 1
Summary of observed excavation trajectory types.

Initial bucket angle Starting Base of pile Middle of pile


position
Tight space Normal space

Signicantly negative Type 1 Type 2


Fig. 16. Sample bucket trajectories calculated using APAS viewed from above the
Near zero Type 3 Type 4
shovel.
R. Tiwari et al. / Automation in Construction 31 (2013) 128139 135

Valve Actuator Angular d = d t


voltage velocity, u u velocity, (change in
(from valve (from angular
calibration actuator position)
curves) kinematics)
d
1 old
unit delay new = old + d
z

Graphical Machine Sensor


new Voltage
Model (Displays Sensor
current position) Simulation

Fig. 17. Sample bucket trajectories calculated using APAS viewed from the side of the Fig. 19. Block diagram for Machine Simulation.
shovel.

3.1.2.1.4. Type 4 trajectory. The bucket starts midway up the pile, the presence of insurmountable obstacles, such as large boulders
and moves in an approximately linear trajectory, which is nearly par- hidden beneath the pile, or due to misjudgment of bucket ll
allel to the slope face, as shown in Fig. 13. The bucket is then lifted rate. Operator redirection can take a variety of forms, including
and moved toward the haul truck as described in Type 2. The bucket sudden lifting or lowering of the bucket, large, sudden changes
angle begins near zero, and increases semi-linearly for most of the in bucket angle, or even temporary reversal of bucket motion.
trajectory with a somewhat sharper increase typically near the end. 3) Trajectory scaling proportional or non-proportional changes in
Four major classes of front shovel trajectories were observed in this overall trajectory dimensions. Proportional scaling does not affect
research. While it is possible that additional types may be found in the the aspect ratio of the trajectory, but changes only its overall size
future, it is believed that the majority of digging trajectories can be to match the size of the pile. Non-proportional scaling allows for
placed into a relatively small number of such classes. Each class can be variations in the slope angle of the pile, by allowing the trajectory
described in terms of a parameterized general trajectory shape, and a to be lengthened in only one direction.
set of possible deviations from the general shape. Distinguishing be-
tween the general shape of a trajectory, and the deviations from that It is assumed that the last two categories of deviation are best
general shape, facilitates the integration of human and robotic control. controlled by the operator, while the general trajectory can be easily
Computerized robotic control is best suited for performance of the gen- performed robotically through the use of pre-programmed, parame-
eral task, while the human operator is much more capable of evaluating terized trajectory shapes, realized by a software-controlled machine
the need for some types of deviations, due to rapidly changing environ- kinematics.
mental factors such as variations in diggability. In addition, it is assumed that there must be a well-dened rela-
Several kinds of deviations that may be commonly observed: tionship between the direction of general trajectory motion and the
bucket angle. Clearly, such a relationship must exist to facilitate ef-
1) Reactive path variations small, transient changes in bucket direc- cient loading of the bucket, and indeed, the data suggest that there
tion due to resistance produced by non-uniform density of mate- is such a relationship. Both the existence of this relationship and its
rial. These are produced by reaction forces exerted on the bucket importance for efciency suggest that the bucket angle might best
by the material, and are facilitated by elastic deformation of machine
links, shaking or bouncing of the machine's undercarriage, and other
similar phenomena. Reactive variations generally appear as noise
on the trajectory plot and do not signicantly affect the overall tra-
jectory shape. Joystick Sensor
Voltage
2) Operator redirection moderate to large changes in bucket direc- Voltage Sensor Machine
Joysticks Signals Calibration (or Machine
tion made by the operator. This type of deviation may occur due to
(or Joystick Simulation)
Simulation)
Measured
Link Angles Valve
(1, 2, 3) voltage
Current desired
Valve
machine angles Kinematics
characteristics
1h = 1hold + d1h Transformation
2h = 2hold + d2h (Real and virtual
3h = 3hold + d3h machine models)

Differential angles
(d1h, d2h, d3h) Desired Actuator
Velocities (u1, u2, u3)

Fig. 18. Sample bucket trajectories calculated using APAS viewed from in front of the
shovel. Fig. 20. Block diagram for Kinematics Transformation.
136 R. Tiwari et al. / Automation in Construction 31 (2013) 128139

Fig. 21. (a) Bucket pin motion control. Motion in Cartesian coordinates at 45 and (b) 0 slope angles.

Fig. 22. Open-loop bucket pin control with improved machine calibration. Cartesian coordinates at (a) 45 and (b) 0 slope angles.

Fig. 23. Motion in Cartesian coordinates at 45 slope angle. Bucket pin motion control without angle stabilization (a) with feedback control and (b) with open-loop control. Three
consecutive cycles are shown.
R. Tiwari et al. / Automation in Construction 31 (2013) 128139 137

be controlled by the robotic portion of the system, allowing the oper- control algorithms in an attempt to further improve the machine per-
ator even more freedom to concentrate on controlling the deviations formance. Recent test results are shown in Fig. 23.
from the general trajectory. Optimization of the relationship between
bucket motion direction and bucket angle is seen to be important for
5. Conclusion
reducing the digging force and related power consumption.

In mining and related development, construction, and restora-


3.1.3. 3D marker tracking
tion, the required exibility of the operation justies the full-time at-
Bucket trajectories represent the loading of one truck by shovel
tention of an operator assigned to a machine. Hence, the operation is
#112 with seven complete digging cycles. The coordinate system for
mostly dependent on the operator skill, visibility and complexity of
these trajectories was constructed with the x-axis parallel to the base
the task itself.
of the shovel (the base does not rotate during this analysis), the y-axis
A multi-camera, 2D and 3D analysis of mining data was performed.
perpendicular to the base of the shovel and the z-axis orthogonal to
Bucket trajectories were classied based on angle of the bucket and
the x and y axis; the point (0,0,0) is located at the center of the
bucket location in the digging pile. The classication and the technique
base-cab turntable. For 3D analysis of a loading cycle 10 control points
discussed in this paper can be applied for different machine automation
were utilized so as to cover a wide area of the image from all the
categories: tele-operation, semi-automation and complete automation.
three cameras as shown in Fig. 14. Since not all the control points fall
The trajectory classication scheme helps us in understanding how an
into the eld of view for each camera at the same time, lines were
operator performs a dig and how the bucket position is changed for
drawn from the visible control points parallel to the machine frame
each digging operation. Using the discussed bucket trajectory for
for determining the location of the invisible control points in order to fa-
machine automation offers the following benet: (1) improved ef-
cilitate the digitizing process. Fig. 15 illustrate this procedure for one of
ciency and energy savings, (2) reduction in time per loading cycle,
the camera views. The projections of these trajectories in the xy (view-
(3) digging cycle less tiresome when a large number of similar (but
ing from above the shovel), xz (viewing from the side of the shovel)
never quite exactly repeated) cycles are to be executed during a shift,
and yz (viewing from in front of the shovel) planes are shown in
and (3) improved precision in terms of the quality of work, contributing
Figs. 16, 17 and 18, respectively.
to a net savings in the gross output of the mining operation, representing
From these results, it can be seen that the path of the bucket changed
a proportional savings in energy and the environment.
in a fairly systematic and predictable way as the digging face advanced.
The results were applied to the developed efciency-enhancing man
Figs. 16 to 18 show, for each successive bucket load, the bucket was po-
machine control algorithms. Renement of the software-kinematics
sitioned just next to the previous digging location and returned to the
based on mine data results will be done.
same location above the truck. Aside from this systematic change, the
bucket trajectories remained very similar with little change in either
the digging cut or the swing. Acknowledgments
These trajectories were for a single point on the bucket, however,
multiple points on the bucket have been tracked using APAS and may The nancial support from the Mining Industry of the Future pro-
be used to provide additional information on bucket angle as well as gram of the U.S. Department of Energy, under grant number DE-FC26-
position during these trajectories. 04NT42087, is greatly appreciated. Co-support from the Bobcat Company
and the Newmont Mining Corporation is also gratefully acknowledged.
4. Parametric trajectory testing

In order to test the feasibility of controlling the machine bucket to Appendix A. EX 3500 shovel marker coordinates
follow the above trajectory, a control algorithm described briey in
Appendix A and described in detail in Reference [8] was used. Paramet- (All dimensions in meters unless otherwise specied)
ric trajectory testing was performed in both simulation platform devel- Track
oped in MATLAB as well as on an experimental Bobcat 435 excavator.
Simulation provided a safe environment in which to test, debug, and
demonstrate the feasibility of new algorithms, to control the bucket
trajectory.
Block diagrams of the machine simulation and the kinematics-
transforming differential control algorithm are shown in Figs. 19 and
20, respectively.
Linear trajectories can be generated at any angle by rotating the x
yz Cartesian system around a horizontal axis. Examples of the simu-
lation output are shown in Fig. 21.

4.1. Parametric trajectory machine tests


Fig. A.1. Marker positions on the track of the machine.
After testing and debugging via the computer simulation, the Carte-
sian kinematics transformation and differential control algorithms were
implemented on the real machine. The machine was moved in its
computer-controlled kinematics mode at different slope angles. Fig. 22
show actual bucket pin trajectories measured during machine tests
Marker Serial number #103 Serial number #112
using APAS. These tests were performed with open-loop control, using
the revolute-to-Cartesian kinematics transformation described in Fig. 20 x y z x y z
to produce coordinated linear motion in Cartesian coordinates. A 3.353 3.200 0 3.353 3.200 0
Recent machine tests with closed-loop control showed improve- B 2.273 3.200 0 2.273 3.200 0
ment in performance compared with earlier tests performed with C 3.353 3.200 0 3.353 3.200 0
D 2.273 3.200 0 2.273 3.200 0
open-loop control. Work is ongoing to develop and test additional
138 R. Tiwari et al. / Automation in Construction 31 (2013) 128139

Markers A and B are on the right side of the machine while Marker Serial number #103 Serial number #112
markers C and D are on the left side of the machine.
x y z x y z
Side of the cab
A 5.893 2.4896 0 5.918 2.413 2.375
B 5.893 2.4896 0 5.906 2.4892 2.324
C 6.464 0 0 6.464 0 1.791
D 6.636 0.0254 0 6.464 0 0.2413

Markers A and B are on the right side of the machine while


markers C and D are on the left side of the machine.
Proposed control architecture
Typical mining shovel operations are semi-repetitive and monoto-
nous and may require a nely pre-programmed machine, and simpli-
ed interactions with the supervisory operator. The chapter describes
a new control architecture based on differentials, which incorporates
operator adjustments into the computer controlled robotic system.
Fig. A.2. Marker positions on the side of the cab the machine. The control architecture is based on determining motion velocities
(and not position). Depending on the operator input, the machine
based on the new control can function as a fully robotized system
with no operator input or, as a fully manual machine under complete
operator control.
Fig. A.4 illustrates the concept of continuous differential control with
Marker Serial number #103 Serial number #112 sensor input. The operator selects a software controlled machine kine-
x y z x y z matics consisting of a family of pre-dened motion trajectories rather
than a specic trajectory for a given task. For the control description,
A 5.588 3.404 0.127 5.639 3.404 0.1905
B 5.055 3.404 2.362 4.890 3.404 2.273 it is assumed that a desired trajectory is given as a three-dimensional
C 1.422 3.404 2.311 1.575 3.404 2.273 digging path of the tip of the bucket.
D 5.613 3.404 0.127 5.652 3.404 0.1905 The operator also sends the main control signal in the form of veloc-
E 4.902 3.404 2.286 4.902 3.404 2.235 ity, through which the tool position to perform a given task is manipulat-
F 1.143 3.404 2.311 1.575 3.404 2.273
G 0.8636 0 1.8796 0.8636 0 1.8796
ed. The resultant movement of the machine and the tool depends on
H 5.207 3.327 5.055 both the operator input and pre-dened signals from a Differential Tra-
I 1.575 3.327 5.055 jectory Generator, which represents a desired or a typical trajectory
J 5.207 3.327 5.055 shape, available to the operator by selection from a suite. A main, Carte-
K 1.575 3.327 5.055
sian (real-world) position control loop is applied through visual feedback
to the supervisory operator, for correction and/or nal adjustment. The
operator's velocity control components are effectively re-distributed
into machine joint velocity components according to a selected and/or
Markers A, B, C, H, I and G are on the right side of the machine
predetermined and/or optimized trajectory characteristic. Therefore,
while D, E, F, J and K are on the left side of the machine.
the desired trajectory is given in the form of a differential relationship
Back of the cab
between partial motion or velocity components either in joint or Carte-
sian (real-world) coordinates.

Table A.1
Notations used in Fig. A.4: differential trajectory generation with coordinate
transformation.
h i
f i
s
s : The pre-dened trajectory feature that is always available to the supervisor
to copy into the motion. This term is a function of the movement along
the trajectory through s. The super-operator can re-wind s and start at the
beginning for a new motion.
v(t): The real-time execution velocity along the pre-dened trajectory, con-
trolled by the operator. If v = 0, the machine becomes fully manual.
J: Machine kinematics Jacobian matrix.
Jc1 : Coordinate transformation feature for manual correction or fully manual
control. Several Jc1 choices can be provided for selection. If Jc1 is the unit
matrix, the machine is controlled directly in the joint space, acting like a
manual machine.
 c
x_ i : Correction or manual motion control velocities guided by the operator.
Fig. A.3. Marker positions on the back of the cab the machine.
R. Tiwari et al. / Automation in Construction 31 (2013) 128139 139

[2] H. Hemami, Modeling, analysis and preliminary studies for automatic scooping,
Journal of Advanced Robotics, Japan 8 (5) (1994) 511529.
[3] X.D. Huang, L.E. Bernold, Control model for robot backhoe excavation and obsta-
cle handling, in: Proc. Robotics for Challenging Environments, 1994, pp. 97105.
[4] J. Maciejewski, A. Jarzebowski, W. Trampczynski, On the efciency of soil digging
process, Engineering Transactions 49 (4) (2001) 599621.
[5] L. Ponecki, W. Trampczynski, J. Cendrowicz, A conception of the assisting system
for the hydraulic excavator operator, Automation in Construction 7 (5) (1998)
401411.
[6] A.S. Hall, P.R. McAree, A study of the interaction between operator style and ma-
chine capability for a hydraulic mining excavator, Proceedings of the Institution
of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 219
(2005) 477.
[7] E. Bisse, A. Hemami, E.K. Boukas, A comparison of the required energy in loading for
four scooping strategies, in: Proceedings the Third International Symposium on Mine
Mechanization and Automation, Golden, Colorado, June 1995, pp. 2-172-28.
[8] G. Danko, Operator control architecture, in: Proceedings of the 4th Regional Sym-
posium on Computer Applications in the Mineral Industries, Tampere, Finland,
2001, pp. 253266.
[9] A. Himami, Fundamental analysis of automatic excavation, Journal of Aerospace
Engineering 8 (4) (1995) 175179.
[10] L.J. Mirabelli, Mining automation program, in: NAANSA/USGS, Automation Con-
ference, 2000.
[11] T.B. Sheridan, Telerobotics, Automation, and Human Supervisory Control, MIT
Press, 1992.
[12] R. Alami, R. Chatila, S. Fleury, M. Ghallab, F. Ingrand, An architecture for autono-
my, International Journal of Robotics Research 17 (4) (1998) 315337.
[13] M.H. Chiang, C.C. Huang, Experimental implementation of complex path tracking
control for large robotic hydraulic excavator, International Journal of Advanced
Manufacturing Technology 23 (2004) 126132.
[14] S.-J. Lee, P.-H. Chang, Y.-M. Kwon, An experimental study on Cartesian tracking
control of automated excavator system using TDC-based robust control design,
in: Proc. of the American Control Conference, 1999, pp. 31803185.
[15] S.-U. Lee, P.-H. Chang, Control of a heavy-duty robotics excavator using time delay
control with switching action with integral sliding surface, in: Proc. of IEEE Inter-
national conference on Robotics and Automation, 2001, pp. 39553960.
[16] B. Yao, J. Zhang, D. Koehler, J. Litherland, High performance swing velocity-tracking
control of hydraulic excavator, in: Proc. of the American Control Conference, 1998,
pp. 818822.
[17] G. Danko, J. Knowles, Rashi Tiwari, Model based camera vision evaluation of ma-
chine motion, in: Proceedings Application of Computer and Operational Research
in Mining Industry, 2005, pp. 513520.
[18] http://www.hitachi-c-m.com/au/products/excavator/face/index.html.
[19] Ariel Dynamics Inc., Ariel performance analysis system, http://www.arielnet.com/
start/apas/default.html 2004.

Fig. A.4. Continuous differential control architecture with workspace sensor input.

References
[1] S. Singh, R. Simmons, Task planning for robotic excavation, in: IEEE/RSJ Interna-
tional Conference on Intelligent Robots and Systems, 1992, pp. 12841291.

You might also like