Roboscan An Automatic System For Accurate and Unattended 3d Scan
Roboscan An Automatic System For Accurate and Unattended 3d Scan
∗ ISTI-CNR, Via Moruzzi, 1 56124 PISA, Italy; email: [email protected] • many range maps (50 ∼ 500) are produced when scan-
† Scienzia Machinale, Pisa, Italy; email: [email protected] ning a complex object; planning this set of views is
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04)
0-7695-2223-8/04 $ 20.00 IEEE
not easy, since we look for the minimal set of views of the surface of scanned artifacts and to process the sam-
covering the whole object’s surface, which should sat- pled data off-line. Therefore, the attention of researchers
isfy a given minimal overlapping factor, and should be has been focused onto the issues of acquisition planning and
defined taking into account the scanner characteristics range map registration, which still remain open problems
and the orientation of the sampled surface portion; and are the crucial obstacles in the design of unattended
scanning systems. Even if a number of solutions address-
• for each view, the scanner has to be moved in the cor- ing the two problems above have been proposed, few of
responding location and orientation. Due to the weight them are feasible in practice. To our knowledge, none of
of standard commercial scanners this action is often the available scanning systems can scan accurately complex
cumbersome and time consuming; objects in a completely unattended way.
• all the acquired range maps have to be integrated in View planning. Planning algorithms can be characterised The
camera is
a single model (either triangle- or point-based); a ba- according to how much a-priori knowledge about the scene activated
continually
sic action is the mutual registration of the range maps. is available. Mainly, planners either assume that a complete
The standard solution is based on a semi-automatic model of the scene is known in advance (e.g. in the indus-
method [17], which requires an intense user contri- trial inspection of mechanical parts, where a CAD reference
bution to select the initial rough registration for each model is used to verify the compliance of manufactured ob-
scans (about 60% over the overall scanning time spent jects [23] ) or, on the other side, that no geometrical infor-
on registration). Current automatic alignment method- mation is available prior to the acquisition (as in standard
ologies (see Section 2), are not sufficiently robust. 3D scanning sessions).
In this case, the object to be scanned is assumed to lie in
Robotic arms or other type of mechanical gantries have a known bounding volume. One of the first algorithms in
been connected to scanners in the past, either with the goal this class was proposed by Connolly [6]. He partitions a
of displacing the scanner in the acquisition space or to track spherical bounding volume using an octree representation.
the scanner position [11, 9, 22]. To our knowledge, our ap- Two planning algorithms are presented. In the “planetar-
proach is innovative since it is the first case in which the ca- ium” algorithm the bounding sphere surface is sampled in
pabilities of a robotic arm are fully exploited to completely a number of candidate viewpoints. Visibility of the surface
automatise the scanning process. Its main features are: for each candidate position is evaluated and the next best
• the scanner has greater freedom thanks to the robotic view (NBV) is selected as the viewpoint maximizing the
arm and the turntable; amount of unseen nodes. The “normal” algorithm counts
the area shared by the faces of nodes which separate re-
• two basic simple solutions have been employed to ar- gions known to be empty from unseen ones. The NBV is
range an initial view planning – starting with minimal the direction maximizing the shared area. Maver and Ba-
information on the object’s shape (the bounding vol- jcsy [13] proposed a planning algorithm tailored to a light
ume extent), either a cylindrical or a planar plan can stripe range sensor constrained to translations and limited
be built. rotations in a plane above the object. Occluded regions are
represented as polygons. Viewpoint visibility constraints
• the relative positions of range maps are known, since are computed from the polygon boundaries. Whaite and
the robot movements are calibrated and accurate – Ferrie [24] developed a model-based approach in which a
ICP [4, 17] can be run without manual alignment. parametric model (superquadric) is fitted to the currently
sensed data. The shape is progressively refined minimising
• a technique is employed which determines automati- the uncertainty in the model, by scanning the region where
cally a set of views to acquire the missing range data – data fit the model worst. In Pito’s approach [15] the scan-
the coverage of the model is improved. ning volume is enclosed by a surface parameterised using
In summary, our system integrates a number of software two bi-dimensional scalar fields (positional space), encod-
techniques (some of them original) to speed-up the acquisi- ing the visibility information for each candidate viewpoint.
tion process and make unattended scanning feasible. The viewpoint which maximizes the unseen volume is cho-
sen as the NBV. Reed and Allen [18] encoded planning con-
straints as operations on sets. They consider the workspace
2 Related work as a set of 3D points. To compute viewing directions, they
apply set operators to the workspace.
The experience of people involved in some remarkable One of the most interesting algorithms for our purposes
scanning projects [11, 2, 19] has demonstrated the substan- is the one proposed by Papadopoulos-Orfanos and Schmitt
tial effort needed to obtain a sufficiently complete coverage [14]. They use a laser stripe scanner mounted on a robot
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04)
0-7695-2223-8/04 $ 20.00 IEEE
with three translational DOF. Their system also incorpo- but also more complex and redundant the set of scanned
rates a path planning algorithm for collision avoidance. The samples), and the need to adopt lower quality acquisition
acquired data is used to guide both the sensor planning and techniques to sustain a real-time throughput. Moreover, if
path planning algorithms. To our knowledge this is the the ICP algorithm fails (e.g. for fast motion of the object
first attempt to assemble a fully automatic system embody- with respect to the scanner, or due to flat regions, where
ing sensor- and path-planning, together with acquisition of range maps can slide) the registration cannot be recovered.
range data. The main drawback of this approach is that the Commercial solutions are limited to the addition of a
scanning system is not allowed to rotate around the opti- tracking system to the scanner, which produces an initial
cal centre. This constrains severely the shape of scannable placement for the data acquired (solving just the registra-
objects. Moreover, their system is intended for very lim- tion problem, and not considering view planning). Stein-
ited applications since it is too tied to a specific commercial bichler’s COMET T-Scan[22] adopts optical tracking (with
hardware, handling only small objects. the inherent line-of-sight problem), while the COMET
Scott et al. [21] worked out a multi-stage approach in which VarioZoom[22] and FARO ScanArm [9] use mechanical po-
a coarse model acquired in a first stage is used to guide sitioning systems.
scene exploration for the fine modeling of the next stage.
An a-priori base model is needed at the first stage (e.g. 3 Architecture of the RoboScan system
a bounding volume). The algorithm partitions the rough
model into cavities, holes and planar patches and for each RoboScan is a fully automatic acquisition system (see
patch a set of candidate viewpoints is generated. A user de- Figure 2), composed by a commercial triangulation-based
fined measurability function is computed for each patch of 3D laser scanner (Minolta VI 910 [10]) and a commercial
the rough model. six-axis robot arm (ABB IRB 4400 [1]). To provide an-
Automatic range maps registration. There are two main other degree of freedom, a PC-controlled turntable is used
registration strategies: the adoption of a tracking system and to rotate the object in front of the robot, enabling to scan
the use of purely software solutions. the object from every direction. This heavy-duty hardware
The scanner pose can be tracked over time and information is controlled by two computational units. The unit PC1, an
about the camera transformation between scans collected. industrial all-integrated machine, is placed directly on the
Common tracking methods employ inertial, magnetic [16], robot arm (due to the limited SCSI cable length). The unit
acoustic, optical [22] or mechanical positioning systems at- PC2 is an off-the-shelf PC connected to the robot arm, to the
tached to the scanner. Tracking systems are used mainly to turntable, and to PC1. It controls the turntable and the robot
provide initial guesses to software systems, since they are arm, sends scanning commands to PC1, and post-processes
seldom sufficiently accurate. the acquired data.
Software methods (e.g. [12]) usually adopt a two-stage ap-
proach, in which a pair-wise registration is followed by a
global optimisation stage [17]. The construction of the ad-
jacency graph (i.e., find all the range maps partially over-
lapping with the one considered) and the initialisation of
the iterative ICP [4] process (i.e. find an approximate place-
ment for a pair of range maps which allows the convergence
of ICP) are the two main issues in software approaches.
Automatic scanning systems and tracked scanners.
Rusinkiewicz et. al [20] demonstrated that an automatic ac-
quisition system can be easily built using a real-time scan-
ning device. Since multiple range maps per second can be
acquired, there is a strong coherence between pairs of con- Figure 2. Hardware components and connec-
secutive range maps. Even if the object is slowly moved in tions of the RoboScan system.
front of the acquisition device, there is still a good chance of
aligning automatically the acquired samples. Furthermore,
since the model is generated on-line, the user gets some
feedback on what is going on, even if no automatic plan- 4 Unassisted 3D scanning
ning strategy has been proposed. The main drawbacks of
this approach are: the limitation on the size and weight of The strategy implemented to support unassisted 3D scan-
the scannable objects, the large number of range maps ac- ning is based on four different steps. First, an automatic
quired (the higher the frame rate, the higher the coherence, initial coverage is performed. Then, the initial set of range
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04)
0-7695-2223-8/04 $ 20.00 IEEE
maps is automatically registered and merged, to produce centage are aligned with ICP. This pair-wise aligment be-
a first digital model. Successively, unsampled regions are tween overlapping range maps is then followed by a the
detected automatically and acquired. Finally, the user can global optimisation stage described in [17].
complete the model by driving manually the placement of After registration convergence, range maps are fused us-
the scanner, in case automatical detection still leaves un- ing a volumetric method based on implicit distance field
sampled regions. The integrated model can be simplified to construction and zero-distance isosurface fitting [7]. A
a user-requested size. medium resolution grid (1-2 millimetres) is used to produce
To start a scanning session, the user has to specify just a a model of adequate resolution for further processing (see
few data. The basic parameters are the target quality level Phase 3).
of the final digital model, and the extents and location of a
bounding volume enclosing the artifact. This latter data is 4.3 Phase 3: selection of missing patches
required by the system to define the initial set of scanning
poses.
The models generated so far are usually incomplete due
to self occlusions and tangential surface sections. A hole
4.1 Phase 1: rough shape acquisition detection phase is thus needed in order to automatically de-
tect unsampled regions and plan appropriate scanner poses.
At the beginning of the acquisition session there is no Different planning algorithms have been outlined in Sec-
knowledge about the shape of the object. For this reason, tion 2. Although the view planning literature is rich, none
the initial probing points have to be planned using only the of the proposed solutions is satisfactory for our needs. The
basic bounding volume data. The idea is to perform a sim- algorithms proposed in [14] and [13] depend heavily on the
ple and regular sampling, which is based on two different particular scanning configuration adopted. Solutions in [6]
scanning modes: planar and cylindrical. The planar mode are too naive and the algorithm in [24] is unable to handle
is mainly adopted when one side of the object has to be ac- complex models. Finally, as our system is designed for ap-
quired (e.g. artistic bas-reliefs or industrial workpieces). In plications working with huge meshes, key features of the
this scenario the turntable is not used. A simple grid of reg- planning algorithm should be time efficiency and low mem-
ularly spaced poses is generated, orthogonal to the side of ory occupancy. These requirements are not fulfilled by the
interest of the bounding volume (see Figure 3: the poses are approaches presented in [15, 18, 21].
represented by yellow dots, and the intermediate results are The idea underlying our planning strategy is to exploit
presented to the user interactively). The cylindrical mode the capabilities of the graphics hardware to detect quickly
is adopted for the acquisition of all-round objects, and uses and accurately unsampled regions. Then, opportune scan-
both the turntable and the robotic arm. For each rotation ner poses can be computed to acquire the missing data. The
step, a set of probing positions are taken at progressively model is rendered from a number of different candidate
increasing height, producing a sort of vertical strip of range scanner positions. Views which exhibit larger unsampled
maps shot from a direction orthogonal to the cylindrical sur- surface portions are selected as new acquisition viewpoints.
face. From each candidate viewpoint we render (disabling
The user can control the number and the density of the shading) the current model, using different colours for the
poses defined by means of two parameters: the shooting dis- background, the front-, and the back-side of its surface (Fig-
tance and the overlap among two consecutive range maps. ure 4(b)). When the back-surface is visible, we are looking
“through” missing data. Thus, back-coloured pixels repre-
4.2 Phase 2: automatic registration of range maps sent missing data. Hence, the number of such pixels is a
measure of the area of unsampled regions. It can happen
To finely register the acquired range maps we exploit the that holes on the front surface and holes on the back sur-
known scanner positional data. The robotic arm used in the face project on the same image pixels and no back-coloured
prototypal implementation has a very good positioning ac- pixel are imaged. Therefore we augment the model with a
curacy (error is 0.1mm), which in our experience is much back-coloured half-box enclosing the back of the acquired
more than the initial rough registration needed for the ICP surface. This simple trick allows to discriminate correctly
convergence. Hence, much less accurate (and cheaper) me- between exterior background and potential holes (see Fig-
chanical positioning systems might be employed as well, ures 4(a) and 4(b)). The planning algorithm starts by testing
without any concern on the quality of the 3D models pro- a set of candidate viewpoints located on a spherical surface,
duced. We build a spatial search structure (called occupancy centred in a point inside the user-defined bounding box. For
grid) to determine which range maps share approximately each rendered image, the percentage of back surface pix-
the same discretised portion of space. All the range maps els bpk is computed and the viewpoints are sorted accord-
pairs with an approximate overlaps greater than a given per- ingly. High bpk values suggest that portions of front sur-
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04)
0-7695-2223-8/04 $ 20.00 IEEE
Figure 3. Automatic initial coverage. On the left, the interface used to define the bounding box and
the view planning parameters. During automatic scanning, the actual position of the robot is shown
and the object appears progressively (middle and right pictures).
face are missing in the current model. Viewpoints whose erative process. High unsampled area ratios are set in the
bpk falls below a user-defined threshold are consequently first stages to generate few good viewpoints, and then the
pruned from the set of candidate viewpoints and the remain- plan is refined lowering the area gain parameter in late it-
ing ones are sorted with respect to bpk . A viewpoint vk is erations. Another critical issue is the choice of the number
accepted if the angle between vk and v j is above a fixed of candidate views. Here the tradeoff is between a fast but
threshold for each viewpoint v j that has already been vali- sparse sampling on the one hand, and a fine but slower one
dated. View selection and testing is adaptive. We start with on the other. In our experiments, we found that a good com-
an initial sparse regular sampling of the spherical surface. promise between time efficiency and quality of plans is to
Then, directions surrounding a good viewpoint are more start with 100 candidate viewpoints, and set the area gain
densely re-sampled and tested. This process can be repeated to 0.1 and the maximum overlap to 0.3. In successive iter-
a number of times, until there is no appreciable change in ations the area gain is halved at each step, while the other
the most significant section of the sorted view list, or up to parameters remain unchanged.
a maximum level of recursion.
The snapshot presented in Figure 5 shows the acquisition
This simple algorithm can be easily extended to handle
of the range map corresponding to the pose shown in Fig-
two-line-of-sight scanners. For each pose vi selected by the
ure 4(b).
procedure described above, we search for an unoccluded
line-of-sight for the laser beam. Taking into account the 3D
Our planning algorithm meets the initial requirements.
scanner geometry, new views are rendered around vi and the
Memory consumption is kept low, since we use a low res-
same thresholding operation is applied to the new frames. If
olution representation of the mesh and a copy of the frame
no good view is found then we assume that no good scanner
buffer. Processing time is low as well, since we take advan-
position exists from that view, and we discard vi . Otherwise
tage of modern GPU rendering speed.
we select the best view as the position of the laser beam.
The planning algorithm can be tuned by setting a few pa-
rameters: the minimum unsampled area ratio (area gain),
the maximum overlap between views, and the number of As the scanning process goes on, more and more empty
candidate viewpoints we are searching in a single iteration. volume is detected in the initial bounding volume. It can
In particular, a low value of area gain allows the planner to be safely be “carved out” of the current bounding volume.
detect even tiny holes, but tends to generate too many valid This allows a more accurate selection of the range maps
viewpoints. Even if this can be partly handled by tuning the to be acquired. Collisions are prevented, since a surface
maximum-overlap parameter the problem still holds. That enclosing the object is always known to the path planning
is why we have designed the view planning stage as an it- unit of the robotic arm.
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04)
0-7695-2223-8/04 $ 20.00 IEEE
(a) The same model is shown with (left) and without (right) shad-
ing. The yellow frame shows the region sampled by scanner from
a given viewpoint.
Figure 5. Automatic selection and acquisition
of missing range maps: before and after the
inclusion of a new range map (in yellow) from
a new viewing direction (red line).
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04)
0-7695-2223-8/04 $ 20.00 IEEE
Acknowledgements We acknowledge the financial support
of the projects EU IST-2001-32641 “ViHAP3D” and Re-
gione Toscana “DigiSculptor”.
References
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04)
0-7695-2223-8/04 $ 20.00 IEEE
Figure 7. The RoboScan system acquiring the “Deposed Christ" high-relief (real size: 2.5 × 1.5 metres),
and the corresponding digital model produced.
Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04)
0-7695-2223-8/04 $ 20.00 IEEE