AUTOMATION IN VIDEOGRAMMETRY

Giuseppe Ganci and Harry Handley

Geodetic Services, Inc.
1511 S. Riverview Drive
Melbourne, Florida USA
E-mail: [email protected]

Commission V, Working Group V/1

KEY WORDS: Industrial Measurement, Intelligent Cameras, Automation, Videogrammetry

ABSTRACT

In the last few years the automation of digital industrial photogrammetric systems has increased dramatically. Due to advancements
in digital image processing software, coded targets and auto-correlating methods, a large number of photogrammetric measurement
tasks can now be fully automated. In many cases a "one button click" is enough to provide 3D-coordinates of measurement points
without any manual interaction, as soon as digital images are acquired. The evolving technology of intelligent cameras is the next
logical step towards automated photogrammetric measurements. An intelligent camera containing an integrated computer can
process the image immediately after it is taken. The technology provides not only a much shorter processing time for the images but
also more control over the measurement process precisely at that time when it is needed most, namely during image acquisition. This
is taking place in the form of real time feedback from the camera itself. This paper describes the role of a digital intelligent camera in
the automation of an industrial photogrammetric measurement system and gives an overview of existing automation techniques in
industrial photogrammetry. As an example of an intelligent camera, the performance of the new digital intelligent camera INCA,
developed and manufactured by Geodetic Services Inc. will be described through reference to a number of example measurement
applications.

film-based systems whilst challenging other metrology systems.

1. INTRODUCTION The push to replace film systems has been largely successful
with all but a few traditional high accuracy users still using film
Close range digital photogrammetry, which is also termed systems. It is interesting to note that many of these users are
vision metrology or videogrammetry, has been successfully using the video systems to provide initial “drive-back” for their
applied in a variety of metrology applications for a significant film-based counterparts.
period of time. The principles of such systems are well known
(Fraser and Shortis, 1995) and they have gained widespread As with most technological developments the initial
acceptance for industrial measurement applications (Ganci and development drive has come from the academic and industry
Shortis, 1995a;b). research sectors where the measurement accuracy envelope is
continually being stretched. However, users of vision metrology
Initially digital photogrammetric equipment and techniques systems, who are usually relatively untrained in the technology
were not sophisticated enough to match the accuracy attainable represent the largest sector of the close range videogrammetry
through the use of other metrology systems such as theodolites, market. Meeting the practical commercial needs of these users
laser trackers and more traditional film-based photogrammetry is the responsibility of vision metrology system manufacturers
systems (Shortis, and Fraser, C. S. 1991). Fortunately and vendors, whose primary objective is to develop the
developments in large area CCD sensors and target image available technology to a point where it can be offered as a fast,
location algorithms have pushed the accuracy attainable using robust, accurate and ultimately useful off-the-shelf measurement
digital photogrammetry to a level where it can comfortably tool. The transition between the laboratory and industrial work
challenge these systems. (Gustafson and Handley, 1992) (Ganci place is often difficult due not only to the change in
and Shortis, 1996). In a modern vision metrology system for environment, but also to the transition from highly trained to
industrial measurement, object space positional accuracies relatively untrained operations personnel. The ultimate goal of
surpassing 1:100,000 of the principal dimension of the object system automation is to make videogrammetry as simple as
are now routinely attainable with large-area CCD cameras and possible for largely untrained practitioners.
photogrammetric data processing (Fraser et al., 1995). High-
resolution digital cameras, such as the Kodak MegaPlus 6.3 The pace of technological change is relentless. Often due to
with a sensor array of 3000 x 2000 pixels, are readily available considerable product development costs and risk, it is difficult
and represent a significant improvement over earlier generations to simply supplant existing product components with new and
of CCD cameras. improved components. Within a research environment it is
obviously far easier to embrace technological advances. For
The general thrust in industry has been to develop sensor example, replacing a sensor in a commercial system may take 6-
technology and computing facilities capable of supplanting 12 months due to the potential commercial repercussions
associated with such a change. Manufacturers also face the The second class of camera system has self-contained data
challenge of finding the middle ground between rapid full-scale storage (usually in the form of a removable PC card hard disk)
development and proven product reliability. and is appropriate for conventional off-line use. In this mode of
operation images are taken from different viewing angles of the
It is obvious that videogrammetry system developers would object being measured. As they are taken, the images are stored
choose to adopt the goal of manufacturing a system that is as on the camera's removable hard disk. When the photography is
automated as possible. The reduction of processing time and complete, the removable hard disk is transferred to a computer
significant simplification of the user interface are two of the system for processing. The Kodak DCS420 and DCS460 are
main advantages of automation. Another less obvious perhaps the most popular examples of this type of still video
advantage is the effect automation has in widening the potential camera. The disadvantages of such systems are that they are not
user base of videogrammetry. Automating many of the suitable for on-line work and lack the structural integrity to
operational aspects significantly reduces the requirement for the absolutely ensure calibration stability between images (Shortis
system operator to possess photogrammetric expertise. The and Ganci, 1997).
videogrammetry user base will thus increase as the level of
system automation increases. Although it is true that some of these off-line camera systems
also have on-line connection capability, this is usually in the
The automated videogrammetry process should commence with form of a serial or SCSI interface that does not offer enough
the camera and end not with the extracted XYZ coordinates but bandwidth to transmit the data for real-time use. A camera
rather with the dimensional data required by the user. This system that would truly satisfy the needs for on-line and off-line
distinction is very important as often the user is not interested in use in photogrammetry should have not only a self-contained
the extracted data per se, but rather the ‘information’ that the storage capability, but also the ability to work in an on-line
data represents in terms of key geometric characteristics such as mode at data rates suitable for real-time operation. The obvious
planes, distances and surfaces, or perhaps the ‘goodness of fit’ solution is a hybrid camera that utilises a true metrology sensor
to an existing data set. This is yet another area where as a base and integrates storage and network components to
automation is applicable. increase its functionality. This camera would have the
flexibility of a hand-held unit whilst retaining the robustness
This paper reports on the process of automation in and reliability of the MegaPlus. The GSI INCA (Geodetic
videogrammetry from the initial photography to the final Services Inc., 1996) is the first commercial example of this type
dimensional analysis. To better illustrate the potential of of camera.
automation in industry, two case studies are considered. In each
videogrammetry measurement case the image data was first Up until only recently, automation of the camera had been
processed using very limited automation, and then processed largely been overlooked by the photogrammetric community.
employing full automation. The V-STARS vision metrology This was perhaps due not only to the difficulty in manufacturing
system from Geodetic Services, Inc.(GSI) was employed for the such a camera, but also to the associated cost of development.
test measurements. Prior to reporting on the results of the case To implement some level of automation within a camera it is
studies, significant recent innovations in automation of vision necessary to incorporate an additional processor or low level
metrology systems will be reviewed. PC. Cameras with this type of processor, as exemplified by the
INCA 6.3 camera shown in Figure 1, are sometimes referred to
as intelligent cameras. The on-board PC is responsible for a
2. SYSTEM AUTOMATION variety of functions including image compression, automatic
exposure setting and image measurement feedback. These
2.1 Intelligent Cameras features can significantly reduce the photographic burden on the
user. Future systems will no doubt be more sophisticated and
Commercially available digital cameras of the price and incorporate network analysis and on-line results verification,
performance suitable for close-range photogrammetry fall into features which will assist the operator in assessing whether all
two basic categories: Those with proprietary interfaces that the requirements of the project have been met prior to leaving
require special frame grabber hardware for acquiring the the site. The potential advantages are therefore very significant
imagery and those with self contained storage capability. The given the tremendous cost of down-time in the type of industries
former is useful for on-line triangulation (Edmundson and that utilise high accuracy vision metrology equipment.
Fraser, 1998) where immediate data acquisition is needed for
real-time processing. These systems require a desktop computer
in order to house the frame grabber hardware. The most notable
examples of camera systems in this category are the MegaPlus
line of digital cameras from Kodak. The MegaPlus is very
suitable for digital photogrammetry based on price and
performance, however, due to the need to be connected to a
computer for data acquisition, these camera systems are not
very suitable for hand-held off-line use. This camera also lacks
any form of onboard storage capability.
 Figure 2a. Examples of an EO device (known as the
AutoBar)

Figure 1. Image showing MegaPlus 6.3 with integrated PC back.

This INtelligent CAmera is known as the INCA.

2.2 Software
Figure 2b. Examples of coded targets.
Advances in automation in system software are clearly central
to the goal of achieving a fully automated system. Automation The second variety of automation feature, which is designed to
software can be broadly defined into two categories. The first alleviate the user of some of the decision making process, is
variety is designed to relieve the user of many of the functions more difficult to implement given the complex nature of
associated with image processing and reduction, whilst the decision making. The most common and time-consuming
second makes decisions for the user. process is that of new point determination. To coordinate new
points it is necessary to locate and label each and every
Three common repetitive functions that are easily automated are unknown point in at least two images. Obviously this is a
the processes of line following, driveback and exterior tedious process which is prone to operator error. To facilitate
orientation determination. The use of uniformly spaced retro- unambiguous labeling it is often necessary to introduce target
reflective target strips within industry is commonplace. This is identification labels.
especially the case in the antenna and aerospace fields. In order
to simplify the labeling or re-labeling process it is possible to To automate the coordination of target points it is first necessary
use two points at the start of a strip to define the separation and to identify potential targets in each image. This identification
direction of the subsequent strip targets. With each subsequent process will also locate false targets such as overhead lights,
target measured the direction of the line is recomputed and the flash hot spots or even return from discarded retro-reflective
search patch modified. The process terminates when no target material. With these potential target locations it is possible to
is found in the reference patch. From a practical point of view, combine matching targets and derive an XYZ coordinate for
strip targets are measured instantaneously. each point that has been qualified as a valid target. In some
cases false targets will be found but with appropriate checks it is
Another feature easily implemented is a process known as possible to almost always identify and remove these errant
driveback. Once an approximate camera location has been points. This process is termed AutoMatching within the V-
established through space resection it is possible to compute the STARS system and as the case studies will show, it is a very
initial target xy locations in the image space. These xy locations powerful automation tool in vision metrology.
are used in combination with a search patch. If a target is found
within this search window then it is measured. If no target is Yet another powerful automation tool is that of the construction
found then the point is skipped in that image. template or macro. Construction templates can be utilised to
complete all manner of repeated analysis. As an example,
Determining the exterior orientation of an image in an consider the situation in Figure 3 where the perpendicularity of
automated fashion is considerably more difficult to implement two planes is required. To solve this problem it is first
than line following or driveback. The orientation of an image is necessary to determine the location of at least three points on
typically determined by identifying four or more points of each plane (four for redundancy), fit planes to the data and then
known approximate XYZ coordinates. Once these have been compute the intersection angles between the planes. This is a
identified, the camera exterior orientation can be computed fairly simple piece of analysis and might only take a few
using a closed-form space resection. To automate the space minutes to complete. Now, consider the case where the same
resection procedure it is necessary to use exterior orientation information is required for 100 planes. It is obvious that the
(EO) devices and/or coded targets. Examples of these are remaining 99 will be solved in the same manner as the first.
shown in Figures 2a and 2b. If either an EO device or coded
targets are seen in any image they are identified and decoded, A construction template would facilitate the automation of the
and if enough object points with approximately known 3D remainder of this analysis. This makes it possible to extend the
coordinates are available the exterior orientation can be “one button” notion for photogrammetric triangulation all the
completed. way to the final 3D coordinate analysis phase. The emergence
of construction templates will bridge the gap between simple
XYZ data and meaningful dimensions. It will also facilitate the Image compression reduces the amount of time required to write
introduction of real-time or quasi real-time measurement the data to disk.
analysis in production facilities Image Transfer: Image transfer involves the transfer of the
acquired images from the PCMCIA disk to the local PC hard
drive. In the automated case these are compressed images while
p1
in the manual case there is no image compression. The time
p2 P1
p3 difference develops due to the difference in image size. This
p5
b4 step is optional given that videogrammetry software can
p7
pn
b7
p6 P2 B1 p4 typically access the image data directly from the PCMCIA disk.
p n-1 p n-2 B5
bn Pn B2 p8
b2
b3 For archiving, it is necessary to transfer the images from the
b n-1
Bn p n-3 b8 PCMCIA disk to long-term storage media.
b6
b n-3 b n-2 Initial Exterior Orientation: The Initial EO is necessary to
approximately locate the camera locations at the time of
exposure. As mentioned earlier, in the automated case this is
p1 p2 p3 p4 → P1 achieved through coded targets and/or an EO device. In the
b1 b2 b3 b4 → B1 manual case this involves manually locating coordinated points.
P1 B1 → α1 Initial Bundle Adjustment: The objective of the Initial Bundle
Adjustment in the automated case is the coordination of coded
p5 p6 p7 p8 → P2 targets. This is based on the coordinate system established by
b5 b6 b7 b8 → B2 the EO device (i.e. the AutoBar). In the manual case the bundle
P2 B2 → α2 adjustment is used to triangulate points measured manually to
facilitate driveback in subsequent images.
:
New Point Determination: In the automated case new points
:
are determined through the use of a set of mathematical rules
pn pn-1 pn-2 pn-3 → Pn governing image point correspondences, which must be satisfied
bn bn-1 bn-2 bn-3 → Bn by AutoMatching. To determine a new point in a manual
Pn Bn → αn system it is necessary to label the point in at least two images in
Where p = vertical plane points which the point is seen.
b = horizontal plane points Final Bundle Adjustment: Once all the points have been
P = vertical plane
B = horizontal plane measured in the image files they can be combined in the Final
α = angle of intersection between P & B Bundle Adjustment to produce the XYZ position of each point.
The time taken to complete this task will not vary between the
Figure 3: Example of construction templates.
automated and manual cases.
Point Renaming: In the automated case the points found are
3. CASE STUDIES
assigned arbitrary labels. If specific labels are required, then
each point must be located and re-labeled. This only has to be
3.1 Overview
completed in one image for each point. Re-labeling is not a
requirement in the manual case as the points are assigned the
In order to quantify the advantages of automation, two case
correct labels at the time of initial measurement. The
studies will be considered. The first involves a modest sized
underlying need for labels in the manual case is driven by the
network whereas the second is more complex, requiring a large
fact that the user needs to establish point correspondence
number of images. The data collected has been processed both
between images. In most cases the labels of the points are not
manually and automatically. Driveback and line following are
important and the step of re-labeling can be bypassed. In the
such common features in modern videogrammetry systems that
case of repeat measurements the new points need only be placed
these have been used in the manual case even though they are
approximately and a label transformation can be utilised to
clearly automation features.
automatically re-label points.
Each of the case studies was carried out using the following Clean up and ‘Re-Bundle’: In some instances, the AutoMatch
equipment: will coordinate points that satisfy the prescribed mathematical
rules, but are not actual target points. These points are typically
Camera: GSI INCA 6.3 (3K x 2K sensor) easily identified and deleted. Once these points are eliminated it
Storage: Viper 340Mb PCMCIA Hard Disk is best, but not necessary, to ‘Re-Bundle’. The corresponding
Processor: Pentium 133Mhz laptop with 48Mb RAM problem in the manual case involves incorrect identification of
Software: GSI V-STARS/S points in the two initial images. This problem will normally
emerge prior to the Final Bundle process.
The analysis was carried out by recording the time taken to
complete the following key functions:

Image Acquisition: The time taken to photograph the object

and store the acquired images to disk. The difference between
the automated and manual case is that image compression and
preliminary measurement are employed in the automated case.
3.2 Case Study 1 – Small Antenna
700
The first case study involved the measurement of a relatively Clean Up and Re-bundle
600
simple object, namely a small antenna with eight orientation Point renaming
500
targets, one AutoBar and 16 rows of strip targeting (272 points Final Bundle

Time(sec)
400
in all). The antenna and target configuration are shown in New Point Determination
300 Initial Bundle
Figure 4.
200 Initial EO
Image Transfer
100
Image Aqusition
0
Manual Automated

Figure 6. Graphical representation of times taken to complete

each key component.

It is clear from the times listed in Table 1 that even in a small

network there are considerable time savings. In this particular
case automation led to a reduction in measurement time from 10
minutes to only 4 minutes. It should be noted that if no line
following or drive back had been employed in the manual case
then the time saving would have been considerably greater. The
biggest savings were achieved at the stages of image acquisition
and new point determination. The final point cloud for the
antenna is shown in Figure 7.

Figure 4. Antenna measured in the first case study.

The object was photographed and eight images were collected

as shown in Figure 5.

Figure 5. Network geometry for antenna measurement.

Four stations were used and the camera was rolled through 90
degrees at each of these stations. The images collected were Figure 7. The final object point cloud for the antenna
processed both manually and with as much automation as measurement.
possible. The times recorded in seconds for each of the steps 3.3 Case Study 2 – Automotive Master Model
described earlier are shown in Table 1 and represented
graphically in Figure 6. The second case study involved the measurement of a more
Table 1. Times recorded for each key component between complex object. The object selected was an automotive master
Manual and Automated Case model cubing block. The block consists of a number of parallel
Category Man(sec) Auto(sec) and perpendicular plates with component-securing points
Image Acquisition 180 30 located in a grid. Each of the master model components are
Image Transfer 75 15 attached to the block in their prescribed locations. These
Initial EO 45 30 components can then be checked against the CAD model that
Initial Bundle 5 5 represents the particular vehicle. Production line parts can also
New Point Determination 270 15 be attached to the block to verify that production components
Final Bundle 15 15 meet design standards. The block consisted of five plates of
Point Renaming 0 135 interest. These plates were targeted with 83 coded targets, an
Clean up and Re-Bundle 15 20 AutoBar and 205 stick-on targets. The object is shown in
Total 605 265 Figure 8.
 These results are represented graphically in Figure 10. It is
clear from the listed times in Table 2 that the timesaving is even
greater for complicated measurement networks. In this
particular case approximately 2.5 hours were saved. Once
again, if no line following or drive back had been employed in
the manual case then the time saving would have been far
greater. Also, if there was no requirement for pre-determined
point labels then a further 30 minutes could have been saved,
reducing the overall time to just over 30 minutes. Figure 11
graphs the results in the case that no pre-defined labels were
required. The biggest savings were achieved at the stages of
image acquisition acquisition and new point determination. At
the new point determination stage a time of 2 hours was saved.

250
Clean Up and Re-bundle

200 Point renaming

Figure 8: Cubing block used in second case study. Note that at Final Bundle

Time (minutes)
this stage the master model was still in place. 150 New Point Determination
Initial Bundle
100 Initial EO
A total of 71 images covering the 293 object points on the 50
Image Transfer

object were recorded from 45 camera stations, as indicated in Image Aqusition

0
Figure 9. Camera rolls were introduced at 26 stations. Once Manual Automated

again the images collected were processed both manually and

with as much automation as possible. The times recorded for Figure 10. Graphical representation of times taken to complete
each of the steps described in the overview are shown in Table each key measurement component in Case 2.
2, this time in minutes.
250

Clean Up and Re-bundle

200
Final Bundle
Time (minutes)

150 New Point Determination

Initial Bundle
100 Initial EO
Image Transfer
50
Image Aqusition
0
Manual Automated

Figure 11. Graphical representation of times taken to complete

each key component in Case 2, with no point re-labeling.

3.4 Evaluation

These case studies clearly indicate that there are significant

advantages to be attained through an automated approach to
Videogrammetry. In both instances the automated approach
Figure 9. Image showing network geometry. proved to be significantly faster. Perhaps the most remarkable
outcome is the time discrepancy in the new-point determination
Table 2. Times recorded for each key component between of the second case study. The time difference between the
Manual and Automated Case. manual and automated processes was two hours. Also of
Category Manual(min) Automated(min) significance is the relative level of skill required to successfully
complete the measurement. It is difficult to accurately quantify
Image Acquisition 30 6
relative skill levels but it is obvious that the amount of skill
Image Transfer 11 2.5
required to successfully reduce a set of images manually is
Initial EO 18 4.5
significantly higher than for automated processing.
Initial Bundle 6 3
New Point Determination 124 2
Final Bundle 8 8
Point Renaming 0 32
Clean up and Re-Bundle 6 6
Total 203 64
 4. CONCLUDING REMARKS

Videogrammetry has matured both on the hardware and

software fronts. The introduction of intelligent cameras and
improved software has had a considerable positive impact on
the performance of commercially available vision metrology
systems. These improvements have lead to a significant degree
of system automation and this has helped widen the technology
user base. The age of the “one button” approach to
videogrammetry has clearly arrived.

5. ACKNOWLEDGEMENTS

The authors wish to acknowledge the assistance provided by Dr

Clive Fraser and Dr Ken Edmundson of the University of
Melbourne in the preparation of this paper. Their suggestions
and patience were greatly appreciated and will no doubt cost us
dearly at the bar.

6. REFERENCES

Edmundson, K.L. and Fraser. C.S., 1998. Performance

evaluation of sequential estimation in vision metrology.
International Archives of Photogrammetry and Remote Sensing,
Hakodate, 32(5) (this volume).
Fraser, C.S. 1997. Innovations in Automation for Vision
Metrology Systems. Photogrammetric Record, 15(90): 901-911.
Fraser, C.S. and Shortis, M.R., 1995. Metric exploitation of still
video imagery. Photogrammetric Record, 15(85) : 107-122.
Fraser, C.S., Shortis, M.R. and G. Ganci 1995. Multi-Sensor
System Calibration. Invited Paper, Videometrics IV Conference,
SPIE Volume 2598, Philadelphia, 25-27 Oct., pp.2-18.
Ganci, G. and Shortis, M.R., 1995a. Videometric as-built
surveys during the manufacture of a furnace hopper.
Proceedings, 3rd Symposium on Surveillance and Monitoring
Surveys, Melbourne, Australia, pp 59-69.
Ganci, G. and Shortis, M. R., 1995b. The Use of Digital
Photogrammetry for Large Scale Metrology. Proceedings, 36th
Australian Surveyors Congress, Singapore, pp 247-261.
Ganci, G. and Shortis, M.R., 1996. A comparison of the utility
and efficiency of digital photogrammetry and industrial
theodolite systems. Proceedings, ISPRS Commission V,
Working Group V/3 Vienna, Austria.
Geodetic Services, Inc., 1996. V-STARS/INCA 4.2, [on-line],
[http://www.geodetic.com/prod01.htm].
Gustafson, P. C. and Handley, H. B., 1992. A video-based
industrial measurement system. Proceedings, ISPRS
Commission V, Volume XXIX, B5 pp501-506, Washington
D.C, USA
Shortis, M. R. and Fraser, C. S., 1991. Current trends in close
range optical 3D measurement for industrial and engineering
applications. Survey Review, 31(242): 188-200.
Shortis, M. R. and Ganci, G., 1997. Calibration Stability of
Digital Still Cameras for Industrial Inspection. Proceedings,
Metrology Society of Australia, Melbourne, Australia.
 Form for Full Paper

Paper Title: AUTOMATION IN VIDEOGRAMMETRY

Author(s): Giuseppe Ganci (AUSTRALIA) Harry Handley(USA)

Address of Presenting Author:

Name: Giuseppe Ganci

Title: Mr.
Affiliation: Geodetic Services, Inc.
Mailing Address: 1511 S. Riverview Drive, Melbourne, Florida
Country: USA
Telephone +1 407 724 6831
Facsimilie +1 407 724 9253
E-mail: [email protected]

Equipment Requested for Presentation

1 PC Projector