2010 IEEE International Conference on Robotics and Automation

Anchorage Convention District

May 3-8, 2010, Anchorage, Alaska, USA

Visual Vein-Finding for Robotic IV Insertion

Reuben D. Brewer and J. Kenneth Salisbury

Abstract— This paper presents a new algorithm for selecting

the optimal needle insertion point in images of hand veins for
robotic IV insertion. The 3D coordinates and orientation of
the vein that the algorithm detects would eventually be fed
to a robot for insertion of the IV needle. The goal of the
algorithm is to identify venous bifurcations and determine an
insertion point and approach angle for the needle in between
their branches. The algorithm uses an annular tracking window
that tracks along the veins and searches for bifurcations. We
describe methods for centering the initial bifurcation estimates,
error-checking, and positioning the needle exactly in between
the bifurcation branches. We conclude with an experimental
study of 50 subjects that shows a 32.4% success rate at detecting
all bifurcations and a 82.6% success rate at finding at least one
bifurcation in each image that contains bifurcations.

I. I NTRODUCTION
(a)
A. Background
Intravenous (IV) catheterization is a medical procedure
wherein a flexible plastic tube, or catheter, is inserted into
a vein for the delivery of medicinal fluids. The catheter
initially surrounds a needle that punctures the wall of a vein
so that the catheter can be slid off of the needle and into the
vein, whereupon the needle is removed. Nearly 1 billion IV
insertions take place in the United States annually, and 28%
of those insertions fail on the first attempt in normal adults,
with appreciably higher failure rates in children [1], [2].
Failed insertions commonly cause bruising and pain, but can
also lead to long-term nerve damage and schlerosis of the
veins.
Robotic IV insertion has been proposed as a possible
solution to increase the insertion success rate through pre-
cise movement of the needle and enhanced sensory abili- (b)
ties [3], [4]. Towards this end, we are currently developing
Fig. 1. 7 DOF robotic IV insertion system currently under development.
a 7-DOF robot, as shown in Figure 1, that can insert either
under teleoperation or autonomously. Such a system could
be used to treat people in remote or hostile locations where
a human practitioner could not be present. Autonomous in- Practitioners often have great difficulty seeing or feeling
sertion could be used in hospitals to increase the success and small veins, as found in women and children, or veins that
through-put of phlebotomists, nurses, and anesthesiologists. lie beneath a layer of fat, as found in children and obese
While good mechanical design can provide precise needle patients. However, detection of the veins through various
motion, the robot must also include robust vein detection sensors could allow a robot to target veins that human
through various sensors to provide the target location and practitioners otherwise could not localize. In [3], robotic
insertion trajectory for the needle. palpation, or tactile sensing, was used to locate veins by
One of the reasons for the high failure rate for human instrumenting a probe with a force sensor and examining
practitioners is the difficulty of locating veins precisely. changes in tissue stiffness across the arm. However, the
pressure of palpation may roll the vein away from the probe,
R. Brewer is with the Department of Mechanical Engineering, Stanford thereby skewing the position information. Further, palpation
University, Stanford, CA, USA, [email protected] suffers from an inability to map more than a small segment
J. Salisbury is with the Departments of Computer Sci-
ence and Surgery, Stanford University, Stanford, CA, USA, of vein or provide real-time tracking of the vein in case the
[email protected] vein/arm moves during insertion. An alternative is the use of

978-1-4244-5040-4/10/$26.00 ©2010 IEEE 4597

 Stereo Camera
infrared imaging, as in [4]. The deoxygenated haemoglobin
in veins absorbs infrared light more than surrounding tissues,
so veins appear as dark on a light background under infrared
light [5]. Such imaging systems are commercially available
for assisting human practitioners and can be readily adapted
to provide images of veins to a robot [6], [7]. However, a
topic of ongoing research is how to provide a robot with the
ability to analyze and interpret these vein images so as to
select an optimal insertion location. IR LEDs, Diffuser Glass
The conventional wisdom of many practitioners is that (a) (b)
inserting at a venous bifurcation minimizes the vein rolling as
opposed to inserting along the midsection of a single vein be-
cause bifurcations are tethered by more connective tissue [8].
For this reason, our algorithm searches the venous network
for bifurcations as the optimal insertion points. Since the
dorsum of the hand contains far more bifurcations than does
the forearm or antecubital fossa (inside of the elbow), we
are restricting our insertions to the hand. Concentrating on
the hand has the additional benefits of allowing for easier
Unseen in Raw Image
mechanical access to the insertion point and the ability to use
simple transillumination of the veins, whereas the forearm is (c)
too thick for transillumination. Fig. 2. (a) Imaging system. (b) Raw image of veins. Veins appear as dark
on a light background. (c) Enhanced image. Veins now appear as bright on
B. Related Work a dark background.
Bifurcation detection has been studied widely for images
of retinal arteries. In [9], the entire image is searched using
the assumption that bifurcations will occur at areas of high macro-structure of the vein instead of a small slice, allowing
variation in Sobel edge direction. Similarly, in [10] and [11] for tracking noisy, variable veins with erratic edges, as
a course grid of seed-points that looks for local gray-scale well as detecting bifurcations during the tracking process.
minima between oppositely-signed edges is applied across We describe a method for centering an initial bifurcation
the entire image. Recursive tracing proceeds from these estimate on the true center of the bifurcation by using a
seed-points by moving in the direction that best fits the set of concentric annular windows. We detail methods for
vein model of a gray-scale minimum between antiparallel error-checking and finding the actual point of insertion in
edges. However, searching the entire image can generate between the branches of a bifurcation. Finally, we discuss
many false positives, especially if the image is noisy or the experimental results of applying our bifurcation detector
has artifacts. In the case of our particular vein images, the to a set of sample images.
variation in gray-scale intensity and contrast of the veins II. I MAGE ACQUISITION
leads to poor edge detection such that the edges cannot
be followed reliably. An alternative to searching the entire A. Imaging System
image is to trace along the arteries from a known landmark, Our imaging system is a low-cost approximation of
hopefully reducing the number of false-positives. In [12], commercially-available infrared imaging systems such as [6]
Canny edge detection is performed, and tracing occurs along and [7]. Our setup transilluminates the hand from the palm
the detected edges, starting from the optic disk. In [13], side and views the illuminated venous structure from the
a steerable Gaussian filter is used to trace from the optic dorsal side, as shown in Figure 2(a). The light source consists
disk. This tracing assumes thin, well-defined vessels with of 5 high-power Osram SFH-4730 infrared LEDs (λpeak =
consistent cross-sectional profiles. Unfortunately, the vessels 850nm) spread out across the palm and covered with a
in our images exhibit cross-sectional intensity profiles of diffusing glass to provide even lighting. It should be noted
varying size, noisiness, and degree of saturation such that that according to [5] and [14], the light-blocking capacity
looking at their cross-sectional profile does not give much of deoxyhaemoglobin (the deoxygenated blood in veins) is
reliable information. highest at a wavelength of 760nm and is reduced by as much
as 50% from its peak value at 850nm. Further, the light-
C. Overview blocking capacity of oxyhaemoglobin (the oxygenated blood
In this paper, we present a new algorithm for finding in arteries) is much lower at 760nm than at 850nm. There-
the position and orientation of venous bifurcations in an fore, the contrast between veins and the surrounding tissues,
image. We first detail a method for locating the wrist veins and in particular arteries, is maximal at 760nm, resulting
as a reliable starting point for vessel tracking. We then in more readily-identifiable veins in the images and less
present a new annular tracking window that examines the ghosting effects from faint arteries. However, the available

4598
 255

High Hyst. Thresh.

Pixel Intensity
200

150
Low Hyst. Thresh.
RL RR

0
300
X distance Identified Wrist Branches
(a)

Vein Crossings
Vein Crossings
Pixel Intensity
255

180 High Thresh.

150 Low Thresh. a a a New Window
N N N
C1 C1 C2
a
a µN
θ a C1
L L
0
0 360
Angle (deg) C2 C3 Last Window

(b) (c) (d) (e) (f)

Fig. 3. (a) Identifying wrist branches with histogram at bottom of image. (b) Polar histogram of annular tracking window. (c) 1 Vein crossing represents
a faded, single vein. (d) 2 Vein crossings represent a solid, single vein. (e) 3 Vein crossings represent a bifurcation. (f) Incremental movement of tracking
window.

light source for a wavelength of 850nm was considerably more likely to be real than a bifurcation that we detect at a
more practical. The camera is a Videre STOC stereo camera random point in the image. To identify the wrist veins, we
(6 cm baseline) with the IR filter removed and provides take a horizontal sample strip (height = 8 pixels, determined
640x480 monochrome images, as shown in Figure 2(b). The empirically) across the bottom of the image and examine
stereo camera allows us to calculate the 3D position and the smoothed histogram of gray-scale intensity, as shown
orientation of the vein that we detect in a 2D image. in Figure 3(a). After collapsing the strip to a single pixel-
width height by taking the median in the vertical direction,
B. Image Enhancement we apply gaussian filtering in the horizontal direction and
The veins are often difficult to identify in the raw images hysteresis-thresholding to mitigate the noise in the histogram.
due to low contrast and fading of some vein sections, as Since veins appear as bright against a dark background,
seen in Figure 2(b). For this reason, we enhance the images they are identified on the histogram as local maxima. Each
to increase the vein vs. non-vein contrast and normalize identified wrist vein becomes a starting point for tracking. We
the appearance of the veins so that all sections appear use the histogram to size each tracking window to be twice
similar. We use Laplace of Gaussian (LOG) filtering (σ = 8, the width of the wrist vein that it will track. The tracking
window width of 5σ, determined empirically) to accentuate window is larger than the vein being tracked so that the
the vein macrostructure over the background and histogram tracking window is never entirely inside the vein.
equalization to improve the overall contrast in the image.
Figure 2(c) shows the enhanced version of the raw image B. Tracking Veins
seen in Figure 2(b). Tracking along the wrist veins employs a steerable window
III. F INDING I NITIAL B IFURCATION E STIMATES that uses information about the macrostructure of the vein
section contained within the window to determine how to
A. Identifying Wrist Veins move along the vein. The tracking window consists of an
Our bifurcation finder operates by finding the most promi- annulus (thickness = 8 pixels, determined empirically) that
nent veins at the base of the wrist and tracking along is centered on a section of the vein. A polar histogram
those veins with a window that looks for bifurcations. The of gray-scale intensity in the annular sample describes the
wrist veins are composed of the basilic, cephalic, and their shape of the vein section contained within the window. As
branches, such as the accessory cephalic vein. The desired before, the veins are identified in the smoothed histogram as
bifurcations occur in the dorsal metacarpal veins in the top local maxima. After collapsing the annulus to a single pixel-
of the hand. Whereas the venous network in the hand varies width thickness by taking the median in the radial direction,
greatly between individuals, the location of wrist veins is we apply gaussian smoothing in the angular direction and
fairly consistent, providing an ideal starting location for hysteresis-thresholding to reduce noise in the histogram.
tracking. As in [12] and [13], we track along the veins from a Figure 3(b) shows the tracking window in detail.
known location instead of searching the entire image because Figures 3(c) - 3(f) show how the number of vein crossings
a bifurcation that we detect while tracking along a vein is detected in the tracking window conveys information about

4599
 Global Maximum = Center of Bifurcation
1

0.8

Correlation
0.6

0.4

0.2

0
20 20
0 0
−20 −20
Y Position (pixels) X Position (pixels)

(a) (b)

(a) (b)

Fig. 4. Tracking along wrist veins. (a) Un-clustered initial bifurcation

estimates. Small black dots show the path of the tracking window, and large
red dots show bifurcation estimates. (b) Clustered estimates of bifurcations
(shown as large red dots).
= Initial Estimate
= Centered Estimate
(c)
the section of vein within the window and how the tracking
window should move along the vein. We assume that one Fig. 5. (a) Concentric annular samples near a bifurcation. (b) Correlation
in the set of annular samples in the vicinity of the bifurcation. (c) Example
vein crossing (C1 ) indicates that the tracking window is of centering algorithm on a real bifurcation.
centered on a faded, single section of vein, two crossings
(C1 , C2 ) indicate that the tracking window is centered on
a solid, single section of vein, and three vein crossings different diameters, as shown in Figure 5(a). These samples
(C1 , C2 , C3 ) indicate that the tracking window is centered are resized versions of the tracking window described above.
on a vein near a bifurcation. In all three cases, the track- If we position this set of annuli at each pixel in the vicinity of
ing window steps a small amount µ (5 pixels, determined the bifurcation and examine the correlation, we find that the
−
→
empirically) in the new direction N that is computed as the global maximum of the correlation corresponds to the true
vector between the window center and the vein crossing most center for bifurcations with 3 branches. Since there are more
→
−
closely aligned with the previous direction of movement L . than two samples, we compute the correlation as the sum of
The degenerate cases are zero vein crossings, which indicates correlations between adjacent rings. Figure 5(b) shows the
a lack of information, and more than three crossings, which correlation plot for the example bifurcation in Figure 5(a).
indicates noise. In either degenerate case, the tracking win- Figure 5(c) shows an example of the centering algorithm on
dow steps in the last known direction of movement. In the a real bifurcation. Note that the initial estimate is appreciably
case of a detected bifurcation, the tracking window continues off-center but that the centered estimate appears directly
tracking along the branch most closely aligned with the last at the center of the bifurcation. For brevity, we omit the
known direction of movement until it reaches an edge of derivation of this centering algorithm.
the image or stalls, whereupon it returns to track along the
alternate (third) branch of the bifurcation. Figure 4(a) shows B. Error-Checking
an example of vein tracking and initial bifurcation detection. We employ two simple methods of error-checking to
Due to the thickness of the veins, there are many points in the detect false-positive bifurcations. In a fashion similar to the
vicinity of a single bifurcation that show three vein crossings, centering algorithm, we examine concentric annular samples
or a detected bifurcation. Clustering all of the bifurcation of each bifurcation over 35 different diameters and count
estimates based on distance from each other reliably thins the number of diameters that elicit the characteristic three
this mass of estimates to one estimate per actual bifurcation vein crossings of a true bifurcation. If our estimate is a
detected, as shown in Figure 4(b). true bifurcation, then it looks like a bifurcation over many
scales, whereas false-positives appear like bifurcations over
IV. R EFINING AND E RROR -C HECKING B IFURCATION
only a few diameters. In practice, false-positives almost
E STIMATES
uniformly return a low number of counts (below 10), whereas
A. Centering the Bifurcation Estimate true bifurcations return a much higher number (typically
The initial bifurcation estimate often does not coincide above 25). A final, simple error-check is to ensure that each
with the true center of the bifurcation due to the asymmetric bifurcation is centered on a bright spot (vein). Figure 6(a)
distribution of initial guesses that were eventually clustered. shows examples of both error-checks. Note that the false
Thus, we need an algorithm for finding the true center of positive is centered on a dark spot (non-vein) and only shows
the bifurcation given a moderately-close initial estimate. We the three vein crossings for the particular diameter shown,
have devised a solution to this problem by looking at the whereas the true bifurcation is centered on a bright spot and
correlation in a set of concentric annular samples taken at shows the three vein crossings over most diameters.

4600
 3 Vein Crossings, 3 Vein Crossings,
False Positive Real Bifurcation

(a) (a) (b)

Bisecting Line
Distal Branch 1 Distal Branch 2 IV Needle Placement

Blue = Vein Edge

= Insertion Point A

Base Branch Nearest Wrist B

(b)

Fig. 6. (a) Error-checking for false-positives. (b) Finding the insertion C

point, angle.
(c) (d)

Fig. 7. (a,b,c) Examples of correctly identified and characterized bifur-

C. Finding the Insertion Point cations. The arrows show the desired position and orientation of the IV
The desired insertion point at a bifurcation is not the center needle. (d) Failure modes. Point A is a false negative, point B is a correctly
identified but misplaced bifurcation, and point C is a false positive that arose
of the bifurcation, but, rather, along the edge in between the because two separate veins appeared to be a bifurcation.
two branches of the bifurcation furthest from the wrist, as
shown in Figure 6(b). To find this point, we use Canny edge
detection to compute the edges of the veins in the vicinity of as is standard practice among practitioners. While these
the bifurcation. This edge detection uses locally determined metrics make sense from an engineering perspective, it will
threshold values because the intensity of the veins varies be necessary in the future to determine additional clinical
somewhat over the image, precluding the usage of global criteria used by practitioners to select amongst bifurcations
threshold values. Once the local vein edges are found, we and implement the same weighting in our algorithm.
sample along the line that bisects the two distal branches
of the bifurcation until we intersect with a vein edge. This V. E XPERIMENTAL M ETHODS AND R ESULTS
intersection at the edge of the vein is the desired insertion
point, and the bisecting line is the desired direction for the We developed our bifurcation-finding algorithm on a set
IV needle. It is acceptable for the bisecting line and needle of thirteen images that provided for iterative refinement.
not to align exactly with the branch nearest the wrist because We determined algorithm parameters such as LOG filter σ,
the needle will not enter the vein far enough to hit the back window step size µ, and annular thickness empirically in
wall, and the catheter is sufficiently flexible to deflect safely these 13 images. To test the general applicability of the
away from the back wall and into the lumen of the base vein algorithm, we applied it to fifty images that were unseen
branch. during development. The test population was 72% male, had
an age range of 22-56 (µ = 31.0, σ = 9.1), and was 72%
D. Weighting Multiple Bifurcations caucasian and 28% asian. To compare the performance of
If multiple bifurcations are detected in the same image, our algorithm with that of human practitioners, we provided
we must be able to decide which is optimal for insertion. four medical doctors with the same images and asked them
Towards this end, we examine each bifurcation and compute to identify all bifurcations, without preference to size or
a weighted score based on a variety of desirable traits. These desirability. Our algorithm successfully detected and char-
traits include the correlation found during the centering pro- acterized 32.4% of the bifurcations identified by the doctors
cess, orientation with respect to the wrist direction, distance with a false positive rate of 11.6%. However, our algorithm
from the center of the hand, and average gray-scale intensity correctly found and characterized at least one bifurcation
in the vicinity of the bifurcation. The correlation and in- in 82.6% of the images that contained bifurcations, which
tensity metrics address our confidence in the authenticity of comprised 92% of all images (4 images did not contain any
the bifurcation. The orientation and distance metrics address bifurcations). Assuming that it is clinically sufficient to find
the practical concerns that we wish to insert the needle in only one (instead of every) bifurcation per hand for the robot
the direction of the wrist and near the center of the hand, to insert an IV, our vein-finding algorithm would allow us to

4601
insert on the majority (82.6%*92% = 76%) of individuals. Real-time visual tracking of the selected insertion point will
Figures 7(a-c) show examples of successfully detected and continue to update the desired insertion trajectory for the
characterized bifurcations, and Figure 7(d) shows several robot throughout the insertion. We will also investigate the
failure cases. Although the algorithm was not tested on effect of light scattering on the accuracy of depth information
children, we expect future pediatric tests to be successful due in the stereo infrared images, as well as possible benefits of
to the algorithm’s ability to auto-size the tracking window switching between infrared and visible-light imaging so as
to differently-sized veins. It should be noted that the large to switch between internal and external views of the hand,
width of the tracking window creates margins at the edges of respectively.
the image that are unsearchable by the algorithm, whereas
VII. ACKNOWLEDGMENTS
humans can search the entire image. In fact, many of the
bifurcations that the doctors identified and our algorithm R. Brewer was supported in part by an ASEE NDSEG
did not were near the edges of the image/hand where the Fellowship and a NSF Graduate Research Fellowship. We
algorithm could not search. These “missed” bifurcations are are thankful to Dr. Greg Hager and Dr. Jana Kosecka for
not as important as others because it would be inconvenient their consultation in this work.
to insert an IV at the edge of the hand. R EFERENCES
VI. D ISCUSSION AND C ONCLUSION [1] R. Lenhardt, “Local warming and insertion of peripheral venous
cannulas: single blinded prospective randomised controlled trial and
In this paper we developed a new visual vein-finding tech- single blinded randomised crossover trial,” British Medical Journal,
nique for use in a robotic IV system, as shown in Figure 1. In vol. 325, no. 7, pp. 409–412, 2002.
[2] R. Lininger, “Pediatric peripheral iv insertion success rates,” Pediatric
such a system, our algorithm would analyze infrared images Nursing, vol. 29, no. 5, pp. 351–355, 2003.
of the hand to locate suitable insertion points at bifurcations [3] A. Zivanovic, “The development of a haptic robot to take blood
and provide the robot with the 3D position and orientation samples from the forearm,” MICCAI, p. 614620, 2001.
[4] V. Paquit, “Near-infrared imaging and structured light ranging for
of the IV needle for insertion. Specific to our vein-finding automatic catheter insertion,” Medical Imaging 2006: Visualization,
algorithm, we discussed a new method for tracking veins Image-Guided Procedures, and Display, vol. SPIE 6141, p. 19, 2006.
that uses an annular tracking window to examine the macro- [5] D. Delpy, “Quantification in tissue near-infrared spectroscopy,” Philos
Trans. of the Royal Soc. B: Biol Sci., vol. 352, pp. 649–659, 1997.
scale structure of local sections of vein. We then discussed [6] InfraRed Imaging Systems, http://www.irimagesys.com/.
a method for centering our initial rough bifurcation estimate [7] Luminetx Corporation, http://www.luminetx.com/.
by maximizing the correlation in a set of concentric annular [8] J. Payne, “Alternative techniques for venipuncture,” The American
Journal of Nursing, vol. 72, no. 4, pp. 702–703, 1972.
samples near the bifurcation. We detailed a method for error- [9] D. Becker, “Image processing algorithms for retinal montage synthe-
checking bifurcation estimates and positioning the IV needle sis, mapping, and real-time location determination,” IEEE Trans. on
exactly in between the branches of a bifurcation. Finally, we Biomed. Eng., vol. 45, no. 1, pp. 105–118, 1998.
[10] C. Tsai, “Model-based method for improving the accuracy and re-
tested the performance of our algorithm on a set of 50 images peatability of estimating vascular bifurcations and crossovers from
and were able to correctly find and characterize 32.4% of retinal fundus images,” IEEE Transactions on Information Technology
all bifurcations with a false positive rate of 11.6% and find in Biomedicine, vol. 8, no. 2, pp. 122–130, 2004.
[11] A. Can, “Rapid automated tracing and feature extraction from retinal
at least one bifurcation in 82.6% of images that contained fundus images using direct exploratory algorithms,” IEEE Transac-
bifurcations. tions on Information Technology in Biomedicine, vol. 3, no. 2, pp.
The experimental results have revealed several insights 125–138, 1999.
[12] L. Gagnon, “Procedure to detect anatomical structures in optical fun-
as to the limitations of our algorithm. The algorithm has dus images,” Proc. of Conf. Medical Imaging 2001 : Image Processing,
particular difficulty in detecting closely-spaced bifurcations pp. 1218–1225, 2001.
because they appear as a single, noisy bifurcation rather than [13] B. Kochner, “Course tracking and contour extraction of retinal ves-
sels from color fundus photographs: Most efficient use of steerable
distinct bifurcations. A limitation of the centering algorithm filters for model based image analysis,” SPIE Conference on Image
is the assumption that bifurcations consist of 3 straight Processing, vol. 3338, pp. 755–761, 1998.
branches. In some cases, there are more than 3 branches, or [14] I. Meglinski, “Quantitative assessment of skin layers absorption and
skin reflectance spectra simulation in the visible and near-infrared
the branches are sufficiently curved to break the centering spectral regions,” Physiol. Meas., vol. 23, pp. 741–753, 2002.
process. A further problem is that each tracking window
maintains a constant size as it tracks along a wrist vein, even
though the veins periodically change diameter. This can lead
to either too much noise for an oversized tracking window
or too little information for an undersized tracking window.
Dynamic resizing of the tracking window is a potential
solution to this problem.
Beyond addressing these limitations, future work will
focus on using depth information from the stereo images
to calculate the desired world-frame position and orientation
(roll, pitch, and yaw) of the IV needle based on the selected
bifurcation in a particular image. The robot will use this
calculated configuration to generate a trajectory for insertion.

4602