I.

INTRODUCTION

We present a new approach for multi-target

Multi-Target/Multi-Sensor detection and tracking, in which information from
multiple, spatially diverse, radar sensors is combined
Tracking using Only Range and to improve track reliability and accuracy. In particular,
we treat the difficult case in which azimuthal
Doppler Measurements measurements are either coarse or unavailable, so
that multiple sensors are required to triangulate
target tracks using range and Doppler measurements
only. However, data association in this problem is
ROSS DEMING, Senior Member, IEEE
extremely complex for several reasons. First, the
General Dynamics Information Technology, Inc.
coarse azimuthal resolution can result in significant
JOHN SCHINDLER, Life Fellow, IEEE
overlap between the signatures of the multiple
General Dynamics Information Technology, Inc.
targets and clutter. Second, increasing the number
LEONID PERLOVSKY, Senior Member, IEEE
of sensor platforms leads to an exponential increase
Air Force Research Laboratory
in the number of mappings between signatures and
hypothesized targets. Thus, it becomes impractical
A new approach is described for combining range and to sort out the associations using combinatorial
Doppler data from multiple radar platforms to perform
approaches.
multi-target detection and tracking. In particular, azimuthal
Multiple hypothesis tracking (MHT) [1, 2], a
measurements are assumed to be either coarse or unavailable,
so that multiple sensors are required to triangulate target tracks
benchmark multi-target tracking algorithm, performs
using range and Doppler measurements only. Increasing the
data association using an exhaustive evaluation of
number of sensors can cause data association by conventional all mappings between targets and data samples, and
means to become impractical due to combinatorial complexity, is therefore subject to a combinatorial explosion
i.e., an exponential increase in the number of mappings as the amount of data and the number of sensors
between signatures and target models. When the azimuthal increase. Pruning or gating are typically used to
resolution is coarse, this problem will be exacerbated by the alleviate the computational burden by eliminating
resulting overlap between signatures from multiple targets and the less likely hypotheses, however valid hypotheses
clutter. In the new approach, the data association is performed may also be eliminated in the process. Therefore,
probabilistically, using a variation of expectation-maximization data having low signal-to-clutter ratio (S/C) may
(EM). Combinatorial complexity is avoided by performing
be discarded, which can lead to missed detections.
an efficient optimization in the space of all target tracks and
mappings between tracks and data. The full, multi-sensor,
An alternative to MHT is joint probabilistic data
version of the algorithm is tested on simulated data. The
association (JPDA) [3—5] which is more efficient
results demonstrate that accurate tracks can be estimated by than MHT because one only needs to evaluate the
exploiting spatial diversity in the sensor locations. Also, as a association probabilities separately at each time step.
proof-of-concept, a simplified, single-sensor range-only version Whereas more simplistic single scan methods (such
of the algorithm is tested on experimental radar data acquired as nearest neighbor approaches) consider only the
with a stretch radar receiver. These results are promising, and single observation closest to the predicted state, JPDA
demonstrate robustness in the presence of nonhomogeneous is more robust since the state of each track is updated
clutter. using a weighted average of all measurements falling
Manuscript received February 7, 2007; revised October 11, 2007;
within its validation region at the current time step.
released for publication February 28, 2008. However, since it is a single scan process, not all
possible data-to-track mappings are considered. A
IEEE Log No. T-AES/45/2/933007.
drawback of JPDA is that while it is appropriate for
Refereeing of this contribution was handled by E. Chornoboy. track maintenance, it lacks an explicit mechanism for
All work was conducted on-site at the Air Force Research track initiation [6]. Both MHT and JPDA have been
Laboratory, Hanscom Air Force Base. adapted for multi-sensor scenarios [5].
An abbreviated version of this work was published in Proceedings
A data association approach based upon linear
of the IEEE Radar Conference, 2007, and the IEEE International programming has been proposed which, like JPDA,
Conference on Integration of Knowledge Intensive Multi-Agent updates the track states using weighted averages of
Systems, 2007. measurements [6]. Results from computer simulations
Authors’ address: Air Force Research Laboratory, Sensors indicate significantly lower computational complexity
Directorate/RYHE, 80 Scott Drive, Hanscom, AFB, MA 01731, than JPDA, as well as improved accuracy. Moreover,
E-mail: ([email protected]; [email protected]; this method provides an explicit mechanism for track
[email protected]). initiation.
Recently, sequential Monte Carlo methods, a.k.a.
0018-9251/09/$25.00 °
c 2009 IEEE “particle filters,” have been adapted for multi-target

IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 45, NO. 2 APRIL 2009 593
tracking problems [7—9]. Whereas Kalman filters is flexible with respect to the choice of track model,
are used in traditional JPDA methods for updating but normally the choice is to include continuous
track states, particle filters are more appropriate for polynomial models (e.g., constant velocity, constant
situations involving nonlinear state and measurement acceleration, etc.), or piecewise polynomial models,
equations and non-Gaussian noise. Data association for target trajectories. The use of polynomial models
for particle filter methods can be performed in various leads to very simple parameter update formulas in the
ways, as described in [8]. M-step, for example, simple matrix inversions which
Both MHT and JPDA assume–often are similar in structure to polynomial regression [22].
correctly–that a target can generate at most one Of course, under certain conditions the discrete-time
measurement per scan. However, if this constraint state transition equations used in PMHT would be
is relaxed, data association can be formulated equivalent to polynomial models. Another difference
using a continuous optimization procedure, notably is in the choice of optimization criterion. Whereas in
expectation-maximization (EM) [10—12], rather than PMHT the goal is maximization of the a posteriori
combinatorics. Thus, an efficient “hill climbing” probability (MAP) [15], Perlovsky uses maximum
optimization is performed in the space of all model likelihood estimation (MLE) in the case of sampled
parameters and all possible mappings between data data, and minimization of the cross-entropy in the
samples and hypothesized targets. An important case of pixelated data [22]. Avitzour’s procedure
advantage of this formulation is that computational is maximum likelihood, like Perlovsky’s, however
complexity scales only linearly with the number different tracking models are specified due to
of targets and sensors, whereas combinatorial differences in the particular applications.
approaches such as MHT scale exponentially. Thus, The algorithm described in the present paper is
trackers based upon the EM formulation would, in developed along the lines of Perlovsky’s general
principle, be practical for scenarios with high clutter approach, and utilizes EM for data association.
and/or high densities of targets. Streit, et al. [13—15] However, a major new aspect considered here is
developed the probabilistic multihypothesis tracker the data model, which is particularly complicated,
(PMHT) that utilizes EM to perform data association and arises from combining data from multiple
while simultaneously estimating tracks based upon sensors, where azimuthal measurements are absent.
multiple scans of data. This approach has also been Thus, the goal here is to use multiple sensors to
extended by other authors, for example to cases triangulate target tracks using range and Doppler
with multiple sensors [16—18] and maneuvering measurements only, while performing data association
targets [19]. Avitzour [20] and Perlovsky [21, 22] probabilistically. An abbreviated version of this work
also, independently, developed maximum-likelihood was published previously [30, 31], although some
procedures for multi-target tracking which use results and most of the mathematical details were not
EM for data association. Perlovsky demonstrated included.
mathematically [22, 23], using Cramer-Rao bound An important consideration for any tracking
analysis, that the utilization of classification features algorithm is computational cost. When applied to
within the tracker is equivalent to an improvement a benchmark (single-sensor, single-target) tracking
in S/C ratio in high-clutter tracking environments. problem, it was found that the computational
Subsequently, he developed a version of the algorithm cost of PMHT is roughly the same order of
in which classification and tracking are performed magnitude as the cost of MHT and JPDA [27]. The
concurrently [22, 24—26]. Here, if they are available, computational cost of PMHT versus other methods
classification features (e.g., radar cross section (RCS), has also been evaluated for multi-target tracking
length, etc.) are placed on an equal footing with [28, 29]. Perlovsky’s approach would likely have
tracking features (e.g., range, Doppler, bearing, etc.). a computational cost similar to PMHT, since they
The model is then a mixture of different types of share a similar structure, and are both based upon
target and clutter components in the combined space EM. For multi-sensor, multi-target applications,
of tracking and classification features. Like PMHT, MHT and JPDA would likely require hypothesis
Perlovsky’s approach is a multi-scan (i.e., “batch”) pruning in order to avoid an exponential increase
algorithm. in computational cost. Since the computational cost
Perlovsky’s approach differs from PMHT chiefly of EM-based approaches scale only linearly with
in the choice of track model. In PMHT the motion increasing numbers of sensors, it is expected that the
of each target is modeled using a set of discrete-time advantages of EM-based approaches would become
state transition equations, and therefore a Kalman increasingly evident for multi-sensor applications,
smoother is used to estimate track parameters such as the one considered in this paper.
in the M-step of the EM iterations [13—15] (in The paper is organized as follows. In
nonlinear/non-Gaussian cases the target states are Sections II—IV the mathematical approach is
obtained using dynamic programming rather than developed for the full, multi-sensor, version of the
Kalman smoothers). In contrast, Perlovsky’s approach algorithm. The EM algorithm is a well-established

594 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 45, NO. 2 APRIL 2009
mathematical tool for estimating the parameters in
mixture models [10—12], and could certainly be
used as a framework for deriving the solution to
our problem, as was done for the PMHT [13—15].
However, in this paper a different approach is
offered, which Redner and Walker [12] refer to as
the “traditional general approach.” Here, a system
of likelihood equations is derived, then solved using
the partial derivatives of the optimization criterion,
as described in [22], [32]. This derivation does not
require an understanding of EM. In addition to the
full, multi-sensor, algorithm, a simplified version
is also derived in Section V which is appropriate
for performing range-Doppler-only tracking from
single-sensor data. In this case, the update formula
for track parameters is particularly simple and Fig. 1. Sample stack of three pixelated scan frames acquired
efficient–it consists of a small matrix inversion for with a particular sensor. Target signature appears as high intensity
each target component. Most of the mathematical (dark) blob centered around its true range/Doppler coordinate.
details are contained in the appendices. Appendix Since target is in motion, signature position is slightly different in
A contains a derivation of the parameter estimation each scan frame. Similar data stacks would be acquired using
other sensors, and entire set of stacks from all sensors is
equations for general mixture models, and also processed jointly to estimate target tracks.
tailors these equations to the tracking model used
in this paper. Appendix B contains a convergence
proof which, again, is within the structure of the frame.” Examples of scan frames are shown in Figs. 2
traditional general approach rather than depending and 3 in Section VI, where the target and clutter
upon EM. Finally, in Section VI results are presented signatures appear as blobs of energy centered upon
from experiments designed to test the algorithm. their true range and Doppler. The brightness (strength)
The simplified, single-sensor version is tested on of each blob is proportional to the range and RCS
experimental radar data, while the full, multi-sensor, of the reflectors. The target and ground clutter may
version is tested on synthetic data. The results, occupy several range-Doppler resolution cells. Of
although preliminary, demonstrate robustness in the course, the range resolution is inversely proportional
presence of nonhomogeneous clutter, and uncertainty to the bandwidth of the transmitted signal, and
in the number of targets present. Section VII provides is degraded by the associated pulse compression
a summary and discusses directions for future processing. Similarly, the Doppler resolution is
research. inversely proportional to the CPI, and is degraded
by sensor platform vibration, system instability, and
processing. The spreading of the target signature
II. DESCRIPTION OF THE SENSORS AND DATA
across several range-Doppler cells may be associated
The sensor model for this discussion incorporates with target speed and acceleration, target size, and
a ground-based radar antenna having poor azimuthal scintillation of target cross section. Ground clutter
resolution. In fact, we will assume the extreme case in appears in the scan frame image as a ridge centered at
which each sensor measures target range and Doppler zero Doppler and spread across all range bins (again,
(range-rate) only, but no azimuth. Thus, the track of refer to Figs. 2 and 3 in Section VI). The spread of
each target can only be estimated by triangulation the ridge beyond the Doppler resolution cell may be
from multiple, spatially diverse, sensor platforms. In caused by antenna scan modulation, sensor platform
this work, we focus on the 2-dimensional tracking vibration, as well as internal motion of the clutter
problem in which targets are assumed to lie on the (e.g., movement of leaves and branches in the wind).
zero-elevation plane, while sensors have arbitrary Depending on its width, this clutter ridge may in some
3-dimensional positions. cases obscure or partially obscure the signatures from
Our method is appropriate for any collection of targets with small radial velocities. Finally, there will
range/Doppler data, although as a working model be a certain amount of background noise (receiver
it is assumed the data are acquired using a stretch noise) which is uniformly distributed in range/Doppler
receiver [33]. Here, a suite of long-duration chirps over the image.
is transmitted from a stationary ground-based radar, It is assumed the data are collected as follows
and the corresponding suite of received signals within (Fig. 1). At each time tj , for j = 1, 2, 3, : : : , J, the
the coherent processing interval (CPI) is processed received signals within the CPI centered at tj are
jointly to produce a two-dimensional digitized image processed jointly to yield a sampled or pixelated
in range/Doppler coordinates, referred to as a “scan range/Doppler image (scan frame) as described above.

DEMING ET AL.: MULTI-TARGET/MULTI-SENSOR TRACKING 595

Thus, for each sensor m = 1, 2, 3, : : : , M we acquire a As described in Section II, the target signatures
stack of J range/Doppler scan frames corresponding appear as blobs of energy in the scan frame images
to the different times tj . Note that, due to the irregular p0 (wjmn ), where recall that wjmn = (rjmn , djmn ) is
overlap in the fields-of-view of the different sensors, the range and Doppler value of the nth pixel in
the signatures from some targets may not appear the frame pertaining to the mth sensor at the jth
in the images from all sensors at all times. Each time. The signature corresponding to target k is
range/Doppler scan frame image is described using roughly centered at the true range Rjkm and Doppler
the notation p0 (wjmn ), where p0 is the pixel intensity Djkm , measured relative to sensor m at time tj .
at the range/Doppler coordinate wjmn = (rjmn , djmn ), We wish to construct a mathematical model that
which is indexed by time index j (i.e., scan frame approximates these signatures, and for target k we
number j), sensor m, and pixel n = 1, 2, 3, : : : , N. denote this model as pk (wjmn j £k ), where £k is the
The total number of pixels in each scan frame is set of model parameters. In this case it makes sense
N = Nr £ Nd , where Nr and Nd are the number of bins to use Gaussian distributions to model the target
in range and Doppler, respectively. The width of each signatures (blobs of energy), so that the signature
pixel with respect to range and Doppler is ¢r and due to target k for sensor m and time tj is modeled
¢d , respectively. The data from the multiple sensor by
platforms are combined noncoherently to estimate
pk (wjmn j £k ) =
target tracks, as described in the following sections.
( "μ ¶2 μ ¶2 #)
¢r ¢d 1 rjnm ¡ Rjkm djnm ¡ Djkm
exp ¡ + :
III. MODEL FOR THE DATA 2¼¾rk ¾dk 2 ¾rk ¾dk

A model for the data p0 (wjmn ) can be developed (5)

which depends upon the target trajectories as well Here, the set of model parameters is £k =
as the sensor parameters and coordinates. Suppose f¾dk2 2
, ¾rk , xk0 , yk0 , xk0 , yk0 , xk00 , yk00 g where, in (5), the
the coordinates of target k at time tj are given by track parameters fxk0 , yk0 , xk0 , yk0 , xk00 , yk00 g are contained
the (east, north) coordinate (xk (tj ), yk (tj )) ´ (xjk , yjk ). implicitly within the quantities Rjkm and Djkm
Using a constant acceleration model, the trajectories according to (3) and (4). Also, ¢r and ¢d define
are described by the pixel size relative to the range and Doppler
coordinates, as discussed in Section II. Note
xjk = xk0 + xk0 tj + xk00 tj2 (1)
that the distribution in (5) is normalized1 so that
P N
yjk = yk0 + yk0 tj + yk00 tj2 : (2) n=1 pk (wjmn j £k ) = 1. The variance parameter
2
¾rk specifies the spread (width) of the signatures
Here (xk0 , yk0 ) is the time-zero position of target k, with respect to the range coordinate in the scan
while (xk0 , yk0 ) is the time-zero velocity and (xk00 , yk00 ) frame image, and is related to the range resolution
are proportional to the x and y components of the of the sensor and the range extent of the target.
2
acceleration. The targets are assumed to lie at zero The other variance parameter ¾dk specifies the
elevation, which simplifies the discussion, as well as width of the signature in Doppler, and depends
reducing the number of track parameters that need both on the sensor resolution and the target motion,
to be estimated. The sensors are allowed to have including internal (micro) motion. It should be
arbitrary 3-dimensional coordinates. Suppose there noted that the model could alternatively have
are M sensor platforms, where sensor m is fixed at the been formulated so that the variance parameters
(east, north, elevation) coordinate (xm , ym , zm ). Then, depend upon time index j or sensor m, as well as
the range Rkm (tj ) ´ Rjkm from sensor m to target k at sensor k.
time tj is given by the equation The interference also requires a model, and we
q approximate this using two components, one for
Rjkm = (xm ¡ xjk )2 + (ym ¡ yjk )2 + zm2 : (3) the uniform background noise (receiver noise), and
one for the stationary clutter. We designate the two
The Doppler (range-rate) Dkm (tj ) = [@Rkm (t)=@t]t=tj is interference indices as k = (K ¡ 1), K and the target
then indices as k = 1, 2, 3, : : : , (K ¡ 2). The background
noise (receiver noise) component k = K ¡ 1, uniform
Dkm (tj ) ´ Djkm
(constant) in both range and Doppler, is described by
à !
xm ¡ xjk 1
=¡ (xk0 + 2xk00 tj ) pK¡1 (wjmn j £K¡1 ) = (6)
Rjkm N
à !
ym ¡ yjk
¡ (yk0 + 2yk00 tj ): (4) 1 Actually,
the normalization is approximate, and assumes the pixel
Rjkm dimensions (¢r , ¢d ) are small relative to (¾rk , ¾dk ).

596 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 45, NO. 2 APRIL 2009
where N is the number of pixels in each we will investigate how model selection strategies
range/Doppler image, as discussed above. The described in the literature can be adapted to the
1=N factor is necessary for normalization, i.e., to tracking problem, for choosing the number of target
P
insure N n=1 pk (wjmn j £k ) = 1. Note, here the set of
components, pruning unneeded components, and
parameters £K¡1 is empty. The other interference adding components when necessary.
component k = K, which models the reflected energy A useful result can be obtained by summing both
from stationary ground clutter, is uniform (constant) in sides of (8) over the pixel index
P n, and using the
range, and zero-mean Gaussian in Doppler, i.e., (previously stated) fact that N n=1 pk (wjmn j £k ) = 1,
( to obtain
μ ¶2 ) X X
¢d 1 djnm p(wjmn j £) = Ekm (9)
pK (wjmn j £K ) = p exp ¡ : n
2¼¾dk Nr 2 ¾dK k

for each j = 1, 2, : : : , J. Here we have assumed that if

(7)
a target is in the field-of-view of a particular sensor
Here, Nr is the number of range bins in each pixel m, it will remain in the field-of-view for the entire
array, as discussed in the previous section. Note, set of scan frames j = 1, 2, : : : , J. Thus, the sum of
2
here the set of parameters is £K = f¾dK g. A further the relative model weights is independent of j. It
discussion of clutter modeling is given in [22]. is sensible to place a constraint so that, for each
The total model for the image data is the weighted scan frame j and sensor m, the total energy in the
summation (mixture) of individual target and clutter modelP equals the totalP energy in the measured data,
components i.e., n p(wjmn j £) = n p0 (wjmn ). Combining this
K
X constraint with (9), we obtain the following constraint
p(wjmn j £) = Ekm pk (wjmn j £k ): (8) on the mixture weight parameters, valid for every scan
k=1 frame j and sensor m:
X X X
Here, Ekm is the relative weighting for each of the p(wjmn j £) = p0 (wjmn ) = Ekm : (10)
model components, which we refer to as the mixture n n k
weight. For target components, the mixture weight is
proportional to the target’s RCS and range as viewed
IV. PARAMETER ESTIMATION
by sensor m. Since RCS can be strongly dependent
upon aspect angle, we allow Ekm to depend upon both The goal is to find the set of model parameters
target k and sensor m. This dependence upon sensor £ = fEkm , £k g providing the best match between
m also allows for the fact that the fields-of-view for the the model p(wjmn j £) and the data p0 (wjmn ).
the different sensors overlap in an irregular manner, so The model, described in Section III, is completely
that a signature from a certain target may not appear specified by the the mixture weights Ekm ,
in the data from all sensors. In (8), the total set of the variances ¾dk 2 2
and ¾rk , and the parameters
model parameters is denoted by £ = fEkm , £k g, where 0 0 0 0 00 00
fxk , yk , xk , yk , xk , yk g describing the target trajectories.
k = 1, 2, : : : , K and m = 1, 2, : : : , M. In this paper the Einsteinian log-likelihood [22,
The well-known problem of estimating the best ch. 4.4.1]
number of components K in mixture models is X
discussed in the literature. For example, mathematical LL(£) = p0 (wjmn ) ln p(wjmn j £)
strategies are described in [34], [35], [36] and the j,m,n
references therein. In our application, this so-called K
X X
“model selection” problem stems from the fact that the = p0 (wjmn ) ln Ekm pk (wjmn j £k )
true number of targets is not known in most contexts, j,m,n k=1
and therefore we must make an educated guess for
the appropriate number of model components K. (11)
If there are more model components than targets, will serve as a criterion to quantify the similarity
unneeded components can presumably be eliminated between the model and the data. Given the constraint
automatically as their mixture weights Ekm adapt in (10), it can be shown that LL will be maximized
to small values. A tracking example is presented in when the model and the data match, i.e., for
Section VI where this type of process is illustrated. p(wjmn j £) = p0 (wjmn ). Note that maximizing LL is
In cases where the true number of targets exceeds equivalent to minimizing the cross-entropy, a.k.a, the
the number of model components, the behavior Kullback-Leibler (KL) divergence, a well-established
of the model during its adaptation is not as well metric from the information theory arena [41].
understood. It is speculated that either the process Mathematical and physical justifications for both
will lock onto K ¡ 2 targets, ignoring the rest, or Einsteinian log-likelihood and the KL divergence
some model components will lock onto multiple have been discussed in detail [22, 37—40]. It should
targets sharing similar parameters. In a future study, be emphasized that the tracking algorithm described

DEMING ET AL.: MULTI-TARGET/MULTI-SENSOR TRACKING 597

here is a “batch” processing approach, i.e., the frames are initialized to large values, the components are
from all time indices j are processed jointly. This is initially fuzzy and indistinct. However, with increasing
apparent in (11) since the log-likelihood includes a iterations, the variances tend to adaptively decrease,
summation over j. and the individual components gradually converge and
The mathematical details of the optimization lock on to the target signatures, or to portions of the
are contained in Appendix A. Briefly, a system clutter. Thus, the model iteratively adapts to fit the
of equations is derived which is satisfied by the data, as we demonstrate with the results in Section VI.
parameters maximizing LL, subject to the constraint If the model is viewed probabilistically
of (10). Unfortunately, an analytical solution is [22, ch. 4.4.8], then (15) is simply a form of
intractable since the system of equations is large, Bayes’ rule, and the quantities P (I) (k j jmn) can
coupled, and nonlinear. However, iterative techniques be construed as the probabilities that the energy
can be employed to solve the system of equations in pixel (j, m, n) originates from target or clutter
and, in Appendix A, an efficient recursive technique component k. Therefore, P (I) (k j jmn) are referred
is described which is a special case of EM. This to as the “association probabilities” [22]. Equation
technique defines a recursive update formula for each (12) therefore makes intuitive sense–it simply states
(I) (I+1)
parameter. For example, using the notation Ekm and that Ekm is the sum of all pixel values p0 (wjmn ),
(I) where the pixels are weighted by their association
£k to indicate the estimates of the parameters at the
Ith iteration, the recursive update formulas for Ekm , probabilities. Similarly, (13) and (14) correspond
2
¾dk 2
, and ¾rk are to the usual definition of sample variance, with
the distinction that the pixels are weighted by their
(I+1) 1X association probabilities. From (15) it is apparent that
Ekm = p0 (wjmn )P (I) (k j jmn) (12)
J X
j,n
P (I) (k j jmn) = 1 (16)
P (I)
2 (I+1) j,m,n p0 (wjmn )P (k j jmn)(djnm ¡ Djkm )2 k
(¾dk ) = P (I)
j,m,n p0 (wjmn )P (k j jmn)
for any iteration I.
For compactness, we define the angled bracket
(13) notation
and X
P (I)
h¤i(I) ´ p0 (wjmn )P (I) (k j jmn)(¤) (17)
2 (I+1) j,m,n p0 (wjmn )P (k j jmn)(rjnm ¡ Rjkm )2 j,n
(¾rk ) = P (I) (k
j,m,n p0 (wjmn )P j jmn)
where the asterisk ¤ denotes a generic quantity. Thus,
(14) (12)—(14) can be rewritten in the compact form
where
(I) (I+1) h1i(I)
Ekm pk (wjmn j £k(I) ) Ekm = (18)
P (I) (k j jmn) = P J
(I)
k 0 Ek 0 m pk 0 (wjmn j £k(I)0 ) P 2 (I)
2 (I+1) m h(djnm ¡ Djkm ) i
(¾dk ) = P (19)
E (I) pk (wjmn j £k(I) ) m h1i
(I)
= km : (15)
p(wjmn j £(I) ) and P
2 (I+1) m h(rjnm¡ Rjkm )2 i(I)
There is an analogous rule for updating the tracking (¾rk ) = P (I)
: (20)
parameters fxk0 , yk0 , xk0 , yk0 , xk00 , yk00 g, which is described m h1i

later in this section. The equations above imply The converged values of Doppler variance
starting with an initial guess for the parameters 2
parameters ¾dk are related to the radar Doppler
(0)
fEkm , £k(0) g, then alternating between updating resolution and target dynamics, and are thus difficult
(I)
P (k j jmn) using (15), then updating the parameters to estimate a priori. On the other hand, the converged
(I+1)
fEkm , £k(I+1) g using (12)—(14). In fact, these steps values of the range variance parameters ¾rk 2
are
correspond to the E-step and the M-step, respectively, related to the radar range resolution, modified by
of the EM algorithm. This iterative procedure is the range extent of the target, and are assumed to
guaranteed to converge to a local maximum of LL, as be better known a priori. Therefore, rather than
shown in Appendix B. Note that the mixture weights adaptively estimating range variance using (20), it
Ekm are updated using (12) for all target and clutter may be advantageous to evolve ¾rk according to a
components k = 1, 2, : : : , K. In fact, (12) is appropriate predetermined schedule [22, 25, 26]. For example,
for updating the weights of arbitrary mixture models, ¾rk can be initialized to a rather large value, so that
not only the tracking model considered here. The the target components can “see” large sections of the
2 2
variances ¾dk and ¾rk are updated for the appropriate pixel data, then decreased according to an exponential
model components, as indicated by the target and decay during EM iterations to a steady state value
clutter model equations. Since the variance parameters corresponding to the appropriate sensor resolution.

598 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 45, NO. 2 APRIL 2009
 Finally, we derive the recursive update formula for are still updated using (18)—(20). However, in the
each of the tracking parameters fxk0 , yk0 , xk0 , yk0 , xk00 , yk00 g, single-sensor case the target track parameters can be
k = 1, 2, 3, : : : , (K ¡ 2). In the single-sensor case, updated using a closed-form formula, which is simpler
described below in Section V, the update formula for and more efficient than the gradient ascent procedure
these parameters is a simple closed-form expression, described by (21) used for the multi-sensor case. Note
analogous to (18)—(20). Unfortunately, a closed-form that the analysis in this section is very similar to the
update formula cannot be derived for the general analysis given in [22, ch. 7.2.6]. However, whereas the
multi-sensor case. However, local convergence is the previous work utilized a constant velocity model,
still guaranteed (see Appendix B) if the tracking here a constant acceleration model is used which is
parameters are simply “nudged” along the uphill slightly more complicated.
gradient of LL during each iteration. This actually Suppose the range between the targets and the
corresponds to a special case of the generalized (single) sensor can be described using a constant
EM (GEM) procedure [11]. Using sk as a generic acceleration model, i.e.,
placeholder for any one of the tracking parameters Rjkm = Rk0 + Rk0 tj + Rk00 tj2 (23)
fxk0 , yk0 , xk0 , yk0 , xk00 , yk00 g, the update formula is given by
· ¸ where Rk0 is the time-zero range of target k, Rk0 is the
@LL time-zero range-rate, and Rk00 is proportional to the
sk(I+1) = sk(I) + h ¢ (21)
@sk range-acceleration. Although there is only a single
where h is the gradient ascent stepsize, and sensor m = 1, the m subscript is maintained in the
quantity Rjkm for consistency with the expression for
¿μ ¶μ ¶À(I)
@LL X rjnm ¡ Rjkm @Rjkm the target model in (5). The Doppler (range-rate) at
= 2 time tj is the derivative of range with respect to time
@sk m
¾rk @sk
evaluated at tj , i.e.,
X ¿μ djnm ¡ Djkm ¶ μ @Djkm ¶À(I)
+ : Djkm = Rk0 + 2Rk00 tj : (24)
2
¾dk @sk
m
In Appendix A, an update formula is derived for
(22) iteratively adjusting the values of fRk0 , Rk0 , Rk00 g in order
This expression is derived in Appendix A. It is to maximize LL which, as in the multi-sensor case, is
straightforward to compute explicit expressions for described by (11). This update formula is described
@Rjkm =@sk and @Djkm =@sk for each of the parameters by the following 3 £ 3 set of linear equations:
8 0 (I+1) 9 8 9
sk = fxk0 , yk0 , xk0 , yk0 , xk00 , yk00 g using (3) and (4). For 2
hrjnm i=¾rk
< (Rk )
> >
= > < >
=
example, @Rjkm =@xk0 = ¡(xm ¡ xjk )=Rjkm . Although
H̄k (Rk0 )(I+1) = 2
hrjnm tj i=¾rk + hdjnm i=¾dk 2
(21) describes a single gradient ascent step during >
: 00 (I+1) >
; > : >
;
each iteration, multiple steps can be taken during each (Rk ) hrjnm tj2 i=¾rk
2
+ h2djnm tj i=¾dk 2

iteration while holding the association probabilities (25)

P(k j jmn) fixed. In practice, the convergence rate
where
can be optimized, using trial-and-error to determine a 8 9 8 9
htj2 i >
suitable combination of stepsize h and steps/iteration. < h1i
> htj i
= <0
> 0 0 >
=
1 1
Although the present study is mainly concerned H̄k = htj i htj2 i htj3 i + 2 0 h1i h2tj i :
with tracking, it is important to consider the issue
2
¾rk >
: 2 >
; ¾dk >
: >
;
htj i htj3 i htj4 i 0 h2tj i h4tj2 i
of automatic detection and target declaration. The
standard detection approach, e.g., described in [22, Here, for example, (Rk0 )(I+1) denotes the updated
sect. 7.2.9], utilizes a log-likelihood ratio test which estimate for Rk0 computed at the (I + 1)th iteration. For
operates on the converged parameter values. The the sake of compactness, the (I) superscript has been
issue of detection will be studied in more detail in the omitted from the angled bracket notation h¤i used for
future. the elements in the above matrices. However it should
be understood that here
X
V. RANGE-ONLY TRACKING FROM A SINGLE h¤i ´ p0 (wjmn )P (I) (k j jmn)(¤): (26)
SENSOR j,n

If only a single sensor platform is available, and It can be shown that, in the absence of Doppler
measurements of azimuth are unavailable, it may still measurements, the set of equations (25) is equivalent
be desirable to perform range-Doppler-only tracking. to second-order polynomial regression in which
Here, the target and interference (clutter plus noise) data samples are weighted by their association
models given by (5)—(7) remain valid, as well as probabilities.
the total model for the data given by (8). Also, the Unlike the update formulas for Ekm , ¾dk , and
2 2
variance (¾dk , ¾rk ) and mixture weight Ekm parameters ¾rk , given by (18)—(20), the matrix expression

DEMING ET AL.: MULTI-TARGET/MULTI-SENSOR TRACKING 599

 Fig. 2. Upper left: scan frame data acquired at time t = 0 s in which a single target signature appears at Doppler and range values of
roughly (¡12 m/s) and (258 m), respectively. Remaining three plots show evolution of model, which adapts to fit data as iterations
increase. White Xs in lower right plot indicate converged range/Doppler coordinates Rk0 and Rk0 for target components in model.

(25) actually represents a set of three coupled target tracks nor detailed information about the radar
update formulas for the three tracking parameters were provided. Nevertheless, this analysis serves
fRk0 , Rk0 , Rk00 g. Upon each iteration, these parameters as a useful proof-of-concept for the algorithm. The
are updated by inverting (25). Note that a separate experimental setup consisted of a stretch receiver
3 £ 3 matrix inversion is performed for each target mounted on a tower overlooking a wooded area,
component k = 1, 2, : : : , (K ¡ 2). and data were collected as various targets moved
through the area. The raw data were converted to a
VI. RESULTS suite of range/Doppler scan frames using standard
processing techniques [33], as discussed in Section II.
In this section, we describe results when the The algorithm described in Section V was used to
algorithm is applied to two cases. First, the algorithm jointly process multiple sets of scan frames, where
is tested against experimental stretch radar data each scan frame is computed from data centered
[33] from a single sensor using only the range and around a different time tj , j = 1, 2, : : : , J.
Doppler data from the sensor. The single sensor The upper left plot of Fig. 2 shows a single scan
results are presented as a proof of concept and to frame acquired at a reference time of t = 0 s. A
illustrate the performance of the algorithm in realistic, target signature appears as a “blob of energy” at
inhomogeneous clutter, variable signal to interference, Doppler and range values of roughly (¡12 m/s) and
and accelerating and maneuvering targets. In the (258 m), respectively. There is a significant ridge
second case, we consider multiple sensors viewing of ground clutter from the stationary background
the same scene. The sensors are assumed to have poor centered at zero Doppler, extending across all range
or no azimuth resolution but good range and Doppler bins. There is also a noise background which is
resolution. Combining the data from multiple sensors roughly uniform in range/Doppler, although with
offers the opportunity to more accurately locate some significant inhomogeneities. For the model
and track targets using triangulation of the sensor we used five components: three target components,
data if the difficult problem of target association one uniform background noise component, plus one
can be solved. In the second case we present results clutter component shaped like a ridge centered at zero
from computational experiments designed to test the Doppler.
algorithm since experimental multi-sensor data are not The remaining three plots of Fig. 2 show how
available to us. the model evolves with increasing iterations. Since
Let us now describe the single-sensor results the model variances are initialized to large values,
computed from experimental data, using the simplified the model is initially broad, fuzzy, and indistinct
version of the algorithm described in Section V. These (upper right). However, as the iterations progress, the
results will be mainly qualitative since neither true parameters adapt in such a way as to eventually define

600 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 45, NO. 2 APRIL 2009
 Fig. 3. Same as Fig. 2, however here scan frame was acquired at a later time of t = 6 s. Target signature has now moved to Doppler
and range values of roughly (¡9 m/s) and (195 m), respectively.

Fig. 4. Left-hand plot shows same scan frame shown in Fig. 3, however a large amount of artificial noise was added which almost
completely obscures target signature. Nevertheless (right-hand plot), one of the model components was able to adapt to range/Doppler
coordinate in close vicinity of target signature.

a model which shares most of the significant features even more extreme than for the t = 0 case shown in
of the data. The lower right plot in Fig. 2 shows Fig. 2. However, the model was able to adapt quite
the converged model after 200 iterations. The white nicely, with one of the target components locking on
Xs indicate the converged range/Doppler estimates to the true target signature, and the extra components
Rk0 and Rk0 for the three target components. Notice adapting to fit the clutter.
that one of the three target components has properly In Figs. 2 and 3, the target signatures were rather
converged to the true target signature, while the extra strong, with signal-to-interference ratios of roughly
two target components have converged to portions 3 dB (computed from the peak value of the signature
of the inhomogeneous background clutter. Although relative to an average background value in the image
Fig. 2 shows the data and model corresponding to frame). Next, the tracking experiment was repeated
only a single scan frame, it is important to note that for the same case shown in Fig. 3. However the
the results were computed by jointly processing problem was intentionally made more difficult by
10 scan frames, starting at t = 0 and proceeding in adding a large amount of artificial noise, so that the
intervals of 0.25 s. signal-to-interference ratio was reduced to roughly
The upper left plot of Fig. 3 shows scan frame 0 dB. This noise is uniformly distributed in range
data corresponding to a different time, t = 6 s. The and Doppler, and is signal-independent, i.e., it was
target signature has now moved to Doppler and range simply added to the intensity image. The data and
values of roughly (¡9 m/s) and (195 m), respectively. the converged model for this noisy experiment are
Here, the inhomogeneities in the background are shown in Fig. 4, which shows that one of the model

DEMING ET AL.: MULTI-TARGET/MULTI-SENSOR TRACKING 601

 Fig. 5. Tracking over extended time interval for smoothly decelerating target. Each scan frame corresponds to different time instance
with 12 s interval. Overlaid is dark line showing track computed using our method.

components locked on to the target signature in the on the path was computed by jointly processing 6
data, despite the fact that the target signature is almost scan frames within the time interval 0:5 · tj · 1:75 s,
completely obscured by noise. etc. Fig. 5 shows results from tracking a smoothly
Although the present study is mainly concerned decelerating target. Four different scan frames are
with tracking, it is important to consider the issue shown corresponding to times of t = 0, 2.5, 6, and
of automatic detection and target declaration. For 12 s. Superimposed on the images is a dark line
example, in the converged model shown in the lower which indicates the extended track computed using
right plot of Fig. 3 there are three target components the sliding window method over a 12 s interval.
which converged to the three different range/Doppler Notice the close agreement between the location of
coordinates indicated with white Xs. One component the target signatures and the computed track. Next,
has locked on to the target signature, while the other Fig. 6 shows results from tracking a maneuvering
two have adapted to fit portions of the clutter. Here, target. Here the target accelerates significantly over
the detection problem consists of automatically the time observation interval, whereas previously
declaring the true target, while disregarding the other the target velocity changed relatively slowly. Each
two components. scan frame in Fig. 6 corresponds to a different time
The standard detection approach, e.g., described instance within a 15.5 s interval. Overlaid is a dark
in reference [22, sect. 7.2.9], utilizes a log-likelihood line showing the track computed using the sliding
ratio test evaluated at the converged track and variance window method which, again, shows close agreement
parameter values. In the examples shown thus far, it between the location of the target signatures and the
appears that perhaps a simple detection rule could computed track.
be devised based upon the compactness of the target In the remainder of this section results are
component, i.e., based upon the converged values presented for the full, multi-sensor, version of the
of the variance parameters ¾dk and ¾rk . The issue of algorithm described in Section IV. Since experimental
detection will be studied in more detail in a future multi-monostatic data were unavailable to us, the
publication. results are based on sets of synthetic range/Doppler
Next, sample results are presented in which scan frames of the type that would be produced from
tracking was performed over longer time intervals. UHF stretch radar data [33]. In these simulations,
Here, a sliding window approach was used to emphasis was more on data association than on
estimate tracks along extended, irregular, paths in the detection performance. Therefore, the data were
range/Doppler space. Overlapping sets (“batches”) of generated with a relatively high signal-to-interference
6 scan frames were used to compute each point along ratio of approximately 8 dB, computed from the
the path. For example, the first point on the path was peak value of the signature relative to an average
computed by jointly processing 6 scan frames within background value in the image frame. However,
the time interval 0 · tj · 1:25 s, the second point the zero-Doppler ridge from stationary clutter was

602 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 45, NO. 2 APRIL 2009
 Fig. 6. Tracking over extended time interval for maneuvering target. Each scan frame corresponds to different time instance within
15.5 s interval. Overlaid is dark line showing track computed using our method.

have constant velocity and two of which are turning

and accelerating. These four targets have x (east)
and y (north) components of their velocities given
by (xk0 , yk0 ) = (0, ¡4), (7, 5), (3, 2), (¡15, 1) m/s, and
corresponding acceleration components proportional
to (xk00 , yk00 ) = (¡0:4, 1), (¡2, 0), (0, 0), (0, 0) m/s2 . Sensors
1—3 are spaced roughly 10 meters apart,2 while the
positions of sensors 4 and 5 are more widely spaced.
The spatial diversity is exploited to “triangulate”
accurate track estimates for the targets. The targets
were observed by all five sensors at 4 different times
separated by 0.25 s. During the 4 frames of data the
targets move a distance comparable to the sensor
Fig. 7. Geometry for multi-sensor simulation. Each arrow shows range resolution. Thus, the algorithm was required
direction of motion for corresponding target, and numbers in to jointly process a total of twenty scan frames in
parentheses are x (east) and y (north) components of velocity order to estimate target tracks. Fig. 8 shows four of
(xk0 , yk0 ).
these twenty scan frames, corresponding to sensors
1 and 5 at times t1 = 0 and t4 = 0:75 s. From this
generated with roughly the same peak amplitude as figure it is apparent that the data association problem
the peak amplitude of the target signatures. Therefore is not trivial. For example, it is not obvious “by eye”
this ridge overlaps and obscures the target signatures which signatures in the bottom-right plot of Fig. 8
having low Doppler, as illustrated in Fig. 8. The correspond to which signatures in the top-left plot.
synthetic data were generated by placing “blobs” of The model was constructed of 8 components, i.e.,
energy at appropriate target and clutter locations at 6 target components plus a uniform noise component,
each frame time without errors or biases, other that plus a zero-Doppler clutter ridge component. We
those associated with the dimensions of the “blobs” assumed no prior knowledge about how many targets
which are related to system resolution and target were actually present, and therefore 2 more target
spreading effects. While errors and biases may be components were used than actual targets present.
important in the ultimate application, we want to The code seemed to converge most quickly by starting
illustrate by this example the native data association off with 30 iterations using data only from sensors
performance of the proposed multi-sensor algorithm 1—3 (these sensors are relatively close together),
with multiple targets and sensors.
Fig. 7 shows the geometry for the experiment. 2 Sensors1—3 do not operate as an array since their data are
There are 5 sensors and 4 targets, two of which processed noncoherently.

DEMING ET AL.: MULTI-TARGET/MULTI-SENSOR TRACKING 603

 Fig. 8. Synthetic range/Doppler image data showing scan frames for sensor 1 (time = 0, 0:75 s) and sensor 5 (time = 0, 0:75 s). There
are 4 target signatures present. Note that signatures of lower velocity targets significantly overlap clutter ridge centered at zero Doppler.

Fig. 9. Model evolution for sensor 1, time = 0 data. Compare converged model (bottom-right plot here) to actual data
(top-left plot of Fig. 8).

followed by 80 additional iterations after adding the of the model. Separate plots are given for the east
data from sensors 4 and 5. Note that for each of these (x) components of time-zero velocity xk0 , time-zero
iterations, the tracking parameters were adjusted using position xk0 , and acceleration xk00 . Similar plots were
50 gradient ascent steps, as discussed in Section V. obtained for the y (north) components yk0 , yk0 , and yk00 ,
The evolution of the model is shown in Fig. 9 but they are not shown here. In Fig. 10, a plot is also
for the scan frame acquired by sensor 1 at t1 = 0 s. given showing the evolution of the mixture weights
The model is initially fuzzy and indistinct (top-left Ek for each of the target components (Ek of clutter
plot), but after 30 or more iterations it converges to an components are not shown).
accurate approximation of the data, as can be seen As mentioned above, the first 30 iterations were
by comparing the bottom-right plot in Fig. 9 with performed using data only from sensors 1—3 while
the top-left plot in Fig. 8. In Fig. 10 it is shown how the remaining iterations were performed using all
the parameter values evolve during the adaptation data from sensors 1—5. There are curves in the

604 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 45, NO. 2 APRIL 2009
 Fig. 10. Evolution of parameter estimates with iterations. Notations c1, c2, etc., in legend refer to target components 1—6 used in
model. Although we only show plots for x (east) components of position, velocity, and acceleration, similar plots were obtained
for y (north) components.

plots of Fig. 10 for each of the 6 target components platforms, for difficult cases in which measurements
labelled “c1—c6,” and these are compared with of azimuth are unavailable. A simplified version of the
the true parameter values (dashed lines). Recall, algorithm is also presented which is appropriate for
there are only 4 targets actually present. From the tracking from a single radar platform.
plots, it can be seen that components c1, c4, and c5, The single-sensor version of the algorithm is tested
converge asymptotically to the true target parameters. on experimental data. These results show the approach
In contrast, component c2 seems to drift around to be very promising, and robust in the presence of
without locking onto a target, and its corresponding significant, inhomogeneous, background clutter. The
mixture weight parameter decreases toward zero full, multi-sensor, algorithm is tested on synthetic
with increasing iterations. Based upon its dwindling data. These results demonstrate that accurate tracks
Ek value, component c2 could easily be pruned can be estimated by exploiting spatial diversity in the
from the model. The most interesting case involves sensor locations. Furthermore, the algorithm appears
components c3 and c6, which seem to lock onto the to be robust in the presence of significant clutter, and
same target, as shown by the fact that their velocity, uncertain knowledge regarding the number of targets
position, and acceleration estimates converge together present.
in order to share a true target track. The mixture It should be emphasized that while the initial
weight parameter Ek plot (right-most plot) shows
results are promising, they are only preliminary.
that the weights for c3 and c6 roughly sum up to the
Significant issues require further study in order to
true mixture weight of 100. Presumably, it would be
make the algorithm practical. For example, it is well
straightforward to detect this condition in which two
known that EM can converge to local maxima in the
components lock onto the same target. Then one of
optimization criterion, and therefore initialization
the two could simply be pruned from the model if
necessary. It is an open question how the model would will likely affect the converged solution. We still
adapt for cases in which the true number of targets need to determine how often these “traps” will occur,
exceeds the number of model components. Strategies under which conditions they can cause the algorithm
for handling this so-called “model selection” problem to fail, and how we can initialize the algorithm to
for general mixture models are described in [34], [35], prevent them. Also, since the true number of targets
[36] and the references therein. is seldom known a priori, more work is needed to
develop strategies for choosing the number of target
VII. CONCLUSIONS components, pruning unneeded components, and
adding components when necessary. Furthermore,
In this paper we describe a mathematical algorithm automatic detection strategies and performance will
for robust multi-target tracking from multiple radar need to be studied to judge our ability to distinguish

DEMING ET AL.: MULTI-TARGET/MULTI-SENSOR TRACKING 605

targets from nonhomogeneous clutter. Finally, more seek to minimize the Lagrangian
study is required to determine the robustness of the à !
X X
algorithm in the presence of sensor platform errors F = ¡LL + ¸ Ek 0 m ¡ p0 (wjmn ) (30)
and vibrations. k0 n

with respect to Ekm , where ¸ is the Lagrange

APPENDIX A. DERIVATION OF PARAMETER multiplier. Setting the partial derivative of F to zero,
ESTIMATION EQUATIONS i.e.,
@F @LL
Here the equations are derived which are satisfied =¡ +¸ = 0 (31)
by the parameters maximizing LL subject to a @Ekm @Ekm
constraint. The general procedure is first developed, the following value for ¸ is obtained
valid for an arbitrary mixture model. These results
@LL
are then used to derive the particular equations ¸= : (32)
@Ekm
corresponding to the tracking model. Good references
for the general procedure described here are An expression for @LL=@Ekm is now needed, and its
Perlovsky’s book [22], and the seminal work by Duda derivation is similar to (27). We obtain
and Hart [32]. The notation we use is very similar to @LL X @
[32]. = p0 (wjmn )P(k j jmn) ln[Ekm pk (wjmn j £k )]
@Ekm @Ekm
First, consider maximizing LL using a general j,n

mixture model. One might seek to maximize LL 1 X

= p0 (wjmn )P(k j jmn):
directly, by finding values for the set of parameters Ekm
j,n
£k for which all partial derivatives @LL=@£k are zero.
Using the expression for LL from (11), Combining this with (32),
" # N
J X
X
@LL X @ X
¸Ekm = p0 (wjmn )P(k j jmn): (33)
= p0 (wjmn ) ln Ek0 m pk0 (wjmn j £k0 )
@£k @£k j=1 n=1
j,m,n k0

X· p0 (wjmn ) @
¸ Next we perform the summation over k on both sides
= P [Ekm pk (wjmn j £k )] of (33), then use (10) and (16) to simplify. This
E p (wjmn j £k0 )
k 0 k0 m k 0
@£ k
j,m,n results in the solution for the Lagrange multiplier
X · ¸ ¸ = J, where J is the number of scan frames (time
Ekm pk (wjmn j £k )
= p0 (wjmn ) P indices) from each sensor used in the estimation.
k0 Ek0 m pk0 (wjmn j £k0 )
j,m,n Substituting this value back into (33), we obtain
@ 1X
[E p (w j £k )] Ekm = p0 (wjmn )P(k j jmn): (34)
@£k km k jmn J
£ j,n
Ekm pk (wjmn j £k )
Equation (34) is satisfied by the set of mixture
X @ weights Ekm that maximize LL, subject to the
= p0 (wjmn )P(k j jmn) ln[Ekm pk (wjmn j £k )]:
@£k constraint in (10).
j,m,n
A set of equations has now been derived governing
(27)
the complete set of model parameters £ = fEkm , £k g,
Here, we made use of the well-known identity for k = 1, 2, : : : , K and m = 1, 2, : : : , M. Equation (29)
@ ln y(x)=@x = [1=y(x)]@y(x)=@x, and the definition of governs the parameters £k , while (34) governs the
Ekm pk (wjmn j £k ) mixture weights Ekm . Unfortunately, solving for
P(k j jmn) = P : (28) the parameters directly from these equations can be
k 0 Ek 0 m pk 0 (wjmn j £k 0 ) problematic. The difficulty arises because P(k j jmn) is
Using (27), the set of parameters £k that maximize LL a function of all of the unknown parameters fEkm , £k g
will satisfy (see (28)). Thus, if there are Q unknown parameters,
it would be required to solve a coupled set of Q
@LL X @ nonlinear equations. However, the problem is not
= p0 (wjmn )P(k j jmn) ln pk (wjmn j £k ) = 0,
@£k @£k hopeless, and the maximum of LL can be reached
j,m,n
in an iterative fashion, alternating between updating
k = 1, 2, : : : , K: (29) P(k j jmn), then updating the set of parameters
The maximization of LL must also be performed fEkm , £k g. We use the notation P (I) (k j jmn) and
(I)
with respect to the mixture weights Ekm . However it is fEkm , £k(I) g to indicate the values of these quantities
now necessary to incorporate the constraint from (10) at the Ith iteration. Then, the explicit update formula
using the method of Lagrange multipliers. Thus we for P (I) (k j jmn) is given by (15), while the parameters

606 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 45, NO. 2 APRIL 2009
are updated in order to satisfy modified versions of The general procedure, developed above, is valid
(29) and (34), i.e., for an arbitrary mixture model. These results are
now used to derive the specific parameter estimation
(I+1) 1X
Ekm = p0 (wjmn )P (I) (k j jmn) (35) equations for the tracking application considered in
J this paper.
j,n
and Variance Parameters: The recursive update
X · ¸ formula is now derived for the variance parameter ¾dk 2
.
@
p0 (wjmn )P (I) (k j jmn) ln pk (wjmn j £k ) = 0: In order to use the update formula given by (36), it
@£k £k =£ (I+1)
j,m,n k is necessary to compute an explicit formula for the
factor within the square brackets, by substituting the
(36) specific model pk (wjmn j £k ) used for the tracking
This iterative procedure is guaranteed to converge to application. Both the target model of (5) and
a local maximum of LL, as shown in Appendix B. It the clutter model of (7) give the following result
will be of later use to note that (neglecting irrelevant terms):
X (I) X @
Ekm = p0 (wjmn ) (37) 2
ln pk (wjmn j £k )
@(1=¾dk )
k n
· ¸
@ 1 1
for all iterations
P I. This follows from (35) by using = 2
2
ln(1=¾dk 2
) ¡ (1=¾dk )(djnm ¡ Djkm )2
@(1=¾dk ) 2 2
the fact that k P (I) (k j jmn) = 1, as can be shown
from (15). 1 2 1
= ¾ ¡ (d ¡ Djkm )2 :
The update formula in (35) for the mixture weight 2 dk 2 jnm
parameter is a simple, closed-form expression. The Note, it is actually more convenient to work with the
precise form of the update equation for £k depends inverse variance 1=¾dk 2
rather than the variance itself,
upon the particular model pk (wjmn j £k ) that gets as shown in the above equation. Substituting this
plugged into (36). In some cases, this leads to expression into (36) leads to the update formula for
closed-form expressions, for example when estimating the variance parameter described by (13). The update
the covariance and mean parameters of Gaussian formula in (14) for the range variance ¾rk 2
is derived in
mixtures. Also, in this paper closed-form equations a similar fashion.
2
are obtained when estimating ¾dk using (13) or when Track Parameters, Multi-Sensor Case: Due
estimating the single-sensor range-only tracking to the model complexity, a closed-form update
parameters using (25). However, sometimes the model formula cannot be derived for the tracking parameters
is specified such that a closed-form expression is in the multi-sensor scenario. However, a GEM
not possible, for example when estimating the track update formula can be derived, starting with (38),
parameters using multi-sensor data, as described at the and computing the factor in square brackets by
end of Section IV. In this case, as an alternative to the inserting the track model given by (5). Using sk as a
closed-form update formula of (36), the parameters £k generic placeholder for any of the track parameters
can be updated by moving along the uphill gradient of fxk0 , xk0 , xk00 , yk0 , yk0 , yk00 g, we compute (neglecting irrelevant
LL, i.e., terms)
· ¸
@LL @
£k(I+1) = £k(I) + h ¢ ln pk (wjmn j £k )
@£k £k =£(I) @sk
k
· ¸
X @ 1 1
= £k(I) +h¢ (I)
p0 (wjmn )P (k j jmn) =
@sk
¡ 2 (rjnm ¡ Rjkm )2 ¡ 2 (djnm ¡ Djkm )2
2¾rk 2¾dk
j,m,n
μ ¶μ ¶ μ ¶μ ¶
· ¸ rjnm ¡ Rjkm @Rjkm djnm ¡ Djkm @Djkm
@ = + :
£ ln pk (wjmn j £k ) : (38) 2
¾rk @sk 2
¾dk @sk
@£k £k =£k(I)
Inserting this expression into (38), the parameter
Here h is the stepsize which can be chosen, for update formula described by (21) and (22) is obtained.
example, by trial and error. Note that the procedure Track Parameters, Single Sensor Case: The
described by (38) is not pure gradient ascent, since problem of range-only tracking using a single sensor
this step is alternated with updating P (I) (k j jmn) via is discussed in Section V. The goal is to find values
(I)
(15) and Ekm using (35). In fact, this procedure is a for the track parameters fRk0 , Rk0 , Rk00 g, for each target
special case of GEM [11]. Note that in some cases component k = 1, 2, 3, : : : , (K ¡ 2), which maximize
the set of parameters £k may be split, with some LL. In this case the target model given by (5) is still
members being updated using gradient ascent (38) valid. However the equations relating fRk0 , Rk0 , Rk00 g
and some being updated using a closed-form equation to the quantities Rjkm and Djkm are now given by
(36). (23) and (24), respectively. Although there is only a

DEMING ET AL.: MULTI-TARGET/MULTI-SENSOR TRACKING 607

single sensor m = 1, the m subscript is maintained in APPENDIX B. CONVERGENCE PROOF
the quantities Rjkm and Djkm for consistency with the
expression for the target model in (5). This convergence proof generally follows the one
The update formulas for fRk0 , Rk0 , Rk00 g, are derived given in [22, ch. 5.6.3], although here we provide
by starting with the general formula of (36), and some details that were left out in the reference. The
substituting an explicit formula for the factor in proof is valid for the general mixture model, and is
square brackets, which is computed using the target not restricted to the tracking model used in this paper.
model pk (wjmn j £k ) in (5). Using sk as a generic In order to show convergence to a local maximum
of LL, we show that LL never decreases from one
placeholder for any of the parameters fRk0 , Rk0 , Rk00 g, the
iteration to the next, i.e., LL(£(I+1) ) ¡ LL(£(I) ) ¸ 0,
square-bracketed factor in (36) is
· where LL(£(I) is the log-likelihood computed in the
@ @ 1 2 Ith iteration. From (11),
ln pk (wjmn j £k ) = ¡ (1=¾rk )(rjnm ¡ Rjkm )2
@sk @sk 2 LL(£(I+1) ) ¡ LL(£(I) )
¸ X
1 2
¡ (1=¾dk )(djnm ¡ Djkm )2 = p0 (wjmn )[ln p(wjmn j £(I+1) ) ¡ ln p(wjmn j £(I) )]:
2 j,m,n
μ ¶μ ¶
rjnm ¡ Rjkm @Rjkm (41)
= 2 P
¾rk @sk It is easy to see from (15) that k P (k j jmn) = 1, (I)
μ ¶μ ¶ and therefore we can insert this unity factor into the
djnm ¡ Djkm @Djkm
+ 2
: above expression, i.e.,
¾dk @sk
LL(£(I+1) ) ¡ LL(£(I) )
Substituting this expression into (36), the parameter X X
update formula = p0 (wjmn ) P (I) (k j jmn)
¿·μ ¶μ ¶ j,m,n k
rjnm ¡ Rjkm @Rjkm
2
¾rk @sk £ [ln p(wjmn j £(I+1) ) ¡ ln p(wjmn j £(I) )]:
μ ¶μ ¶¸ +(I)
djnm ¡ Djkm @Djkm (42)
+ 2
=0 We can further expand the expression by noting
¾dk @sk s =s(I+1) k k
(I)
ln p(wjmn j £(I) ) = ln(Ekm pk (wjmn j £k(I) )) ¡ ln P (I) (k j jmn)
(39)
as can be seen from the definition of P (I) (k j jmn) in
is obtained where, for compactness, we have used
(15). Using this fact, and rearranging terms, we can
the bracket notation h¤i(I) defined in (17). Note that
obtain
(23) and (24) are used to express Rjkm and Djkm and
their derivatives with respect to sk = fRk0 , Rk0 , Rk00 g so, LL(£(I+1) ) ¡ LL(£(I) )
for example, @Rjkm =@Rk0 = 1 and @Rjkm =@Rk0 = tj . For X X · ¸
P (I) (k j jmn)
each target component k, (39) is used to produce = p0 (wjmn ) P (I) (k j jmn) ln
P (I+1) (k j jmn)
three different equations for the three track parameters j,m,n k

sk = fRk0 , Rk0 , Rk00 g. First, substituting sk = Rk0 into (39), X

(I+1)
+ p0 (wjmn )P (I) (k j jmn) ln(Ekm pk (wjmn j £k(I+1) ))
we obtain
j,k,m,n

(Rk0 )(I+1) h1i(I) + (Rk0 )(I+1) htj i(I) + (Rk00 )(I+1) htj2 i(I) (I)
= hrjnm i : X
(I)
¡ p0 (wjmn )P (I) (k j jmn) ln(Ekm pk (wjmn j £k(I) )):
(40) j,k,m,n

Here, for example, (Rk0 )(I+1) denotes the updated (43)

estimate for Rk0 computed at the (I + 1)th iteration. P (I) (I)
Since k P (k j jmn) = 1 and P (k j jmn) ¸ 0
Note that (40) is linear with respect to the three for all iterations I, the log-sum inequality [41] or
parameters Rk0 , Rk0 , and Rk00 . By substituting sk = Rk0 , Jensen’s inequality can be used to prove the first term
then sk = Rk00 , into (39) we obtain two additional on the right-hand side of (43) is ¸ 0. Thus, in order
equations which are also linear with respect to Rk0 , to show convergence we need to show the sum of
Rk0 , and Rk00 . Thus there are three equations that are the second and third terms is ¸ 0, which can be done
linear with respect to three unknowns which, together, by analyzing the rules governing the updates of the
can be expressed using matrix notation as shown parameters fEkm , £k g. The rule governing the update
in (25) given in Section V. Note there is a separate of the weights Ekm is given by (35), which can be
3 £ 3 set of equations for each target component rewritten as
k = 1, 2, : : : , (K ¡ 2). In each iteration the tracking X 1
parameters are updated by inverting these matrix p0 (wjmn )P (I) (k j jmn) (I+1) ¡ J = 0: (44)
equations. j,n Ekm

608 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 45, NO. 2 APRIL 2009
Then, since It should be noted that if GEM is used to update £k
· ¸ according to (38), then the inequality in (47) is still
1 @
(I+1)
= ln(Ekm pk (wjmn j £k )) true (given a small enough stepsize) since (38) implies
Ekm @Ekm (I+1)
Ekm =Ekm gradient ascent of the function in curly brackets of
therefore (44) can be rewritten as3 (46). Equation (47) can be modified by summing both
2 8 sides over k, using (37) to simplify, then cancelling
<X terms. The result is
4 @ p (w 0 )P (I) (k j jm0 n) X
@Ekm : 0 0 jm n (I+1)
p0 (wjmn )P (I) (k j jmn) ln(Ekm pk (wjmn j £k(I+1) ))
j,m ,n
93 j,k,m,n
= X
£ ln(Ekm0 pk (wjm0 n j £k )) ¡ JEkm 5 = 0: ¸ (I)
p0 (wjmn )P (I) (k j jmn) ln(Ekm pk (wjmn j £k(I) )):
;
(I+1) j,k,m,n
Ekm =Ekm

(45) Thus, the sum of the second and third terms on

the right-hand side of (43) is ¸ 0, which means
The rule governing the update of £k is given by (36), LL(£(I+1) ) ¡ LL(£(I) ) ¸ 0. This implies convergence
which can be rewritten as to a local maximum of LL.
2 8
<X
4 @ p (w 0 )P (I) (k j jm0 n)
@£k : 0 0 jm n ACKNOWLEDGMENT
j,m ,n
93
= This research was supported by Dr. John Sjogren
£ ln(Ekm0 pk (wjm0 n j £k )) ¡ JEkm 5 = 0: of the Air Force Office of Scientific Research. The
;
£k =£k(I+1)
authors would like to thank Jen King Jao and Jason
Franz from Lincoln Laboratory for supplying the
(46) experimental radar data and for technical assistance.
Observe that (45) and (46) contain the same
expression within the curly brackets, which is a
function of Ekm and £k for indices k = 1, 2, : : : , K REFERENCES
and m = 1, 2, : : : , M. Furthermore, (45) and (46) [1] Reid, D.
imply the function within curly brackets has an An algorithm for tracking multiple targets.
(I+1)
extremum at the point (Ekm = Ekm , £k = £k(I+1) ). IEEE Transactions on Automatic Control, AC-24 (1979),
By computing the second derivatives of the curly 843—854.
bracketed function with respect to Ekm and £k (not [2] Blackman, S.
Multiple hypothesis tracking for multiple target tracking.
shown here), it is straightforward to show that the IEEE Aerospace and Electronic Systems Magazine, 19, 1,
(I+1)
point (Ekm = Ekm , £k = £k(I+1) ) represents a global pt. 2 (2004), 5—17.
maximum, given the Gaussian components used in [3] Fortmann, T., Bar-Shalom, Y., and Scheffe, M.
our model, and given that P (I) has been fixed using Sonar tracking of multiple targets using joint probabilistic
parameter values from the previous iteration. Thus, data association.
2 IEEE Journal of Oceanic Engineering, OE-8 (1983),
X 173—183.
4 p0 (wjm0 n )P (I) (k j jm0 n) [4] Bar-Shalom, Y., Kirubarajan, T., and Lin, X.
j,m0 ,n
Probabilistic data association techniques for target
3 tracking with applications to sonar, radar and EO sensors.
IEEE Aerospace and Electronic Systems Magazine, 20, 8,
(I+1) (I+1) 5
£ ln(Ekm 0 pk (wjm0 n j £k(I+1) )) ¡ JEkm pt. 2 (2005), 37—55.
[5] Bar-Shalom, Y., and Li, X. R.
2 Multitarget-Multisensor Tracking: Principles and
X Techniques.
¸4 p0 (wjm0 n )P (I) (k j jm0 n) Storrs, CT: YBS Publishing, 1995.
[6] Li, X., Wong, K. M., and Bosse, E.
j,m0 ,n
3 An interior point linear programming approach to
two-scan data association.
(I) (I) (I) 5 IEEE Transactions on Aerospace and Electronic Systems,
£ ln(Ekm 0 pk (wjm0 n j £k )) ¡ JEkm :
35 (1999), 474—490.
[7] Hue, C., Le Cadre, J. P., and Perez, P.
(47) Tracking multiple objects with particle filters.
IEEE Transactions on Aerospace and Electronic Systems,
3 Itis important to recall that P (I) (k j jmn) is fixed using the 38 (2002), 791—812.
parameter values Ekm (I)
and £k(I) which were computed in the [8] Hue, C., Le Cadre, J. P., and Perez, P.
Sequential Monte Carlo methods for multiple target
previous Ith iteration (refer to (15)). Therefore P (I) (k j jmn) is not
tracking and data fusion.
a function of the variables Ekm and £k which we seek to optimize
IEEE Transactions on Signal Processing, 50 (2002),
in iteration I + 1. 309—326.

DEMING ET AL.: MULTI-TARGET/MULTI-SENSOR TRACKING 609

 [9] Arulampalam, M. S., Maskell, S., Gordon, N., and Clapp, T. [25] Perlovsky, L. I., Schoendorf, W. H., Tye, D. M., and
A tutorial on particle filters for online Chang, W.
nonlinear/non-Gaussian Bayesian tracking. Concurrent classification and tracking using maximum
IEEE Transactions on Signal Processing, 50 (2002), likelihood adaptive neural system.
174—188. Journal of Underwater Acoustics, 45 (1995), 399—414.
[10] Dempster, A. P., Laird, N. M., and Rubin, D. B. [26] Perlovsky, L. I., Schoendorf, W. H., Garvin, L. C., Chang,
Maximum likelihood from incomplete data via the EM W., and Monti, J.
algorithm. Development of concurrent classification and tracking.
Journal of Royal Statistical Society, Series B, 39 (1977), Journal of Underwater Acoustics, 47 (1997), 202—210.
1—38. [27] Ruan, Y., and Willett, P.
[11] McLachlan, G. J., and Krishnan, T. Multiple model PMHT and its application to the second
The EM Algorithm and Extensions. benchmark radar tracking problem.
New York: Wiley, 1997. IEEE Transactions on Aerospace and Electronic Systems,
[12] Redner, R. A., and Walker, H. F. 40 (2004), 1337—1347.
Mixture densities, maximum likelihood and the EM [28] Dunham, D. T., Dempster, R. J., and Blackman, S. S.
algorithm. Tracking algorithm speed comparisons between MHT and
SIAM Review, 26 (1984), 195—239. PMHT.
[13] Streit, R. L., and Luginbuhl, T. E. Proceedings of the Fifth International Conference on
Maximum likelihood method for probabilistic Information Fusion, vol. 2, July 8—11, 2002, 846—851.
multi-hypothesis tracking. [29] Yanhua, R., Willett, P., and Streit, R.
Proceedings of SPIE International Symposium, Signal and Comparison of PMHT and S-D assignment trackers.
Data Processing of Small Targets 1994, vol. 2335-24, IEE Colloquium on Target Tracking: Algorithms and
Orlando, FL, Apr. 5—7, 1994. Applications, Nov. 11—12 1999, 12/1—12/4.
[14] Streit, R. L., and Luginbuhl, T. E. [30] Deming, R. W., Schindler, J., and Perlovsky, L. I.
Probabilistic multi-hypothesis tracking. Track-before-detect of multiple slowly moving targets.
NUWC-NPT technical report 10428, Naval Undersea Proceedings of IEEE Radar Conference 2007, Boston, MA,
Warfare Center, Newport, RI, Feb. 1995. Apr. 17—20, 2007.
[15] Willett, P., Ruan, Y., and Streit, R. [31] Deming, R. W., Schindler, J., and Perlovsky, L. I.
PMHT: Problems and some solutions. Concurrent tracking and detection of slowly moving
IEEE Transactions on Aerospace and Electronic Systems, targets using dynamic logic.
38 (2002), 738—754. IEEE International Conference on Integration of
[16] Rago, C., Willett, P., and Streit, R. Knowledge Intensive Multi-Agent Systems (KIMAS ’07),
Direct data fusion using the PMHT. Waltham, MA, Apr. 29—May 3, 2007.
Proceedings of the 1995 American Control Conference, vol.
[32] Duda, R. O., and Hart, P. E.
3, Seattle, WA, June 1995, 1698—1702.
Pattern Classification and Scene Analysis.
[17] Kreig, M., and Gray, D. New York: Wiley, 1973, sect. 6.4.
Multi-sensor probabilistic multi-hypothesis tracking using
[33] Stimson, G. W.
dissimilar sensors.
Introduction to Airborne Radar.
Proceedings of SPIE Conference 3086, Apr. 1997,
Mendham, NJ: Scitech Publishing, 1998, ch. 13,
129—138.
165—167.
[18] Molnar, K. J., and Modestino, J. W.
Application of the EM algorithm for the [34] Roberts, S. J., Husmeier, D., Rezek, I., and Penny, W.
multitarget/multisensor tracking problem. Bayesian approaches to Gaussian mixture modeling.
IEEE Transactions on Signal Processing, 46 (1998), IEEE Transactions on Pattern Analysis and Machine
115—129. Intelligence, 20 (1998), 1133—1142.
[19] Pulford, G. W., and La Scala, B. F. [35] Constantinopoulis, C.
MAP estimation of target manoeuver sequence with the Bayesian feature and model selection for Gaussian
expectation-maximization algorithm. mixture models.
IEEE Transactions on Aerospace and Electronic Systems, IEEE Transactions on Pattern Analysis and Machine
38 (2002), 367—377. Intelligence, 28 (2006), 1013—1018.
[20] Avitzour, D. [36] Figueiredo, M. A. T., and Jain, A. K.
A maximum likelihood approach to data association. Unsupervised learning of finite mixture models.
IEEE Transactions on Aerospace and Electronic Systems, IEEE Transactions on Pattern Analysis and Machine
28 (1992), 560—565. Intelligence, 24 (2002), 381—396.
[21] Perlovsky, L. I. [37] Perlovsky, L. I., Plum, C. P., Franchi, P. R., Tichovolsky,
Fuzzy tracking of multiple objects. E. J., Choi, D. S., and Weijers, B.
Proceedings of SPIE, Signal Processing, Sensor Fusion, Einsteinian neural network for spectrum estimation.
and Target Recognition, vol. 1699, Apr. 21, 1992, 18—20. Neural Networks, 10 (1997), 1541—1546.
[22] Perlovsky, L. I. [38] Shore, J. E.
Neural Networks and Intellect. Minimum cross-entropy spectral analysis.
New York: Oxford University Press, 2001, ch. 7 and 9. IEEE Transactions on Acoustics, Speech, and Signal
[23] Perlovsky, L. I. Processing, ASSP-29 (1981), 230—237.
Cramer-Rao bound for tracking in clutter and tracking [39] Kullback, S.
multiple objects. Information Theory and Statistics.
Pattern Recognition Letters, 18 (1997), 283—288. New York: Wiley, 1959.
[24] Perlovsky, L. I., Kwauk, R., and Tye, D. [40] Titterington, D.
MLANS application to classification while tracking. On the iterative image space reconstruction algorithm for
Proceedings of SPIE, Signal Processing, Sensor Fusion, ECT.
and Target Recognition III, vol. 2232, Apr. 4, 1994, IEEE Transactions on Medical Imaging, MI-6 (1987),
105—110. 52—56.

610 IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS VOL. 45, NO. 2 APRIL 2009
 Ross Deming (M’04–SM’07) received the B.S. degree from Cornell University,
Ithaca, NY, in 1985, the M.S. degree from the University of Vermont, Burlington,
in 1993, and the Ph.D. degree from Northeastern University, Boston, MA, in
1996, all in electrical engineering.
Currently, he is a consultant for General Dynamics, Inc., where he develops
radar and optical signal processing tools for the Air Force Research Laboratory.
He was previously employed as an electrical engineer at General Electric, Inc.
and Teradyne, Inc., and as a senior scientist at Nichols Research Corp. and Witten
Technologies, Inc. While at Witten Technologies he was part of a small team that
won, among other awards, the Wall Street Journal’s 2004 Technology Innovation
Award (1st place software category, honorable mention overall) for its technology
that creates detailed images of objects and conditions underground using radar.
He is also coauthor on a patent for this radar system.

John K. Schindler (S’62–M’64–SM’89–LF’91) received the SB degree from

the Massachusetts Institute of Technology, Cambridge, and the M.S. and Ph.D.
degrees from Purdue University, Lafayette, all in electrical engineering.
Prior to his retirement, he was the Director of Electromagnetics and Reliability
at the Air Force Rome Laboratory where he directed research and development
activities in electromagnetics, solid state electronics and reliability. In addition,
he served on a defense task force to organize and manage the DoD electronics
program. He is a lecturer in electrical engineering at Northeastern University,
teaching graduate courses in radar, digital communications and statistical and
adaptive signal processing. He has consulted on radar target classification,
antenna design and radar cross section measurements. He has been an Adjunct
Professor of Electrical Engineering at the Air Force Institute of Technology.
Dr. Schindler is a recipient of the IEEE Millennium Medal, the Meritorious
Civilian Service Award and the Presidential Rank Award of Meritorious
Executive.

Leonid Perlovsky (M’86–SM’95) is principal research physicist and technical

advisor at the Air Force Research Laboratory. Previously, from 1985 to 1999,
he served as chief scientist at Nichols Research, a $0.5 B high-tech organization,
leading the corporate research in intelligent systems, neural networks, information
science, sensor fusion, and algorithm development. He served as professor at
Novosibirsk University and New York University. He participated as a principal
in commercial startups developing tools for text understanding, biotechnology,
and financial predictions. He published more than 250 papers, delivered invited
keynote plenary talks, tutorial lectures, and authored a book Neural Networks and
Intellect, (Oxford University Press, 2001 (currently in the 3rd printing)).
Dr. Perlovsky organizes conferences on computational intelligence, serves
on multiple IEEE committees, chairs IEEE Boston Computational Intelligence
Chapter, and received the Distinguished Member of the IEEE Boston Section
Award. He serves as editor-at-large for Natural Computations journal and as
editor-in-chief for Physics of Life Reviews. He received the IEEE Distinguished
Member of Boston Section Award 2005; the USAF Research Lab Charles Ryan
Memorial Award for Basic Research 2007; and International Neural Network
Society Gabor Award for achievement in engineering/applications 2007.

DEMING ET AL.: MULTI-TARGET/MULTI-SENSOR TRACKING 611