Method and system for tracking multiple regional objects by multi-dimensional relaxation

Abstract

A method and system for real-time tracking of objects is are disclosed. A region is repeatedly scanned providing a plurality of images or data sets having points corresponding to ojbects objects in the region to be tracked. Given a previously determined track for each object in the region, an M-dimensional combinatorial optimization assignment problem is formulated using the points from M-1 of the images or data sets, wherein each point is preferably used in extending at most one track. The M-dimensional problem is subsequently solved for an optimal or near-optimal assignment of the points to the tracks, extending the tracking of the objects so that a response to each object can be initiated by the system in real-time. Speed and accuracy is provided by an iterative Lagrangian Relaxation technique wherein a plurality of constraint dimensions are relaxed simultaneously to yield a reduced dimensional optimization problem whose solution is used to formulate an assignment problem of dimensionality less than M. The iterative reducing of dimensions terminates when exact solutions are determined for two-dimensional cases. A recovery procedure is used for determining a higher dimensional assignment problem solution from a problem having one less dimension. The procedure is useful when the reduced dimensional optimizational problem has two constraint dimensions.

Description

RELATED 3-dimensional[1.0]

2-dimensional
Consequently, the
This relationship means that either Φ_k

$\left[[\Psi \Psi {\Psi }_k{\Psi }_k{u}^k{\Psi }_k]\right]$
determine the gradient of each Ψ
(where v^T is the transpose of v),
is calculated as an estimate of u^k_j+1. To determine if the current step size is valid the we evaluate Ψ_k u³ u^k−2u₊ If the result represents an improvement then we double the step size. Otherwise we halve the step size. In either case a new u₊ is calculated. The doubling or halving continues until the step becomes too large to improve the result, or until it becomes small enough to not degrade the result. The resulting suitable step size is saved with d as part of the subgradient bundle. The last acceptable u₊ is assigned to u^k_j+1.

Three distinct M

For each dimension k=1M there exists a set of lists, with each list element being a pointer to a particular hypothesis:
L_ki={p_kj|k_j=1, . . . , N_ki}^i=Nk,k=M_i=1,k=1
where N_ki is N_ki(
Γ is a discrete random element defined on Γ*, γ⁰ is a reference partition, and
respectively. In multisensor data fusion and multitarget tracking the data sets Z(k) may represent different classes of objects, and each data set can arise from different sensors. For track initiation the objects are measurements that must be partitioned into tracks and false alarms. In the formulation of track extensions a moving window over time of observations sets is used. The observation sets will be measurements which are: (a) assigned to existing tracks, (b) designated as false measurements, or (c) used for initiating tracks. However, note that alternative data objects instead of observation sets may also be fused such as in sensor level tracking wherein each sensor forms tracks from its own measurements and then the tracks from the sensors are fused in a central location. Note that, as one skilled in the art will appreciate, both embodiments of the present invention may be used for this type of data fusion.

The data assignment problem considered presently is represented as a case of set partitioning defined in the following way. First, for notational convenience in representing tracks, a zero index is added to each of the index sets in m<n<M[6.1][6.1]
If, in addition, w^m is optimal for

With the exception of one equality being converted to an inequality, the following Fact is a converse of Fact A.2 and has also been shown to be true.

Fact A.3. Let wⁿ be a feasible solution to problem
{circumflex over (Φ)}_mn(u^m+1, . . . , uⁿ)−Σⁿ_k=m+1Σ^Mk_ik−ou^k_ik.

II.2.2. The Recovery Procedure

The next objective is to explain a recovery procedure or method for recovering a solution to the n-dimensional problem of [5.51]
where u_o^k=0 for all k=m+1, . . . ,n. The evaluation of Φ^nmγ^{m+1 n} corresponding minimization problem
Furthermore,
Φ_m−1nγ^{m m+1 n≧Φ}_mnγ^{n+1 n}   
for all u^kR^Mk+1 with u_o^k=0 and k=m, . . . n.

Most of the procedures for non-smooth optimization are based on generalized gradients called subgradients, given by the following definition.

A vector qεδΦ_mnγ
The actual function value used in the optimization phase is
denote the set of assignable variables. Define an undirected graph G(N,A) where the set of nodes is
N={z_in^min=1, . . . ,n; in=1, . . . , Nm}
and arcs,

Note that the nodes corresponding to zero index have not been Included in the above defined graph, since two variables that have only the zero index in common can be assigned independently. Connected components of the graph are then easily found by constructing a spanning forest via a depth first search. (A detailed algorithm can be found in the book by Aho, Hopcroft and Ullman cited above). Furthermore, this procedure is used at each level in the relaxation. i.e., is applied to each assignment problem
where Z^N+1 represents N+1 data sets, γ
respectively. (The dummy report z_o^k serves several purposes in the representation of missing data, false reports, initiating tracks, and terminating tracks
be an enumeration of all those zero-one variables in the solution of the assignment problem (3.5) (i.e., z_{i1i2 . . . iN+1=1}) excluding all the false reports in the solution (i.e., all those zero-one variables with exactly one nonzero index) and zero-one variables in the solution for which (i₁,i₂)=(0,0). (The latter can correspond to tracks that initiate on frames three and higher.) These denote our initial tracks.

Consider only the first two index sets in this enumeration
with the following excpetions.

a. All

Then, for l_k+1=1, . . . ,L_k+1, the 1_lk+1^th such track is denoted by {T_k+1(l_k+1)={T_k(l_k(l_k+1)),zk+1_ik+1(l_k+1)} and the (N+1)-tuple {T_k+1(1_k+1),z_ik+2^k+2, . . . ,z_lk+1+N^k+1+N} will denote a track T_k+1(1_k+1) plus a set of reports {z_lk+2^k+2, . . . ,z_ik+1+N^k+1+N}, actual or dummy, that are feasible with the track T_k+1(1_k+1). The (N+1)-tuple {T_k+1(0),z_ik+2^k+2, . . . ,z_ik+1+N^k+1+N} will denote a track that initiates in the sliding window, i.e., on subsequent frames. A false report in the sliding window is one with all but one non-zero index i_p for some p=k+2, . . . , k+1+N in the (N+1)-tuple {T_k+1(0),z_ik+2^k+2, . . . ,z_ik+1+N^k+1+N}.

The corresponding hypothesis about a partition γεΓ* being true is now conditioned on the truth of the L_k+1 tracks existing at the beginning of the N-frame window. (Thus the assignments prior to this sliding window are fixed.) The likelihood function is given by

$\begin{array}{cc}\begin{array}{c}{L}_{l_{k+1}{i}_{k+2}\dots i_{k+1+N}}=L_{T_{k+1}\left(l_{k+1}\right)}{L}_{z_{i_{k+2}}^{k+2},\dots ,z_{i_{k+1+N}}^{k+1+N},}\\ L_{T_{k+1}\left(0\right)}=1.\end{array}& (\left[\right[4.7a\left]\right]3.12)\end{array}$
Next, define the cost and the zero-one variable by

where the index l_k is used for an enumeration of those reports paired together. We also use the notation T_p,k(l_k) to denote the sequence of reports belonging to track T_k(l_k) but restricted to frames prior to and including p. Thus,
T_k(l_k)=T_k,k(l_k)
T_k(l_k)={z_i1¹(l_k), . . . ,z_ip^p_(lk)}
T_k,k(l_k)=T_p,k(l_k)∪{z_ip+1^p+1, . . . ,z_ik−1^k−1_(lk),z_ik^k_(lk)} for p≦k−1.  
Given this notation for the tracks and partition of the data in the frames {k−I, . . . ,k, . . . ,k+J}. L_Tp,k)lk) will denote the accumulated likelihood ratio up to and including frame p(p≦k) for a track that is declared as existing on frame k as a solution of the assignment problem. In this notation, the likelihood for T_p,k(l_k) and that of the association of {z_lk+1^k+1, . . . , z_lk+N^k+N} with track T_k(l_k) is given by
L_tk(lk)=L_Tp,k(lk)L_{ip+1(lk) . ik(ik)} for any p≦k−1,
L_{Tk(lk)k+1 ik+N}=L_Tpk(lk)L_{ip+1(lk) ik(ik)i+N} for any p≦k−1,   
respectively.

The cost for the assignment of {k_lk+1^k+1, . . . ,z_lk+N^k+N} to track T_k(l_k) and the corresponding
be an enumeration of the solution set of (3.5), i.e., those zero-one variables z_{i1(l2)i2(l2) . . . iN+1(l2)}=1, including z_{OO . . . O}=1 corresponding to l₂=0, but excluding all those zero-one variables that are assigned to one and correspond to false reports (i.e.. there is exactly one nonzero index in the subscript of z_{i1i2 . . . iN+1}), all those zero-one variables that are assigned to one and correspond to tracks that initiate on frames higher than I+2. Then we fix the data association decisions corresponding to the reports in our list of tracks prior to and including frame k+1=I+2. This defines the k for problem (4.4) and one can then continue the development by adding a frame to the window as in the general case.

If I+J>N, then one possibility is to start the process with N+1 frames, and assuming J≦N, proceed as before replacing I by N−J for the moment, and continue to add frames without lopping off the first frame in the window until reaches a window of length I+J+1. Then we proceed as in the previous paragraph.

If I+J<N, then one can solve the track initiation problem (3.5), formulate the problem with the center of the window at k+1=N+1−J, enumerate the solutions as above, and lop off the first N−J−I frames. Then, we proceed just as in the case I+J=N.

A primary objective in this work has been to demonstrate how multidimensional assignment problems arise in the tracking environment. The problem of track initiation and maintenance has been formulated within the framework of a moving window over the frames of data. The solution of these NP-hard, noisy, large scale, and sparse problems to the noise level in the problem is fundamental to superior track estimation and identification. Thus, one must utilize the special structure in the problems as well take advantage of special information that is available. Since these moving windows are overlapping, there are some algorithm efficiencies that can be identified and that take advantages of the overlap in the windows from one frame of reports to the next. Here is an example of the use of a primal solution of one problem to warm start the solution of the next problem in the sequence.

Claims

1. A method for tracking a plurality of objects, comprising:

repeatedly scanning a region containing a set consisting of one or more moving objects and generating n sequential images or data sets of said region, a plurality of observations in said images or data sets providing positional information for objects in said set;

determining a plurality of tracks, at least one track for each object in said set;

determining a plurality of costs, wherein each cost is for assigning one of said observations to one of said tracks;

defining a linear programming problem:

2. A method for tracking a plurality of objects, comprising:

using remote scanning apparatus to repeatedly scan a region containing a set consisting of one or more moving objects and generating n sequential images or data sets of said region, a plurality of observations in said images or data sets providing positional information for objects in said set;

using a computer system to determine a plurality of tracks, at least one track for each object in said set;

using a computer system to determine a plurality of costs, wherein each cost is for assigning one of said observations to one of said tracks;

defining a linear programming problem:

3. The method of claim 2, wherein the remote scanning apparatus for repeatedly scanning a region containing a set consisting of one or more moving objects comprises radar apparatus.