Overall Objectives
Scientific Foundations
Application Domains
New Results
Contracts and Grants with Industry
Other Grants and Activities

Section: Scientific Foundations

Object tracking with non-linear probabilistic filtering

Tracking problems that arise in target motion analysis ( tma ) and video analysis are highly non-linear and multi-modal, which precludes the use of Kalman filter and its classic variants. A powerful way to address this class of difficult filtering problems has become increasingly successful in the last ten years. It relies on sequential Monte Carlo ( smc ) approximations and on importance sampling. The resulting sample-based filters, also called particle filters, can, in theory, accommodate any kind of dynamical models and observation models, and permit an efficient tracking even in high dimensional state spaces. In practice, there is however a number of issues to address when it comes to difficult tracking problems such as long-term visual tracking under drastic appearance changes, or multi-object tracking.

The detection and tracking of single or multiple targets is a problem that arises in a wide variety of contexts. Examples include sonar or radar tma and visual tracking of objects in videos for a number of applications (e.g., visual servoing, tele-surveillance, video editing, annotation and search). The most commonly used framework for tracking is that of Bayesian sequential estimation. This framework is probabilistic in nature, and thus facilitates the modeling of uncertainties due to inaccurate models, sensor errors, environmental noise, etc. The general recursions update the posterior distribution of the target state Im8 ${p(\#119857 _t|\#119858 _{1:t})}$ , also known as the filtering distribution, where Im9 ${\#119858 _{1:t}={(\#119858 _1\#8943 \#119858 _t)}}$ denotes all the observations up to the current time step, through two stages:

Im10 $\mtable{...}$(5)

where the prediction step follows from marginalization, and the new filtering distribution is obtained through a direct application of Bayes' rule. The recursion requires the specification of a dynamic model describing the state evolution Im11 ${p(\#119857 _t|\#119857 _{t-1})}$ , and a model for the state likelihood in the light of the current measurements Im12 ${p(\#119858 _t|\#119857 _t)}$ . The recursion is initialized with some distribution for the initial state Im13 ${p(\#119857 _0)}$ . Once the sequence of filtering distributions is known, point estimates of the state can be obtained according to any appropriate loss function, leading to, e.g., Maximum A Posteriori (map ) and Minimum Mean Square Error (mmse ) estimates.

The tracking recursion yields closed-form expressions in only a small number of cases. The most well-known of these is the Kalman Filter (kf ) for linear and Gaussian dynamic and likelihood models. For general non-linear and non-Gaussian models the tracking recursion becomes analytically intractable, and approximation techniques are required. Sequential Monte Carlo (smc ) methods [46] , [52] , [51] , otherwise known as particle filters, have gained a lot of popularity in recent years as a numerical approximation strategy to compute the tracking recursion for complex models. This is due to their efficiency, simplicity, flexibility, ease of implementation, and modeling success over a wide range of challenging applications.

The basic idea behind particle filters is very simple. Starting with a weighted set of samples Im14 ${{w_{t-1}^{(n)},\#119857 _{t-1}^{(n)}}}_{n=1}^N$ approximately distributed according to Im15 ${p(\#119857 _{t-1}|\#119858 _{1:t-1})}$ , new samples are generated from a suitably designed proposal distribution, which may depend on the old state and the new measurements, i.e., Im16 ${\#119857 _t^{(n)}\#8764 q{(\#119857 _t|\#119857 _{t-1}^{(n)},\#119858 _t)}}$ , Im17 ${n=1\#8943 N}$ . Importance sampling theory indicates that a consistent sample is maintained by setting the new importance weights to

Im18 ${w_t^{(n)}\#8733 w_{t-1}^{(n)}\mfrac {p{(\#119858 _t|\#119857 _t^{(n)})}p{(\#119857 _t^{(n)}|\#11985 ... })}}{q(\#119857 _t^{(n)}|\#119857 _{t-1}^{(n)},\#119858 _t)},~\munderover \#8721 {n=1}Nw_t^{(n)}=1,}$(6)

where the proportionality is up to a normalizing constant. The new particle set Im19 ${{w_t^{(n)},\#119857 _t^{(n)}}}_{n=1}^N$ is then approximately distributed according to Im8 ${p(\#119857 _t|\#119858 _{1:t})}$ . Approximations to the desired point estimates can then be obtained by Monte Carlo techniques. From time to time it is necessary to resample the particles to avoid degeneracy of the importance weights. The resampling procedure essentially multiplies particles with high importance weights, and discards those with low importance weights.

In many applications, the filtering distribution is highly non-linear and multi-modal due to the way the data relate to the hidden state through the observation model. Indeed, at the heart of these models usually lies a data association component that specifies which part, if any, of the whole current data set is “explained” by the hidden state. This association can be implicit, like in many instances of visual tracking where the state specifies a region of the image plane. The data, e.g., raw color values or more elaborate descriptors, associated to this region only are then explained by the appearance model of the tracked entity. In case measurements are the sparse outputs of some detectors, as with edgels in images or bearings in tma , associations variables are added to the state space, whose role is to specify which datum relates to which target (or clutter).

In this large context of smc tracking techniques, two sets of important open problems are of particular interest for Vista:


Logo Inria