Multiple Tracking of Moving Objects with Kalman Filtering and PCA-GMM Method

In this article we propose to combine an integrated method, the PCA-GMM method that generates a relatively improved segmentation outcome as compared to conventional GMM with Kalman Filtering (KF). The combined new method the PCA-GMM-KF attempts tracking multiple moving objects; the size and position of the objects along the sequence of their images in dynamic scenes. The obtained experimental results successfully illustrate the tracking of multiple moving objects based on this robust combination

• Component; Pixels; Gaussian Mixture Model; Principle Component Analysis; Background Model; Noise Process; Segmentation; Tracking; Kalman Filtering

1. Introduction

Object tracking is an important technique used in many systems, especially in the field of computer vision. Background subtraction involves calculating the reference image and labeling the pixels corresponding to foreground objects. N. Friedman and S. Russell were proposed a Gaussian mixture model (GMM) for the background subtraction [1]. C. Stauffer and W. Grimson developed an algorithm for foreground segmentation based on the Gaussian mixture model [2,3]. W. Grimson and et al. used tracking information in multicamera calibration, for object detection and classification [4]. T. Bouwmans, F. ElBaf and B. Vachon surveyed the background modeling using mixture of Gaussians for object detection [5]. F. Zbu and K. Fujimura demonstrated a face tracking method using GMM and EM algorithm [6]. P. KaewTraKulPong and R. Bowden proposed a method to detect moving shadows using Gaussian mixture model. The shadow detection also reduces the effect of small repetitive motions in the background scene [7]. On the other hand, the binding ellipsis, due to S. Cheung and C. Kamath is inappropriate for complex-shaped objects such as human beings. They used frame-differencing as it produces minimal false foreground trails behind objects [8]. R. Chan and J. Lin introduce mixture Kalman filtering for on-line estimation and prediction in conditional dynamic linear model [9]. A. Yilmaz, O. Javed, and M. Shah surveyed detection of object and tracking [10]. K. Quast and A. Kaup presented Auto GMM-SAMT video surveillance system and used the mean shift algorithm to track the contour of objects of changing shape without the help of any predefined shape model [11].

The principal component analysis (PCA) is as a linear transformation filter can be used to simplify a dataset and creates new classifiers. H. Moon and P. Jonathan introduced a statistical pattern recognition PCA algorithm which has become the famous benchmark PCA algorithm [12]. H. Kim, D. Kim, and S. Yang proposed PCA mixture models applicable to many areas, where applied as the alphabet recognition [13]. B. Antoni, M. Vijay and V. Nuno introduced a generalization of the Stauffer-Grimson background subtraction algorithm. However, the background subtraction algorithm was evaluated through extensive experiments, involving scenes with dynamic backgrounds, and high detection. Moreover, in their generalization the low false positive rates were obtained regardless of object scale suggesting that it may be suitable for surveillance on dynamic scenes [14]. I. Gómez, D. Olivieri, X. Vila, and S. Orozco proposed a method for foreground, background segmentation and created a feature vector for discriminating and tracking several people in the scene [15]. Y. Sheikh and M. Shah Introduced single distribution with kernel density [16]. Y. Yong and W. Ya-Fei used Gaussian mixture model with spatial local correlation and Markov Chain [17]. R. Hassanpour, A. Shahbahrami and S. Wong proposed Gaussian mixture model and considered the higher level a priori knowledge of spatial and temporal relationship between the pixels in a video sequence [18]. G. Rajkumar, K. Srinivasarho and P. Sribi-Vasa proposed a new image segmentation algorithm used the finite mixture of doubly truncated bivariate Gaussian distribution by integrating this distribution with the hierarchical clustering [19]. K. Panta proposed novel data association schemes for the probability hypothesis density [20].

In our work, we show that the segmentation of moving objects generated by PCA-GMM performed relatively better than the conventional GMM. This improvement in performance is based on the following: given that GMM segments the foreground from the background labeling for each sequence frame, where the background model can be updated using sufficient statistics and all the estimated parameter can be obtained efficiently. However, when the foreground is not available, or can be changed due to critical situations like illumination changes or it has been removed from the scene, then the GMM needs to consider such situations. The PCA-GMM is an extension to GMM which considers these limitations [21]. The PCA-GMM which integrates PCA with GMM produces precise and clear features of the shape in the scenes and provides good classification of the object. The tracking of moving object is then based on applying KF on this integrated method.

The remainder of the paper is organized as follows: In the Section 2, we introduced the PCA-GMM that links GMM with PCA, followed by brief description of Kalman filtering method, the experiment results of tracking moving objects by applying KF to PCA-GMM are placed in Section 4, followed by a conclusion in Section 5.

2. The PCA-GMM Method

First, in the adaptive mixture of Gaussian method, GMM, we considered the pixel processes as time series such that each pixel assumed to be an independent statistical process, incorporates all pixel observations. The mixture model does not distinguish components that correspond to background from those associated with foreground objects and record the observed intensity at each pixel over the previous N frames.

In the same manner as C. Stauffer and W. Grimson [3], we used the same number k of components. We consider the values of a particular pixel, with initial over time t and we can define the history of this particular pixel as, where is the image sequence. The probability of pixels observation data is defined as follows:

where is the weight of each component at time, is the mean and is the covariance mixture of each component at time respectively, is a Gaussian probability density defined as follows:

For simplicity we use, where I is the identity matrix. Next, we update the mixture of each pixel consecutively by reading in new pixel values and update each matching mixture component. We initialize the parameters and. They are updated for each frame in the mixture. We calculate W for each Gaussian component in the mixture. When the current pixel value matches none of the distributions, the least probable distribution is updated with the current pixel values. Then the parameters of the mixture are updated as follows:

where the constant is a learning rate to update the components weights, is a dummy variable with value 1 for matching models and 0 otherwise. We introduce the following criterion for screening the preliminary Gaussian matching distribution with Mahalanobis distance less than a threshold T

to collect the targeted pixels, with the update recursive equations

and

where indicates how to accelerate the update. Thus, we can find the foreground resulted from the background and then find the best segment of the image. The non active background Gaussian is treated as foreground. In the ordering, we selected the ratios Large values of ratios are associated with distributions which have a high weight, low variance. The first B distributions chosen under the expression:

where the threshold T is a prior probability estimated from the background process. Movements caused by the background (such as tree branches shaking, surface fluctuations, etc.) were considered to segment for the foreground objects.

Next, with data where. First we consider each of the N frames in one color such that we can define the frame vectors for fixed with respect to the pixels index:

The arithmetic average of pixels in each component for the frame vector can be defined as follows:

where is number of pixels in the frame i. Next, we compute the difference for the image and construct the difference matrix A as:

The covariance matrix of each frame vector is defined by:

We compute the eigenvectors and corresponding eigenvalues by:

using Singular Value Decomposition (SVD) method, where V is the set of eigenvectors associated with the corresponding eigenvalues λ. To apply the method of PCA we need to define the projection by applying the translation matrix to the vectors as follows:

Next, we construct each frame to obtain the features of the shape. Then we can define the estimated data by:

One property of PCA is that the projection onto the principal subspace minimizes squared errors. To find the feature principal subspecies of PCA, the sequential reconstruction errors are estimated from:

the major advantage of the PCA method comes from its generalization ability. It reduces the feature space dimension by considering the variance of the input data. This method determines which projections are preferable for representing the structure of input data. Those projections are selected in such a way that the maximum amount of information (i.e. maximum values of variances, where Eigenvalues represent the largest variances) is obtained in the smallest dimension of feature space.

The PCA-GMM addresses the major limitation of the GMM, which is the illumination changes and noise. The PCA-GMM method considers observing pixel data from (1, 2), using the method of PCA. The estimated is obtained by projecting the data on the axis, then with the new data we propose to compute based on such that:

and again for simplicity we assume, where I is identity matrix. In linking PCA with GMM it is evident that the integrated based on by its physical structure, contains the maximum amount of information or has the minimum squared errors as compared to based on of the GMM in handling the limitation and shadow that contribute to noise in the adaptive Gaussian model.

To update each mixture of each pixel entails reading the new pixel values and select the matching components, then compute the mean obtained with and finally injecting that mean into the covariance of PCA method form (5, 6). Symbolically the update is based on:

and

where indicates how to accelerate the update. In the integrated method we insert the estimates of the mean and covariance of each component in PCA into GMM respectively. We simply select the components with largest weights and re-estimate with (7) as follows:

And their Gaussian matching parameters must satisfy:

with the estimates of the ratios. The large values of ratios are associated with distributions which have a high weights or low variances. We select the first Gaussian distributions:

then for estimating the distribution of data by PCAGMM method, we need to perform both the appropriate partitioning of the components and the estimation of model parameters. We can perform this task successfully due to the important property of the PCA method in data manipulation.

3. Kalman Filtering: The Version of PCA-GMM-KF Method

A Kalman Filter (KF) is applied to estimate the state of a linear system where the state is assumed to be Gaussian distributed. This filter not only provides an efficient computational solution to sequential systems but also provides an optimal solution for the discrete data for linear filtering problem [22]. On tracking objects from frame to frame in long sequences of images, the continuity of the motion of the observed scene, allows the prediction of the image, at any instant, based on their previous trajectories. Because the moving state changes little in the neighboring consecutive frames, we model the system as linear Gaussian with the state parameters of Kalman filter given by the object location, its velocity and its size of the object respectively. A discrete-time dynamic equation state is given by

where the state vector,

,

and are the predicted coordinates of the object, and are velocities in respective direction, represent the width of the object rectangle, represents the change in time and, is the white Gaussian noise with zero means and covariance matrix, that is. The position of obtained PCAGMM algorithm will be the measurement vector.that can be injected in the measurement model of KF. Accordantly:

Equation (23) combines PCA-GMM with KF. Such a combination proposes what we may call PCA-GMM-KF, as a version of Kalman Filtering in tracking multiple moving objects. The predicted coordinates and dimensions ω of the rectangle in the measurement model of Equation (23) under PCA-GMM-KF are used to locate the object in the present frame. Where, is white Gaussian noise, is covariance matrix and is the design matrix such that

.

The corresponding covariance matrices and are the inputs to the Kalman filter. In the sequential image, if the dynamics of the moving object is known, prediction can be made about the positions of the objects in the current image. The Kalman filter state prediction and state covariance prediction are defined by:

where and denotes the estimated state vector and error covariance matrix respectively at time t. Then the Kalman filter update steps are as follows:

KF algorithm starts with initial conditions with and. is the Kalman gain, which defines the updating weight between the new measurements and the prediction from the dynamic model.

4. Experimental Results and Analysis

To demonstrate the effectiveness of the algorithm, in our application, we first show how the PCA-GMM out-performs the conventional GMM algorithm. In our experiment, each pixel is checked against the current frame of the background model by comparing it with every component in the PCA-GMM. This relatively better performance of PCA-GMM is mainly due to its ability to manipulate the variation to ease the segmentation of the foreground from the background. Moreover, the PCAGMM method is based on the minimum reconstruction error over the data needed to be estimated which contribute for the better segmentation [22].

It is evident from viewing the segmentation using the GMM (Figure 1(a)), we can achieve remarkable improve by using the segmentation on PCA-GMM (Figure 1(b)). Next, based on this segmentation result we are motivated to tackle the tracking problem based on PCAGMM rather than the GMM.

The tracking experimental results based on PCA-GMMKF method, are arranged into two cases of tracking multiple moving objects, where we have constructed two frame’s sequences in the different backgrounds. The frame’s dimensions were 167 × 120.

The tracking algorithm represented by performing the following: First we apply the PCA-GMM algorithm to extract the moving objects from background in each video frame, and then we predict the next position candidates in the next frame by predicting it from KF in Equations (22) and (23). Sequentially the tracking results based on the routine loop of detecting the objects by using data association function and the add new hypotheses function in KF algorithm respectively and finely we update the frame using the Kalman update function which in turn provides new input. Consequently, the resulted PCA-GMM-KF method reflects more accurate detection of object results at each level of the proposed approach (Figures 2(a) and (b)).

After we applied the PCA to solve the problem of Gaussian mixture and obtained improved result in the two cases: (Figures 2(a) and (b)), the performance enhanced remarkably with the PCA-GMM proposed detection method.

5. Conclusion

In this article, we attempt to improve tracking of multiple moving objects within the framework of Kalman Filtering. Given that the integrated combined PCA-GMM outperforms the conventional GMM in generating relatively improved performance in segmentation, we have been motivated to address the tracking of multiple objects within the framework of KF. The experimental results in addition of illustrating the outperformance of segmentation results of the PCA-GMM algorithm over GMM, have also considered the tracking of multiple moving objects. The proposed PCA-GMM-KF method which utilized the desirable segmentation results due to PCAGMM can be viewed as a version of KF. The experimental results of tracking objects have shown that that the application of PCA-GMM-KF method to each case and under sequence of images, successfully captured the objects and produces a satisfactory results.

References

1. N. Friedman and S. Russell, “Image Segmentation in Video Sequences,” 13th Conference on Uncertainty in Artificial Intelligence, 1997, pp. 175-181.

2. C. Stauffer and W. Grimson “Adaptive Background Mixture Models for Real-Time Tracking,” Proceedings of IEEE Computer Vision and Pattern Recognition, 1999, pp. 246-252.

3. C. Stauffer and W. Grimson, “Learning Patterns of Activity Using Real-Time Tracking,” IEEE Transactions on Pattern Analysis & Machine Intelligence, Vol. 22, No. 8, 2000, pp. 747-757. doi:10.1109/34.868677

4. W. Grimson, C. Stauffer, R. Romano and L. Lee, “Using Adaptive Tracking to Classify and Monitor Activities in a Site,” Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1998.

5. T. Bouwmans, F. El Baf and B. Vachon, “Background Modeling Using Mixture of Gaussians for Foreground Detection,” Recent Patents on Computer Science, 2008, pp. 219-237.

6. Y. Zhu and K. Fujimura, “Driver Face Tracking Using Gaussian Mixture model (GMM),” Proceedings of IEEE on Intelligent Vehicles Symposium, 2003, pp. 587-592.doi:10.1109/IVS.2003.1212978

7. P. KaewTraKulPong and R. Bowden, “An Improved Adaptive Background Mixture Model for Real-Time Tracking with Shadow Detection,” 2nd European Workshop on Advanced Video Based Surveillance Systems, September 2001.

8. S. Cheung and C. Kamath, “Robust Background Subtraction with Foreground Validation for Urban Traffic Video,” EURASIP Journal on Applied Signal Processing, Vol. 14, 2005, pp. 2330-2340.doi:10.1155/ASP.2005.2330

9. R. Chen and J. S. Liu, “Mixture Kalman Filters,” Journal of the Royal Statistical Society, Series B-Statistical Methodology, Vol. 62, No. 3, 2000, pp. 493-508.doi:10.1111/1467-9868.00246

10. A. Yilmaz, O. Javed and M. Shah, “Object Tracking: A Survey,” ACM Computing Surveys, Vol. 38, No. 4, 2006, pp. 1-45. doi:10.1145/1177352.1177355

11. K. Quast and A. Kaup, “AUTO GMM-SAMT: An Automatic Object Tracking System for Video Surveillance in Traffic Scenarios,” EURASIP Journal on Image and Video Processing, Vol. 2011, 2011, pp. 2-14.

12. H. Moon and P. Jonathan, “Computational and Performance Aspects of PCA-Based Face-Recognition Algorithms,” Perception, Vol. 30, No. 3, 2001, pp. 303-321.doi:10.1068/p2896

13. H. Kim, D. Kim and S. Yang, “An Efficient Model Order Selection for PCA Mixture Model,” Pattern Recognition, Vol. 24, No. 9-10, 2003, pp. 1385-1393.doi:10.1016/S0167-8655(02)00379-3

14. B. Antoni, M. Vijay and V. Nuno, “Generalized StaufferGrimson Background Subtraction for Dynamic Scenes,” Journal of Machine Vision and Applications, Vol. 22, No. 5, 2011, pp. 751-766.

15. I. Gómez, D. Olivieri, X. Vila and S. Orozco, “Simple Human Gesture Detection and Recognition Using a Feature Vector and a Real-Time Histogram Based Algorithm,” Journal of Signal and Information Processing, Vol. 2, No. 4, 2011, pp. 279-286.doi:10.4236/jsip.2011.24040

16. Y. Sheikh and M. Shah, “Bayesian Modeling of Dynamic Scenes for Object Detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 1, 2005, pp. 1778-1792. doi:10.1109/TPAMI.2005.213

17. Y. Yong and W. Ya-Fei, “Moving Object Detection Based on Spatial Local Correlation,” Opto-Electronic Engineering, Vol. 36, No. 2, 2009.

18. R. Hassanpour, A. Shahbahrami and S. Wong, “Adaptive Gaussian Mixture Model for Skin Color Segmentation,” Proceeding of World Academic of Science Engineering and Technology, Vol. 31, 2008, pp.1307-6884.

19. G. Rajkumar, K. Srinivasarho and P. Sribivasa, “Image Segmentation Method Based on Finite Doubly Truncated Bivariate Gaussian Mixture Model with Hierarchical Clustering,” International Journal of Computer Science Issues, Vol. 8, No. 2, 2011, pp. 1694-0814.

20. K. Panta, “Novel Data Association Schemes for the Probability Hypothesis Density Filter,” IEEE Transactions on Aerospace and Electronic Systems, Vol. 43, No. 2, 2007, pp. 556-570. doi:10.1109/TAES.2007.4285353

21. N. Emadeldeen, M. Jedra and N. Zahid, “On Segmentation of Moving Objects by Integrating PCA Method with the Adaptive Background Model,” Journal of Signal and Information Processing, Vol. 3, No. 3, 2012, pp. 387-393. doi:10.4236/jsip.2012.33051

22. S. Grewal and P. Andrews, “Kalman Filtering Theory and Practice Using Matlab Grewal,” 2nd Edition, John Wiley & Sons Inc., New York, 2001.