{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T15:36:54Z","timestamp":1780501014508,"version":"3.54.1"},"reference-count":56,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,1,18]],"date-time":"2019-01-18T00:00:00Z","timestamp":1547769600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, we present a novel 2D\u20133D pedestrian tracker designed for applications in autonomous vehicles. The system operates on a tracking by detection principle and can track multiple pedestrians in complex urban traffic situations. By using a behavioral motion model and a non-parametric distribution as state model, we are able to accurately track unpredictable pedestrian motion in the presence of heavy occlusion. Tracking is performed independently, on the image and ground plane, in global, motion compensated coordinates. We employ Camera and LiDAR data fusion to solve the association problem where the optimal solution is found by matching 2D and 3D detections to tracks using a joint log-likelihood observation model. Each 2D\u20133D particle filter then updates their state from associated observations and a behavioral motion model. Each particle moves independently following the pedestrian motion parameters which we learned offline from an annotated training dataset. Temporal stability of the state variables is achieved by modeling each track as a Markov Decision Process with probabilistic state transition properties. A novel track management system then handles high level actions such as track creation, deletion and interaction. Using a probabilistic track score the track manager can cull false and ambiguous detections while updating tracks with detections from actual pedestrians. Our system is implemented on a GPU and exploits the massively parallelizable nature of particle filters. Due to the Markovian nature of our track representation, the system achieves real-time performance operating with a minimal memory footprint. Exhaustive and independent evaluation of our tracker was performed by the KITTI benchmark server, where it was tested against a wide variety of unknown pedestrian tracking situations. On this realistic benchmark, we outperform all published pedestrian trackers in a multitude of tracking metrics.<\/jats:p>","DOI":"10.3390\/s19020391","type":"journal-article","created":{"date-parts":[[2019,1,18]],"date-time":"2019-01-18T11:26:55Z","timestamp":1547810815000},"page":"391","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":68,"title":["Behavioral Pedestrian Tracking Using a Camera and LiDAR Sensors on a Moving Vehicle"],"prefix":"10.3390","volume":"19","author":[{"given":"Martin","family":"Dimitrievski","sequence":"first","affiliation":[{"name":"TELIN-IPI, Ghent University - imec, St-Pietersnieuwstraat 41, B-9000 Gent, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4746-9087","authenticated-orcid":false,"given":"Peter","family":"Veelaert","sequence":"additional","affiliation":[{"name":"TELIN-IPI, Ghent University - imec, St-Pietersnieuwstraat 41, B-9000 Gent, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wilfried","family":"Philips","sequence":"additional","affiliation":[{"name":"TELIN-IPI, Ghent University - imec, St-Pietersnieuwstraat 41, B-9000 Gent, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,1,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets Robotics: The KITTI Dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res. (IJRR)"},{"key":"ref_2","unstructured":"Leal-Taixe, L., Milan, A., Schindler, K., Cremers, D., Reid, I.D., and Roth, S. (arXiv, 2017). Tracking the Trackers: An Analysis of the State of the Art in Multiple Object Tracking, arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Pirsiavash, H., Ramanan, D., and Fowlkes, C.C. (2011, January 20\u201325). Globally-optimal Greedy Algorithms for Tracking a Variable Number of Objects. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR \u201911), Washington, DC, USA.","DOI":"10.1109\/CVPR.2011.5995604"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2054","DOI":"10.1109\/TPAMI.2015.2505309","article-title":"Multi-Target Tracking by Discrete-Continuous Energy Minimization","volume":"38","author":"Milan","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Hua, G., and J\u00e9gou, H. (2016). Tracking Multiple Persons Based on a Variational Bayesian Model. Computer Vision\u2014ECCV 2016 Workshops: Amsterdam, The Netherlands, 8\u201310 and 15\u201316 October 2016, Proceedings, Part II, Springer International Publishing.","DOI":"10.1007\/978-3-319-48881-3"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Dicle, C., Camps, O.I., and Sznaier, M. (2013, January 1\u20138). The Way They Move: Tracking Multiple Targets with Similar Appearance. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Sydney, NSW, Australia.","DOI":"10.1109\/ICCV.2013.286"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1109\/TPAMI.2013.103","article-title":"Continuous Energy Minimization for Multitarget Tracking","volume":"36","author":"Milan","year":"2014","journal-title":"IEEE TPAMI"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Leal-Taixe, L., Fenzi, M., Kuznetsova, A., Rosenhahn, B., and Savarese, S. (2014, January 23\u201328). Learning an Image-based Motion Context for Multiple People Tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.453"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). Improving Multi-frame Data Association with Sparse Representations for Robust Near-online Multi-object Tracking. Computer Vision\u2014ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11\u201314 October 2016, Proceedings, Part VIII, Springer International Publishing.","DOI":"10.1007\/978-3-319-46484-8"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kim, C., Li, F., Ciptadi, A., and Rehg, J.M. (2015, January 7\u201313). Multiple Hypothesis Tracking Revisited. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Washington, DC, USA.","DOI":"10.1109\/ICCV.2015.533"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kieritz, H., Becker, S., Hubner, W., and Arens, M. (2016, January 23\u201326). Online multi-person tracking using Integral Channel Features. Proceedings of the 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Colorado Springs, CO, USA.","DOI":"10.1109\/AVSS.2016.7738059"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Choi, W. (arXiv, 2015). Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor, arXiv.","DOI":"10.1109\/ICCV.2015.347"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Osep, A., Mehner, W., Mathias, M., and Leibe, B. (June, January 29). Combined Image- and World-Space Tracking in Traffic Scenes. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989230"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sadeghian, A., Alahi, A., and Savarese, S. (arXiv, 2017). Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies, arXiv.","DOI":"10.1109\/ICCV.2017.41"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Tang, S., Andres, B., Andriluka, M., and Schiele, B. (arXiv, 2016). Multi-Person Tracking by Multicut and Deep Matching, arXiv.","DOI":"10.1007\/978-3-319-48881-3_8"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1023\/A:1008103604354","article-title":"Integrated Person Tracking Using Stereo, Color, and Pattern Detection","volume":"37","author":"Darrell","year":"2000","journal-title":"Int. J. Comput. Vis."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Mart\u00ednez-Trinidad, J.F., Carrasco-Ochoa, J.A., Ben-Youssef Brants, C., and Hancock, E.R. (2011). People Detection Using Color and Depth Images. Proceedings of the Pattern Recognition: Third Mexican Conference, MCPR 2011, Cancun, Mexico, 29 June\u20132 July 2011, Springer.","DOI":"10.1007\/978-3-642-21587-2"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Bansal, M., Jung, S.H., Matei, B., Eledath, J., and Sawhney, H. (2010, January 3\u20138). A real-time pedestrian detection system based on structure and appearance classification. Proceedings of the 2010 IEEE International Conference on Robotics and Automation, Anchorage, AK, USA.","DOI":"10.1109\/ROBOT.2010.5509841"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1013","DOI":"10.1109\/TCE.2012.6311350","article-title":"Robust people counting system based on sensor fusion","volume":"58","author":"Dan","year":"2012","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1109\/TCE.2012.6227420","article-title":"Employing a RGB-D sensor for real-time tracking of humans across multiple re-entries in a smart environment","volume":"58","author":"Han","year":"2012","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Bajracharya, M., Moghaddam, B., Howard, A., Brennan, S., and Matthies, L.H. (2009). A Fast Stereo-based System for Detecting and Tracking Pedestrians from a Moving Vehicle. Int. J. Robot. Res., 28.","DOI":"10.1177\/0278364909341884"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1429","DOI":"10.1109\/TCYB.2013.2275291","article-title":"Real-Time Multiple Human Perception With Color-Depth Cameras on a Mobile Robot","volume":"43","author":"Zhang","year":"2013","journal-title":"IEEE Trans. Cybern."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, J., Liu, Y., Cui, Y., and Chen, Y.Q. (2013, January 15\u201318). Real-time human detection and tracking in complex environments using single RGBD camera. Proceedings of the 2013 IEEE International Conference on Image Processing, Melbourne, Australia.","DOI":"10.1109\/ICIP.2013.6738636"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.patrec.2014.09.013","article-title":"Detecting and tracking people in real time with RGB-D camera","volume":"53","author":"Liu","year":"2015","journal-title":"Pattern Recognit. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"438","DOI":"10.1007\/s11633-013-0740-y","article-title":"Visual Person Identification Using a Distance-dependent Appearance Model for a Person Following Robot","volume":"10","author":"Junji","year":"2013","journal-title":"Int. J. Autom. Comput."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1016\/j.imavis.2006.07.012","article-title":"People Detection and Tracking Using Stereo Vision and Color","volume":"25","author":"Aguirre","year":"2007","journal-title":"Image Vis. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Vo, D.M., Jiang, L., and Zell, A. (2014, January 5\u201310). Real time person detection and tracking by mobile robots using RGB-D images. Proceedings of the 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO 2014), Bali, Indonesia.","DOI":"10.1109\/ROBIO.2014.7090411"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.imavis.2003.07.009","article-title":"Stereo person tracking with adaptive plan-view templates of height and occupancy statistics","volume":"22","author":"Harville","year":"2004","journal-title":"Image Vis. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3665","DOI":"10.1016\/j.patcog.2008.06.013","article-title":"A Bayesian plan-view map based approach for multiple-person detection and tracking","volume":"41","year":"2008","journal-title":"Pattern Recognit."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1016\/j.jvcir.2009.03.005","article-title":"People detection and tracking with multiple stereo cameras using particle filters","volume":"20","year":"2009","journal-title":"J. Visual Commun. Image Represent."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.jvcir.2007.07.004","article-title":"Adaptive multi-modal stereo people tracking without background modelling","volume":"19","year":"2008","journal-title":"J. Visual Commun. Image Represent."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Choi, W., Pantofaru, C., and Savarese, S. (2011, January 6\u201313). Detecting and Tracking People using an RGB-D Camera via Multiple Detector Fusion. Proceedings of the Workshop on Challenges and Opportunities in Robot Perception, at the International Conference on Computer Vision (ICCV), Barcelona, Spain.","DOI":"10.1109\/ICCVW.2011.6130370"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1577","DOI":"10.1109\/TPAMI.2012.248","article-title":"A General Framework for Tracking Multiple People from a Moving Camera","volume":"35","author":"Choi","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"626","DOI":"10.5220\/0005719006260633","article-title":"Robust Matching of Occupancy Maps for Odometry in Autonomous Vehicles","volume":"Volume 3","author":"Dimitrievski","year":"2016","journal-title":"Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications"},{"key":"ref_35","unstructured":"Liu, C. (2009). Beyond Pixels: Exploring New Representations and Application for Motion Analysis. [Ph.D. Thesis, Massachusetts Institute of Technology]."},{"key":"ref_36","unstructured":"Blackman, S., and Popoli, R. (1999). Design and Analysis of Modern Tracking Systems, Artech House Radar Library, Artech House."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Cappe, O., Moulines, E., and Ryden, T. (2005). Inference in Hidden Markov Models (Springer Series in Statistics), Springer Inc.","DOI":"10.1007\/0-387-28982-8"},{"key":"ref_38","first-page":"584","article-title":"The Unscented Particle Filter","volume":"Volume 13","author":"Doucet","year":"2001","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_39","first-page":"893","article-title":"ICONDENSATION: Unifying Low-Level and High-Level Tracking in a Stochastic Framework","volume":"Volume I","author":"Isard","year":"1998","journal-title":"Proceedings of the 5th European Conference on Computer Vision"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Xiang, Y., Choi, W., Lin, Y., and Savarese, S. (2017, January 24\u201331). Subcategory-Aware Convolutional Neural Networks for Object Proposals and Detection. Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA.","DOI":"10.1109\/WACV.2017.108"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Gong, S., Cristani, M., Yan, S., and Loy, C.C. (2014). Evaluating Feature Importance for Re-identification. Person Re-Identification, Springer.","DOI":"10.1007\/978-1-4471-6296-4"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Forsyth, D., Torr, P., and Zisserman, A. (2008). Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features. Computer Vision\u2014ECCV 2008: 10th European Conference on Computer Vision, Marseille, France, 12\u201318 October 2008, Proceedings, Part I, Springer.","DOI":"10.1007\/978-3-540-88688-4"},{"key":"ref_43","unstructured":"Burnham, K.P., and Anderson, D.R. (2002). Model Selection and Multi-Model Inference, Springer Inc.. [2nd ed.]."},{"key":"ref_44","first-page":"116","article-title":"Grafok es matrixok","volume":"38","year":"1931","journal-title":"Matematikai es Fizikai Lapok"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1002\/nav.3800020109","article-title":"The Hungarian Method for the assignment problem","volume":"2","author":"Kuhn","year":"1955","journal-title":"Naval Res. Logist. Q."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1109\/MAES.2004.1263228","article-title":"Multiple hypothesis tracking for multiple target tracking","volume":"19","author":"Blackman","year":"2004","journal-title":"IEEE Aerosp. Electron. Syst. Mag."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Xiang, Y., Alahi, A., and Savarese, S. (2015, January 7\u201313). Learning to Track: Online Multi-object Tracking by Decision Making. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.534"},{"key":"ref_48","unstructured":"Milan, A., Leal-Taixe, L., Reid, I.D., Roth, S., and Schindler, K. (arXiv, 2016). MOT16: A Benchmark for Multi-Object Tracking, arXiv."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"246309","DOI":"10.1155\/2008\/246309","article-title":"Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics","volume":"2008","author":"Bernardin","year":"2008","journal-title":"EURASIP J. Image Video Process."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1007\/s11263-006-0027-7","article-title":"Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet based Part Detectors","volume":"75","author":"Wu","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"ref_51","unstructured":"De Vylder, J., and Goossens, B. (2016). Quasar: A Programming Framework for Rapid Prototyping. Presented at the GPU Technology Conference, Silicon Valley, CA, USA, 4\u20137 April 2016, NVIDIA."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Hua, G., and J\u00e9gou, H. (2016). Multi-class Multi-object Tracking Using Changing Point Detection. Computer Vision\u2014ECCV 2016 Workshops: Amsterdam, The Netherlands, 8\u201310 and 15\u201316 October 2016, Proceedings, Part II, Springer International Publishing.","DOI":"10.1007\/978-3-319-48881-3"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Tian, W., and Lauer, M. (September, January 29). Joint tracking with event grouping and temporal constraints. Proceedings of the 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy.","DOI":"10.1109\/AVSS.2017.8078515"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Yoon, J.H., Lee, C.R., Yang, M.H., and Yoon, K.J. (2016, January 27\u201330). Online Multi-object Tracking via Structural Constraint Event Aggregation. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.155"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Yoon, J.H., Yang, M.H., Lim, J., and Yoon, K.J. (2015, January 5\u20139). Bayesian Multi-object Tracking Using Motion Context from Multiple Objects. Proceedings of the 2015 IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV.2015.12"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Wang, S., and Fowlkes, C.C. (arXiv, 2016). Learning Optimal Parameters for Multi-target Tracking with Contextual Interactions, arXiv.","DOI":"10.5244\/C.29.4"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/2\/391\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:27:10Z","timestamp":1760185630000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/2\/391"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,18]]},"references-count":56,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,1]]}},"alternative-id":["s19020391"],"URL":"https:\/\/doi.org\/10.3390\/s19020391","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,18]]}}}