{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T01:27:08Z","timestamp":1782437228486,"version":"3.54.5"},"reference-count":57,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2020,8,26]],"date-time":"2020-08-26T00:00:00Z","timestamp":1598400000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100011878","name":"Flemish Government","doi-asserted-by":"publisher","award":["AI Research Program"],"award-info":[{"award-number":["AI Research Program"]}],"id":[{"id":"10.13039\/501100011878","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper presents a vulnerable road user (VRU) tracking algorithm capable of handling noisy and missing detections from heterogeneous sensors. We propose a cooperative fusion algorithm for matching and reinforcing of radar and camera detections using their proximity and positional uncertainty. The belief in the existence and position of objects is then maximized by temporal integration of fused detections by a multi-object tracker. By switching between observation models, the tracker adapts to the detection noise characteristics making it robust to individual sensor failures. The main novelty of this paper is an improved imputation sampling function for updating the state when detections are missing. The proposed function uses a likelihood without association that is conditioned on the sensor information instead of the sensor model. The benefits of the proposed solution are two-fold: firstly, particle updates become computationally tractable and secondly, the problem of imputing samples from a state which is predicted without an associated detection is bypassed. Experimental evaluation shows a significant improvement in both detection and tracking performance over multiple control algorithms. In low light situations, the cooperative fusion outperforms intermediate fusion by as much as 30%, while increases in tracking performance are most significant in complex traffic scenes.<\/jats:p>","DOI":"10.3390\/s20174817","type":"journal-article","created":{"date-parts":[[2020,8,26]],"date-time":"2020-08-26T09:05:37Z","timestamp":1598432737000},"page":"4817","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Cooperative Multi-Sensor Tracking of Vulnerable Road Users in the Presence of Missing Detections"],"prefix":"10.3390","volume":"20","author":[{"given":"Martin","family":"Dimitrievski","sequence":"first","affiliation":[{"name":"TELIN-IPI, Ghent University\u2014Imec, St-Pietersnieuwstraat 41, B-9000 Gent, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Van Hamme","sequence":"additional","affiliation":[{"name":"TELIN-IPI, Ghent University\u2014Imec, St-Pietersnieuwstraat 41, B-9000 Gent, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4746-9087","authenticated-orcid":false,"given":"Peter","family":"Veelaert","sequence":"additional","affiliation":[{"name":"TELIN-IPI, Ghent University\u2014Imec, St-Pietersnieuwstraat 41, B-9000 Gent, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wilfried","family":"Philips","sequence":"additional","affiliation":[{"name":"TELIN-IPI, Ghent University\u2014Imec, St-Pietersnieuwstraat 41, B-9000 Gent, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,8,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Gavrila, D. (2000). Pedestrian Detection from a Moving Vehicle. Proceedings of the 6th European Conference on Computer Vision\u2014Part II, Springer. ECCV 2000.","DOI":"10.1007\/3-540-45053-X_3"},{"key":"ref_2","unstructured":"Gavrila, D.M., Kunert, M., and Lages, U. (2001, January 21\u201323). A multi-sensor approach for the protection of vulnerable traffic participants the PROTECTOR project. Proceedings of the 18th IEEE Instrumentation and Measurement Technology Conference, Rediscovering Measurement in the Age of Informatics (Cat. No.01CH 37188), IMTC 2001, Budapest, Hungary."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Macioszek, E., Swierk, P., and Kurek, A. (2020). The Bike-Sharing System as an Element of Enhancing Sustainable Mobility\u2014A Case Study based on a City in Poland. Sustainability, 12.","DOI":"10.3390\/su12083285"},{"key":"ref_4","unstructured":"(2020, August 25). Regulation (EC) No 78\/2009 of the European Parliament and of the Council of 14 January 2009 on the type-approval of motor vehicles with regard to the protection of pedestrians and other vulnerable road users, amending Directive 2007\/46\/EC and repealing Directives 2003\/102\/EC and 2005\/66\/EC. Off. J. Eur. Union., Available online: https:\/\/eur-lex.europa.eu\/legal-content\/EN\/ALL\/?uri=CELEX%3A32009R0078."},{"key":"ref_5","unstructured":"Milan, A., Leal-Taixe, L., Reid, I.D., Roth, S., and Schindler, K. (2020, August 25). MOT16: A Benchmark for Multi-Object Tracking. Available online: https:\/\/motchallenge.net\/results\/MOT16\/."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets Robotics: The KITTI Dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robots Res. (IJRR)"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Dimitrievski, M., Shopovska, I., Hamme, D.V., Veelaert, P., and Philips, W. (2020, January 20\u201323). Weakly supervised deep learning method for vulnerable road user detection in FMCW radar. Proceedings of the 23rd IEEE International Conference on Intelligent Transportation Systems, Rhodes, Greece.","DOI":"10.1109\/ITSC45102.2020.9294399"},{"key":"ref_8","unstructured":"Redmon, J., and Farhadi, A. (2020, August 25). YOLOv3: An Incremental Improvement. Available online: https:\/\/arxiv.org\/abs\/1804.02767."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1002\/nav.3800020109","article-title":"The Hungarian Method for the assignment problem","volume":"2","author":"Kuhn","year":"1955","journal-title":"Nav. Res. Logist. Q."},{"key":"ref_10","unstructured":"Blackman, S., and Popoli, R. (1999). Design and Analysis of Modern Tracking Systems, Artech House Radar Library."},{"key":"ref_11","unstructured":"Polychronopoulos, A., Scheunert, U., and Tango, F. (July, January 28). Centralized data fusion for obstacle and road borders tracking in a collision warning system. Proceedings of the International Conference on Information Fusion, Stockholm, Sweden."},{"key":"ref_12","unstructured":"Blanc, C., Le, B.T., Le, Y., and Moreira, G.R. (July, January 28). Track to Track Fusion Method Applied to Road Obstacle Detection. Proceedings of the International Conference on Information Fusion, Stockholm, Sweden."},{"key":"ref_13","unstructured":"Cou\u00e9, C., Fraichard, T., Bessiere, P., and Mazer, E. (2002, January 18\u201320). Using Bayesian Programming for Multi-Sensor Data Fusion in Automotive Applications. Proceedings of the IEEE Intelligent Vehicle Symposium, Versailles, France."},{"key":"ref_14","unstructured":"Park, S.B., Tango, F., Aycard, O., Polychronopoulos, A., Scheunert, U.D.I., and Tatschke, T. (2020, August 25). ProFusion2\u2014Sensor Data Fusion for Multiple Active Safety Applications. Available online: https:\/\/trid.trb.org\/view\/843357."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yenkanchi, S., and Wu, Q.M.J. (2016, January 12\u201315). Cooperative fusion for road obstacles detection using laser scanner and camera. Proceedings of the 2016 12th World Congress on Intelligent Control and Automation (WCICA), Guilin, China.","DOI":"10.1109\/WCICA.2016.7578528"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Gruyer, D., Cord, A., and Belaroussi, R. (2013, January 3\u20137). Vehicle Detection and Tracking by Collaborative Fusion Between Laser Scanner and Camera. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6697109"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1007\/s10514-005-0611-7","article-title":"Cooperative Fusion for Multi-Obstacles Detection With Use of Stereovision and Laser Scanner","volume":"19","author":"Labayrade","year":"2005","journal-title":"Auton. Robots"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Bao, T., Zhang, Z., and Sabahi, M. (2019, January 9\u201311). An Improved Radar and Infrared Sensor Tracking Fusion Algorithm Based on IMM-UKF. Proceedings of the 2019 IEEE 16th International Conference on Networking, Sensing and Control (ICNSC), Banff, AB, Canada.","DOI":"10.1109\/ICNSC.2019.8743212"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Burlet, J., and Fontana, M.D. (2012, January 25\u201326). Robust and efficient multi-object detection and tracking for vehicle perception systems using radar and camera sensor fusion. Proceedings of the IET and ITS Conference on Road Transport Information and Control (RTIC 2012), London, UK.","DOI":"10.1049\/cp.2012.1553"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lee, K., Kanzawa, Y., Derry, M., and James, M.R. (2018, January 26\u201330). Multi-Target Track-to-Track Fusion Based on Permutation Matrix Track Association. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500433"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Farahnakian, F., Haghbayan, M., Poikonen, J., Laurinen, M., Nevalainen, P., and Heikkonen, J. (2018, January 17\u201320). Object Detection Based on Multi-sensor Proposal Fusion in Maritime Environment. Proceedings of the 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, USA.","DOI":"10.1109\/ICMLA.2018.00158"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Han, J., Liao, Y., Zhang, J., Wang, S., and Li, S. (2018). Target fusion detection of LiDAR and camera based on the improved YOLO algorithm. Mathematics, 6.","DOI":"10.3390\/math6100213"},{"key":"ref_23","unstructured":"Redmon, J., and Farhadi, A. (2020, August 25). YOLO9000: Better, Faster, Stronger, Available online: http:\/\/www.lanl.gov\/abs\/1612.08242."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Frossard, D., and Urtasun, R. (2018, January 21\u201325). End-to-end Learning of Multi-sensor 3D Tracking by Detection. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8462884"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Liang, M., Yang, B., Chen, Y., Hu, R., and Urtasun, R. (2019, January 16). Multi-Task Multi-Sensor Fusion for 3D Object Detection. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00752"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Liang, M., Yang, B., Zeng, W., Chen, Y., Hu, R., Casas, S., and Urtasun, R. (2020, August 25). PnPNet: End-to-End Perception and Prediction with Tracking in the Loop, Available online: http:\/\/www.lanl.gov\/abs\/2005.14711.","DOI":"10.1109\/CVPR42600.2020.01157"},{"key":"ref_27","unstructured":"Feng, D., Haase-Schuetz, C., Rosenbaum, L., Hertlein, H., Duffhauss, F., Glaser, C., Wiesbeck, W., and Dietmayer, K. (2020, August 25). Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges, Available online: http:\/\/www.lanl.gov\/abs\/1902.07830."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2847","DOI":"10.1109\/ACCESS.2019.2962554","article-title":"Multi-Sensor Fusion in Automated Driving: A Survey","volume":"8","author":"Wang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_29","unstructured":"Chen, X., Ma, H., Wan, J., Li, B., and Xia, T. (2020, August 25). Multi-View 3D Object Detection Network for Autonomous Driving, Available online: http:\/\/www.lanl.gov\/abs\/1611.07759."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ha, Q., Watanabe, K., Karasawa, T., Ushiku, Y., and Harada, T. (2017, January 24\u201328). MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8206396"},{"key":"ref_31","unstructured":"Ku, J., Mozifian, M., Lee, J., Harakeh, A., and Waslander, S.L. (2020, August 25). Joint 3D Proposal Generation and Object Detection from View Aggregation, Available online: http:\/\/www.lanl.gov\/abs\/1712.02294."},{"key":"ref_32","unstructured":"Mees, O., Eitel, A., and Burgard, W. (2020, August 25). Choosing Smartly: Adaptive Multimodal Fusion for Object Detection in Changing Environments, Available online: http:\/\/xxx.lanl.gov\/abs\/1707.05733."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Luber, M., Spinello, L., and Arras, K. (2011, January 25\u201330). People tracking in RGB-D Data with on-line boosted target models. Proceedings of the 2011 IEEE\/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6048836"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Allebosch, G., Van den Bossche, S., Veelaert, P., and Philips, W. (2020). Camera-Based System for Drafting Detection While Cycling. Sensors, 20.","DOI":"10.3390\/s20051241"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Pereira Barata, A., Takes, F.W., van den Herik, H.J., and Veenman, C.J. (2019, January 8\u201311). Imputation Methods Outperform Missing-Indicator for Data Missing Completely at Random. Proceedings of the 2019 International Conference on Data Mining Workshops (ICDMW), Beijing, China.","DOI":"10.1109\/ICDMW.2019.00066"},{"key":"ref_36","unstructured":"Silbert, M.E., and Rea, C.A. (2012, January 9\u201312). Track-to-track fusion with missing information: Empirical study: Tracking a variety of target types. Proceedings of the 2012 15th International Conference on Information Fusion, Singapore."},{"key":"ref_37","first-page":"82","article-title":"The probabilistic data association filter","volume":"29","author":"Daum","year":"2010","journal-title":"Control Syst. IEEE"},{"key":"ref_38","unstructured":"Fansi Tchango, A., Thomas, V., Buffet, O., Flacher, F., and Dutech, A. (2014, January 7\u201310). Tracking Multiple Interacting Targets Using a Joint Probabilistic Data Association Filter. Proceedings of the 17th International Conference on Information Fusion (FUSION), Salamanca, Spain."},{"key":"ref_39","unstructured":"Housfater, A., Zhang, X.P., and Zhou, Y. (2006, January 14\u201319). Nonlinear Fusion of Multiple Sensors with Missing Data. Proceedings of the 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, Toulouse, France."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1536","DOI":"10.1109\/JSTSP.2015.2465360","article-title":"Multiple Imputations Particle Filters: Convergence and Performance Analyses for Nonlinear State Estimation With Missing Data","volume":"9","author":"Zhang","year":"2015","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Claveria, R.M., Acuna, D., M\u2019endez, R.A., Silva, J.F., and Orchard, M.E. (2016, January 3\u20136). Application of Multiple-imputation-particle-filter for Parameter Estimation of Visual Binary Stars with Incomplete Observations. Proceedings of the Annual Conference of the Prognostics and Health Management Society 2016, Denver, CO, USA.","DOI":"10.36001\/phmconf.2016.v8i1.2585"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Jin, S., Huang, H., Li, Y., Ren, Y., Wang, Y., and Zhong, R. (2019, January 17\u201320). An Improved Particle Filter Based Track-Before-Detect Method for Underwater Target Bearing Tracking. Proceedings of the OCEANS 2019\u2014Marseille, Marseille, France.","DOI":"10.1109\/OCEANSE.2019.8867231"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2703","DOI":"10.1109\/TSP.2007.893914","article-title":"Particle Filtering for Multisensor Data Fusion With Switching Observation Models: Application to Land Vehicle Positioning","volume":"55","author":"Caron","year":"2007","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Kim, C., Li, F., Ciptadi, A., and Rehg, J.M. (2015, January 7\u201313). Multiple Hypothesis Tracking Revisited. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.533"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Willner, D., Chang, C.B., and Dunn, K.P. (1976, January 1\u20133). Kalman filter algorithms for a multi-sensor system. Proceedings of the 1976 IEEE Conference on Decision and Control including the 15th Symposium on Adaptive Processes, Clearwater, FL, USA.","DOI":"10.1109\/CDC.1976.267794"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Siciliano, B., and Khatib, O. (2008). Multisensor Data Fusion. Springer Handbook of Robotics, Springer.","DOI":"10.1007\/978-3-540-30301-5"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Dimitrievski, M., Veelaert, P., and Philips, W. (2019). Behavioral Pedestrian Tracking Using a Camera and LiDAR Sensors on a Moving Vehicle. Sensors, 19.","DOI":"10.3390\/s19020391"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Andrieu, C., Doucet, A., and Punskaya, E. (2001). Sequential Monte Carlo Methods in Practice, Springer.","DOI":"10.1007\/978-1-4757-3437-9"},{"key":"ref_49","unstructured":"Van Der Merwe, R., Doucet, A., De Freitas, N., and Wan, E. (2001). The Unscented Particle Filter. Advances in Neural Information Processing Systems, Massachusetts Institute of Technology Press."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Rubin, D.B. (1987). Multiple Imputation for Nonresponse in Surveys, Wiley.","DOI":"10.1002\/9780470316696"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1080\/01621459.1994.10476469","article-title":"Sequential Imputations and Bayesian Missing Data Problems","volume":"89","author":"Kong","year":"1994","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., and Beijbom, O. (2020, August 25). nuScenes: A Multimodal Dataset for Autonomous Driving, Available online: http:\/\/www.lanl.gov\/abs\/1903.11027.","DOI":"10.1109\/CVPR42600.2020.01164"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Mimouna, A., Alouani, I., Ben Khalifa, A., El Hillali, Y., Taleb-Ahmed, A., Menhaj, A., Ouahabi, A., and Ben Amara, N.E. (2020). OLIMP: A Heterogeneous Multimodal Dataset for Advanced Environment Perception. Electronics, 9.","DOI":"10.3390\/electronics9040560"},{"key":"ref_54","unstructured":"Meyer, M., and Kuschk, G. (2019, January 2\u20134). Automotive Radar Dataset for Deep Learning Based 3D Object Detection. Proceedings of the 2019 16th European Radar Conference (EuRAD), Paris, France."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1109\/LRA.2020.2967272","article-title":"CNN based Road User Detection using the 3D Radar Cube","volume":"5","author":"Palffy","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Dimitrievski, M., Hamme, D.V., Veelaert, P., and Philips, W. (2016). Robust Matching of Occupancy Maps for Odometry in Autonomous Vehicles. Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications\u2014Volume 3: VISAPP, (VISIGRAPP 2016). INSTICC, SciTePress.","DOI":"10.5220\/0005719006260633"},{"key":"ref_57","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Doll\u00e1r, P. (2020, August 25). Microsoft COCO: Common Objects in Context, Available online: http:\/\/www.lanl.gov\/abs\/arXiv:1405.0312."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/17\/4817\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:06:55Z","timestamp":1760177215000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/17\/4817"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,26]]},"references-count":57,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["s20174817"],"URL":"https:\/\/doi.org\/10.3390\/s20174817","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,26]]}}}