{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T05:00:22Z","timestamp":1779253222873,"version":"3.51.4"},"reference-count":43,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2021,11,30]],"date-time":"2021-11-30T00:00:00Z","timestamp":1638230400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006595","name":"Unitatea Executiva Pentru Finantarea Invatamantului Superior a Cercetarii Dezvoltarii si Inovarii","doi-asserted-by":"publisher","award":["PN-III-P4-ID-PCE-2020-1700"],"award-info":[{"award-number":["PN-III-P4-ID-PCE-2020-1700"]}],"id":[{"id":"10.13039\/501100006595","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Object tracking is an essential problem in computer vision that has been extensively researched for decades. Tracking objects in thermal images is particularly difficult because of the lack of color information, low image resolution, or high similarity between objects of the same class. One of the main challenges in multi-object tracking, also referred to as the data association problem, is finding the correct correspondences between measurements and tracks and adapting the object appearance changes over time. We addressed this challenge of data association for thermal images by proposing three contributions. The first contribution consisted of the creation of a data-driven appearance score using five Siamese Networks, which operate on the image detection and on parts of it. Secondly, we engineered an original edge-based descriptor that improves the data association process. Lastly, we proposed a dataset consisting of pedestrian instances that were recorded in different scenarios and are used for training the Siamese Networks. The data-driven part of the data association score offers robustness, while feature engineering offers adaptability to unknown scenarios and their combination leads to a more powerful tracking solution. Our approach had a running time of 25 ms and achieved an average precision of 86.2% on publicly available benchmarks, containing real-world scenarios, as shown in the evaluation section.<\/jats:p>","DOI":"10.3390\/s21238005","type":"journal-article","created":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T01:45:02Z","timestamp":1638323102000},"page":"8005","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Robust Data Association Using Fusion of Data-Driven and Engineered Features for Real-Time Pedestrian Tracking in Thermal Images"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0315-3507","authenticated-orcid":false,"given":"Mircea Paul","family":"Muresan","sequence":"first","affiliation":[{"name":"Computer Science Department, Technical University of Cluj-Napoca, 28 Memorandumului Street, 400114 Cluj Napoca, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2018-4647","authenticated-orcid":false,"given":"Sergiu","family":"Nedevschi","sequence":"additional","affiliation":[{"name":"Computer Science Department, Technical University of Cluj-Napoca, 28 Memorandumului Street, 400114 Cluj Napoca, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4515-8114","authenticated-orcid":false,"given":"Radu","family":"Danescu","sequence":"additional","affiliation":[{"name":"Computer Science Department, Technical University of Cluj-Napoca, 28 Memorandumului Street, 400114 Cluj Napoca, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Berg, A., Ahlberg, J., and Felsberg, M. (2015, January 25\u201328). A thermal Object Tracking benchmark. Proceedings of the 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Karlsruhe, Germany.","DOI":"10.1109\/AVSS.2015.7301772"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Muresan, M.P., and Nedevschi, S. (2019, January 5\u20137). Multi-Object Tracking of 3D Cuboids Using Aggregated Features. Proceedings of the 2019 IEEE 15th International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania.","DOI":"10.1109\/ICCP48234.2019.8959552"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"223346","DOI":"10.1109\/ACCESS.2020.3043414","article-title":"Stereovision Tracking System for Monitoring Loader Crane Tip Position","volume":"8","author":"Grudzinski","year":"2020","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Tang, S., Andriluka, M., Andres, B., and Schiele, B. (2017, January 21\u201326). Multiple people tracking by lifted multicut and person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.394"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16\u201321). Are we ready for autonomous driving? The KITTI vision benchmark suite. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"ref_6","unstructured":"Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, P.R., Zajc, L.C., Voj\u00edr, T., Bhat, G., Lukezic, A., and Eldesokey, A. (, January 8\u201314). The Sixth Visual Object Tracking vot2018 Challenge Results. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, Germany."},{"key":"ref_7","unstructured":"Hua, G., and J\u00e9gou, H. (2016). Multi-Class Multi-Object Tracking Using Changing Point Detection, Springer. Computer Vision\u2014ECCV Workshops."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1657","DOI":"10.1109\/TVT.2004.834876","article-title":"Pedestrian detection using stereo night vision","volume":"53","author":"Liu","year":"2004","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Mure\u0219an, M.P., Giosan, I., and Nedevschi, S. (2020). Stabilization and Validation of 3D Object Position Using Multimodal Sensor Fusion and Semantic Segmentation. Sensors, 20.","DOI":"10.3390\/s20041110"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kim, D., and Kwon, D. (, January 28\u201330). Pedestrian detection and tracking in thermal images using shape features. Proceedings of the 2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Goyangi, Korea.","DOI":"10.1109\/URAI.2015.7358920"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"G\u00fcnd\u00fcz, G., and Acarman, T. (2018, January 26\u201330). A lightweight online multiple object vehicle tracking method. Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500386"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"104423","DOI":"10.1109\/ACCESS.2019.2932301","article-title":"Multiple Object Tracking With Attention to Appearance, Structure, Motion and Size","volume":"7","author":"Karunasekera","year":"2019","journal-title":"IEEE Access"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1002\/nav.3800020109","article-title":"The Hungarian method for the assignment problem","volume":"2","author":"Kuhn","year":"1955","journal-title":"Nav. Res. Logist. Q."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Choi, W. (2015, January 7\u201313). Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.347"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1666","DOI":"10.1109\/TVT.2004.834878","article-title":"IR pedestrian detection for advanced driver assistance systems","volume":"53","author":"Bertozzi","year":"2004","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1109\/TITS.2008.922943","article-title":"Pedestrian Detection and Tracking Using a Mixture of View-Based Shape\u2013Texture Models","volume":"9","author":"Munder","year":"2008","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kallhammer, J.E., Eriksson, D., Granlund, G., Felsberg, M., Moe, A., Johansson, B., Wiklund, J., and Forssen, P.E. (2007, January 13\u201315). Near Zone Pedestrian Detection using a Low-Resolution FIR Sensor. Proceedings of the 2007 IEEE Intelligent Vehicles Symposium, Istanbul, Turkey.","DOI":"10.1109\/IVS.2007.4290137"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1109\/TITS.2016.2569159","article-title":"Pedestrian Tracking Using Online Boosted Random Ferns Learning in Far-Infrared Imagery for Safe Driving at Night","volume":"18","author":"Kwak","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1016\/j.patrec.2017.10.026","article-title":"Dense structural learning for infrared object tracking at 200+ Frames per Second","volume":"100","author":"Yu","year":"2017","journal-title":"Pattern Recognit. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhu, G., Porikli, F., and Li, H. (2016, January 27\u201330). Beyond local search: Tracking objects everywhere with instance-specific proposals. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.108"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"74302","DOI":"10.1109\/ACCESS.2021.3080822","article-title":"Pedestrian Street-Cross Action Recognition in Monocular Far Infrared Sequences","volume":"9","author":"Brehar","year":"2021","journal-title":"IEEE Access"},{"key":"ref_22","unstructured":"Nam, H., Baek, M., and Han, B. (2016). Modeling and propagating cnns in a tree structure for visual tracking. arXiv."},{"key":"ref_23","unstructured":"Alahari, K., Berg, A., Hager, G., Ahlberg, J., Kristan, M., Matas, J., Leonardis, A., Cehovin, L., Fernandez, G., and Vojir, T. (2015, January 7\u201313). The thermal infrared visual object tracking vot-tir2015 challenge results. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Danelljan, M., Hager, G., Khan, F.S., and Felsberg, M. (2015, January 7\u201313). Learning spatially regularized correlation filters for visual tracking. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.490"},{"key":"ref_25","first-page":"850","article-title":"Fully-convolutional siamese networks for object tracking","volume":"Volume 9914","author":"Hua","year":"2016","journal-title":"Computer Vision\u2014ECCV 2016 Workshops. ECCV 2016. Lecture Notes in Computer Science"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Liu, Q., Yuan, D., and He, Z. (2017, January 7\u201313). Thermal infrared object tracking via Siamese convolutional neural networks. Proceedings of the 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), Shenzhen, China.","DOI":"10.1109\/SPAC.2017.8304241"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhang, X., Chen, R., Liu, G., Li, X., Luo, S., and Fan, X. (2020, January 22\u201324). Thermal Infrared Tracking using Multi-stages Deep Features Fusion. Proceedings of the 2020 Chinese Control And Decision Conference (CCDC), Hefei, China.","DOI":"10.1109\/CCDC49329.2020.9164750"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Guo, Q., Feng, W., Zhou, C., Huang, R., Wan, L., and Wang, S. (2017, January 22\u201329). Learning Dynamic Siamese Network for Visual Object Tracking. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.196"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, G., Luo, C., Xiong, Z., and Zeng, W. (2019, January 15\u201320). SPM-Tracker: Series-Parallel Matching for Real-Time Visual Object Tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00376"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1837","DOI":"10.1109\/TIP.2018.2879249","article-title":"Synthetic Data Generation for End-to-End Thermal Infrared Tracking","volume":"28","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Valmadre, J., Bertinetto, L., Henriques, J., Vedaldi, A., and Torr, P.H. (2017, January 21\u201326). End-to-end representation learning for correlation filter-based tracking. Computer Vision and Pattern Recognition (CVPR). Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.531"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2114","DOI":"10.1109\/TMM.2020.3008028","article-title":"Learning Deep Multi-Level Similarity for Thermal Infrared Object Tracking","volume":"23","author":"Liu","year":"2020","journal-title":"IEEE Trans. Multimedia"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2013\/429347","article-title":"Towards Understanding the Formation of Uniform Local Binary Patterns","volume":"2013","author":"Lahdenoja","year":"2013","journal-title":"ISRN Mach. Vis."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"10166","DOI":"10.3390\/s150510166","article-title":"Infrared Thermal Imaging System on a Mobile Phone","volume":"15","author":"Lee","year":"2015","journal-title":"Sensors"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, RealTime Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_37","unstructured":"FLIR (2021, November 29). Flir Thermal Dataset for Algorithm Training. Available online: https:\/\/www.flir.com\/oem\/adas\/adas-dataset-form\/."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"666","DOI":"10.1109\/TMM.2019.2932615","article-title":"PTB-TIR: A Thermal Infrared Pedestrian Tracking Benchmark","volume":"22","author":"Liu","year":"2019","journal-title":"IEEE Trans. Multimedia"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Song, Y., Ma, C., Wu, X., Gong, L., Bao, L., Zuo, W., Shen, C., Lau, R.W.H., and Yang, M.-H. (2018, January 18\u201323). VITAL: VIsual tracking via adversarial learning. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00937"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Li, X., Ma, C., Wu, B., He, Z., and Yang, M.H. (2019, January 16\u201320). Target-aware deep tracking. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition CVPR 2019, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00146"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/j.knosys.2017.07.032","article-title":"Deep convolutional neural networks for thermal infrared object tracking","volume":"134","author":"Liu","year":"2017","journal-title":"Knowl.-Based Syst."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Qi, Y., Zhang, S., Qin, L., Yao, H., Huang, Q., Lim, J., and Yang, M.-H. (2016, January 27\u201330). Hedged deep tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.466"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Ma, C., Huang, J.-B., Yang, X., and Yang, M.-H. (2015, January 7\u201313). Hierarchical convolutional features for visual tracking. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.352"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/23\/8005\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:38:08Z","timestamp":1760168288000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/23\/8005"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,30]]},"references-count":43,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["s21238005"],"URL":"https:\/\/doi.org\/10.3390\/s21238005","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,30]]}}}