{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T00:33:10Z","timestamp":1773880390426,"version":"3.50.1"},"reference-count":33,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T00:00:00Z","timestamp":1730851200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia (FCT)"},{"name":"NSERC Discovery and Canada Research Chair Programs"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Drones"],"abstract":"<jats:p>Given the recent proliferation of Unmanned Aerial Systems (UASs) and the consequent importance of counter-UASs, this project aims to perform the detection and tracking of small non-cooperative UASs using Electro-optical (EO) and Infrared (IR) sensors. Two data integration techniques, at the decision and pixel levels, are compared with the use of each sensor independently to evaluate the system robustness in different operational conditions. The data are submitted to a YOLOv7 detector merged with a ByteTrack tracker. For training and validation, additional efforts are made towards creating datasets of spatially and temporally aligned EO and IR annotated Unmanned Aerial Vehicle (UAV) frames and videos. These consist of the acquisition of real data captured from a workstation on the ground, followed by image calibration, image alignment, the application of bias-removal techniques, and data augmentation methods to artificially create images. The performance of the detector across datasets shows an average precision of 88.4%, recall of 85.4%, and mAP@0.5 of 88.5%. Tests conducted on the decision-level fusion architecture demonstrate notable gains in recall and precision, although at the expense of lower frame rates. Precision, recall, and frame rate are not improved by the pixel-level fusion design.<\/jats:p>","DOI":"10.3390\/drones8110650","type":"journal-article","created":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T11:13:37Z","timestamp":1730891617000},"page":"650","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Infrared and Visible Camera Integration for Detection and Tracking of Small UAVs: Systematic Evaluation"],"prefix":"10.3390","volume":"8","author":[{"given":"Ana","family":"Pereira","sequence":"first","affiliation":[{"name":"Instituto Superior T\u00e9cnico, Universidade de Lisboa, 1049-001 Lisbon, Portugal"}]},{"given":"Stephen","family":"Warwick","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, University of Victoria, Victoria, BC V8P 5C2, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4271-7996","authenticated-orcid":false,"given":"Alexandra","family":"Moutinho","sequence":"additional","affiliation":[{"name":"IDMEC, Mechanical Engineering Institute, Instituto Superior T\u00e9cnico, Universidade de Lisboa, 1049-001 Lisbon, Portugal"}]},{"given":"Afzal","family":"Suleman","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, University of Victoria, Victoria, BC V8P 5C2, Canada"},{"name":"IDMEC, Mechanical Engineering Institute, Instituto Superior T\u00e9cnico, Universidade de Lisboa, 1049-001 Lisbon, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,6]]},"reference":[{"key":"ref_1","unstructured":"(2024, January 19). Worldwide Drone Incidents. Available online: https:\/\/www.dedrone.com\/resources\/incidents-new\/all."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Castrillo, V.U., Manco, A., Pascarella, D., and Gigante, G. (2022). A Review of Counter-UAS Technologies for Cooperative Defensive Teams of Drones. Drones, 6.","DOI":"10.3390\/drones6030065"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"42635","DOI":"10.1109\/ACCESS.2021.3065926","article-title":"Survey on Anti-Drone Systems: Components, Designs, and Challenges","volume":"9","author":"Park","year":"2021","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/MAES.2020.3015537","article-title":"Counter-Unmanned Aircraft System(s) (C-UAS): State of the Art, Challenges, and Future Trends","volume":"36","author":"Wang","year":"2021","journal-title":"IEEE Aerosp. Electron. Syst. Mag."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wang, B., Li, Q., Mao, Q., Wang, J., Chen, C.L.P., Shangguan, A., and Zhang, H. (2024). A Survey on Vision-Based Anti Unmanned Aerial Vehicles Methods. Drones, 8.","DOI":"10.3390\/drones8090518"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6285","DOI":"10.1109\/TGRS.2020.3028654","article-title":"Micro-Doppler Signature-Based Detection, Classification, and Localization of Small UAV with Long Short-Term Memory Neural Network","volume":"59","author":"Sun","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Passafiume, M., Rojhani, N., Collodi, G., and Cidronali, A. (2021). Modeling small UAV micro-doppler signature using millimeter-wave FMCW radar. Electronics, 10.","DOI":"10.3390\/electronics10060747"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yan, J., Hu, H., Gong, J., Kong, D., and Li, D. (2021). Exploring Radar Micro-Doppler Signatures for Recognition of Drone Types. Drones, 7.","DOI":"10.3390\/drones7040280"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3062","DOI":"10.1109\/LRA.2022.3145498","article-title":"Drone Detection Using Sparse Lidar Measurements","volume":"7","author":"Dogru","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Nelega, R., Belean, B., Valeriu, R., Turcu, F., and Puschita, E. (2023, January 21\u201323). Radio Frequency-Based Drone Detection and Classification using Deep Learning Algorithms. Proceedings of the 2023 International Conference on Software, Telecommunications and Computer Networks (SoftCOM), Split, Croatia.","DOI":"10.23919\/SoftCOM58365.2023.10271669"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Fu, Y., and He, Z. (2024). Radio Frequency Signal-Based Drone Classification with Frequency Domain Gramian Angular Field and Convolutional Neural Network. Drones, 8.","DOI":"10.3390\/drones8090511"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2731","DOI":"10.1109\/TVT.2020.2964110","article-title":"An Acoustic-Based Surveillance System for Amateur Drones Detection and Localization","volume":"69","author":"Shi","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ahmed, C.A., Batool, F., Haider, W., Asad, M., and Raza Hamdani, S.H. (2022, January 28\u201331). Acoustic Based Drone Detection Via Machine Learning. Proceedings of the 2022 International Conference on IT and Industrial Technologies (ICIT), Shanghai, China.","DOI":"10.1109\/ICIT56493.2022.9989229"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"25323","DOI":"10.1109\/TITS.2022.3177627","article-title":"Vision-Based Anti-UAV Detection and Tracking","volume":"23","author":"Zhao","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ghosh, S., Patrikar, J., Moon, B., Hamidi, M.M., and Scherer, S. (June, January 29). AirTrack: Onboard Deep Learning Framework for Long-Range Aircraft Detection and Tracking. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10160627"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Coluccia, A., Fascista, A., Schumann, A., Sommer, L., Dimou, A., Zarpalas, D., M\u00e9ndez, M., de la Iglesia, D., Gonz\u00e1lez, I., and Mercier, J.P. (2021). Drone vs. Bird detection: Deep learning algorithms and results from a grand challenge. Sensors, 21.","DOI":"10.3390\/s21082824"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"102949","DOI":"10.1016\/j.dsp.2020.102949","article-title":"Detection and tracking of infrared small target by jointly using SSD and pipeline filter","volume":"110","author":"Ding","year":"2021","journal-title":"Digit. Signal Process. Rev. J."},{"key":"ref_18","first-page":"1","article-title":"Infrared Small UAV Target Detection Based on Depthwise Separable Residual Dense Network and Multiscale Feature Fusion","volume":"71","author":"Fang","year":"2022","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1109\/MGRS.2021.3115137","article-title":"Deep Learning for Unmanned Aerial Vehicle-Based Object Detection and Tracking: A survey","volume":"10","author":"Wu","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Svanstr\u00f6m, F., Alonso-Fernandez, F., and Englund, C. (2022). Drone Detection and Tracking in Real-Time by Fusion of Different Sensing Modalities. Drones, 6.","DOI":"10.3390\/drones6110317"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Alldieck, T., Bahnsen, C.H., and Moeslund, T.B. (2016). Context-aware fusion of RGB and thermal imagery for traffic monitoring. Sensors, 16.","DOI":"10.3390\/s16111947"},{"key":"ref_22","unstructured":"Yang, L., Ma, R., and Zakhor, A. (2022, January 17\u201320). Drone Object Detection Using RGB\/IR Fusion. Proceedings of the Symposium on Electronic Imaging: Computational Imaging XX, Online."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2023, January 18\u201322). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00721"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Sun, P., Jiang, Y., Yu, D., Weng, F., Yuan, Z., Luo, P., Liu, W., and Wang, X. (2022, January 23\u201327). YOLOv7: ByteTrack: Multi-Object Tracking by Associating Every Detection Box. Proceedings of the European Conference on Computer Vision (ECCV), Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-20047-2_1"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1858","DOI":"10.1109\/TPAMI.2008.113","article-title":"Parametric image alignment using enhanced correlation coefficient maximization","volume":"30","author":"Evangelidis","year":"2008","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","unstructured":"Lopes, J.P.D., Suleman, A., and Figueiredo, M.A.T. (2022). Detection and Tracking of Non-Cooperative UAVs: A Deep Learning Moving-Object Tracking Approach. [MsC Thesis, Instituto Superior T\u00e9cnico]."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sun, C., Zhang, C., and Xiong, N. (2020). Infrared and visible image fusion techniques based on deep learning: A review. Electronics, 9.","DOI":"10.3390\/electronics9122162"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.inffus.2018.09.004","article-title":"FusionGAN: A generative adversarial network for infrared and visible image fusion","volume":"48","author":"Ma","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Szeliski, R. (2021). Computer Vision: Algorithms and Applications, Springer. [2nd ed.].","DOI":"10.1007\/978-3-030-34372-9"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1768","DOI":"10.1017\/aer.2021.63","article-title":"Design and performance quantification of VTOL systems for a canard aircraft","volume":"125","author":"Pedro","year":"2021","journal-title":"Aeronaut. J."},{"key":"ref_31","unstructured":"Castellani, N., Pedrosa, F., Matlock, J., Mazur, A., Lowczycki, K., Widera, P., Zawadzki, K., Lipka, K., and Suleman, A. (2023, January 5\u20138). Development of a Series Hybrid Multirotor. Proceedings of the 13th EASN International Conference on Innovation in Aviation & Space for opening New Horizons, Salerno, Italy."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2243","DOI":"10.1109\/TPAMI.2008.263","article-title":"Single-Image Vignetting Correction","volume":"31","author":"Zheng","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zheng, Y., Grossman, M., Awate, S., and Gee, J. (2009, January 20\u201324). Automatic Correction of Intensity Nonuniformity From Sparseness of Gradient Distribution in Medical Images. Proceedings of the 12th International Conference on Medical Image Computing and Computer Assisted Intervention, London, UK.","DOI":"10.1007\/978-3-642-04271-3_103"}],"container-title":["Drones"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-446X\/8\/11\/650\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:27:33Z","timestamp":1760113653000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-446X\/8\/11\/650"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,6]]},"references-count":33,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["drones8110650"],"URL":"https:\/\/doi.org\/10.3390\/drones8110650","relation":{},"ISSN":["2504-446X"],"issn-type":[{"value":"2504-446X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,6]]}}}