{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,3]],"date-time":"2026-11-03T16:46:12Z","timestamp":1793724372773,"version":"4.6.1"},"reference-count":37,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T00:00:00Z","timestamp":1774396800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Basic Science (Nature Science) Research Project of Jiangsu Province for Colleges and Universities","award":["24KJD510006"],"award-info":[{"award-number":["24KJD510006"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Red\u2013Green\u2013Blue\u2013Thermal (RGBT) object detection is critical for robust all-weather perception. However, deploying dual-stream networks on resource-constrained edge devices is severely hindered by insufficiently adaptive multimodal fusion, the loss of small-object features during downsampling, and substantial computational overhead. To address these challenges, we propose GEM-YOLO, a real-time and lightweight RGBT detector. Specifically, an Adaptive Multimodal Gated Fusion Mechanism (GFM) is designed to dynamically calibrate modality weights and suppress noise. Furthermore, Space-to-Depth (SPD) convolutions are integrated into the backbone to achieve lossless downsampling, preventing the feature collapse of small targets. Finally, a lightweight Ghost-Neck is constructed using Ghost modules and GSConv to eliminate computational redundancy. Extensive experiments on the Forward-Looking Infrared (FLIR) and Multi-Modal Multispectral Fusion Dataset (M3FD) datasets demonstrate the effectiveness of the proposed method. With only 7.58 Giga Floating-Point Operations (GFLOPs) and 3.44 million parameters (M), GEM-YOLO reduces the computational cost by 18.6% relative to the dual-stream YOLOv11n baseline. Concurrently, it achieves competitive mean Average Precision at IoU = 0.5 (mAP@50) scores of 82.8% and 69.0% on FLIR and M3FD, respectively, with more evident gains on small-target localization. In practice, GEM-YOLO maintains competitive detection performance while keeping computational overhead low, making it promising for real-time multispectral perception on resource-constrained edge platforms.<\/jats:p>","DOI":"10.3390\/s26072035","type":"journal-article","created":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T10:07:13Z","timestamp":1774433233000},"page":"2035","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["GEM-YOLO: A Lightweight and Real-Time RGBT Object Detector with Gated Multimodal Fusion"],"prefix":"10.3390","volume":"26","author":[{"given":"Lijuan","family":"Wang","sequence":"first","affiliation":[{"name":"School of Electronic and Optical Engineering, Nanjing University of Science and Technology ZiJin College, Nanjing 210023, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zuchao","family":"Bao","sequence":"additional","affiliation":[{"name":"School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dongming","family":"Lu","sequence":"additional","affiliation":[{"name":"School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,3,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1109\/JPROC.2023.3238524","article-title":"Object Detection in 20 Years: A Survey","volume":"111","author":"Zou","year":"2023","journal-title":"Proc. IEEE"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_3","unstructured":"Jocher, G., Chaurasia, A., and Qiu, J. (2025, January 10). Ultralytics YOLO. Available online: https:\/\/docs.ultralytics.com\/."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Sun, C., Chen, Y., Qiu, X., Li, R., and You, L. (2024). MRD-YOLO: A Multispectral Object Detection Algorithm for Complex Road Scenes. Sensors, 24.","DOI":"10.3390\/s24103222"},{"key":"ref_5","unstructured":"Li, C., Song, D., Tong, R., and Tang, M. (2018, January 3\u20136). Multispectral Pedestrian Detection via Simultaneous Detection and Segmentation. Proceedings of the 29th British Machine Vision Conference (BMVC), Newcastle, UK."},{"key":"ref_6","unstructured":"Zhang, L., Zhu, X., Chen, X., Yang, X., Lei, Z., and Liu, Z. (November, January 27). Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zhang, H., Fromont, E., Lefevre, S., and Aviniti, B. (2020). Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blocks. Proceedings of the IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, 25\u201328 October 2020, IEEE.","DOI":"10.1109\/ICIP40778.2020.9191080"},{"key":"ref_8","unstructured":"Wagner, J., Fischer, V., Herman, M., and Behnke, S. (2016, January 27\u201329). Multispectral Pedestrian Detection Using Deep Fusion Convolutional Neural Networks. Proceedings of the 24th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges, Belgium."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Bao, C., Cao, J., Hao, Q., Cheng, Y., Ning, Y., and Zhao, T. (2023). Dual-YOLO Architecture from Infrared and Visible Images for Object Detection. Sensors, 23.","DOI":"10.3390\/s23062934"},{"key":"ref_10","unstructured":"FLIR Systems (2025, January 15). Free FLIR Thermal Dataset for Algorithm Training. Available online: https:\/\/www.flir.com\/industries\/automotive\/."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Liu, J., Fan, X., Huang, Z., Wu, G., Liu, R., Zhong, W., and Luo, Z. (2022, January 18\u201324). Target-Aware Dual-Adversarial Learning and a Multi-Scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00571"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Sunkara, R., and Luo, T. (2022, January 19\u201323). No More Strided Convolutions or Pooling: A New CNN Building Block for Low-Resolution Images and Small Objects. Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), Grenoble, France.","DOI":"10.1007\/978-3-031-26409-2_27"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Han, K., Wang, Y., Tian, Q., Guo, J., and Xu, C. (2020). GhostNet: More Features from Cheap Operations. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13\u201319 June 2020, IEEE.","DOI":"10.1109\/CVPR42600.2020.00165"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1007\/s11554-024-01436-6","article-title":"Slim-neck by GSConv: A lightweight-design for real-time detector architectures","volume":"21","author":"Li","year":"2024","journal-title":"J. Real-Time Image Process."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhao, P., Ye, X., and Du, Z. (2024). Object Detection in Multispectral Remote Sensing Images Based on Cross-Modal Cross-Attention. Sensors, 24.","DOI":"10.3390\/s24134098"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Jia, X., Zhu, C., Li, M., Tang, W., and Zhou, W. (2021, January 11\u201317). LLVIP: A Visible-Infrared Paired Dataset for Low-Light Vision. Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, QC, Canada.","DOI":"10.1109\/ICCVW54120.2021.00389"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Lee, S., Kim, T., Shin, J., Kim, N., and Choi, Y. (2024). INSANet: INtra-INter Spectral Attention Network for Effective Feature Fusion of Multispectral Pedestrian Detection. Sensors, 24.","DOI":"10.3390\/s24041168"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"852","DOI":"10.1109\/TMM.2023.3272471","article-title":"Multiscale Cross-Modal Homogeneity Enhancement and Confidence-Aware Fusion for Multispectral Pedestrian Detection","volume":"26","author":"Li","year":"2024","journal-title":"IEEE Trans. Multimedia"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Shen, J., He, J., Liu, Q., Zhang, Z., Wang, G., and Lu, D. (2025). MSDF-Mamba: Mutual-Spectrum Perception Deformable Fusion Mamba for Drone-Based Visible\u2013Infrared Cross-Modality Vehicle Detection. Remote Sens., 17.","DOI":"10.3390\/rs17244037"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2576","DOI":"10.1109\/LRA.2019.2904733","article-title":"RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes","volume":"4","author":"Sun","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhou, K., Chen, L., and Cao, X. (2020, January 23\u201328). Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK.","DOI":"10.1007\/978-3-030-58523-5_46"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Chen, Y.-T., Shi, J., Ye, Z., Mertz, C., Ramanan, D., and Kong, S. (2022, January 23\u201327). Multimodal Object Detection via Probabilistic Ensembling. Proceedings of the European Conference on Computer Vision (ECCV), Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-20077-9_9"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhang, H., Fromont, E., Lef\u00e8vre, S., and Avignon, B. (2021, January 5\u20139). Guided Attentive Feature Fusion for Multispectral Pedestrian Detection. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.","DOI":"10.1109\/WACV48630.2021.00012"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Fang, Q., Han, D., and Wang, Z. (2021). Cross-Modality Fusion Transformer for Multispectral Object Detection. arXiv, Available online: https:\/\/arxiv.org\/abs\/2111.00273.","DOI":"10.2139\/ssrn.4227745"},{"key":"ref_25","first-page":"1","article-title":"YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information","volume":"Volume 15089","author":"Leonardis","year":"2025","journal-title":"Computer Vision\u2014ECCV 2024"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yang, C., Huang, Z., and Wang, N. (2022, January 18\u201324). QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01330"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"105475","DOI":"10.1016\/j.infrared.2024.105475","article-title":"MDCENet: Multi-Dimensional Cross-Enhanced Network for Infrared Small Target Detection","volume":"141","author":"Ma","year":"2024","journal-title":"Infrared Phys. Technol."},{"key":"ref_28","unstructured":"Wang, A., Chen, H., Liu, L., Chen, K., Lin, Z., Han, J., and Ding, G. (2024, January 10\u201315). YOLOv10: Real-Time End-to-End Object Detection. Proceedings of the 38th Annual Conference on Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chen, J., Kao, S.-H., He, H., Zhuo, W., Wen, S., Lee, C.-H., and Chan, S.-H.G. (2023, January 18\u201322). Run, Don\u2019t Walk: Chasing Higher FLOPS for Faster Neural Networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01157"},{"key":"ref_30","unstructured":"Qin, D., Leichner, C., Delakis, M., Fornoni, M., Luo, S., Yang, F., Wang, W., Banbury, C., Ye, C., and Akin, B. (October, January 29). MobileNetV4: Universal Models for the Mobile Ecosystem. Proceedings of the 18th European Conference on Computer Vision (ECCV), Milan, Italy."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2080","DOI":"10.1109\/TIV.2024.3443264","article-title":"CDC-YOLOFusion: Leveraging Cross-Scale Dynamic Convolution Fusion for Visible-Infrared Object Detection","volume":"10","author":"Wang","year":"2025","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ma, N., Zhang, X., Zheng, H.-T., and Sun, J. (2018, January 8\u201314). ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_8"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4240","DOI":"10.1109\/TPAMI.2025.3538473","article-title":"YOLO-MS: Rethinking Multi-Scale Representation Learning for Real-Time Object Detection","volume":"47","author":"Chen","year":"2025","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhao, Y., Lv, W., Xu, S., Wei, J., Wang, G., and Dang, Q. (2024, January 17\u201321). DETRs Beat YOLOs on Real-time Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.01605"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Liu, S., Qi, L., Qin, H., Shi, J., and Jia, J. (2018, January 18\u201322). Path Aggregation Network for Instance Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00913"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8\u201314). CBAM: Convolutional Block Attention Module. Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2023). YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 18\u201322 June 2023, IEEE.","DOI":"10.1109\/CVPR52729.2023.00721"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/7\/2035\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T10:11:48Z","timestamp":1774433508000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/7\/2035"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,25]]},"references-count":37,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,4]]}},"alternative-id":["s26072035"],"URL":"https:\/\/doi.org\/10.3390\/s26072035","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,25]]}}}