{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T18:10:11Z","timestamp":1788891011296,"version":"build-2803163510"},"reference-count":48,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T00:00:00Z","timestamp":1788825600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>To address the challenges of modality heterogeneity, scale inconsistency, and background interference in dense small object detection under multimodal conditions, this paper proposes a novel detection framework based on cross-modality guidance and hierarchical scale refinement. Built upon the RT-DETR backbone, the framework integrates a Cross-Modality Guided Dynamic Fusion (CMG-DF) module, which performs semantic-level recalibration between infrared and visible features via a learnable modality attention mechanism, and a Hierarchical Scale Refinement Network (HSRN), which enhances semantic consistency and boundary continuity across scales through bidirectional residual flow and graph-based relational modeling. To validate the effectiveness of the proposed method, extensive comparison and ablation studies are conducted on two public multimodal benchmarks, SMOD and LLVIP. Experimental results show that the proposed method achieves 92.7% mAP@50 and 69.5% mAP@50:95 on SMOD, as well as 77.3% mAP@50 and 43.1% mAP@50:95 on LLVIP, consistently outperforming existing state-of-the-art multimodal detection algorithms. Qualitative visualizations further confirm the robustness and enhancement capability of the method for small objects under low illumination, occlusion, and complex background conditions, highlighting its strong structural generalization and practical deployment potential.<\/jats:p>","DOI":"10.1371\/journal.pone.0348533","type":"journal-article","created":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T17:49:11Z","timestamp":1788889751000},"page":"e0348533","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Research on multimodal dense small object detection algorithms guided by cross-modal information"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-4906-8114","authenticated-orcid":true,"given":"Jiacheng","family":"Hu","sequence":"first","affiliation":[{"name":"A. B. Freeman School of Business, Tulane University","place":["New Orleans, United States of America"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yumeng","family":"Ma","sequence":"additional","affiliation":[{"name":"Ira A. Fulton School of Engineering, Arizona State University","place":["Tempe, United States of America"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yue","family":"Xing","sequence":"additional","affiliation":[{"name":"School of Engineering and Applied Science, University of Pennsylvania","place":["Philadelphia, United States of America"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yun","family":"Zi","sequence":"additional","affiliation":[{"name":"School of Computer Science, Georgia Institute of Technology","place":["Atlanta, United States of America"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yingnan","family":"Deng","sequence":"additional","affiliation":[{"name":"School of Computer Science, Georgia Institute of Technology","place":["Atlanta, United States of America"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ming","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Graduate and Professional Studies, Trine University","place":["Phoenix, United States of America"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Heyao","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Engineering, Northeastern University","place":["Boston, United States of America"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4306-6339","authenticated-orcid":true,"given":"Junliang","family":"Du","sequence":"additional","affiliation":[{"name":"MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University","place":["Shanghai, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,9,8]]},"reference":[{"key":"pone.0348533.ref001","doi-asserted-by":"crossref","unstructured":"Cao Y, Bin J, Hamari J, Blasch E, Liu Z. Multimodal object detection by channel switching and spatial attention. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 2023. p. 403\u2013411.","DOI":"10.1109\/CVPRW59228.2023.00046"},{"key":"pone.0348533.ref002","article-title":"Weakly Aligned Feature Fusion for Multimodal Object Detection","author":"L Zhang","year":"2021","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"pone.0348533.ref003","doi-asserted-by":"crossref","unstructured":"Sindagi VA, Zhou Y, Tuzel O. MVX-Net: Multimodal VoxelNet for 3D Object Detection. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE; 2019. p. 7276\u201382.","DOI":"10.1109\/ICRA.2019.8794195"},{"issue":"6","key":"pone.0348533.ref004","doi-asserted-by":"crossref","first-page":"133","DOI":"10.3390\/computation13060133","article-title":"Advancing Object Detection in Transportation with Multimodal Large Language Models (MLLMs): A Comprehensive Review and Empirical Testing","volume":"13","author":"HI Ashqar","year":"2025","journal-title":"Computation"},{"issue":"7","key":"pone.0348533.ref005","doi-asserted-by":"crossref","first-page":"071017","DOI":"10.1115\/1.4043222","article-title":"Multimodal Fusion Object Detection System for Autonomous Vehicles","volume":"141","author":"M Person","year":"2019","journal-title":"J Dyn Syst Measur Control"},{"issue":"11","key":"pone.0348533.ref006","doi-asserted-by":"crossref","first-page":"3980","DOI":"10.1109\/TCYB.2016.2593940","article-title":"On-Board Object Detection: Multicue, Multimodal, and Multiview Random Forest of Local Experts","volume":"47","author":"A Gonzalez","year":"2017","journal-title":"IEEE Trans Cybern"},{"key":"pone.0348533.ref007","doi-asserted-by":"crossref","unstructured":"Sapkota R, Karkee M. Object detection with multimodal large vision-language models: An in-depth review. 2025. Available at: https:\/\/ssrn.com\/abstract=5233953","DOI":"10.2139\/ssrn.5233953"},{"key":"pone.0348533.ref008","article-title":"A transformer-based multimodal object detection system for real-world applications","author":"S Ikram","year":"2025","journal-title":"IEEE Access"},{"key":"pone.0348533.ref009","doi-asserted-by":"crossref","first-page":"125826","DOI":"10.1016\/j.eswa.2024.125826","article-title":"Differential multimodal fusion algorithm for remote sensing object detection through multi-branch feature extraction","volume":"265","author":"W Zhao","year":"2025","journal-title":"Exp Syst Appl"},{"key":"pone.0348533.ref010","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2025.3530085","article-title":"Dual-Dynamic Cross-Modal Interaction Network for Multimodal Remote Sensing Object Detection","volume":"63","author":"W Bao","year":"2025","journal-title":"IEEE Trans Geosci Remote Sens"},{"key":"pone.0348533.ref011","first-page":"1","article-title":"Convolutional Neural Networks for Multimodal Remote Sensing Data Classification","volume":"60","author":"X Wu","year":"2021","journal-title":"IEEE Trans Geosci Remote Sens"},{"key":"pone.0348533.ref012","doi-asserted-by":"crossref","unstructured":"Chen YT, Shi J, Ye Z, Mertz C, Ramanan D, Kong S. Multimodal object detection via probabilistic ensembling. In: European Conference on Computer Vision. Springer; 2022. p. 139\u201358.","DOI":"10.1007\/978-3-031-20077-9_9"},{"key":"pone.0348533.ref013","doi-asserted-by":"crossref","first-page":"1497","DOI":"10.1109\/JSTARS.2020.3041316","article-title":"YOLOrs: Object Detection in Multimodal Remote Sensing Imagery","volume":"14","author":"M Sharma","year":"2020","journal-title":"IEEE J Sel Top Appl Earth Observations Remote Sens"},{"key":"pone.0348533.ref014","first-page":"1","article-title":"SuperYOLO: Super Resolution Assisted Object Detection in Multimodal Remote Sensing Imagery","volume":"61","author":"J Zhang","year":"2023","journal-title":"IEEE Trans Geosci Remote Sens"},{"key":"pone.0348533.ref015","doi-asserted-by":"crossref","first-page":"113856","DOI":"10.1016\/j.rse.2023.113856","article-title":"Cross-city matters: A multimodal remote sensing benchmark dataset for cross-city semantic segmentation using high-resolution domain adaptation networks","volume":"299","author":"D Hong","year":"2023","journal-title":"Remote Sens Environ"},{"key":"pone.0348533.ref016","doi-asserted-by":"crossref","unstructured":"Xu S, Zhou D, Fang J, Yin J, Bin Z, Zhang L. FusionPainting: Multimodal Fusion with Adaptive Attention for 3D Object Detection. In: 2021 IEEE International Intelligent Transportation Systems Conference (ITSC). IEEE; 2021. p. 3047\u201354.","DOI":"10.1109\/ITSC48978.2021.9564951"},{"key":"pone.0348533.ref017","doi-asserted-by":"crossref","first-page":"108450","DOI":"10.1016\/j.neunet.2025.108450","article-title":"CMMDL: Cross-modal multi-domain learning method for image fusion","volume":"196","author":"D Yuan","year":"2025","journal-title":"Neural Networks"},{"key":"pone.0348533.ref018","doi-asserted-by":"crossref","unstructured":"Wu H, Wen C, Shi S, Li X, Wang C. Virtual Sparse Convolution for Multimodal 3D Object Detection. In: 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2023. p. 21653\u201362.","DOI":"10.1109\/CVPR52729.2023.02074"},{"key":"pone.0348533.ref019","doi-asserted-by":"crossref","unstructured":"Valverde FR, Valeria Hurtado J, Valada A. There is More than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking with Sound by Distilling Multimodal Knowledge. In: 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021. 11607\u201316.","DOI":"10.1109\/CVPR46437.2021.01144"},{"key":"pone.0348533.ref020","doi-asserted-by":"crossref","first-page":"4119","DOI":"10.1109\/LSP.2025.3624079","article-title":"Multi-Scale Adaptive Cascaded Tracking for Vision-Language Integration","volume":"32","author":"G Guo","year":"2025","journal-title":"IEEE Signal Process Lett"},{"key":"pone.0348533.ref021","first-page":"1","article-title":"A Comprehensive Review of RGBT Tracking","volume":"73","author":"H Zhang","year":"2024","journal-title":"IEEE Trans Instrum Meas"},{"issue":"2","key":"pone.0348533.ref022","doi-asserted-by":"crossref","first-page":"825","DOI":"10.1007\/s11263-024-02214-4","article-title":"Contextual Object Detection with Multimodal Large Language Models","volume":"133","author":"Y Zang","year":"2025","journal-title":"Int J Comput Vis"},{"key":"pone.0348533.ref023","doi-asserted-by":"crossref","unstructured":"Wu W, Li C, Wang X, Luo B, Liu Q. Large Language Model Guided Progressive Feature Alignment for Multimodal UAV Object Detection. arXiv preprint arXiv:250306948. 2025.","DOI":"10.1109\/TIP.2026.3702419"},{"issue":"2","key":"pone.0348533.ref024","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1109\/MGRS.2022.3145502","article-title":"Single-Frame Infrared Small-Target Detection: A survey","volume":"10","author":"M Zhao","year":"2022","journal-title":"IEEE Geosci Remote Sens Mag"},{"key":"pone.0348533.ref025","first-page":"1","article-title":"RISTDnet: Robust Infrared Small Target Detection Network","volume":"19","author":"Q Hou","year":"2022","journal-title":"IEEE Geosci Remote Sens Lett"},{"issue":"4","key":"pone.0348533.ref026","doi-asserted-by":"crossref","first-page":"9686","DOI":"10.1109\/TAES.2025.3558181","article-title":"OIPF: An Orthogonal Inputs Perception Fusion Framework for Infrared Small Target Detection","volume":"61","author":"Q Ma","year":"2025","journal-title":"IEEE Trans Aerosp Electron Syst"},{"key":"pone.0348533.ref027","doi-asserted-by":"crossref","unstructured":"Liu Q, Liu R, Zheng B, Wang H, FU Y. Infrared Small Target Detection with Scale and Location Sensitivity. In: 2024 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2024. 17490\u20139.","DOI":"10.1109\/CVPR52733.2024.01656"},{"issue":"7","key":"pone.0348533.ref028","doi-asserted-by":"crossref","first-page":"7717","DOI":"10.1109\/TITS.2022.3193909","article-title":"An Improving Faster-RCNN With Multi-Attention ResNet for Small Target Detection in Intelligent Autonomous Transport With 6G","volume":"24","author":"L Yang","year":"2022","journal-title":"IEEE Trans Intell Transport Syst"},{"issue":"13","key":"pone.0348533.ref029","doi-asserted-by":"crossref","first-page":"6823","DOI":"10.3390\/app12136823","article-title":"Research on Tiny Target Detection Technology of Fabric Defects Based on Improved YOLO","volume":"12","author":"X Yue","year":"2022","journal-title":"Appl Sci"},{"issue":"10","key":"pone.0348533.ref030","article-title":"Application of local fully Convolutional Neural Network combined with YOLO v5 algorithm in small target detection of remote sensing image","volume":"16","author":"W Wu","year":"2021","journal-title":"PLoS One"},{"key":"pone.0348533.ref031","doi-asserted-by":"crossref","first-page":"115595","DOI":"10.1016\/j.measurement.2024.115595","article-title":"DCANet: Dense Convolutional Attention Network for infrared small target detection","volume":"240","author":"D Zang","year":"2025","journal-title":"Measurement"},{"key":"pone.0348533.ref032","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1016\/j.aej.2024.10.114","article-title":"Intelligent monitoring of small target detection using YOLOv8","volume":"112","author":"L Sun","year":"2025","journal-title":"Alex Eng J"},{"issue":"4","key":"pone.0348533.ref033","first-page":"122","article-title":"Survey of Video Based Small Target Detection","volume":"9","author":"Y Liu","year":"2021","journal-title":"JOIG"},{"key":"pone.0348533.ref034","doi-asserted-by":"crossref","unstructured":"Zhao Y, Lv W, Xu S, Wei J, Wang G, Dang Q, et al. Detrs beat yolos on real-time object detection. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. 2024. p. 16965\u201374.","DOI":"10.1109\/CVPR52733.2024.01605"},{"key":"pone.0348533.ref035","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. p. 770\u20138.","DOI":"10.1109\/CVPR.2016.90"},{"key":"pone.0348533.ref036","doi-asserted-by":"crossref","unstructured":"Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE; 2009. p. 248\u201355.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"pone.0348533.ref037","doi-asserted-by":"crossref","unstructured":"Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2014. p. 580\u20137.","DOI":"10.1109\/CVPR.2014.81"},{"key":"pone.0348533.ref038","doi-asserted-by":"crossref","unstructured":"Sohan M, Sai Ram T, Rami Reddy CV. A review on yolov8 and its advancements. In: International Conference on Data Intelligence and Cognitive Informatics. Springer; 2024. p. 529\u201345.","DOI":"10.1007\/978-981-99-7962-2_39"},{"key":"pone.0348533.ref039","doi-asserted-by":"crossref","first-page":"107984","DOI":"10.52202\/079017-3429","article-title":"Yolov10: Real-time end-to-end object detection","volume":"37","author":"A Wang","year":"2024","journal-title":"Adv Neural Inform Process Syst"},{"key":"pone.0348533.ref040","unstructured":"Khanam R, Hussain M. Yolov11: An overview of the key architectural enhancements. arXiv preprint arXiv:241017725. 2024."},{"key":"pone.0348533.ref041","unstructured":"Qingyun F, Dapeng H, Zhaokui W. Cross-modality fusion transformer for multispectral object detection. arXiv preprint arXiv:211100273. 2021."},{"issue":"2","key":"pone.0348533.ref042","doi-asserted-by":"crossref","first-page":"443","DOI":"10.3390\/electronics13020443","article-title":"Multispectral Object Detection Based on Multilevel Feature Fusion and Dual Feature Modulation","volume":"13","author":"J Sun","year":"2024","journal-title":"Electronics"},{"key":"pone.0348533.ref043","doi-asserted-by":"crossref","unstructured":"Song X, Zhang J, Liang D, Zhang Z, Cai J, Hu L. Adaptive Dual Cross-Attention Network for Multispectral Object Detection. In: International Conference on Intelligent Computing. Springer; 2025. p. 139\u201350.","DOI":"10.1007\/978-981-96-9961-2_12"},{"key":"pone.0348533.ref044","doi-asserted-by":"crossref","unstructured":"Cai W, Liu Q, Wang Y. HIPTrack: Visual Tracking with Historical Prompts. In: 2024 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2024. p. 19258\u201367.","DOI":"10.1109\/CVPR52733.2024.01822"},{"key":"pone.0348533.ref045","first-page":"1","article-title":"SHRS-YOLO: Small Target Detection in Hazy Remote Sensing Images","volume":"64","author":"R Yang","year":"2026","journal-title":"IEEE Trans Geosci Remote Sens"},{"key":"pone.0348533.ref046","doi-asserted-by":"crossref","first-page":"111958","DOI":"10.1016\/j.patcog.2025.111958","article-title":"DFINet: Dynamic feedback iterative network for infrared small target detection","volume":"169","author":"J Wu","year":"2026","journal-title":"Pattern Recognit"},{"issue":"4","key":"pone.0348533.ref047","doi-asserted-by":"crossref","first-page":"2947","DOI":"10.1609\/aaai.v38i4.28076","article-title":"Object-Aware Domain Generalization for Object Detection","volume":"38","author":"W Lee","year":"2024","journal-title":"AAAI"},{"key":"pone.0348533.ref048","doi-asserted-by":"crossref","unstructured":"Wu A, Deng C. Single-domain generalized object detection in urban scene via cyclic-disentangled self-distillation. In: 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2022. p. 837\u2013846.","DOI":"10.1109\/CVPR52688.2022.00092"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0348533","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T17:49:21Z","timestamp":1788889761000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0348533"}},"subtitle":[],"editor":[{"given":"Joanna","family":"Tindall","sequence":"first","affiliation":[{"name":"PLOS: Public Library of Science","place":["UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,9,8]]},"references-count":48,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9,8]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0348533","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,8]]}}}