{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T17:08:03Z","timestamp":1792861683060,"version":"4.5.0"},"reference-count":35,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T00:00:00Z","timestamp":1781568000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012216","name":"Jiangsu University of Science and Technology","doi-asserted-by":"publisher","award":["BY20240031"],"award-info":[{"award-number":["BY20240031"]}],"id":[{"id":"10.13039\/501100012216","id-type":"DOI","asserted-by":"publisher"}]},{"award":["BY20240031"],"award-info":[{"award-number":["BY20240031"]}],"id":[{"id":"https:\/\/ror.org\/00tyjp878","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Effective cross-modal feature alignment and interaction are central challenges in multispectral object detection. Although global cross-attention provides strong long-range modeling ability, its quadratic complexity with respect to feature size limits deployment on resource-constrained platforms. We therefore propose Progressive Pixel-Neighborhood Deformable Cross-Attention for multispectral feature fusion, termed PNAFusion. The proposed framework is motivated by two observations: weak misalignment between visible and thermal images is usually concentrated around local neighborhoods, and semantic correspondence across modalities often follows non-linear spatial mappings that fixed receptive fields cannot model well. To address these issues, PNAFusion incorporates local spatial priors into its architectural design to concentrate feature interaction and alignment on the most relevant neighborhoods. Specifically, a Pixel-Neighborhood Cross-Attention (PNCA) module is introduced to avoid redundant global feature matching and suppress background noise. Meanwhile, an Adaptive Deformable Alignment (ADA) module captures non-linear spatial correspondences through learned pixel-wise offsets. These components are further integrated through an iterative feedback mechanism to progressively refine cross-modal feature alignment. Experiments on FLIR, M3FD, and DroneVehicle show that PNAFusion achieves 84.2, 90.5, and 85.5 mAP@0.5, respectively, under the YOLOv5 detector, and further reaches 86.8 mAP@0.5 on FLIR and 90.8 mAP@0.5 on M3FD when transferred to Co-DETR. Efficiency analysis indicates that PNAFusion reduces allocated GPU memory by 33.0% compared with ICAFusion and reduces theoretical FLOPs from 194.8 G to 156.4 G, although the deformable sampling and iterative refinement introduce additional latency. These results demonstrate that PNAFusion provides a practical accuracy\u2013memory trade-off for weakly aligned multispectral object detection.<\/jats:p>","DOI":"10.3390\/s26123825","type":"journal-article","created":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T01:58:38Z","timestamp":1781661518000},"page":"3825","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Progressive Pixel-Neighborhood Deformable Cross-Attention for Multispectral Object Detection"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-3371-6106","authenticated-orcid":false,"given":"Tian","family":"Qiu","sequence":"first","affiliation":[{"name":"School of Electrical and Information Engineering, Jiangsu University, Zhenjiang 212013, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4356-1831","authenticated-orcid":false,"given":"Jifeng","family":"Shen","sequence":"additional","affiliation":[{"name":"School of Electrical and Information Engineering, Jiangsu University, Zhenjiang 212013, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9093-0102","authenticated-orcid":false,"given":"Xin","family":"Zuo","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Jiangsu University of Science and Technology, Zhenjiang 212003, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.inffus.2018.02.004","article-title":"Infrared and visible image fusion methods and applications: A survey","volume":"45","author":"Ma","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ma, W., Wang, K., Li, J., Yang, S.X., Li, J., Song, L., and Li, Q. (2023). Infrared and visible image fusion technology and application: A review. Sensors, 23.","DOI":"10.3390\/s23020599"},{"key":"ref_3","first-page":"79","article-title":"Combining UAV-based plant height from crop surface models, visible, and near infrared vegetation indices for biomass monitoring in barley","volume":"39","author":"Bendig","year":"2015","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Fang, Q., Han, D., and Wang, Z. (2022). Cross-modality fusion transformer for multispectral object detection. arXiv.","DOI":"10.2139\/ssrn.4227745"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"5518615","DOI":"10.1109\/TGRS.2021.3130716","article-title":"SpectralFormer: Rethinking hyperspectral image classification with transformers","volume":"60","author":"Hong","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"109913","DOI":"10.1016\/j.patcog.2023.109913","article-title":"ICAFusion: Iterative cross-attention guided feature fusion for multispectral object detection","volume":"145","author":"Shen","year":"2024","journal-title":"Pattern Recognit."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Guo, R., Guo, X., Sun, X., Zhou, P., Sun, B., and Su, S. (2024). Background-aware cross-attention multiscale fusion for multispectral object detection. Remote Sens., 16.","DOI":"10.3390\/rs16214034"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_10","unstructured":"Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. (2020). YOLOv4: Optimal speed and accuracy of object detection. arXiv."},{"key":"ref_11","unstructured":"Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Ke, Z., Li, Q., Cheng, M., and Nie, W. (2022). YOLOv6: A single-stage object detection framework for industrial applications. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, A., Chen, H., Liu, L., Chen, K., Lin, Z., Han, J., and Ding, G. (2024). YOLOv10: Real-Time End-to-End Object Detection. arXiv.","DOI":"10.52202\/079017-3429"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhang, H., Fromont, E., Lef\u00e8vre, S., and Avignon, B. (2021). Guided attentive feature fusion for multispectral pedestrian detection. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, IEEE.","DOI":"10.1109\/WACV48630.2021.00012"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"5010014","DOI":"10.1109\/TGRS.2025.3636590","article-title":"Illumination-aware multimodal hierarchical fusion network for RGB-infrared object detection","volume":"63","author":"Lu","year":"2025","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zuo, X., Qu, C., Zhan, H., Shen, J., and Yang, W. (2025). SFFR: Spatial-Frequency Feature Reconstruction for Multispectral Aerial Object Detection. arXiv.","DOI":"10.1109\/TGRS.2025.3631708"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Shen, J., Zhan, H., Dong, S., Zuo, X., Yang, W., and Ling, H. (2025). Multispectral State-Space Feature Fusion: Bridging Shared and Cross-Parametric Interactions for Object Detection. arXiv.","DOI":"10.1016\/j.inffus.2025.103895"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"7392","DOI":"10.1109\/TMM.2025.3599020","article-title":"Fusion-Mamba for cross-modality object detection","volume":"27","author":"Dong","year":"2025","journal-title":"IEEE Trans. Multimed."},{"key":"ref_18","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 3\u20137). An image is worth 16\u00d716 words: Transformers for image recognition at scale. Proceedings of the International Conference on Learning Representations (ICLR), Virtual."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wang, W., Xie, E., Li, X., Fan, D.P., Song, K., Liang, D., Lu, T., Luo, P., and Shao, L. (2021, January 10\u201317). Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00061"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 10\u201317). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Hassani, A., Walton, S., Li, J., Li, S., and Shi, H. (2023). Neighborhood attention transformer. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.","DOI":"10.1109\/CVPR52729.2023.00599"},{"key":"ref_22","unstructured":"Zhu, X., Su, W., Lu, L., Li, B., Wang, X., and Dai, J. (2020). Deformable DETR: Deformable transformers for end-to-end object detection. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2447","DOI":"10.1109\/TIP.2024.3378473","article-title":"Learning temporal distribution and spatial correlation toward universal moving object segmentation","volume":"33","author":"Dong","year":"2024","journal-title":"IEEE Trans. Image Process."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Cao, Y., Bin, J., Hamari, J., Blasch, E., and Liu, Z. (2023). Multimodal object detection by channel switching and spatial attention. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPRW59228.2023.00046"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1109\/TCSVT.2024.3454631","article-title":"Multidimensional fusion network for multispectral object detection","volume":"35","author":"Yang","year":"2024","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"16300","DOI":"10.1109\/TITS.2024.3412417","article-title":"Rethinking self-attention for multispectral object detection","volume":"25","author":"Hu","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Xiao, Y., Meng, F., Wu, Q., Xu, L., He, M., and Li, H. (2024). Gm-detr: Generalized muiltispectral detection transformer with efficient fusion encoder for visible-infrared detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPRW63382.2024.00563"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"13276","DOI":"10.1109\/TNNLS.2024.3443455","article-title":"TFDet: Target-aware fusion for RGB-T pedestrian detection","volume":"36","author":"Zhang","year":"2024","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Guo, J., Gao, C., Liu, F., Meng, D., and Gao, X. (2024). DAMSDet: Dynamic adaptive multispectral detection transformer with competitive query selection and adaptive feature fusion. Proceedings of the European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-031-73383-3_27"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2121","DOI":"10.1109\/JAS.2022.106082","article-title":"SuperFusion: A versatile image registration and fusion network with semantic awareness","volume":"9","author":"Tang","year":"2022","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5401013","DOI":"10.1109\/TGRS.2025.3530085","article-title":"Dual-dynamic cross-modal interaction network for multimodal remote sensing object detection","volume":"63","author":"Bao","year":"2025","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3284","DOI":"10.1109\/TITS.2024.3522086","article-title":"CCLDet: A Cross-Modality and Cross-Domain Low-Light Detector","volume":"26","author":"Shang","year":"2025","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhou, M., Li, T., Qiao, C., Xie, D., Wang, G., Ruan, N., Mei, L., and Yang, Y. (2024). DMM: Disparity-guided Multispectral Mamba for Oriented Object Detection in Remote Sensing. arXiv.","DOI":"10.1109\/TGRS.2025.3578309"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"103414","DOI":"10.1016\/j.inffus.2025.103414","article-title":"COMO: Cross-mamba interaction and offset-guided fusion for multimodal object detection","volume":"125","author":"Liu","year":"2026","journal-title":"Inf. Fusion"},{"key":"ref_35","first-page":"5404215","article-title":"Reflectance-Guided Progressive Feature Alignment Network for All-Day UAV Object Detection","volume":"63","author":"Zhao","year":"2025","journal-title":"IEEE Trans. Geosci. Remote Sens."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/12\/3825\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T04:19:10Z","timestamp":1782101950000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/12\/3825"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,16]]},"references-count":35,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2026,6]]}},"alternative-id":["s26123825"],"URL":"https:\/\/doi.org\/10.3390\/s26123825","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,16]]}}}