{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T02:21:12Z","timestamp":1771467672963,"version":"3.50.1"},"reference-count":49,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T00:00:00Z","timestamp":1696982400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>In light of advancing socio-economic development and urban infrastructure, urban traffic congestion and accidents have become pressing issues. High-resolution remote sensing images are crucial for supporting urban geographic information systems (GIS), road planning, and vehicle navigation. Additionally, the emergence of robotics presents new possibilities for traffic management and road safety. This study introduces an innovative approach that combines attention mechanisms and robotic multimodal information fusion for retrieving traffic scenes from remote sensing images. Attention mechanisms focus on specific road and traffic features, reducing computation and enhancing detail capture. Graph neural algorithms improve scene retrieval accuracy. To achieve efficient traffic scene retrieval, a robot equipped with advanced sensing technology autonomously navigates urban environments, capturing high-accuracy, wide-coverage images. This facilitates comprehensive traffic databases and real-time traffic information retrieval for precise traffic management. Extensive experiments on large-scale remote sensing datasets demonstrate the feasibility and effectiveness of this approach. The integration of attention mechanisms, graph neural algorithms, and robotic multimodal information fusion enhances traffic scene retrieval, promising improved information extraction accuracy for more effective traffic management, road safety, and intelligent transportation systems. In conclusion, this interdisciplinary approach, combining attention mechanisms, graph neural algorithms, and robotic technology, represents significant progress in traffic scene retrieval from remote sensing images, with potential applications in traffic management, road safety, and urban planning.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1267231","type":"journal-article","created":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T07:53:21Z","timestamp":1697010801000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Remote sensing traffic scene retrieval based on learning control algorithm for robot multimodal sensing information fusion and human-machine interaction and collaboration"],"prefix":"10.3389","volume":"17","author":[{"given":"Huiling","family":"Peng","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nianfeng","family":"Shi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guoqiang","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,10,11]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1016\/j.isprsjprs.2017.11.011","article-title":"Beyond RGB: very high resolution urban remote sensing with multimodal deep networks","volume":"140","author":"Audebert","year":"2018","journal-title":"ISPRS J. Photogram. Remote Sens."},{"key":"B2","first-page":"8102","article-title":"\u201cBLOCK: bilinear superdiagonal fusion for visual question answering and visual relationship detection,\u201d","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Ben-younes","year":"2019"},{"key":"B3","doi-asserted-by":"publisher","first-page":"2615","DOI":"10.1109\/JSTARS.2018.2849363","article-title":"Building footprint extraction from VHR remote sensing images combined with normalized dsms using fused fully convolutional networks","volume":"11","author":"Bittner","year":"2018","journal-title":"IEEE J. Select. Top. Appl. Earth Observ. Remote Sens."},{"key":"B4","doi-asserted-by":"publisher","first-page":"118380","DOI":"10.1016\/j.eswa.2022.118380","article-title":"Semantic segmentation of clouds in satellite images based on U-Net++ architecture and attention mechanism","volume":"209","author":"Buttar","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"B5","doi-asserted-by":"publisher","first-page":"5817","DOI":"10.3390\/rs14225817","article-title":"On the co-selection of vision transformer features and images for very high-resolution image scene classification","volume":"14","author":"Chaib","year":"2022","journal-title":"Remote Sens."},{"key":"B6","doi-asserted-by":"publisher","first-page":"893","DOI":"10.1016\/j.rse.2009.01.007","article-title":"Summary of current radiometric calibration coefficients for Landsat MSS, TM, ETM+, and EO-1 ALI sensors","volume":"113","author":"Chander","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"B7","doi-asserted-by":"publisher","first-page":"1538","DOI":"10.1109\/JSTARS.2012.2199085","article-title":"Semi-automated road detection from high resolution satellite images by directional morphological enhancement and segmentation techniques","volume":"5","author":"Chaudhuri","year":"2012","journal-title":"IEEE J. Select. Top. Appl. Earth Observ. Remote Sens."},{"key":"B8","first-page":"485","article-title":"\u201cGated residual recurrent graph neural networks for traffic prediction,\u201d","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Chen","year":"2019"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1023","DOI":"10.3390\/atmos14061023","article-title":"Sea ice extent prediction with machine learning methods and subregional analysis in the Arctic","volume":"14","author":"Chen","year":"2023","journal-title":"Atmosphere"},{"key":"B10","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1145\/3511808.3557429","article-title":"\u201cRelax: reinforcement learning agent explainer for arbitrary predictive models,\u201d","volume-title":"Proceedings of the 31st ACM International Conference on Information & Knowledge Management","author":"Chen","year":"2022"},{"key":"B11","doi-asserted-by":"publisher","first-page":"4883","DOI":"10.1109\/TITS.2019.2950416","article-title":"Traffic graph convolutional recurrent neural network: a deep learning framework for network-scale traffic learning and forecasting","volume":"21","author":"Cui","year":"2019","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"B12","doi-asserted-by":"publisher","first-page":"3192","DOI":"10.3390\/rs13163192","article-title":"Ship object detection of remote sensing image based on visual attention","volume":"13","author":"Dong","year":"2021","journal-title":"Remote Sens."},{"key":"B13","doi-asserted-by":"publisher","first-page":"2200213","DOI":"10.1002\/aisy.202200213","article-title":"Multimodal sensors and ML-based data fusion for advanced robots","volume":"4","author":"Duan","year":"2022","journal-title":"Adv. Intell. Syst."},{"key":"B14","article-title":"\u201cHuman computer confluence,\u201d","volume-title":"Human Computer Confluence","author":"Gaggioli","year":"2016"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1142","DOI":"10.1002\/celc.201600153","article-title":"Polyaniline nanorods grown on hollow carbon fibers as high-performance supercapacitor electrodes","volume":"3","author":"Gao","year":"2016","journal-title":"ChemElectroChem"},{"key":"B16","doi-asserted-by":"publisher","first-page":"2965","DOI":"10.3390\/rs13152965","article-title":"Effect of attention mechanism in deep learning-based remote sensing image processing: a systematic literature review","volume":"13","author":"Ghaffarian","year":"2021","journal-title":"Remote Sens."},{"key":"B17","doi-asserted-by":"publisher","first-page":"778","DOI":"10.3390\/coatings13040778","article-title":"Bacterial detection and differentiation of Staphylococcus aureus and Escherichia coli utilizing long-period fiber gratings functionalized with nanoporous coated structures","volume":"13","author":"He","year":"2023","journal-title":"Coatings"},{"key":"B18","doi-asserted-by":"publisher","first-page":"2518","DOI":"10.1109\/TIP.2018.2886997","article-title":"CONNNet: a long-range relation-aware pixel-connectivity network for salient segmentation","volume":"28","author":"Kampffmeyer","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"B19","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1016\/j.jaridenv.2006.03.029","article-title":"Evapo-transpiration estimation over the river basin of the loess plateau of China based on remote sensing","volume":"68","author":"Kimura","year":"2007","journal-title":"J. Arid Environ."},{"key":"B20","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1002\/rob.21535","article-title":"Robust data fusion of multimodal sensory information for mobile robots","volume":"32","author":"Kubelka","year":"2015","journal-title":"J. Field Robot."},{"key":"B21","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1080\/01431161.2022.2161852","article-title":"A double-layer feature fusion convolutional neural network for infrared small target detection","volume":"44","author":"Li","year":"2023","journal-title":"Int. J. Remote Sens."},{"key":"B22","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s12369-019-00535-w","article-title":"Multimodal information fusion for automatic aesthetics evaluation of robotic dance poses","volume":"12","author":"Li","year":"2020","journal-title":"Int. J. Soc. Robot."},{"key":"B23","first-page":"4189","article-title":"\u201cSpatial-temporal fusion graph neural networks for traffic flow forecasting,\u201d","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Li","year":"2021"},{"key":"B24","doi-asserted-by":"publisher","first-page":"11","DOI":"10.3390\/rs11010011","article-title":"Large-scale oil palm tree detection from high-resolution satellite images using two-stage convolutional neural networks","volume":"11","author":"Li","year":"2018","journal-title":"Remote Sens."},{"key":"B25","first-page":"848","article-title":"\u201cDeep variation-structured reinforcement learning for visual relationship and attribute detection,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Liang","year":"2017"},{"key":"B26","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/j.inffus.2019.11.001","article-title":"Multi-sensor fusion for body sensor network in medical human\u2013robot interaction scenario","volume":"57","author":"Lin","year":"2020","journal-title":"Inform. Fusion"},{"key":"B27","doi-asserted-by":"publisher","first-page":"13132","DOI":"10.1002\/ange.201706647","article-title":"Observation and identification of an atomic oxygen structure on catalytic gold nanoparticles","volume":"129","author":"Liu","year":"2017","journal-title":"Angew. Chem."},{"key":"B28","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1109\/SACI.2015.7208262","article-title":"\u201cMultimodal information fusion for human robot interaction,\u201d","volume-title":"2015 IEEE 10th Jubilee International Symposium on Applied Computational Intelligence and Informatics","author":"Luo","year":"2015"},{"key":"B29","doi-asserted-by":"publisher","first-page":"7092","DOI":"10.1109\/TGRS.2017.2740362","article-title":"High-resolution aerial image labeling with convolutional neural networks","volume":"55","author":"Maggiori","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"B30","doi-asserted-by":"publisher","first-page":"1790","DOI":"10.1109\/LGRS.2015.2426112","article-title":"Semiautomatic object-space road extraction combining a stereoscopic image pair and a tin-based DTM","volume":"12","author":"Martins","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2202.07732","article-title":"Multi-modal data fusion in enhancing human-machine interaction for robotic applications: a survey","author":"Mohd","year":"2022","journal-title":"arXiv preprint arXiv:2202.07732"},{"key":"B32","doi-asserted-by":"publisher","first-page":"684","DOI":"10.3390\/rs11060684","article-title":"A novel object-based deep learning framework for semantic segmentation of very high-resolution remote sensing data: comparison with convolutional and fully convolutional networks","volume":"11","author":"Papadomanolaki","year":"2019","journal-title":"Remote Sens."},{"key":"B33","first-page":"1928","article-title":"\u201cPhrase localization and visual relationship detection with comprehensive image-language cues,\u201d","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Plummer","year":"2017"},{"key":"B34","doi-asserted-by":"publisher","first-page":"1432","DOI":"10.3390\/rs12091432","article-title":"Small-object detection in remote sensing images with end-to-end edge-enhanced GAN and object detector network","volume":"12","author":"Rabbi","year":"2020","journal-title":"Remote Sens."},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1049\/itr2.12257","article-title":"Deep learning serves traffic safety analysis: a forward-looking review","author":"Razi","year":"2022","journal-title":"IET Intell. Transport Syst."},{"key":"B36","doi-asserted-by":"publisher","first-page":"1100","DOI":"10.1109\/TGRS.2018.2864716","article-title":"Buildings detection in VHR SAR images using fully convolution neural networks","volume":"57","author":"Shahzad","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"B37","doi-asserted-by":"publisher","first-page":"4985","DOI":"10.1021\/acsnano.2c12592","article-title":"Soft robotic perception system with ultrasonic auto-positioning and multimodal sensory intelligence","volume":"17","author":"Shi","year":"2023","journal-title":"ACS Nano"},{"key":"B38","unstructured":"TangZ.\n          Molecular fundamentals of upgrading biomass-derived feedstocks over platinum-molybdenum catalysts2022"},{"key":"B39","doi-asserted-by":"publisher","first-page":"11603","DOI":"10.1021\/acs.langmuir.1c01941","article-title":"Atomic, molecular and hybrid oxygen structures on silver","volume":"37","author":"Tang","year":"2021","journal-title":"Langmuir"},{"key":"B40","doi-asserted-by":"publisher","first-page":"976","DOI":"10.1177\/0962280220983381","article-title":"Online control of the familywise error rate","volume":"30","author":"Tian","year":"2021","journal-title":"Stat. Methods Med. Res."},{"key":"B41","first-page":"1","article-title":"\u201cRSG-search: semantic traffic scene retrieval using graph-based scene representation,\u201d","volume-title":"2023 IEEE Intelligent Vehicles Symposium (IV)","author":"Tian","year":"2023"},{"key":"B42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2366145.2366206","article-title":"Lightweight binocular facial performance capture under uncontrolled lighting","volume":"31","author":"Valgaerts","year":"2012","journal-title":"ACM Trans. Graph."},{"key":"B43","doi-asserted-by":"crossref","first-page":"1082","DOI":"10.1145\/3366423.3380186","article-title":"\u201cTraffic flow prediction via spatial temporal graph neural network,\u201d","volume-title":"Proceedings of the Web Conference 2020","author":"Wang","year":"2020"},{"key":"B44","doi-asserted-by":"publisher","first-page":"101446","DOI":"10.1016\/j.ecoser.2022.101446","article-title":"On the comparative use of social media data and survey data in prioritizing ecosystem services for cost-effective governance","volume":"56","author":"Wang","year":"2022","journal-title":"Ecosyst. Serv."},{"key":"B45","article-title":"Extended agriculture-vision: an extension of a large aerial image dataset for agricultural pattern analysis","author":"Wu","year":"2023","journal-title":"arXiv preprint arXiv:2303.02460"},{"key":"B46","first-page":"1712","article-title":"\u201cOptimizing nitrogen management with deep reinforcement learning and crop simulations,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Wu","year":"2022"},{"key":"B47","doi-asserted-by":"publisher","first-page":"75742","DOI":"10.1109\/ACCESS.2022.3192026","article-title":"Fine segmentation on faces with masks based on a multistep iterative segmentation algorithm","volume":"10","author":"Zhang","year":"2022","journal-title":"IEEE Access"},{"key":"B48","doi-asserted-by":"publisher","first-page":"28821","DOI":"10.1109\/ACCESS.2022.3157823","article-title":"Fast 3D visualization of massive geological data based on clustering index fusion","volume":"10","author":"Zhang","year":"2022","journal-title":"IEEE Access"},{"key":"B49","doi-asserted-by":"publisher","first-page":"4036","DOI":"10.1039\/D2CC00283C","article-title":"Spectroscopic observation and structure-insensitivity of hydroxyls on gold","volume":"58","author":"Zheng","year":"2022","journal-title":"Chem. Commun."}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1267231\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T07:53:32Z","timestamp":1697010812000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1267231\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,11]]},"references-count":49,"alternative-id":["10.3389\/fnbot.2023.1267231"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1267231","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,11]]},"article-number":"1267231"}}