{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T21:55:24Z","timestamp":1793915724518,"version":"4.8.0"},"reference-count":118,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T00:00:00Z","timestamp":1780444800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"Ministry of Education of the Republic of Korea","doi-asserted-by":"publisher","award":["RS-2025-25399600"],"award-info":[{"award-number":["RS-2025-25399600"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"award":["RS-2025-25399600"],"award-info":[{"award-number":["RS-2025-25399600"]}],"id":[{"id":"https:\/\/ror.org\/02bffdv67","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Recent Vision\u2013Language\u2013Action (VLA) models have rapidly emerged as general-purpose robotic policies that integrate language understanding, visual perception, and robot control. However, prior studies and surveys have primarily emphasized backbone architectures, action decoders, training recipes, and benchmark performance, whereas relatively limited systematic attention has been given to sensor modality selection, heterogeneous signal alignment and fusion, and their connection to action generation, all of which are critical to the performance and safety of real-world robotic manipulation. This survey addresses this gap by reinterpreting VLA within the framework of a sensor\u2013fusion\u2013action pipeline. This study first presents a systematic taxonomy of major sensor modalities, including RGB, depth, tactile sensing, force\/torque, proprioception and inertial measurement unit, multi-spectral\/thermal, and event-based vision, and compares them in terms of the physical information they provide, their characteristic failure modes, and their deployment constraints. This survey further reviews teleoperation-, human video-, and simulation-based data collection pipelines, together with representative dataset configurations, and analyzes the multi-modal design space from a sensor-centric perspective, including early and late fusion, cross-attention, token-level fusion, adapters, mixture of experts, and multi-rate action representations. In addition, this study identifies a strong bias in existing benchmarks toward RGB-centric inputs and single success-rate metrics and emphasizes the need for a multidimensional evaluation framework incorporating robustness, worst-case performance, safety, latency, and efficiency. By shifting the focus away from a model-centric narrative and explicitly accounting for real-world sensor complexity, this survey seeks to establish a sensor-centered foundation for the next generation of Physical AI.<\/jats:p>","DOI":"10.3390\/s26113541","type":"journal-article","created":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T09:33:14Z","timestamp":1780479194000},"page":"3541","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Sensing the Action: Rethinking Sensor Modalities and Multi-Modal Fusion in Vision\u2013Language\u2013Action Models for Robotic Manipulation"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7284-0768","authenticated-orcid":false,"given":"Byoung Chul","family":"Ko","sequence":"first","affiliation":[{"name":"Department of Computer Engineering, Keimyung University, Daegu 42601, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., and Quillen, D. (2016). Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection. arXiv.","DOI":"10.1007\/978-3-319-50115-4_16"},{"key":"ref_2","first-page":"991","article-title":"BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning","volume":"Volume 164","author":"Eric","year":"2022","journal-title":"Proceedings of the 5th Conference on Robot Learning"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, T., McCarthy, Z., Jow, O., Lee, D., Chen, X., Goldberg, K., and Abbeel, P. (2017). Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation. arXiv.","DOI":"10.1109\/ICRA.2018.8461249"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3300","DOI":"10.1109\/LRA.2018.2852779","article-title":"More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch","volume":"4","author":"Calandra","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Pinto, L., and Gupta, A. (2016, January 16\u201321). Supersizing Self-Supervision: Learning to Grasp from 50K Tries and 700 Robot Hours. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden.","DOI":"10.1109\/ICRA.2016.7487517"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Tirado-Garin, A., and Civera, J. (2025). AnyCalib: On-Manifold Learning for Model-Agnostic Single-View Camera Calibration. Proceedings of the IEEE\/CVF International Conference on Computer Vision, IEEE.","DOI":"10.1109\/ICCV51701.2025.00754"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Yuan, W., Dong, S., and Adelson, E.H. (2017). GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force. Sensors, 17.","DOI":"10.3390\/s17122762"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1146\/annurev-control-060117-104848","article-title":"Toward Robotic Manipulation","volume":"1","author":"Mason","year":"2018","journal-title":"Annu. Rev. Control Robot. Auton. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Anthony, B., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., and Hsu, J. (2022). RT-1: Robotics Transformer for Real-World Control at Scale. arXiv.","DOI":"10.15607\/RSS.2023.XIX.025"},{"key":"ref_10","first-page":"2165","article-title":"RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control","volume":"Volume 229","author":"Brianna","year":"2023","journal-title":"Conference on Robot Learning"},{"key":"ref_11","unstructured":"Kim, M.J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., and Sanketi, P. (2024). OpenVLA: An Open-Source Vision-Language-Action Model. arXiv."},{"key":"ref_12","unstructured":"Reed, S., Zolna, K., Parisotto, E., Colmenarejo, S.G., Novikov, A., Barth-Maron, G., Gimenez, M., Sulsky, Y., Kay, J., and Springenberg, J.T. (2022). A Generalist Agent (Gato). arXiv."},{"key":"ref_13","first-page":"10","article-title":"Diffusion policy: Visuomotor policy learning via action diffusion","volume":"44","author":"Chi","year":"2024","journal-title":"Int. J. Robot. Res."},{"key":"ref_14","unstructured":"Lipman, Y., Chen, R.T.Q., Ben-Hamu, H., Nickel, M., and Le, M. (2023, January 1\u20135). Flow Matching for Generative Modeling. Proceedings of the 11th International Conference on Learning Representations (ICLR 2023), Kigali, Rwanda."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ghosh, D., Walke, H., Pertsch, K., Black, K., Mees, O., Dasari, S., Hejna, J., Kreiman, T., Xu, C., and Luo, J. (2024, January 15\u201319). Octo: An Open-Source Generalist Robot Policy. Proceedings of the Robotics: Science and Systems (RSS), Delft, The Netherlands.","DOI":"10.15607\/RSS.2024.XX.090"},{"key":"ref_16","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_17","unstructured":"O\u2019Neill, A., Rehman, A., Maddukuri, A., Gupta, A., Padalkar, A., Lee, A., Pooley, A., Gupta, A., Mandlekar, A., and Jain, A. (2024, January 13\u201317). Open X-Embodiment: Robotic Learning Datasets and RT-X Models. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan."},{"key":"ref_18","unstructured":"Kevin, B., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., and Ichter, B. (2024). \u03c00: A Vision-Language-Action Flow Model for General Robot Control. arXiv."},{"key":"ref_19","unstructured":"Li, X., Liu, M., Zhang, H., Yu, C., Xu, J., Wu, H., and Kong, T. (2023). Vision-Language Foundation Models as Effective Robot Imitators (RoboFlamingo). arXiv."},{"key":"ref_20","unstructured":"Awadalla, A., Gao, I., Gardner, J., Hessel, J., Hanafy, Y., Zhu, W., Marathe, K., Bitton, Y., Gadre, S., and Sagawa, S. (2023). OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ud Din, M., Akram, W., Saad Saoud, L., Rosell, J., and Hussain, I. (2025). Vision Language Action Models in Robotic Manipulation: A Systematic Review. arXiv.","DOI":"10.1016\/j.inffus.2025.104062"},{"key":"ref_22","unstructured":"Ma, Y., Song, Z., Zhuang, Y., Hao, J., and King, I. (2024). A Survey on Vision-Language-Action Models for Embodied AI. arXiv."},{"key":"ref_23","unstructured":"Zhong, Y., Bai, F., Cai, S., Huang, X., Chen, Z., Zhang, X., Wang, Y., Guo, S., Guan, T., and Lui, K.N. (2025). A Survey on Vision-Language-Action Models: An Action Tokenization Perspective. arXiv."},{"key":"ref_24","unstructured":"Shao, R., Li, W., Zhang, L., Zhang, R., Liu, Z., Chen, R., and Nie, L. (2025). Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey. arXiv."},{"key":"ref_25","unstructured":"Li, H., Chen, Y., Cui, W., Liu, W., Liu, K., Zhou, M., Zhang, Z., and Zhao, D. (2025). Survey of Vision-Language-Action Models for Embodied Manipulation. arXiv."},{"key":"ref_26","unstructured":"Wang, Z., Wang, B., Zhang, H., Du, T., Chen, T., Sun, G., He, Y., Shen, Z., Ye, W., and Li, A. (2026). Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"103652","DOI":"10.1016\/j.inffus.2025.103652","article-title":"Multimodal Fusion and Vision\u2013Language Models: A Survey for Robot Vision","volume":"126","author":"Han","year":"2026","journal-title":"Inf. Fusion"},{"key":"ref_28","first-page":"1097","article-title":"ImageNet classification with deep convolutional neural networks","volume":"Volume 25","author":"Krizhevsky","year":"2012","journal-title":"Advances in Neural Information Processing Systems 25 (NIPS 2012)"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Palazzo, L., Suglia, V., Grieco, S., Buongiorno, D., Pagano, G., Bevilacqua, V., and D\u2019Addio, G. (2025, January 28\u201330). Optimized Deep Learning-Based Pathological Gait Recognition Explored Through Network Analysis of Inertial Data. Proceedings of the 2025 IEEE Medical Measurements & Applications (MeMeA), Chania, Greece.","DOI":"10.1109\/MeMeA65319.2025.11067994"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Buongiorno, D., Prunella, M., Grossi, S., Hussain, S.M., Rennola, A., Longo, N., Di Stefano, G., Bevilacqua, V., and Brunetti, A. (2022). Inline Defective Laser Weld Identification by Processing Thermal Image Sequences with Machine and Deep Learning Techniques. Appl. Sci., 12.","DOI":"10.3390\/app12136455"},{"key":"ref_31","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An Image is Worth 16\u00d716 Words: Transformers for Image Recognition at Scale. arXiv."},{"key":"ref_32","first-page":"287","article-title":"Do As I Can, Not As I Say: Grounding Language in Robotic Affordances","volume":"Volume 205","author":"Ahn","year":"2023","journal-title":"Proceedings of the 6th Conference on Robot Learning (CoRL)"},{"key":"ref_33","unstructured":"Driess, D., Xia, F., Sajjadi, M.S.M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., and Yu, T. (2023). PaLM-E: An Embodied Multimodal Language Model. arXiv."},{"key":"ref_34","unstructured":"Wu, T., Jing, Y., Cheang, C., Chen, G., Xu, J., Li, X., Liu, M., Li, H., and Kong, T. (2025, January 24\u201328). GR-1: Unleashing Large-Scale Video Generative Pre-Training for Visual Robot Manipulation. Proceedings of the International Conference on Learning Representations, Singapore."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Bharadhwaj, H., Vakil, J., Sharma, M., Gupta, A., Tulsiani, S., and Kumar, V. (2024, January 13\u201317). RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentation and Action Chunking. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan.","DOI":"10.1109\/ICRA57147.2024.10611293"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhao, H., Jiang, L., Fu, C.-W., and Jia, J. (2021, January 11\u201317). Point Transformer. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.01595"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Yu, X., Tang, L., Rao, Y., Huang, T., Zhou, J., and Lu, J. (2021). Point-BERT: Pre-Training 3D Point Cloud Transformers with Masked Point Modeling. arXiv.","DOI":"10.1109\/CVPR52688.2022.01871"},{"key":"ref_38","unstructured":"Shridhar, M., Manuelli, L., and Fox, D. (2022). PerAct: Perceiver-Actor for Robotics Manipulation. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1109\/TPAMI.2020.3008413","article-title":"Event-Based Vision: A Survey","volume":"44","author":"Gallego","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"566","DOI":"10.1109\/JSSC.2007.914337","article-title":"A 128\u00d7128 120 dB 15 \u03bcs Latency Asynchronous Temporal Contrast Vision Sensor","volume":"43","author":"Lichtsteiner","year":"2008","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Rebecq, H., Ranftl, R., Koltun, V., and Scaramuzza, D. (2019). Events-to-Video: Bringing Modern Computer Vision to Event Cameras. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR.2019.00398"},{"key":"ref_42","first-page":"341","article-title":"ESS: Learning Event-Based Semantic Segmentation from Still Images","volume":"Volume 13694","author":"Avidan","year":"2022","journal-title":"Computer Vision\u2014ECCV 2022, Lecture Notes in Computer Science"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"4947","DOI":"10.1109\/LRA.2021.3068942","article-title":"DSEC: A Stereo Event Camera Dataset for Driving Scenarios","volume":"6","author":"Gehrig","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_44","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18\u201324). Learning Transferable Visual Models from Natural Language Supervision. Proceedings of the 38th International Conference on Machine Learning (ICML 2021), Virtual."},{"key":"ref_45","unstructured":"Wu, Z., Liu, X., and Gilitschenski, I. (2023). EventCLIP: Adapting CLIP for Event-Based Vision. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Khazatsky, A., Pertsch, K., Nair, S., Balakrishna, A., Dasari, S., Karamcheti, S., Nasiriany, S., Srirama, M.K., Chen, L.Y., and Ellis, K. (2024). DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset. arXiv.","DOI":"10.15607\/RSS.2024.XX.120"},{"key":"ref_47","unstructured":"Shafiullah, N.M.M., Rai, A., Etukuru, H., Liu, Y., Misra, I., Chintala, S., and Pinto, L. (2023). On Bringing Robots Home. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Dal Cin, A., Dikov, G., Ju, J., and Ghafoorian, M. (2025). AnyMap: Learning a General Camera Model for Structure-from-Motion with Unknown Cameras. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.","DOI":"10.1109\/CVPR52734.2025.01554"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"666","DOI":"10.1038\/s44287-024-00089-8","article-title":"Wearable Haptics for Virtual Reality and Beyond","volume":"1","author":"Frisoli","year":"2024","journal-title":"Nat. Rev. Electr. Eng."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1186\/s12984-020-00771-6","article-title":"Immersive Virtual Environments and Wearable Haptic Devices in Rehabilitation of Children with Neuromotor Impairments: A Single-Blind Randomized Controlled Crossover Pilot Study","volume":"17","author":"Bortone","year":"2020","journal-title":"J. NeuroEng. Rehabil."},{"key":"ref_51","unstructured":"Calandra, R., Owens, A., Upadhyaya, M., Yuan, W., Lin, J., Adelson, E.H., and Levine, S. (2017). The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?. arXiv."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Dong, S., Yuan, W., and Adelson, E.H. (2017). Improved GelSight Tactile Sensor for Measuring Geometry and Slip. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE.","DOI":"10.1109\/IROS.2017.8202149"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"3838","DOI":"10.1109\/LRA.2020.2977257","article-title":"DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation","volume":"5","author":"Lambeta","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_54","unstructured":"Hao, P., Zhang, C., Li, D., Cao, X., Hao, X., Cui, S., and Wang, S. (2025). TLA: Tactile-Language-Action Model. arXiv."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Bi, J., Ma, K.Y., Hao, C., Zheng, M.S., and Soh, H. (2025). VLA-Touch: Enhancing Vision-Language-Action Models with Dual-Level Tactile Feedback. arXiv.","DOI":"10.1109\/LRA.2026.3692345"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Liu, W., Wang, J., Wang, Y., Wang, W., and Lu, C. (2024). ForceMimic: Force-Centric Imitation Learning with Force-Motion Capture System for Contact-Rich Manipulation. arXiv.","DOI":"10.1109\/ICRA55743.2025.11128061"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Suglia, V., Palazzo, L., Bevilacqua, V., Passantino, A., Pagano, G., and D\u2019Addio, G. (2024). A Novel Framework Based on Deep Learning Architecture for Continuous Human Activity Recognition with Inertial Sensors. Sensors, 24.","DOI":"10.3390\/s24072199"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Jaramillo, I.E., Jeong, J.G., Lopez, P.R., Lee, C.-H., Kang, D.-Y., Ha, T.-J., Oh, J.-H., Jung, H., Lee, J.H., and Lee, W.H. (2022). Real-Time Human Activity Recognition with IMU and Encoder Sensors in Wearable Exoskeleton Robot via Deep Learning Networks. Sensors, 22.","DOI":"10.3390\/s22249690"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Zhao, T.Z., Kumar, V., Levine, S., and Finn, C. (2023, January 10\u201314). Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. Proceedings of the Robotics: Science and Systems (RSS) XIX, Daegu, Republic of Korea.","DOI":"10.15607\/RSS.2023.XIX.016"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"H\u00f8eg, S.H., Du, Y., and Egeland, O. (2024). Streaming diffusion policy: Fast policy synthesis with variable noise diffusion models. arXiv.","DOI":"10.1109\/ICRA55743.2025.11127858"},{"key":"ref_61","unstructured":"Ahn, D., Jung, C., Baek, J., Yoo, S., and Ko, B.C. Shifted flow policy: Uncertainty-aware time reparameterization for visuomotor learning. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), to appear."},{"key":"ref_62","unstructured":"Shrivastava, A., Gangani, K., Jain, L., Goel, M., and Batra, N. (2026). ThermEval: A Structured Benchmark for Evaluation of Vision-Language Models on Thermal Imagery. arXiv."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Signoroni, A., Savardi, M., Baronio, A., and Benini, S. (2019). Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review. J. Imaging, 5.","DOI":"10.3390\/jimaging5050052"},{"key":"ref_64","unstructured":"Murray, R.M., Li, Z., and Sastry, S.S. (1994). A Mathematical Introduction to Robotic Manipulation, CRC Press."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Lynch, K.M., and Park, F.C. (2017). Modern Robotics: Mechanics, Planning, and Control, Cambridge University Press.","DOI":"10.1017\/9781316661239"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Furgale, P., Rehder, J., and Siegwart, R. (2013). Unified Temporal and Spatial Calibration for Multi-Sensor Systems. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, IEEE.","DOI":"10.1109\/IROS.2013.6696514"},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"1215","DOI":"10.1109\/TRO.2016.2596771","article-title":"Motion-Based Calibration of Multimodal Sensor Extrinsics and Timing Offset Estimation","volume":"32","author":"Taylor","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_68","first-page":"119","article-title":"General Robot-Camera Synchronization Based on Reprojection Error Minimization","volume":"Volume 2019","author":"Koide","year":"2019","journal-title":"Proceedings of the ARW & OAGM Workshop 2019"},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"1021","DOI":"10.1109\/LRA.2019.2893612","article-title":"General Hand\u2013Eye Calibration Based on Reprojection Error Minimization","volume":"4","author":"Koide","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"1196","DOI":"10.1109\/TRO.2022.3214350","article-title":"Probabilistic Framework for Hand-Eye and Robot-World Calibration AX=YB","volume":"39","author":"Ha","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"7309","DOI":"10.1109\/LRA.2021.3098942","article-title":"Learning to Calibrate\u2014Estimating the Hand-eye Transformation without Calibration Objects","volume":"6","author":"Pachtrachai","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_72","first-page":"1723","article-title":"BridgeData V2: A Dataset for Robot Learning at Scale","volume":"Volume 229","author":"Walke","year":"2023","journal-title":"Proceedings of The 7th Conference on Robot Learning"},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Porcini, F., Chiaradia, D., Marcheschi, S., Solazzi, M., and Frisoli, A. (August, January 31). Evaluation of an Exoskeleton-Based Bimanual Teleoperation Architecture with Independently Passivated Slave Devices. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197079"},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1109\/TOH.2023.3274388","article-title":"A Mechanical Hand-Tracking System with Tactile Feedback Designed for Telemanipulation","volume":"16","author":"Palagi","year":"2023","journal-title":"IEEE Trans. Haptics"},{"key":"ref_75","first-page":"892","article-title":"R3M: A Universal Visual Representation for Robot Manipulation","volume":"Volume 205","author":"Nair","year":"2023","journal-title":"Proceedings of the 6th Conference on Robot Learning"},{"key":"ref_76","unstructured":"Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., and Liu, X. (2022, January 18\u201324). Ego4D: Around the World in 3000 Hours of Egocentric Video. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA."},{"key":"ref_77","doi-asserted-by":"crossref","unstructured":"Chen, A.S., Nair, S., and Finn, C. (2021). Learning Generalizable Robotic Reward Functions from In-The-Wild Human Videos. arXiv.","DOI":"10.15607\/RSS.2021.XVII.012"},{"key":"ref_78","unstructured":"Makoviychuk, V., Wawrzyniak, L., Guo, Y., Lu, M., Storey, K., Macklin, M., Hoeller, D., Rudin, N., Allshire, A., and Handa, A. (2021). Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv."},{"key":"ref_79","unstructured":"Mayank, M., Roth, P., Tigue, J., Richard, A., Zhang, O., Du, P., Serrano-Mu\u00f1oz, A., Yao, X., Zurbr\u00fcgg, R., and Rudin, N. (2025). Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning. arXiv."},{"key":"ref_80","unstructured":"Mandlekar, A., Nasiriany, S., Wen, B., Akinola, I., Narang, Y., Fan, L., Zhu, Y., and Fox, D. (2023). MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations. arXiv."},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Fang, H.-S., Fang, H., Tang, Z., Liu, J., Wang, C., Wang, J., Zhu, H., and Lu, C. (2024, January 13\u201317). RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan.","DOI":"10.1109\/ICRA57147.2024.10611615"},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"3019","DOI":"10.1109\/LRA.2020.2974707","article-title":"RLBench: The Robot Learning Benchmark & Learning Environment","volume":"5","author":"James","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_83","doi-asserted-by":"crossref","unstructured":"Fu, Z., Zhao, T.Z., and Finn, C. (2023, January 6\u20139). Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation. Proceedings of the 7th Annual Conference on Robot Learning (CoRL 2023), Atlanta, GA, USA.","DOI":"10.15607\/RSS.2023.XIX.016"},{"key":"ref_84","doi-asserted-by":"crossref","unstructured":"Merriaux, P., Dupuis, Y., Boutteau, R., Vasseur, P., and Savatier, X. (2017). A Study of Vicon System Positioning Performance. Sensors, 17.","DOI":"10.3390\/s17071591"},{"key":"ref_85","doi-asserted-by":"crossref","unstructured":"Wang, J., and Olson, E. (2016, January 9\u201314). AprilTag 2: Efficient and Robust Fiducial Detection. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Republic of Korea.","DOI":"10.1109\/IROS.2016.7759617"},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"2280","DOI":"10.1016\/j.patcog.2014.01.005","article-title":"Automatic Generation and Detection of Highly Reliable Fiducial Markers under Occlusion","volume":"47","year":"2014","journal-title":"Pattern Recognit."},{"key":"ref_87","doi-asserted-by":"crossref","unstructured":"Wen, B., Yang, W., Kautz, J., and Birchfield, S. (2024, January 17\u201321). FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.01692"},{"key":"ref_88","doi-asserted-by":"crossref","unstructured":"Wen, B., Tremblay, J., Blukis, V., Tyree, S., M\u00fcller, T., Evans, A., Fox, D., Kautz, J., and Birchfield, S. (2023, January 18\u201322). BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00066"},{"key":"ref_89","doi-asserted-by":"crossref","unstructured":"Moya Rueda, F., Grzeszick, R., Fink, G.A., Feldhorst, S., and Ten Hompel, M. (2018). Convolutional Neural Networks for Human Activity Recognition Using Body-Worn Sensors. Informatics, 5.","DOI":"10.3390\/informatics5020026"},{"key":"ref_90","doi-asserted-by":"crossref","unstructured":"Sheng, H., Xuanqi, W., Chang, Z., Jiacheng, W., Pingxia, D., and Yuwei, W. (2026). AIGC video detection based on the fusion of spatial-frequency-optical flow multimodal features. J. Syst. Eng. Electron., 1\u201315.","DOI":"10.23919\/JSEE.2026.000049"},{"key":"ref_91","doi-asserted-by":"crossref","first-page":"3820","DOI":"10.1109\/TNNLS.2024.3357118","article-title":"Adaptive sparse memory networks for efficient and robust video object segmentation","volume":"36","author":"Dang","year":"2025","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_92","doi-asserted-by":"crossref","unstructured":"Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., and Reynolds, M. (2022). Flamingo: A Visual Language Model for Few-Shot Learning. arXiv.","DOI":"10.52202\/068431-1723"},{"key":"ref_93","unstructured":"Li, J., Li, D., Savarese, S., and Hoi, S.C.H. (2023, January 23\u201329). BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. Proceedings of the 40th International Conference on Machine Learning (ICML 2023), Honolulu, HI, USA."},{"key":"ref_94","unstructured":"Cheang, C.-L., Chen, G., Jing, Y., Kong, T., Li, H., Li, Y., Liu, Y., Wu, H., Xu, J., and Yang, Y. (2024). GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation. arXiv."},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Ma, M., Ren, J., Zhao, L., Testuggine, D., and Peng, X. (2022, January 19\u201324). Are Multimodal Transformers Robust to Missing Modality?. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01764"},{"key":"ref_96","doi-asserted-by":"crossref","unstructured":"Bednarek, P., Kicki, P., and Walas, K. (2020). On Robustness of Multi-Modal Fusion\u2014Robotics Perspective. Electronics, 9.","DOI":"10.3390\/electronics9071152"},{"key":"ref_97","doi-asserted-by":"crossref","first-page":"102648","DOI":"10.1016\/j.inffus.2024.102648","article-title":"Deep Evidential Fusion with Uncertainty Quantification and Reliability Learning for Multimodal Medical Image Segmentation","volume":"113","author":"Huang","year":"2025","journal-title":"Inf. Fusion"},{"key":"ref_98","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1016\/j.comcom.2023.12.003","article-title":"Robust Multimodal Federated Learning for Incomplete Modalities","volume":"214","author":"Yu","year":"2024","journal-title":"Comput. Commun."},{"key":"ref_99","first-page":"523","article-title":"Adaptive Multi-Scale Attention for Robust Visual\u2013Tactile Feature Fusion","volume":"58","author":"Li","year":"2025","journal-title":"IFAC-Pap."},{"key":"ref_100","unstructured":"Ma, X., Patidar, S., Haughton, I., and James, S. (June, January 16). Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA."},{"key":"ref_101","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.automatica.2019.03.009","article-title":"Event-Triggered Communication and Control of Networked Systems for Multi-Agent Consensus","volume":"105","author":"Nowzari","year":"2019","journal-title":"Automatica"},{"key":"ref_102","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1007\/s11633-024-1523-3","article-title":"End-to-End Identification of Autoregressive with Exogenous Input (ARX) Models Using Neural Networks","volume":"22","author":"Dong","year":"2025","journal-title":"Mach. Intell. Res."},{"key":"ref_103","unstructured":"Liu, B., Zhu, Y., Gao, C., Feng, Y., Liu, Q., Zhu, Y., and Stone, P. (2023). LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning. Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, Curran Associates."},{"key":"ref_104","unstructured":"Srivastava, S., Li, C., Lingelbach, M., Mart\u00edn-Mart\u00edn, R., Xia, F., Vainio, K., Lian, Z., Gokmen, C., Buch, S., and Liu, K. (2021, January 8\u201311). BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments. Proceedings of the 5th Conference on Robot Learning (CoRL 2021), London, UK."},{"key":"ref_105","unstructured":"Li, C., Xia, F., Mart\u00edn-Mart\u00edn, R., Lingelbach, M., Srivastava, S., Shen, B., Vainio, K., Gokmen, C., Dharan, G., and Jain, T. (2022, January 14\u201318). BEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation. Proceedings of the 6th Annual Conference on Robot Learning (CoRL 2022), Auckland, New Zealand."},{"key":"ref_106","doi-asserted-by":"crossref","unstructured":"Zhang, S., Xu, Z., Liu, P., Yu, X., Li, Y., Gao, Q., Fei, Z., Yin, Z., Wu, Z., and Jiang, Y.-G. (2025, January 19\u201320). VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Honolulu, HI, USA.","DOI":"10.1109\/ICCV51701.2025.01037"},{"key":"ref_107","unstructured":"Li, X., Hsu, K., Gu, J., Pertsch, K., Mees, O., Walke, H.R., Fu, C., Lunawat, I., Sieh, I., and Kirmani, S. (2024, January 6\u20139). Evaluating Real-World Robot Manipulation Policies in Simulation. Proceedings of the 8th Annual Conference on Robot Learning (CoRL 2024), Munich, Germany."},{"key":"ref_108","unstructured":"Liu, M., Sheng, J., Li, P., Wang, Z., Xu, T., Xu, T., and Liu, H. (2026). Trustworthy Evaluation of Robotic Manipulation: A New Benchmark and AutoEval Methods. arXiv."},{"key":"ref_109","doi-asserted-by":"crossref","unstructured":"Yang, F., Feng, C., Chen, Z., Park, H., Wang, D., Dou, Y., Zeng, Z., Chen, X., Gangopadhyay, R., and Owens, A. (2024, January 17\u201321). Binding Touch to Everything: Learning Unified Multimodal Tactile Representations. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.02488"},{"key":"ref_110","unstructured":"Zhang, K., Zhang, H., Xu, Z., Zhang, Z., Prince, M.R.I., Li, X., Han, X., Zhou, Y., Ajoudani, A., and She, Y. (2026). TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulation. arXiv."},{"key":"ref_111","unstructured":"Huang, Y., Lin, P., Li, W., Li, D., Li, J., Jiang, J., Xiao, C., and Jiao, Z. (2026). TaF-VLA: Tactile-Force Alignment in Vision-Language-Action Models for Force-aware Manipulation. arXiv."},{"key":"ref_112","unstructured":"Huang, J., Wang, S., Lin, F., Hu, Y., Wen, C., and Gao, Y. (2025). Tactile-VLA: Unlocking Vision-Language-Action Model\u2019s Physical Knowledge for Tactile Generalization. arXiv."},{"key":"ref_113","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/0951-8320(94)90132-5","article-title":"Robotic Fault Detection and Fault Tolerance: A Survey","volume":"46","author":"Visinsky","year":"1994","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_114","doi-asserted-by":"crossref","first-page":"362","DOI":"10.1007\/s10462-025-11327-2","article-title":"Fault-Tolerant Control Strategies for Industrial Robots: State of the Art and Future Perspective on AI-Based Fault Management","volume":"58","author":"Khan","year":"2025","journal-title":"Artif. Intell. Rev."},{"key":"ref_115","doi-asserted-by":"crossref","unstructured":"Samarathunga, S.M.B.P.B., Valori, M., Legnani, G., and Fassi, I. (2025). Assessing Safety in Physical Human\u2013Robot Interaction in Industrial Settings: A Systematic Review of Contact Modelling and Impact Measuring Methods. Robotics, 14.","DOI":"10.3390\/robotics14030027"},{"key":"ref_116","unstructured":"(2011). Robots and Robotic Devices\u2014Safety Requirements for Industrial Robots\u2014Part 1: Robots (Standard No. ISO 10218-1:2011)."},{"key":"ref_117","unstructured":"(2011). Robots and Robotic Devices\u2014Safety Requirements for Industrial Robots\u2014Part 2: Robot Systems and Integration (Standard No. ISO 10218-2:2011)."},{"key":"ref_118","unstructured":"(2010). Functional Safety of Electrical\/Electronic\/Programmable Electronic Safety-related Systems\u2014Part 1: General Requirements. (Standard No. IEC 61508-1:2010)."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/11\/3541\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T09:38:12Z","timestamp":1780479492000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/11\/3541"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,3]]},"references-count":118,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2026,6]]}},"alternative-id":["s26113541"],"URL":"https:\/\/doi.org\/10.3390\/s26113541","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,3]]}}}