{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T22:23:42Z","timestamp":1769725422487,"version":"3.49.0"},"reference-count":43,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T00:00:00Z","timestamp":1763510400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Deployment of robots into hazardous environments typically involves a \u201chuman\u2013robot teaming\u201d (HRT) paradigm, in which a human supervisor interacts with a remotely operating robot inside the hazardous zone. Situational awareness (SA) is vital for enabling HRT, to support navigation, planning, and decision-making. In this paper, we explore issues of higher-level \u201csemantic\u201d information and understanding in SA. In semi-autonomous or variable-autonomy paradigms, different types of semantic information may be important, in different ways, for both the human operator and an autonomous agent controlling the robot. We propose a generalizable framework for acquiring and combining multiple modalities of semantic-level SA during remote deployments of mobile robots. We demonstrate the framework with an example application of search and rescue (SAR) in disaster-response robotics. We propose a set of \u201cenvironment semantic indicators\u201d that can reflect a variety of different types of semantic information, such as indicators of risk or signs of human activity (SHA), as the robot encounters different scenes. Based on these indicators, we propose a metric to describe the overall situation of the environment, called \u201cSituational Semantic Richness\u201d (SSR). This metric combines multiple semantic indicators to summarize the overall situation. The SSR indicates whether an information-rich, complex situation has been encountered, which may require advanced reasoning by robots and humans and, hence, the attention of the expert human operator. The framework is tested on a Jackal robot in a mock-up disaster-response environment. Experimental results demonstrate that the proposed semantic indicators are sensitive to changes in different modalities of semantic information in different scenes, and the SSR metric reflects the overall semantic changes in the situations encountered.<\/jats:p>","DOI":"10.3389\/frobt.2025.1694123","type":"journal-article","created":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T05:18:16Z","timestamp":1763529496000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["A framework for semantics-based situational awareness during mobile robot deployments"],"prefix":"10.3389","volume":"12","author":[{"given":"Tianshu","family":"Ruan","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aniketh","family":"Ramesh","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hao","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alix","family":"Johnstone-Morfoisse","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gokcenur","family":"Altindal","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul","family":"Norman","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Grigoris","family":"Nikolaou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rustam","family":"Stolkin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Manolis","family":"Chiou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,11,19]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1109\/IVS.2014.6856509","article-title":"Ontology-based context awareness for driving assistance systems","volume-title":"2014 IEEE intelligent vehicles symposium proceedings","author":"Armand","year":"2014"},{"key":"B2","doi-asserted-by":"publisher","first-page":"4849","DOI":"10.3390\/s23104849","article-title":"From slam to situational awareness: challenges and survey","volume":"23","author":"Bavle","year":"2023","journal-title":"Sensors"},{"key":"B3","doi-asserted-by":"crossref","DOI":"10.1109\/ICCV.2019.00925","article-title":"Yolact: real-time instance segmentation","volume-title":"ICCV","author":"Bolya","year":"2019"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3472206","article-title":"Mixed-initiative variable autonomy for remotely operated mobile robots","volume":"10","author":"Chiou","year":"2021","journal-title":"ACM Trans. Human-Robot Interact. (THRI)"},{"key":"B5","first-page":"4545","article-title":"Robot-assisted nuclear disaster response: report and insights from a field exercise","author":"Chiou","year":"2022"},{"key":"B6","volume-title":"Pearson\u2019s versus Spearman\u2019s and Kendall\u2019s correlation coefficients for continuous data","author":"Chok","year":"2010"},{"key":"B7","first-page":"4354","article-title":"Measurement and prediction of situation awareness in human-robot interaction based on a framework of probabilistic attention","author":"Dini","year":"2017"},{"key":"B8","doi-asserted-by":"crossref","first-page":"789","DOI":"10.1109\/NAECON.1988.195097","article-title":"Situation awareness global assessment technique (sagat)","volume-title":"Proceedings of the IEEE 1988 national aerospace and electronics conference","author":"Endsley","year":"1988"},{"key":"B9","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1518\/001872095779049499","article-title":"Measurement of situation awareness in dynamic systems","volume":"37","author":"Endsley","year":"1995","journal-title":"Hum. factors"},{"key":"B10","doi-asserted-by":"crossref","first-page":"9","DOI":"10.4324\/9781315087924-3","article-title":"Toward a theory of situation awareness in dynamic systems","volume-title":"Situational awareness","author":"Endsley","year":"2017"},{"key":"B11","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1109\/ICPICS50287.2020.9201954","article-title":"Uav security situation awareness method based on semantic analysis","volume-title":"2020 IEEE international conference on power, intelligent computing and systems (ICPICS)","author":"Gao","year":"2020"},{"key":"B12","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1109\/ICARCV.2014.7064347","article-title":"Rsaw: a situation awareness system for autonomous robots","volume-title":"2014 13th international conference on control automation robotics &amp; vision (ICARCV)","author":"Ghezala","year":"2014"},{"key":"B13","first-page":"3144","article-title":"Autonomous task planning and situation awareness in robotic surgery","author":"Ginesi","year":"2020"},{"key":"B14","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1007\/978-88-470-1821-1_22","article-title":"Modeling pilot situation awareness","volume-title":"Human modelling in assisted transportation: models, tools and risk methods","author":"Hooey","year":"2011"},{"key":"B15","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1109\/ACIRS.2019.8935984","article-title":"Ontology-based driving scene modeling, situation assessment and decision making for autonomous vehicles","volume-title":"2019 4th Asia-Pacific conference on intelligent robot systems (ACIRS)","author":"Huang","year":"2019"},{"key":"B16","first-page":"1","article-title":"Rescue robots at earthquake-hit mirandola, Italy: a field report","volume-title":"2012 IEEE international symposium on safety, security, and rescue robotics (SSRR)","author":"Kruijff","year":"2012"},{"key":"B17","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1109\/ITSC.2017.8317942","article-title":"Fast semi-dense 3d semantic mapping with monocular visual slam","volume-title":"2017 IEEE 20th international conference on intelligent transportation systems (ITSC)","author":"Li","year":"2017"},{"key":"B18","doi-asserted-by":"crossref","first-page":"3431","DOI":"10.1109\/CVPR.2015.7298965","article-title":"Fully convolutional networks for semantic segmentation","volume-title":"2015 IEEE conference on computer vision and pattern recognition (CVPR)","author":"Long","year":"2015"},{"key":"B19","doi-asserted-by":"publisher","first-page":"733","DOI":"10.1017\/aer.2018.14","article-title":"Quantifying situation awareness for small unmanned aircraft: towards routine beyond visual line of sight operations","volume":"122","author":"McAree","year":"2018","journal-title":"Aeronautical J."},{"key":"B20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3645090","article-title":"Who\u2019s in charge here? a survey on trustworthy ai in variable autonomy robotic systems","volume":"56","author":"Methnani","year":"2024","journal-title":"ACM Comput. Surv."},{"key":"B21","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/9407.001.0001","volume-title":"Disaster robotics","author":"Murphy","year":"2014"},{"key":"B22","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1137\/1109020","article-title":"On estimating regression","volume":"9","author":"Nadaraya","year":"1964","journal-title":"Theory Probab. & Its Appl."},{"key":"B23","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1002\/rob.21439","article-title":"Emergency response to the nuclear accident at the fukushima daiichi nuclear power plants using mobile rescue robots","volume":"30","author":"Nagatani","year":"2013","journal-title":"J. Field Robotics"},{"key":"B24","doi-asserted-by":"publisher","first-page":"3590","DOI":"10.1109\/jsyst.2019.2918283","article-title":"A review of situation awareness assessment approaches in aviation environments","volume":"13","author":"Nguyen","year":"2019","journal-title":"IEEE Syst. J."},{"key":"B25","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1109\/SSRR50563.2020.9292585","article-title":"Vfh+ based shared control for remotely operated mobile robots","volume-title":"2020 IEEE international symposium on safety, security, and rescue robotics (SSRR)","author":"Pappas","year":"2020"},{"key":"B26","doi-asserted-by":"publisher","first-page":"10729","DOI":"10.1109\/lra.2022.3192612","article-title":"Robot vitals and robot health: towards systematically quantifying runtime performance degradation in robots under adverse conditions","volume":"7","author":"Ramesh","year":"2022","journal-title":"IEEE Robotics Automation Lett."},{"key":"B27","doi-asserted-by":"crossref","DOI":"10.1109\/SMC53992.2023.10394184","article-title":"Experimental evaluation of model predictive mixed-initiative variable autonomy systems applied to human-robot teams","volume-title":"2023 IEEE international conference on systems, man, and cybernetics (SMC), October 1-4, 2023, Hyatt Maui, Hawaii, USA","author":"Ramesh","year":"2023"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3636432","article-title":"Variable autonomy through responsible robotics: design guidelines and research agenda","volume":"13","author":"Reinmund","year":"2024","journal-title":"ACM Trans. Human-Robot Interact."},{"key":"B29","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1109\/SSRR56537.2022.10018727","article-title":"A taxonomy of semantic information in robot-assisted disaster response","volume-title":"2022 IEEE international symposium on safety, security, and rescue robotics (SSRR)","author":"Ruan","year":"2022"},{"key":"B30","unstructured":"An exploratory study on human-robot interaction using semantics-based situational awareness\n          \n          \n            \n              Ruan\n              T.\n            \n            \n              Ramesh\n              A.\n            \n            \n              Stolkin\n              R.\n            \n            \n              Chiou\n              M.\n            \n          \n          arXiv preprint arXiv:2507.17376\n          \n          2025"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1763","DOI":"10.1213\/ane.0000000000002864","article-title":"Correlation coefficients: appropriate use and interpretation","volume":"126","author":"Schober","year":"2018","journal-title":"Anesth. & analgesia"},{"key":"B32","volume-title":"A framework for dynamic situational awareness in human robot teams: an interview study","author":"Senaratne","year":"2025"},{"key":"B33","first-page":"117","volume-title":"A quantitative situational awareness model of pilot","author":"Shuang","year":"2014"},{"key":"B34","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1016\/s0378-4754(00)00270-6","article-title":"Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates","volume":"55","author":"Sobol","year":"2001","journal-title":"Math. Comput. Simul."},{"key":"B35","doi-asserted-by":"publisher","first-page":"717","DOI":"10.1016\/j.autcon.2004.04.012","article-title":"A fuzzy based multi-objective path planning of construction sites","volume":"13","author":"Soltani","year":"2004","journal-title":"Automation Constr."},{"key":"B36","doi-asserted-by":"publisher","first-page":"449","DOI":"10.1080\/00140139.2017.1278796","article-title":"State-of-science: situation awareness in individuals, teams and systems","volume":"60","author":"Stanton","year":"2017","journal-title":"Ergonomics"},{"key":"B37","article-title":"Status, barriers and cost-benefits of robotic and remote systems applications in nuclear decommissioning and radioactive waste management","author":"Stolkin","year":"2023"},{"key":"B38","doi-asserted-by":"crossref","first-page":"111","DOI":"10.4324\/9781315087924-8","article-title":"Situational awareness rating technique (sart): the development of a tool for aircrew systems design","volume-title":"Situational awareness","author":"Taylor","year":"2017"},{"key":"B39","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1016\/j.artint.2015.05.010","article-title":"Representations for robot knowledge in the knowrob framework","volume":"247","author":"Tenorth","year":"2017","journal-title":"Artif. Intell."},{"key":"B40","first-page":"5998","article-title":"Attention is all you need","volume-title":"Advances in neural information processing systems","author":"Vaswani","year":"2017"},{"key":"B41","doi-asserted-by":"publisher","first-page":"23","DOI":"10.24018\/ejers.2017.2.11.517","article-title":"Inverse square law for light and radiation: a unifying educational approach","volume":"2","author":"Voudoukis","year":"2017","journal-title":"Eur. J. Eng. Technol. Res."},{"key":"B42","first-page":"359","article-title":"Smooth regression analysis","author":"Watson","year":"1964","journal-title":"Sankhya Indian J. Statistics, Ser."},{"key":"B43","article-title":"Visual semantic navigation using scene priors","author":"Yang","year":"2018"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1694123\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T05:18:22Z","timestamp":1763529502000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1694123\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,19]]},"references-count":43,"alternative-id":["10.3389\/frobt.2025.1694123"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2025.1694123","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,19]]},"article-number":"1694123"}}