{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T04:22:05Z","timestamp":1780633325956,"version":"3.54.1"},"reference-count":52,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,1,5]],"date-time":"2020-01-05T00:00:00Z","timestamp":1578182400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The design of human\u2013robot interactions is a key challenge to optimize operational performance. A promising approach is to consider mixed-initiative interactions in which the tasks and authority of each human and artificial agents are dynamically defined according to their current abilities. An important issue for the implementation of mixed-initiative systems is to monitor human performance to dynamically drive task allocation between human and artificial agents (i.e., robots). We, therefore, designed an experimental scenario involving missions whereby participants had to cooperate with a robot to fight fires while facing hazards. Two levels of robot automation (manual vs. autonomous) were randomly manipulated to assess their impact on the participants\u2019 performance across missions. Cardiac activity, eye-tracking, and participants\u2019 actions on the user interface were collected. The participants performed differently to an extent that we could identify high and low score mission groups that also exhibited different behavioral, cardiac and ocular patterns. More specifically, our findings indicated that the higher level of automation could be beneficial to low-scoring participants but detrimental to high-scoring ones, and vice versa. In addition, inter-subject single-trial classification results showed that the studied behavioral and physiological features were relevant to predict mission performance. The highest average balanced accuracy (74%) was reached using the features extracted from all input devices. These results suggest that an adaptive HRI driving system, that would aim at maximizing performance, would be capable of analyzing such physiological and behavior markers online to further change the level of automation when it is relevant for the mission purpose.<\/jats:p>","DOI":"10.3390\/s20010296","type":"journal-article","created":{"date-parts":[[2020,1,6]],"date-time":"2020-01-06T03:48:48Z","timestamp":1578282528000},"page":"296","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Towards Mixed-Initiative Human\u2013Robot Interaction: Assessment of Discriminative Physiological and Behavioral Features for Performance Prediction"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3578-4186","authenticated-orcid":false,"given":"Caroline P. C.","family":"Chanel","sequence":"first","affiliation":[{"name":"ISAE-SUPAERO, Universit\u00e9 de Toulouse, 31400 Toulouse, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rapha\u00eblle N.","family":"Roy","sequence":"additional","affiliation":[{"name":"ISAE-SUPAERO, Universit\u00e9 de Toulouse, 31400 Toulouse, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fr\u00e9d\u00e9ric","family":"Dehais","sequence":"additional","affiliation":[{"name":"ISAE-SUPAERO, Universit\u00e9 de Toulouse, 31400 Toulouse, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicolas","family":"Drougard","sequence":"additional","affiliation":[{"name":"ISAE-SUPAERO, Universit\u00e9 de Toulouse, 31400 Toulouse, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,5]]},"reference":[{"key":"ref_1","unstructured":"Finzi, A., and Orlandini, A. (2005). A Mixed-Initiative Approach to Human-Robot Interaction in Rescue Scenarios, American Association for Artificial Intelligence."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1002\/rob.21439","article-title":"Emergency response to the nuclear accident at the Fukushima Daiichi Nuclear Power Plants using mobile rescue robots","volume":"30","author":"Nagatani","year":"2013","journal-title":"J. Field Robot."},{"key":"ref_3","first-page":"75","article-title":"Mixed-initiative planning in space mission operations","volume":"28","author":"Bresina","year":"2007","journal-title":"AI Mag."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Gateau, T., Chanel, C.P.C., Le, M.H., and Dehais, F. (2016, January 9\u201314). Considering human\u2019s non-deterministic behavior and his availability state when designing a collaborative human-robots system. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759646"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Bevacqua, G., Cacace, J., Finzi, A., and Lippiello, V. (2015, January 7\u201311). Mixed-Initiative Planning and Execution for Multiple Drones in Search and Rescue Missions. Proceedings of the Twenty-Fifth International Conference in Automated Planning and Scheduling (ICAPS), Jerusalem, Israel.","DOI":"10.1609\/icaps.v25i1.13700"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Dehais, F., Dupr\u00e8s, A., Blum, S., Drougard, N., Scannella, S., Roy, R.N., and Lotte, F. (2019). Monitoring Pilot\u2019s Mental Workload Using ERPs and Spectral Power with a Six-Dry-Electrode EEG System in Real Flight Conditions. Sensors, 19.","DOI":"10.3390\/s19061324"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Dehais, F., Dupres, A., Di Flumeri, G., Verdiere, K., Borghini, G., Babiloni, F., and Roy, R. (2018, January 7\u201310). Monitoring Pilot\u2019s Cognitive Fatigue with Engagement Features in Simulated and Actual Flight Conditions Using an Hybrid fNIRS-EEG Passive BCI. Proceedings of the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Miyazaki, Japan.","DOI":"10.1109\/SMC.2018.00102"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Roy, R.N., Bonnet, S., Charbonnier, S., and Campagne, A. (2013, January 3\u20137). Mental fatigue and working memory load estimation: interaction and implications for EEG-based passive BCI. Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan.","DOI":"10.1109\/EMBC.2013.6611070"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Dehais, F., Rida, I., Roy, R.N., Iversen, J., Mullen, T., and Callan, D. (2019, January 6\u20139). A pBCI to Predict Attentional Error Before it Happens in Real Flight Conditions. Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy.","DOI":"10.1109\/SMC.2019.8914010"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1016\/j.neubiorev.2019.03.006","article-title":"Momentary lapse of control: A cognitive continuum approach to understanding and mitigating perseveration in human error","volume":"100","author":"Dehais","year":"2019","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"438","DOI":"10.1109\/TSMCA.2005.850587","article-title":"Validating human\u2013robot interaction schemes in multitasking environments","volume":"35","author":"Crandall","year":"2005","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"de Souza, P.E.U., Chanel, C.P.C., and Dehais, F. (2015, January 9\u201311). MOMDP-based target search mission taking into account the human operator\u2019s cognitive state. Proceedings of the IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI), Mare, Italy.","DOI":"10.1109\/ICTAI.2015.109"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Nikolaidis, S., Ramakrishnan, R., Gu, K., and Shah, J. (2015, January 2\u20135). Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks. Proceedings of the ACM\/IEEE International Conference on Human-Robot Interaction, Portland, OR, USA.","DOI":"10.1145\/2696454.2696455"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Jiang, S., and Arkin, R.C. (2015, January 9\u201312). Mixed-Initiative Human-Robot Interaction: Definition, Taxonomy, and Survey. Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Hong Kong, China.","DOI":"10.1109\/SMC.2015.174"},{"key":"ref_15","unstructured":"de Souza, P.E.U. (2017). Towards Mixed-Initiative Human-Robot Interaction: A Cooperative Human-Drone Team Framework. [Ph.D. Thesis, Universit\u00e9 de Toulouse]."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"776","DOI":"10.15837\/ijccc.2014.6.1484","article-title":"Dissonance engineering: A new challenge to analyse risky knowledge when using a system","volume":"9","author":"Vanderhaegen","year":"2014","journal-title":"Int. J. Comput. Commun. Control"},{"key":"ref_17","unstructured":"Scerbo, M.W. (2008). Adaptive Automation, Oxford University Press."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1177\/1555343411410160","article-title":"Adaptive aiding of human-robot teaming: Effects of imperfect automation on performance, trust, and workload","volume":"5","author":"Parasuraman","year":"2011","journal-title":"J. Cogn. Eng. Decis. Mak."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Mower, E., Feil-Seifer, D.J., Mataric, M.J., and Narayanan, S. (2007, January 26\u201329). Investigating implicit cues for user state estimation in human-robot interaction using physiological measurements. Proceedings of the RO-MAN 2007\u2014The 16th IEEE International Symposium on Robot and Human Interactive Communication, Jeju Island, Korea.","DOI":"10.1109\/ROMAN.2007.4415249"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"187","DOI":"10.3389\/fnhum.2018.00187","article-title":"In silico versus over the clouds: On-the-fly mental state estimation of aircraft pilots, using a functional near infrared spectroscopy based passive-BCI","volume":"12","author":"Gateau","year":"2018","journal-title":"Front. Hum. Neurosci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1080\/01449290701288791","article-title":"Multiple robot\/single human interaction: Effects on perceived workload","volume":"28","author":"Adams","year":"2009","journal-title":"Behav. Inf. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Harriott, C.E., Zhang, T., and Adams, J.A. (2011, January 15\u201316). Evaluating the applicability of current models of workload to peer-based human-robot teams. Proceedings of the 2011 6th ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Ulm, Germany.","DOI":"10.1145\/1957656.1957670"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.eswa.2016.10.044","article-title":"Analysing temporal performance profiles of UAV operators using time series clustering","volume":"70","author":"Camacho","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1109\/THMS.2014.2307258","article-title":"Formal Detection of Attentional Tunneling in Human Operator-Automation Interactions","volume":"44","author":"Regis","year":"2014","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1007\/s10044-006-0025-y","article-title":"An empirical study of machine learning techniques for affect recognition in human\u2013robot interaction","volume":"9","author":"Rani","year":"2006","journal-title":"Pattern Anal. Appl."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Singh, G., Berm\u00f9dez i Badia, S., Ventura, R., and Silva, J.L. (2018, January 19\u201321). Physiologically Attentive User Interface for Robot Teleoperation\u2014Real Time Emotional State Estimation and Interface Modification Using Physiology, Facial Expressions and Eye Movements. Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC), Funchal, Madeira, Portugal.","DOI":"10.5220\/0006733002940302"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Heard, J., Harriott, C.E., and Adams, J.A. (2018). A Survey of Workload Assessment Algorithms. IEEE Trans. Hum. Mach. Syst.","DOI":"10.1109\/THMS.2017.2782483"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1016\/j.ergon.2006.04.002","article-title":"Using eye movement activity as a correlate of cognitive workload","volume":"36","author":"Ahlstrom","year":"2006","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_29","unstructured":"Chanel, C.P.C., Wilson, M.D., and Scannella, S. (2019, January 6\u20139). Online ECG-based Features for Cognitive Load Assessment. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Bari, Italy."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1016\/j.apergo.2015.07.009","article-title":"Effects of mental workload on physiological and subjective responses during traffic density monitoring: A field study","volume":"52","author":"Fallahi","year":"2016","journal-title":"Appl. Ergon."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1016\/j.apergo.2016.09.013","article-title":"Classification of a Driver\u2019s cognitive workload levels using artificial neural network on ECG signals","volume":"59","author":"Tjolleng","year":"2017","journal-title":"Appl. Ergon."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.apergo.2019.04.012","article-title":"Global difficulty modulates the prioritization strategy in multitasking situations","volume":"80","author":"Matton","year":"2019","journal-title":"Appl. Ergon."},{"key":"ref_33","unstructured":"Drougard, N., Chanel, C.P.C., Roy, R.N., and Dehais, F. (2017, January 24). Mixed-initiative mission planning considering human operator state estimation based on physiological sensors. Proceedings of the IROS-2017 Workshop on Human-Robot Interaction in Collaborative Manufacturing Environments (HRI-CME), Vancouver, BC, Canada."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Charles, J.A., Chanel, C.P.C., Chauffaut, C., Chauvin, P., and Drougard, N. (2018, January 15\u201318). Human-Agent Interaction Model Learning based on Crowdsourcing. Proceedings of the 6th International Conference on Human-Agent Interaction, Southampton, UK.","DOI":"10.1145\/3284432.3284471"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"29","DOI":"10.3109\/00207459008994241","article-title":"Subjective and objective sleepiness in the active individual","volume":"52","author":"Gillberg","year":"1990","journal-title":"Int. J. Neurosci."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/S0166-4115(08)62386-9","article-title":"Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research","volume":"Volume 52","author":"Hart","year":"1988","journal-title":"Advances in Psychology"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1093\/oxfordjournals.eurheartj.a014868","article-title":"Heart rate variability: Standards of measurement, physiological interpretation, and clinical use","volume":"17","author":"Malik","year":"1996","journal-title":"Eur. Heart J."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"S\u00f6rnmo, L., and Laguna, P. (2005). Bioelectrical Signal Processing in Cardiac and Neurological Applications, Elsevier Academic Press.","DOI":"10.1016\/B978-012437552-9\/50007-6"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1635","DOI":"10.1016\/j.jmva.2008.01.005","article-title":"Nonparametric methods for unbalanced multivariate data and many factor levels","volume":"99","author":"Harrar","year":"2008","journal-title":"J. Multivar. Anal."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wickens, C.D., and Dehais, F. (2019). Expertise in Aviation, Oxford University Press.","DOI":"10.1093\/oxfordhb\/9780198795872.013.29"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/j.apergo.2017.10.009","article-title":"How we can measure the non-driving-task engagement in automated driving: comparing flow experience and workload","volume":"67","author":"Ko","year":"2018","journal-title":"Appl. Ergon."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.bbr.2013.10.042","article-title":"Using near infrared spectroscopy and heart rate variability to detect mental overload","volume":"259","author":"Durantin","year":"2014","journal-title":"Behav. Brain Res."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1518\/001872095779049516","article-title":"How in the world did we ever get into that mode? Mode error and awareness in supervisory control","volume":"37","author":"Sarter","year":"1995","journal-title":"Hum. Factors"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.mechatronics.2018.07.012","article-title":"Relieving operators\u2019 workload: Towards affective robotics in industrial scenarios","volume":"54","author":"Landi","year":"2018","journal-title":"Mechatronics"},{"key":"ref_45","unstructured":"Huang, H.M., Messina, E., and Albus, J. (2003). Toward a Generic Model for Autonomy Levels for Unmanned Systems (ALFUS), Technical Report."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Tang, F., and Ito, E. (2017, January 29\u201331). Human-assisted navigation through sliding autonomy. Proceedings of the 2017 2nd International Conference on Robotics and Automation Engineering (ICRAE), Shanghai, China.","DOI":"10.1109\/ICRAE.2017.8291347"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Tang, F., Mohammed, M., and Longazo, J. (2016, January 12\u201315). Experiments of human-robot teaming under sliding autonomy. Proceedings of the 2016 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), Banff, AB, Canada.","DOI":"10.1109\/AIM.2016.7576752"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1033","DOI":"10.1080\/00140139.2018.1441449","article-title":"Human-automation interaction for multiple robot control: The effect of varying automation assistance and individual differences on operator performance","volume":"61","author":"Wright","year":"2018","journal-title":"Ergonomics"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1518\/001872008X312198","article-title":"Humans: Still Vital After All These Years of Automation","volume":"50","author":"Parasuraman","year":"2008","journal-title":"Hum. Factors"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1016\/j.apergo.2011.09.004","article-title":"Cognitive conflict in human\u2013automation interactions: A psychophysiological study","volume":"43","author":"Dehais","year":"2012","journal-title":"Appl. Ergon."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Dehais, F., Peysakhovich, V., Scannella, S., Fongue, J., and Gateau, T. (2015, January 18\u201323). Automation surprise in aviation: Real-time solutions. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Korea.","DOI":"10.1145\/2702123.2702521"},{"key":"ref_52","unstructured":"Lyons, J.B. (2013, January 25\u201327). Being transparent about transparency: A model for human-robot interaction. Proceedings of the 2013 AAAI Spring Symposium Series, Palo Alto, CA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/296\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:03:39Z","timestamp":1760364219000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/296"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,5]]},"references-count":52,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["s20010296"],"URL":"https:\/\/doi.org\/10.3390\/s20010296","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1,5]]}}}