{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,25]],"date-time":"2026-07-25T19:50:36Z","timestamp":1785009036014,"version":"3.55.0"},"reference-count":44,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T00:00:00Z","timestamp":1764892800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000780","name":"Horizon Europe project Fluently","doi-asserted-by":"publisher","award":["101058680"],"award-info":[{"award-number":["101058680"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Eurostars project Singularity","award":["2309"],"award-info":[{"award-number":["2309"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Human\u2013robot collaboration (HRC) is a key focus of Industry 5.0, aiming to enhance worker productivity while ensuring well-being. The ability to perceive human psycho-physical states, such as stress and cognitive load, is crucial for adaptive and human-aware robotics. This paper introduces MultiPhysio-HRC, a multimodal dataset containing physiological, audio, and facial data collected during real-world HRC scenarios. The dataset includes electroencephalography (EEG), electrocardiography (ECG), electrodermal activity (EDA), respiration (RESP), electromyography (EMG), voice recordings, and facial action units. The dataset integrates controlled cognitive tasks, immersive virtual reality experiences, and industrial disassembly activities performed manually and with robotic assistance, to capture a holistic view of the participants\u2019 mental states. Rich ground truth annotations were obtained using validated psychological self-assessment questionnaires. Baseline models were evaluated for stress and cognitive load classification, demonstrating the dataset\u2019s potential for affective computing and human-aware robotics research. MultiPhysio-HRC is publicly available to support research in human-centered automation, workplace well-being, and intelligent robotic systems.<\/jats:p>","DOI":"10.3390\/robotics14120184","type":"journal-article","created":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T10:50:38Z","timestamp":1764931838000},"page":"184","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["MultiPhysio-HRC: A Multimodal Physiological Signals Dataset for Industrial Human\u2013Robot Collaboration"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1191-7765","authenticated-orcid":false,"given":"Andrea","family":"Bussolan","sequence":"first","affiliation":[{"name":"ARM-Lab, Scuola Universitaria Professionale della Svizzera Italiana, 6900 Lugano, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7941-8287","authenticated-orcid":false,"given":"Stefano","family":"Baraldo","sequence":"additional","affiliation":[{"name":"ARM-Lab, Scuola Universitaria Professionale della Svizzera Italiana, 6900 Lugano, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1228-5950","authenticated-orcid":false,"given":"Oliver","family":"Avram","sequence":"additional","affiliation":[{"name":"ARM-Lab, Scuola Universitaria Professionale della Svizzera Italiana, 6900 Lugano, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3880-1842","authenticated-orcid":false,"given":"Pablo","family":"Urcola","sequence":"additional","affiliation":[{"name":"Bitbrain, 50006 Zaragoza, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1183-349X","authenticated-orcid":false,"given":"Luis","family":"Montesano","sequence":"additional","affiliation":[{"name":"Bitbrain, 50006 Zaragoza, Spain"},{"name":"Departamento de Inform\u00e1tica e Ingenier\u00eda de Sistemas, Universidad de Zaragoza, 50009 Zaragoza, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-2555-1762","authenticated-orcid":false,"given":"Luca Maria","family":"Gambardella","sequence":"additional","affiliation":[{"name":"Faculty of Informatics, Universit\u00e0 della Svizzera Italiana, 6900 Lugano, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3257-4482","authenticated-orcid":false,"given":"Anna","family":"Valente","sequence":"additional","affiliation":[{"name":"ARM-Lab, Scuola Universitaria Professionale della Svizzera Italiana, 6900 Lugano, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,12,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Lorenzini, M., Lagomarsino, M., Fortini, L., Gholami, S., and Ajoudani, A. (2023). Ergonomic human-robot collaboration in industry: A review. Front. Robot. AI, 9.","DOI":"10.3389\/frobt.2022.813907"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"612","DOI":"10.1016\/j.jmsy.2022.02.001","article-title":"Outlook on human-centric manufacturing towards Industry 5.0","volume":"62","author":"Lu","year":"2022","journal-title":"J. Manuf. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.cirp.2022.03.045","article-title":"Deliberative robotics \u2013 a novel interactive control framework enhancing human-robot collaboration","volume":"71","author":"Valente","year":"2022","journal-title":"CIRP Ann."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Spezialetti, M., Placidi, G., and Rossi, S. (2020). Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives. Front. Robot. AI, 7.","DOI":"10.3389\/frobt.2020.532279"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1038\/s41597-024-03128-z","article-title":"Physiological data for affective computing in HRI with anthropomorphic service robots: The AFFECT-HRI data set","volume":"11","author":"Heinisch","year":"2024","journal-title":"Sci. Data"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"34298","DOI":"10.1109\/JSEN.2025.3597329","article-title":"Physiological Sensor Technologies in Workload Estimation: A Review","volume":"25","author":"Tamantini","year":"2025","journal-title":"IEEE Sens. J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1109\/TITS.2005.848368","article-title":"Detecting Stress During Real-World Driving Tasks Using Physiological Sensors","volume":"6","author":"Healey","year":"2005","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Schmidt, P., Reiss, A., Duerichen, R., Marberger, C., and Van Laerhoven, K. (2018, January 2). Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection. Proceedings of the Proceedings of the 20th ACM International Conference on Multimodal Interaction, Boulder, CO, USA.","DOI":"10.1145\/3242969.3242985"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1109\/JBHI.2017.2688239","article-title":"DREAMER: A Database for Emotion Recognition Through EEG and ECG Signals From Wireless Low-cost Off-the-Shelf Devices","volume":"22","author":"Katsigiannis","year":"2018","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_10","first-page":"76","article-title":"AVCAffe: A Large Scale Audio-Visual Dataset of Cognitive Load and Affect for Remote Work","volume":"37","author":"Sarkar","year":"2023","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_11","first-page":"29798","article-title":"StressID: A Multimodal Dataset for Stress Identification","volume":"Volume 36","author":"Oh","year":"2023","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_12","unstructured":"SenseCobot (2025, December 02). SenseCobot. Available online: https:\/\/zenodo.org\/records\/8363762."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"104418","DOI":"10.1016\/j.apergo.2024.104418","article-title":"Assessing operator stress in collaborative robotics: A multimodal approach","volume":"123","author":"Borghi","year":"2025","journal-title":"Appl. Ergon."},{"key":"ref_14","unstructured":"Bussolan, A., Baraldo, S., Gambardella, L.M., and Valente, A. (2023, January 26\u201327). Assessing the Impact of Human-Robot Collaboration on Stress Levels and Cognitive Load in Industrial Assembly Tasks. Proceedings of the ISR Europe 2023, 56th International Symposium on Robotics, Stuttgart, Germany."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Nenna, F., Zanardi, D., Orlando, E.M., Nannetti, M., Buodo, G., and Gamberini, L. (2024, January 11). Getting Closer to Real-world: Monitoring Humans Working with Collaborative Industrial Robots. Proceedings of the Companion of the 2024 ACM\/IEEE International Conference on Human-Robot Interaction, New York, NY, USA. HRI \u201924.","DOI":"10.1145\/3610978.3640632"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Scarpina, F., and Tagini, S. (2017). The Stroop Color and Word Test. Front. Psychol., 8.","DOI":"10.3389\/fpsyg.2017.00557"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Meule, A. (2017). Reporting and Interpreting Working Memory Performance in n-back Tasks. Front. Psychol., 8.","DOI":"10.3389\/fpsyg.2017.00352"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Schmidtke, K. (2010). Tower of Hanoi Problem. The Corsini Encyclopedia of Psychology, John Wiley & Sons, Ltd.","DOI":"10.1002\/9780470479216.corpsy1002"},{"key":"ref_19","unstructured":"Secchi, C., and Marconi, L. (2024). Advancing Human-Robot Collaboration by Robust Speech Recognition in Smart Manufacturing. Proceedings of the European Robotics Forum 2024, Springer."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bansod, Y., Patra, S., Nau, D., and Roberts, M. (2022). HTN Replanning from the Middle. Int. FLAIRS Conf. Proc., 35.","DOI":"10.32473\/flairs.v35i.130732"},{"key":"ref_21","unstructured":"Coleman, D., Sucan, I., Chitta, S., and Correll, N. (2014). Reducing the barrier to entry of complex robotic software: A moveit! case study. arXiv."},{"key":"ref_22","unstructured":"Spielberger, C., and Gorsuch, R. (1983). Manual for the State-Trait Anxiety Inventory (form Y) (\u201cSelf-Evaluation Questionnaire\u201d), Consulting Psychologists Press."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/S0166-4115(08)62386-9","article-title":"Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research","volume":"Volume 52","author":"Hart","year":"1988","journal-title":"Advances in Psychology"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/0005-7916(94)90063-9","article-title":"Measuring emotion: The self-assessment manikin and the semantic differential","volume":"25","author":"Bradley","year":"1994","journal-title":"J. Behav. Ther. Exp. Psychiatry"},{"key":"ref_25","unstructured":"Nomura, T., Kanda, T., Suzuki, T., and Kato, K. (2004, January 22\u201322). Psychology in Human-Robot Communication: An Attempt through Investigation of Negative Attitudes and Anxiety toward Robots. Proceedings of the RO-MAN 2004, 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759), Kurashiki, Okayama, Japan."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Loizaga, E., Bastida, L., Sillaurren, S., Moya, A., and Toledo, N. (2024). Modelling and Measuring Trust in Human\u2013Robot Collaboration. Appl. Sci., 14.","DOI":"10.3390\/app14051919"},{"key":"ref_27","unstructured":"Stegeman, D.F., and Hermens, H.J. (2025, December 02). Standards for Surface Electromyography: The European Project \u201cSurface EMG for Non-Invasive Assessment of Muscles (SENIAM)\u201d. Available online: https:\/\/www.researchgate.net\/publication\/398119434_Preparatory_use_of_neurodynamics_to_enhance_upper_limb_function_in_patients_with_acquired_brain_injury_a_randomized_controlled_trial."},{"key":"ref_28","first-page":"797","article-title":"cvxEDA: A Convex Optimization Approach to Electrodermal Activity Processing","volume":"63","author":"Greco","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1689","DOI":"10.3758\/s13428-020-01516-y","article-title":"NeuroKit2: A Python toolbox for neurophysiological signal processing","volume":"53","author":"Makowski","year":"2021","journal-title":"Behav. Res. Methods"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Pham, T., Lau, Z.J., Chen, S.H.A., and Makowski, D. (2021). Heart Rate Variability in Psychology: A Review of HRV Indices and an Analysis Tutorial. Sensors, 21.","DOI":"10.20944\/preprints202105.0070.v1"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Orguc, S., Khurana, H.S., Stankovic, K.M., Leel, H., and Chandrakasan, A. (2018, January 18\u201321). EMG-based Real Time Facial Gesture Recognition for Stress Monitoring. Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA.","DOI":"10.1109\/EMBC.2018.8512781"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"857","DOI":"10.1109\/TAFFC.2019.2901673","article-title":"Feature Extraction and Selection for Emotion Recognition from Electrodermal Activity","volume":"12","author":"Shukla","year":"2021","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Raufi, B., and Longo, L. (2022). An Evaluation of the EEG Alpha-to-Theta and Theta-to-Alpha Band Ratios as Indexes of Mental Workload. Front. Neuroinformatics, 16.","DOI":"10.3389\/fninf.2022.861967"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/TAU.1967.1161901","article-title":"The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms","volume":"15","author":"Welch","year":"1967","journal-title":"IEEE Trans. Audio Electroacoust."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"781","DOI":"10.1007\/s42761-023-00191-4","article-title":"Py-Feat: Python Facial Expression Analysis Toolbox","volume":"4","author":"Cheong","year":"2023","journal-title":"Affect. Sci."},{"key":"ref_36","unstructured":"Team, S. (2025, December 02). Silero VAD: Pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier. Available online: https:\/\/github.com\/snakers4\/silero-vad."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Tomba, K., Dumoulin, J., Mugellini, E., Abou Khaled, O., and Hawila, S. (2018, January 26\u201328). Stress Detection Through Speech Analysis. Proceedings of the 15th International Joint Conference on e-Business and Telecommunications, Porto, Portugal.","DOI":"10.5220\/0006855803940398"},{"key":"ref_38","unstructured":"Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. (2023, January 23\u201329). Robust speech recognition via large-scale weak supervision. Proceedings of the International Conference on Machine Learning, Honolulu, HI, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Reimers, N., and Gurevych, I. (2019, January 3\u20137). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Hong Kong, China.","DOI":"10.18653\/v1\/D19-1410"},{"key":"ref_40","unstructured":"Procopio, N. (2025, December 02). sentence-bert-base-italian-xxl-cased. Available online: https:\/\/huggingface.co\/nickprock\/sentence-bert-base-italian-xxl-uncased."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1006\/jcss.1997.1504","article-title":"A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting","volume":"55","author":"Freund","year":"1997","journal-title":"J. Comput. Syst. Sci."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA. KDD \u201816\u2019.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Bussolan, A., Baraldo, S., Gambardella, L.M., and Valente, A. (2024, January 26\u201330). Multimodal fusion stress detector for enhanced human-robot collaboration in industrial assembly tasks. Proceedings of the 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN), Pasadena, CA, USA.","DOI":"10.1109\/RO-MAN60168.2024.10731373"}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/14\/12\/184\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T12:49:51Z","timestamp":1764938991000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/14\/12\/184"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,5]]},"references-count":44,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["robotics14120184"],"URL":"https:\/\/doi.org\/10.3390\/robotics14120184","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,5]]}}}