{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T13:54:04Z","timestamp":1771336444530,"version":"3.50.1"},"reference-count":72,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,12,13]],"date-time":"2023-12-13T00:00:00Z","timestamp":1702425600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2023,12,31]]},"abstract":"<jats:p>As robots have become more pervasive in our everyday life, social aspects of robots have attracted researchers\u2019 attention. Because emotions play a crucial role in social interactions, research has been conducted on conveying emotions via speech. Our study sought to investigate the synchronization of multimodal interaction in human-robot interaction (HRI). We conducted a within-subjects exploratory study with 40 participants to investigate the effects of non-speech sounds (natural voice, synthesized voice, musical sound, and no sound) and basic emotions (anger, fear, happiness, sadness, and surprise) on user perception with emotional body gestures of an anthropomorphic robot (Pepper). While listening to a fairytale with the participant, a humanoid robot responded to the story with recorded emotional non-speech sounds and gestures. Participants showed significantly higher emotion recognition accuracy from the natural voice than from other sounds. The confusion matrix showed that happiness and sadness had the highest emotion recognition accuracy, which is in line with previous research. The natural voice also induced higher trust, naturalness, and preference compared to other sounds. Interestingly, the musical sound mostly showed lower perception ratings, even compared to no sound. Results are discussed with design guidelines for emotional cues from social robots and future research directions.<\/jats:p>","DOI":"10.1145\/3626185","type":"journal-article","created":{"date-parts":[[2023,10,17]],"date-time":"2023-10-17T21:32:04Z","timestamp":1697578324000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Robots\u2019 \u201cWoohoo\u201d and \u201cArgh\u201d Can Enhance Users\u2019 Emotional and Social Perceptions: An Exploratory Study on Non-lexical Vocalizations and Non-linguistic Sounds"],"prefix":"10.1145","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-8675-9935","authenticated-orcid":false,"given":"Xiaozhen","family":"Liu","sequence":"first","affiliation":[{"name":"Virginia Polytechnic Institute and State University, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7253-8868","authenticated-orcid":false,"given":"Jiayuan","family":"Dong","sequence":"additional","affiliation":[{"name":"Virginia Polytechnic Institute and State University, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2908-671X","authenticated-orcid":false,"given":"Myounghoon","family":"Jeon","sequence":"additional","affiliation":[{"name":"Virginia Polytechnic Institute and State University, USA"}]}],"member":"320","published-online":{"date-parts":[[2023,12,13]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2015.1093856"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1162\/pres.16.4.337"},{"key":"e_1_3_1_4_2","volume-title":"Proceedings of the Human Factors and Ergonomics Society's 2022 International Annual Meeting (HFES \u201822)","author":"Dong J.","year":"2022","unstructured":"J. Dong, A. Santiago-Anaya, and M. Jeon. 2022. Facial expressions increase emotion recognition accuracy and clarity on a humanoid robot without adding the uncanny valley. In Proceedings of the Human Factors and Ergonomics Society's 2022 International Annual Meeting (HFES \u201822)."},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/HRI.2013.6483602"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2013.6630793"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3319502.3374814"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2018.2874986"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAMD.2012.2208962"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2012.6343883"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2017.01.001"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1196\/annals.1280.012"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/RO-MAN46459.2019.8956386"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2013.00353"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/2070481.2070545"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2006.2004"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2022.102802"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jesp.2014.01.005"},{"key":"e_1_3_1_19_2","author":"Lohse M.","unstructured":"M. Lohse, F. Hegel, A. Swadzba, K. Rohlfing, S. Wachsmuth, and B. Wrede. 2007. What can I do for you? Appearance and application of robots. In Workshop on The Reign of Catz and Dogz? The Role of Virtual Creatures in a Computerised Society. 121\u2013126.","journal-title":"Workshop on The Reign of Catz and Dogz? The Role of Virtual Creatures in a Computerised Society"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12193-009-0033-5"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2011.6005285"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(02)00373-1"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10846-015-0259-2"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2009.5326255"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2010.1"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2016.2536360"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2007.4415161"},{"key":"e_1_3_1_28_2","first-page":"469","volume-title":"Proceedings of the 15th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN \u201806)","author":"Blow M. P.","year":"2006","unstructured":"M. P. Blow, K. Dautenhahn, A. Appleby, C. L. Nehaniv, and D. Lee. 2006. Perception of robot smiles and dimensions for human\u2013robot interaction design. In Proceedings of the 15th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN \u201806). 469\u2013474."},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/2559636.2559680"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-015-0304-0"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2017.2774815"},{"key":"e_1_3_1_32_2","first-page":"325","volume-title":"SP \u201904","author":"Ward N.","year":"2004","unstructured":"N. Ward. 2004. Pragmatic functions of prosodic features in non-lexical utterances. In SP \u201904. 325\u2013328."},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.129.5.770"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2013.00105"},{"key":"e_1_3_1_35_2","volume-title":"Proceedings of the Workshop on Affective Social Speech Signals","author":"Yilmazyildiz S.","year":"2013","unstructured":"S. Yilmazyildiz, D. Henderickx, B. Vanderborght, W. Verhelst, E. Soetens, and D. Lefeber. 2013. Multi-modal emotion expression for affective human\u2013robot interaction. In Proceedings of the Workshop on Affective Social Speech Signals."},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1097\/WNR.0b013e32834ea757"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-6393(02)00078-X"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10919-017-0253-4"},{"key":"e_1_3_1_39_2","author":"Frid E.","unstructured":"E. Frid and R. Bresin 2022. Perceptual evaluation of blended sonification of mechanical robot sounds produced by emotionally expressive gestures: Augmenting consequential sounds to improve non-verbal robot communication. International Journal of Social Robotics, 14, 2 (2022), 357\u2013372.","journal-title":"International Journal of Social Robotics"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/RO-MAN47096.2020.9223452"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2005.03.007"},{"key":"e_1_3_1_42_2","first-page":"637","article-title":"Composition of musical sound expressing an emotion of robot based on musical factors","author":"Jee E.-S.","year":"2007","unstructured":"E.-S. Jee, C. H. Kim, S.-Y. Park, and K.-W. Lee. 2007. Composition of musical sound expressing an emotion of robot based on musical factors. In Proceedings of the 16th International Symposium on Robot and Human Interactive Communication. 637\u2013641.","journal-title":"Proceedings of the 16th International Symposium on Robot and Human Interactive Communication"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753619"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-21605-3_57"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-60117-1_13"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/1877826.1877843"},{"key":"e_1_3_1_47_2","first-page":"219","volume-title":"Proceedings of the 7th International Conference on Human\u2013robot Interaction (HRI \u201912)","author":"Read R.","year":"2012","unstructured":"R. Read and T. Belpaeme. 2012. How to use non-linguistic utterances to convey emotion in child\u2013robot interaction. In Proceedings of the 7th International Conference on Human\u2013robot Interaction (HRI \u201912). ACM\/IEEE, 219\u2013220."},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2017.2658726"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.79.6.941"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1037\/amp0000399"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3344274"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1207\/s15327051hci0401_1"},{"key":"e_1_3_1_53_2","first-page":"600","volume-title":"Proceedings of the 6th International Conference of Cognitive Science (ICCS \u201808)","author":"Jeon M.","year":"2008","unstructured":"M. Jeon, U. Heo, J. H. Ahn, and J. Kim. 2008. Emotional palette: Affective user experience elements for product design according to user segmentation. In Proceedings of the 6th International Conference of Cognitive Science (ICCS \u201808). 600\u2013603."},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-07230-2_60"},{"key":"e_1_3_1_55_2","volume-title":"Cultural Differences in Preference of Auditory Emoticons: USA and South Korea","author":"Jeon M.","year":"2015","unstructured":"M. Jeon, J. H. Lee, J. Sterkenburg, and C. Plummer. 2015. Cultural Differences in Preference of Auditory Emoticons: USA and South Korea. Georgia Institute of Technology."},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3371382.3378377"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.99.3.550"},{"key":"e_1_3_1_58_2","first-page":"106","article-title":"Ten common misunderstandings, misconceptions, persistent myths and urban legends about Likert scales and Likert response formats and their antidotes","volume":"2","author":"Carifio J.","year":"2007","unstructured":"J. Carifio and R. Perla. 2007. Ten common misunderstandings, misconceptions, persistent myths and urban legends about Likert scales and Likert response formats and their antidotes. Journal of Social Sciences 2 (2007), 106\u2013116.","journal-title":"Journal of Social Sciences"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10459-010-9222-y"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3171221.3171261"},{"key":"e_1_3_1_61_2","author":"Latupeirissa A. B.","unstructured":"A. B. Latupeirissa, C. Panariello, and R. Bresin. 2020. Exploring emotion perception in sonic HRI. In Sound and Music Computing Conference, Torino, Zenodo, 434\u2013441.","journal-title":"Sound and Music Computing Conference"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.1999.812787"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICHR.2004.1442120"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1013215010749"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0033031"},{"key":"e_1_3_1_66_2","volume-title":"Interaction Design: Beyond Human-computer Interaction","author":"Jennifer P.","year":"2002","unstructured":"P. Jennifer, R. Yvonne, and S. Helen. 2002. Interaction Design: Beyond Human-computer Interaction. Wiley, New York."},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2008.4650714"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15760-8_74"},{"key":"e_1_3_1_69_2","author":"Ko S.","unstructured":"S. Ko, J. Barnes, J. Dong, C. H. Park, A. Howard, and M. Jeon. 2023. The effects of robot voices and appearances on users' emotion recognition and subjective perception. International Journal of Humanoid Robotics, 20, 1 (2023), 2350001.","journal-title":"International Journal of Humanoid Robotics"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-008-0001-3"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/2909824.3020208"},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1177\/1541931215591094"},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3250278"}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626185","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3626185","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:57:03Z","timestamp":1750291023000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626185"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,13]]},"references-count":72,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12,31]]}},"alternative-id":["10.1145\/3626185"],"URL":"https:\/\/doi.org\/10.1145\/3626185","relation":{},"ISSN":["2573-9522"],"issn-type":[{"value":"2573-9522","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,13]]},"assertion":[{"value":"2022-05-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-08-31","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}