{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T09:55:56Z","timestamp":1780394156990,"version":"3.54.1"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2012,1,19]],"date-time":"2012-01-19T00:00:00Z","timestamp":1326931200000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/2.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J AUDIO SPEECH MUSIC PROC."],"published-print":{"date-parts":[[2012,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>This study proposes a music-aided framework for affective interaction of service robots with humans. The framework consists of three systems, respectively, for perception, memory, and expression on the basis of the human brain mechanism. We propose a novel approach to identify human emotions in the perception system. The conventional approaches use speech and facial expressions as representative bimodal indicators for emotion recognition. But, our approach uses the mood of music as a supplementary indicator to more correctly determine emotions along with speech and facial expressions. For multimodal emotion recognition, we propose an effective decision criterion using records of bimodal recognition results relevant to the musical mood. The memory and expression systems also utilize musical data to provide natural and affective reactions to human emotions. For evaluation of our approach, we simulated the proposed human-robot interaction with a service robot, iRobiQ. Our perception system exhibited superior performance over the conventional approach, and most human participants noted favorable reactions toward the music-aided affective interaction.<\/jats:p>","DOI":"10.1186\/1687-4722-2012-5","type":"journal-article","created":{"date-parts":[[2012,1,19]],"date-time":"2012-01-19T19:18:36Z","timestamp":1327000716000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Music-aided affective interaction between human and service robot"],"prefix":"10.1186","volume":"2012","author":[{"given":"Jeong-Sik","family":"Park","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gil-Jin","family":"Jang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yong-Ho","family":"Seo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2012,1,19]]},"reference":[{"key":"40_CR1","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1086\/209499","volume":"24","author":"M Richins","year":"1997","unstructured":"Richins M: Measuring emotions in the consumption experience. J Consum Res 1997, 24: 127-146. 10.1086\/209499","journal-title":"J Consum Res"},{"key":"40_CR2","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1109\/79.911197","volume":"18","author":"R Cowie","year":"2001","unstructured":"Cowie R, Cowie E, Tsapatsoulis N, Votsis G, Kollias S, Fellenz W, Taylor J: Emotion recognition in human-computer interaction. IEEE Signal Process Mag 2001, 18: 32-80. 10.1109\/79.911197","journal-title":"IEEE Signal Process Mag"},{"issue":"4","key":"40_CR3","doi-asserted-by":"publisher","first-page":"603","DOI":"10.1016\/S0167-6393(03)00099-2","volume":"41","author":"T Nwe","year":"2003","unstructured":"Nwe T, Foo S, Silva L: Speech emotion recognition using hidden Markov models. Speech Commun 2003, 41(4):603-623. 10.1016\/S0167-6393(03)00099-2","journal-title":"Speech Commun"},{"key":"40_CR4","first-page":"174","volume-title":"Proc Conf Image Video Retrieval, Xi'an, China","author":"M Paleari","year":"2010","unstructured":"Paleari M, Huet B, Chellali R: Towards multimodal emotion recognition: a new approach. Proc Conf Image Video Retrieval, Xi'an, China 2010, 174-181."},{"key":"40_CR5","doi-asserted-by":"publisher","first-page":"332","DOI":"10.1109\/AFGR.2000.840655","volume-title":"Proc of Fourth IEEE Int Conf Automatic Face Gesture Recog","author":"LC De Silva","year":"2000","unstructured":"De Silva LC, Ng PC: Bimodal emotion recognition. In Proc of Fourth IEEE Int Conf Automatic Face Gesture Recog. Grenoble, France; 2000:332-335."},{"key":"40_CR6","first-page":"1971","volume-title":"Proc of Interspeech","author":"LM Ignacio","year":"2009","unstructured":"Ignacio LM, Carlos OR, Joaquin GR, Daniel R: Speaker dependent emotion recognition using prosodic supervectors. In Proc of Interspeech. Brighton, UK; 2009:1971-1974."},{"key":"40_CR7","volume-title":"Music as the Language of Emotion: a Lecture delivered in the Whittall Pavilion of the Library of Congress","author":"CC Pratt","year":"1952","unstructured":"Pratt CC: Music as the Language of Emotion: a Lecture delivered in the Whittall Pavilion of the Library of Congress. US Govt. Print. Off., Washington; 1952."},{"key":"40_CR8","first-page":"149","volume":"2001-2002","author":"KR Scherer","year":"2002","unstructured":"Scherer KR, Zentner MR: A Schacht, Emotional states generated by music: an exploratory study of music experts. Musicae Scientiae 2002, 2001-2002: 149-171.","journal-title":"Musicae Scientiae"},{"key":"40_CR9","first-page":"477","volume":"2003","author":"A Makiko","year":"2003","unstructured":"Makiko A, Toshie N, Satoshi K, Chika N, Tomotsugu K: Psychological research on emotions in strong experiences with music. Human Interface 2003, 2003: 477-480.","journal-title":"Human Interface"},{"issue":"2","key":"40_CR10","first-page":"77","volume":"2002","author":"A Gabrielsson","year":"2002","unstructured":"Gabrielsson A: Some reflections on links between music psychology and music education. Res Higher Music Educ 2002, 2002(2):77-86.","journal-title":"Res Higher Music Educ"},{"key":"40_CR11","first-page":"130","volume-title":"Proc Human Comp Conf","author":"C Bartneck","year":"2001","unstructured":"Bartneck C, Okada M: Robotic user interfaces. In Proc Human Comp Conf. Aizu-Wakamatsu, Japan; 2001:130-140."},{"key":"40_CR12","first-page":"1146","volume-title":"Proc Sixteenth Int Joint Conf Art Intel","author":"C Breazeal","year":"1999","unstructured":"Breazeal C, Scassellati B: A context-dependent attention system for a Social Robot. In Proc Sixteenth Int Joint Conf Art Intel. Stockholm, Sweden; 1999:1146-1151."},{"key":"40_CR13","doi-asserted-by":"publisher","first-page":"191","DOI":"10.1016\/S0921-8890(02)00375-5","volume":"42","author":"RC Arkin","year":"2003","unstructured":"Arkin RC, Fujita M, Takagi T, Hasegawa R: An ethological and emotional basis for human-robot interaction. Robot Autonomous Syst 2003, 42: 191-201. 10.1016\/S0921-8890(02)00375-5","journal-title":"Robot Autonomous Syst"},{"key":"40_CR14","doi-asserted-by":"publisher","first-page":"140","DOI":"10.1016\/j.artint.2005.03.005","volume":"166","author":"CL Sidner","year":"2005","unstructured":"Sidner CL, Lee C, Kidds CD, Lesh N, Rich C: Explorations in engagement for humans and robots. Artif Intell 2005, 166: 140-164. 10.1016\/j.artint.2005.03.005","journal-title":"Artif Intell"},{"key":"40_CR15","first-page":"2868","volume-title":"Proc Int Conf Robotics Automation","author":"T Shibata","year":"2000","unstructured":"Shibata T, Tashima T, Tanie K: Emergence of emotional behavior through physical interaction between human and artificial emotional creatures. In Proc Int Conf Robotics Automation. San Francisco, USA; 2000:2868-2873."},{"key":"40_CR16","first-page":"12","volume-title":"Proc Int Conf Multi Comp Syst","author":"N Tosa","year":"1996","unstructured":"Tosa N, Nakatsu R: Life-like communication agent-emotion sensing character \"MIC\" & feeling session character \"MUSE\". In Proc Int Conf Multi Comp Syst. Hiroshima, Japan; 1996:12-19."},{"key":"40_CR17","first-page":"439","volume-title":"Proc IEEE Int Workshop Multi Signal Process","author":"R Nakatsu","year":"1999","unstructured":"Nakatsu R, Nicholson J, Tosa N: Emotion recognition and its application to computer agents with spontaneous interactive capabilities. In Proc IEEE Int Workshop Multi Signal Process. Copenhagen, Denmark; 1999:439-444."},{"key":"40_CR18","volume-title":"The Emotional Brain: The Mysterious Underpinning of Emotional Life","author":"J Ledoux","year":"1996","unstructured":"Ledoux J: The Emotional Brain: The Mysterious Underpinning of Emotional Life. Simon & Schuster, New York; 1996."},{"key":"40_CR19","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1145\/1743384.1743431","volume-title":"Proc ACM SIGMM Int Conf Multimedia Info Retrieval","author":"EM Schmidt","year":"2010","unstructured":"Schmidt EM, Turnbull D, Kim YE: Feature selection for content-based, time-varying musical emotion regression. In Proc ACM SIGMM Int Conf Multimedia Info Retrieval. Philadelphia, USA; 2010:267-274."},{"key":"40_CR20","first-page":"465","volume-title":"Proc Int Soc Music Inform Retrieval Conf","author":"EM Schmidt","year":"2010","unstructured":"Schmidt EM, Kim YE: Prediction of time-varying musical mood distributions from audio. In Proc Int Soc Music Inform Retrieval Conf. Utrecht, Netherlands; 2010:465-470."},{"issue":"2","key":"40_CR21","doi-asserted-by":"publisher","first-page":"448","DOI":"10.1109\/TASL.2007.911513","volume":"16","author":"YH Yang","year":"2008","unstructured":"Yang YH, Lin YC, Su YF, Chen HH: A regression approach to music emotion recognition. IEEE Trans Audio Speech Lang Process 2008, 16(2):448-457.","journal-title":"IEEE Trans Audio Speech Lang Process"},{"key":"40_CR22","first-page":"255","volume-title":"Proc Int Soc Music Inform Retrieval Conf","author":"YE Kim","year":"2010","unstructured":"Kim YE, Schmidt EM, Migneco R, Morton BG, Richardson P, Scott J, Speck JA, Turnbull D: Music emotion recognition: a state of the art review. In Proc Int Soc Music Inform Retrieval Conf. Utrecht, Netherlands; 2010:255-266."},{"key":"40_CR23","volume-title":"Ph.D. dissertation, Technical University, Denmark","author":"P Ahrendt","year":"2006","unstructured":"Ahrendt P: Music genre classification systems--a computational approach. Ph.D. dissertation, Technical University, Denmark 2006."},{"issue":"3","key":"40_CR24","doi-asserted-by":"publisher","first-page":"1590","DOI":"10.1109\/TCE.2009.5278031","volume":"55","author":"JS Park","year":"2009","unstructured":"Park JS, Kim JH, Oh YH: Feature vector classification based speech emotion recognition for service robots. IEEE Trans Consum Electron V 2009, 55(3):1590-1596.","journal-title":"IEEE Trans Consum Electron V"},{"key":"40_CR25","first-page":"599","volume-title":"Proc Int Conf Elect Control Eng","author":"X Yang","year":"2010","unstructured":"Yang X, Tan B, Ding J, Zhang J, Gong J: Comparative study on voice activity detection algorithm. In Proc Int Conf Elect Control Eng. Wuhan, China; 2010:599-602."},{"key":"40_CR26","doi-asserted-by":"crossref","first-page":"125","DOI":"10.21437\/Eurospeech.2003-80","volume-title":"Proc Eurospeech","author":"O Kwon","year":"2003","unstructured":"Kwon O, Chan K, Hao J, Lee T: Emotion recognition by speech signals. In Proc Eurospeech. Geneva, Switzerland; 2003:125-128."},{"key":"40_CR27","first-page":"1204","volume-title":"Proc Int Conf Pattern Recog","author":"R Huang","year":"2006","unstructured":"Huang R, Ma C: Toward a speaker-independent real time affect detection system. In Proc Int Conf Pattern Recog. Hong Kong, China; 2006:1204-1207."},{"key":"40_CR28","volume-title":"Facial Action Coding System: Investigator's Guide","author":"P Ekman","year":"1978","unstructured":"Ekman P, Friesen WV: Facial Action Coding System: Investigator's Guide. Consulting Psychologists Press, Palo Alto; 1978."},{"key":"40_CR29","first-page":"1","volume-title":"Proc Int Conf Cog Neural Syst","author":"S Giripunje","year":"2009","unstructured":"Giripunje S, Bajaj P, Abraham A: Emotion recognition system using connectionist models. In Proc Int Conf Cog Neural Syst. Boston, USA; 2009:1-2."},{"key":"40_CR30","first-page":"628","volume-title":"Proc Int Symposium Image Signal Process. Anal","author":"L Franco","year":"2001","unstructured":"Franco L, Treves A: A neural network facial expression recognition system using unsupervised local processing. In Proc Int Symposium Image Signal Process. Anal. Pula, Croatia; 2001:628-632."},{"issue":"2","key":"40_CR31","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1007\/BF03025294","volume":"20","author":"HA Rowley","year":"1998","unstructured":"Rowley HA, Baluja S, Kanade T: Neural network-based face detection. IEEE Trans. Pattern Anal. Mach. Intell 1998, 20(2):23-38. 10.1007\/BF03025294","journal-title":"Intell"},{"key":"40_CR32","first-page":"227","volume-title":"Proc Int Symposium Network. Network Security","author":"X Zhu","year":"2010","unstructured":"Zhu X: Emotion recognition of EMG based on BP neural network. In Proc Int Symposium Network. Network Security. Jinggangshan, China; 2010:227-229."},{"key":"40_CR33","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/5236.001.0001","volume-title":"Parallel Distributed Processing: Explorations in the Microstructure of Cognition","author":"DE Rumelhart","year":"1986","unstructured":"Rumelhart DE, McClelland JL: Parallel Distributed Processing: Explorations in the Microstructure of Cognition. MIT Press, Cambridge; 1986."},{"key":"40_CR34","first-page":"318","volume-title":"18th IEEE Int Symposium Robot Human Inter Comm","author":"J Han","year":"2009","unstructured":"Han J, Lee S, Hyun E, Kang B, Shin K: The birth story of robot, IROBIQ for children's tolerance. In 18th IEEE Int Symposium Robot Human Inter Comm. Toyama, Japan; 2009:318."},{"key":"40_CR35","first-page":"41","volume-title":"Proc Int Symposium Adv Robotics Machine Intell","author":"HG Lee","year":"2006","unstructured":"Lee HG, Baeg MH, Lee DW, Lee TG, Park HS: Development of an android for emotional communication between human and machine: EveR-2. In Proc Int Symposium Adv Robotics Machine Intell. Beijing, China; 2006:41-47."},{"issue":"9","key":"40_CR36","doi-asserted-by":"publisher","first-page":"1162","DOI":"10.1016\/j.specom.2006.04.003","volume":"48","author":"D Ververidis","year":"2006","unstructured":"Ververidis D, Kotropoulos C: Emotional speech recognition: resources, features, and methods. Speech Commun 2006, 48(9):1162-1181. 10.1016\/j.specom.2006.04.003","journal-title":"Speech Commun"},{"issue":"1","key":"40_CR37","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/S0167-6393(02)00070-5","volume":"40","author":"E Cowie","year":"2003","unstructured":"Cowie E, Campbell N, Cowie R: Roach P, Emotional speech: towards a new generation of databases. Speech Commun 2003, 40(1):33-60. 10.1016\/S0167-6393(02)00070-5","journal-title":"Speech Commun"},{"key":"40_CR38","first-page":"103","volume-title":"Proc of the 24th Symposium on Information Theory","author":"P Vanroose","year":"2003","unstructured":"Vanroose P: Blind source separation of speech and background music for improved speech recognition. In Proc of the 24th Symposium on Information Theory. Yokohama, Japan; 2003:103-108."}],"container-title":["EURASIP Journal on Audio, Speech, and Music Processing"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/1687-4722-2012-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/1687-4722-2012-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/1687-4722-2012-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,13]],"date-time":"2023-06-13T20:06:46Z","timestamp":1686686806000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmp-eurasipjournals.springeropen.com\/articles\/10.1186\/1687-4722-2012-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,1,19]]},"references-count":38,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2012,12]]}},"alternative-id":["40"],"URL":"https:\/\/doi.org\/10.1186\/1687-4722-2012-5","relation":{},"ISSN":["1687-4722"],"issn-type":[{"value":"1687-4722","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,1,19]]},"assertion":[{"value":"2 April 2011","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 January 2012","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 January 2012","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"5"}}