{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T03:52:53Z","timestamp":1727063573921},"reference-count":18,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2014,1,13]],"date-time":"2014-01-13T00:00:00Z","timestamp":1389571200000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/2.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J AUDIO SPEECH MUSIC PROC."],"published-print":{"date-parts":[[2014,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We propose an integrative method of recognizing gestures such as pointing, accompanying speech. Speech generated simultaneously with gestures can assist in the recognition of gestures, and since this occurs in a complementary manner, gestures can also assist in the recognition of speech. Our integrative recognition method uses a probability distribution which expresses the distribution of the time interval between the starting times of gestures and of the corresponding utterances. We evaluate the rate of improvement of the proposed integrative recognition method with a task involving the solution of a geometry problem.<\/jats:p>","DOI":"10.1186\/1687-4722-2014-2","type":"journal-article","created":{"date-parts":[[2014,1,13]],"date-time":"2014-01-13T02:06:13Z","timestamp":1389578773000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Improvement of multimodal gesture and speech recognition performance using time intervals between gestures and accompanying speech"],"prefix":"10.1186","volume":"2014","author":[{"given":"Madoka","family":"Miki","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Norihide","family":"Kitaoka","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chiyomi","family":"Miyajima","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Takanori","family":"Nishino","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kazuya","family":"Takeda","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2014,1,13]]},"reference":[{"key":"97_CR1","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1145\/1027933.1027957","volume-title":"Proceedings of ICMI","author":"S Oviatt","year":"2004","unstructured":"Oviatt S, Coulston R, Lundsford R: When do we interact multimodally? Cognitive load and multimodal communication patterns. In Proceedings of ICMI. New York: ACM; 2004:129-136."},{"key":"97_CR2","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1145\/2305484.2305490","volume-title":"Proceedings of 4th ACM SIGCHI Symposium on Engineering Interactive Computing Systems","author":"B Dumas","year":"2012","unstructured":"Dumas B, Signer B, Lalanne D: Fusion in multimodal interactive systems: an HMM-Based algorithm for user-induced adaptation. In Proceedings of 4th ACM SIGCHI Symposium on Engineering Interactive Computing Systems. New York: ACM,; 2012:15-24."},{"key":"97_CR3","first-page":"161","volume-title":"Proceedings of ICME","author":"S Kettebekov","year":"2002","unstructured":"Kettebekov S, Yeasin M, Sharma R: Prosody based co-analysis for continuous recognition of co-verbal gestures. In Proceedings of ICME. Washington DC: IEEE Computer Society,; 2002:161-166."},{"issue":"5","key":"97_CR4","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1016\/0097-8493(94)90157-0","volume":"18","author":"M Fukumoto","year":"1994","unstructured":"Fukumoto M, Suenaga Y, Mase K: Finger-pointer: pointing interface by image processing. ACM Comput. Graph 1994, 18(5):633-642. 10.1016\/0097-8493(94)90157-0","journal-title":"ACM Comput. Graph"},{"issue":"3","key":"97_CR5","doi-asserted-by":"publisher","first-page":"262","DOI":"10.1145\/965105.807503","volume":"14","author":"RA Bolt","year":"1980","unstructured":"Bolt RA: Put-that-there: voice and gesture at the graphics interface. ACM Comput. Graph 1980, 14(3):262-270. 10.1145\/965105.807503","journal-title":"ACM Comput. Graph"},{"key":"97_CR6","first-page":"1197","volume-title":"INTERSPEECH","author":"P Hui","year":"2006","unstructured":"Hui P, Meng H: Joint interpretation of input speech and pen gestures for multimodal human computer interaction. In INTERSPEECH. Pittsburgh: ISCA,; 2006:1197-1200."},{"key":"97_CR7","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1145\/1180995.1181036","volume-title":"Proceedings of ICMI","author":"S Qu","year":"2006","unstructured":"Qu S, Chai JY: Salience modeling based on non-verbal modalities for spoken language understanding. In Proceedings of ICMI. New York: ACM,; 2006:193-200."},{"key":"97_CR8","first-page":"349","volume-title":"Proceedings of ICMI","author":"N Krahnstoever","year":"2002","unstructured":"Krahnstoever N, Kettebekov S, Yeasin M, Sharma R: A real-time framework for natural multimodal interaction with large screen displays. In Proceedings of ICMI. Piscataway: IEEE; 2002:349-354."},{"key":"97_CR9","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1145\/1647314.1647343","volume-title":"Proceedings of ICMI-MLMI","author":"D Lalanne","year":"2009","unstructured":"Lalanne D, Nigay L, Palanque P, Robinson P, Vanderdonckt J, Ladry J: Fusion engines for multimodal input: a survey. In Proceedings of ICMI-MLMI. New York: ACM; 2009:153-160."},{"key":"97_CR10","first-page":"1","volume-title":"Proceedings of ACL","author":"J Chai","year":"2004","unstructured":"Chai J, Hong P, Zhou M, Prasov Z: Optimization in multimodal interpretation. In Proceedings of ACL. Stroudsburg: Association for Computational Linguistics; 2004:1-8."},{"key":"97_CR11","first-page":"369","volume-title":"Proceedings of COLING","author":"M Johnston","year":"2000","unstructured":"Johnston M: Finite-state multimodal parsing and understanding. In Proceedings of COLING. Stroudsburg: Association for Computational Linguistics; 2000:369-375."},{"key":"97_CR12","first-page":"624","volume-title":"Proceedings of COLING-ACL","author":"M Johnston","year":"1998","unstructured":"Johnston M: Unification-based multimodal parsing. In Proceedings of COLING-ACL. Stroudsburg: Association for Computational Linguistics; 1998:624-630."},{"issue":"4","key":"97_CR13","doi-asserted-by":"publisher","first-page":"334","DOI":"10.1109\/6046.807953","volume":"1","author":"L Wu","year":"1999","unstructured":"Wu L, Oviatt L, Cohen PR: Multimodal integration - a statistical view. Trans. Multimedia 1999, 1(4):334-341. 10.1109\/6046.807953","journal-title":"Trans. Multimedia"},{"key":"97_CR14","first-page":"943","volume-title":"Proceedings of ICSLP","author":"H Ban","year":"2004","unstructured":"Ban H, Miyajima C, Itou K, Takeda K, Itakura F: Speech recognition using synchronization between speech and finger tapping. In Proceedings of ICSLP. Pittsburgh: ISCA; 2004:943-946."},{"issue":"3","key":"97_CR15","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1016\/j.specom.2010.10.001","volume":"53","author":"K Shinoda","year":"2011","unstructured":"Shinoda K, Watanabe Y, Iwata K, Liang Y, Nakagawa R, Furui S: Semi-synchronous speech and pen input for mobile user interfaces. Speech Commun 2011, 53(3):283-291. 10.1016\/j.specom.2010.10.001","journal-title":"Speech Commun"},{"key":"97_CR16","first-page":"93","volume-title":"Proceedings of ICMI","author":"M Miki","year":"2008","unstructured":"Miki M, Miyajima C, Nishino T, Kitaoka N, Takeda K: An integrative recognition method for speech and gestures. In Proceedings of ICMI. New York: ACM; 2008:93-96."},{"key":"97_CR17","doi-asserted-by":"crossref","first-page":"1691","DOI":"10.21437\/Eurospeech.2001-396","volume-title":"Proceedings of EUROSPEECH","author":"A Lee","year":"2001","unstructured":"Lee A, Kawahara T, Shikano K: Julius \u2014 an open source real-time large vocabulary recognition engine. In Proceedings of EUROSPEECH. Aalborg: ISCA; 2001:1691-1694."},{"key":"97_CR18","first-page":"7","volume-title":"Proceedings of SSPR","author":"K Maekawa","year":"2003","unstructured":"Maekawa K: Corpus of spontaneous Japanese: its design and evaluation. In Proceedings of SSPR. Tokyo: ISCA and IEEE; 2003:7-12."}],"container-title":["EURASIP Journal on Audio, Speech, and Music Processing"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/1687-4722-2014-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/1687-4722-2014-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/1687-4722-2014-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,9]],"date-time":"2023-07-09T19:26:34Z","timestamp":1688930794000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmp-eurasipjournals.springeropen.com\/articles\/10.1186\/1687-4722-2014-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,1,13]]},"references-count":18,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2014,12]]}},"alternative-id":["97"],"URL":"https:\/\/doi.org\/10.1186\/1687-4722-2014-2","relation":{},"ISSN":["1687-4722"],"issn-type":[{"value":"1687-4722","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,1,13]]},"assertion":[{"value":"28 February 2013","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 January 2014","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 January 2014","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"2"}}