{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:31:05Z","timestamp":1750307465024,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":36,"publisher":"ACM","license":[{"start":{"date-parts":[[2010,10,16]],"date-time":"2010-10-16T00:00:00Z","timestamp":1287187200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["MO 1038\/6-1"],"award-info":[{"award-number":["MO 1038\/6-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2010,10,16]]},"DOI":"10.1145\/1868914.1868949","type":"proceedings-article","created":{"date-parts":[[2010,10,19]],"date-time":"2010-10-19T12:41:35Z","timestamp":1287492095000},"page":"286-294","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Evaluating multimodal systems"],"prefix":"10.1145","author":[{"given":"Christine","family":"K\u00fchnel","sequence":"first","affiliation":[{"name":"Technische Universit\u00e4t Berlin, Berlin, Germany"}]},{"given":"Tilo","family":"Westermann","sequence":"additional","affiliation":[{"name":"Universit\u00e4t Hamburg, Hamburg, Germany"}]},{"given":"Benjamin","family":"Weiss","sequence":"additional","affiliation":[{"name":"Technische Universit\u00e4t Berlin, Berlin, Germany"}]},{"given":"Sebastian","family":"M\u00f6ller","sequence":"additional","affiliation":[{"name":"Technische Universit\u00e4t Berlin, Berlin, Germany"}]}],"member":"320","published-online":{"date-parts":[[2010,10,16]]},"reference":[{"key":"e_1_3_2_1_1_1","first-page":"77","volume-title":"Proc. of the LREC Workshop on Multimodal Resources and Multimodal Systems Evaluation, Las Palmas","author":"Beringer N.","year":"2002","unstructured":"N. Beringer , U. Kartal , K. Louka , F. Schiel , and U. T\u00fcrk . PROMISE: -- A procedure for multimodal interactive system evaluation . In Proc. of the LREC Workshop on Multimodal Resources and Multimodal Systems Evaluation, Las Palmas , pages 77 -- 80 , 2002 . N. Beringer, U. Kartal, K. Louka, F. Schiel, and U. T\u00fcrk. PROMISE: -- A procedure for multimodal interactive system evaluation. In Proc. of the LREC Workshop on Multimodal Resources and Multimodal Systems Evaluation, Las Palmas, pages 77--80, 2002."},{"key":"e_1_3_2_1_2_1","volume-title":"Proc. of the UXEM Workshop at INTERACT","author":"Bevan N.","year":"2009","unstructured":"N. Bevan . What is the difference between the purpose of usability and user experience evaluation methods? In Proc. of the UXEM Workshop at INTERACT , 2009 . N. Bevan. What is the difference between the purpose of usability and user experience evaluation methods? In Proc. of the UXEM Workshop at INTERACT, 2009."},{"key":"e_1_3_2_1_3_1","first-page":"189","volume-title":"Usability evaluation in industry","author":"Brooke J.","year":"1996","unstructured":"J. Brooke . SUS: A quick and dirty usability scale . In P. W. Jordan, B. Weerdmeester, A. Thomas, and I. L. McClelland, editors, Usability evaluation in industry , pages 189 -- 194 . Taylor and Francis , London , 1996 . J. Brooke. SUS: A quick and dirty usability scale. In P. W. Jordan, B. Weerdmeester, A. Thomas, and I. L. McClelland, editors, Usability evaluation in industry, pages 189--194. Taylor and Francis, London, 1996."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/57167.57203"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/978-1-5041-2896-4_19","volume-title":"Human-Computer Interaction, Interact","author":"Coutaz J.","year":"1995","unstructured":"J. Coutaz , L. Nigay , D. Salber , A. Blandford , J. May , and R. Young . Four easy pieces for assessing the usability of multimodal interaction: The CARE properties . In Human-Computer Interaction, Interact , pages 115 -- 120 . Chapman & amp; Hall, London, 1995 . J. Coutaz, L. Nigay, D. Salber, A. Blandford, J. May, and R. Young. Four easy pieces for assessing the usability of multimodal interaction: The CARE properties. In Human-Computer Interaction, Interact, pages 115--120. Chapman & Hall, London, 1995."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-00437-7_1"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/332040.332455"},{"key":"e_1_3_2_1_8_1","first-page":"187","volume-title":"Interaktion in Bewegung","author":"Hassenzahl M.","year":"2003","unstructured":"M. Hassenzahl , M. Burmester , and F. Koller . AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualit\u00e4t {a questionnaire for measuring perceived hedonic and pracmatic quality}. Mensch & Computer 2003 . Interaktion in Bewegung , pages 187 -- 196 , 2003 . M. Hassenzahl, M. Burmester, and F. Koller. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualit\u00e4t {a questionnaire for measuring perceived hedonic and pracmatic quality}. Mensch & Computer 2003. Interaktion in Bewegung, pages 187--196, 2003."},{"key":"e_1_3_2_1_9_1","first-page":"275","volume-title":"Proc. of the 6th international conference on Work With Display Units WWDU 2002","author":"Hassenzahl M.","year":"2002","unstructured":"M. Hassenzahl , R. Kekez , and M. Burmester . The importance of a software's pragmatic quality depends on usage modes. In H. Luczak, A. E. Cakir, and G. Cakir, editors , Proc. of the 6th international conference on Work With Display Units WWDU 2002 , pages 275 -- 276 , Berlin , 2002 . ERGONOMIC Institut f\u00fcr Arbeits- und Sozialforschung. M. Hassenzahl, R. Kekez, and M. Burmester. The importance of a software's pragmatic quality depends on usage modes. In H. Luczak, A. E. Cakir, and G. Cakir, editors, Proc. of the 6th international conference on Work With Display Units WWDU 2002, pages 275--276, Berlin, 2002. ERGONOMIC Institut f\u00fcr Arbeits- und Sozialforschung."},{"key":"e_1_3_2_1_10_1","volume-title":"The inference of perceived usability from beauty. Human-Computer Interaction (accepted)","author":"Hassenzahl M.","year":"2010","unstructured":"M. Hassenzahl and A. Monk . The inference of perceived usability from beauty. Human-Computer Interaction (accepted) , 2010 . M. Hassenzahl and A. Monk. The inference of perceived usability from beauty. Human-Computer Interaction (accepted), 2010."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324900002497"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240722"},{"key":"e_1_3_2_1_13_1","author":"Kirakowski J.","year":"1993","unstructured":"J. Kirakowski and M. Corbett . SUMI: The Software Usability Measurement Inventory. British Journal of Educational Technology, (3):210--212 , 1993 . J. Kirakowski and M. Corbett. SUMI: The Software Usability Measurement Inventory. British Journal of Educational Technology, (3):210--212, 1993.","journal-title":"SUMI: The Software Usability Measurement Inventory. British Journal of Educational Technology, (3):210--212"},{"key":"e_1_3_2_1_14_1","volume-title":"Parameters describing multimodal interaction --- definitions and three usage scenarios. In accepted for Interspeech","author":"K\u00fchnel C.","year":"2010","unstructured":"C. K\u00fchnel , B. Weiss , and S. M\u00f6ller . Parameters describing multimodal interaction --- definitions and three usage scenarios. In accepted for Interspeech 2010 , 2010. C. K\u00fchnel, B. Weiss, and S. M\u00f6ller. Parameters describing multimodal interaction --- definitions and three usage scenarios. In accepted for Interspeech 2010, 2010."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518813"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447319509526110"},{"key":"e_1_3_2_1_17_1","volume-title":"Spoken, multilingual and multimodal dialogue systems: development and assessment","author":"L\u00f3pez-C\u00f3zar Delgado R.","year":"2005","unstructured":"R. L\u00f3pez-C\u00f3zar Delgado and M. Araki . Spoken, multilingual and multimodal dialogue systems: development and assessment . John Wiley & amp; Sons, Chichester, 2005 . R. L\u00f3pez-C\u00f3zar Delgado and M. Araki. Spoken, multilingual and multimodal dialogue systems: development and assessment. John Wiley & Sons, Chichester, 2005."},{"key":"e_1_3_2_1_18_1","volume-title":"Measuring usability with the USE questionnaire. Usability Interface, 8(2)","author":"Lund A. M.","year":"2001","unstructured":"A. M. Lund . Measuring usability with the USE questionnaire. Usability Interface, 8(2) , 2001 . A. M. Lund. Measuring usability with the USE questionnaire. Usability Interface, 8(2), 2001."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-374708-2.00014-0"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2008.03.001"},{"key":"e_1_3_2_1_21_1","first-page":"1603","volume-title":"Proc. 4th Int. Conf. on Language Resources and Evaluation (LREC)","volume":"5","author":"M\u00f6ller S.","year":"2004","unstructured":"S. M\u00f6ller , J. Krebber , A. Raake , P. Smeele , M. Rajman , M. Melichar , V. Pallotta , G. Tsakou , B. Kladis , A. Vovos , J. Hoonhout , D. Schuchardt , N. Fakotakis , T. Ganchev , and I. Potamitis . INSPIRE: Evaluation of a Smart-Home System for Infotainment Management and Device Control . In Proc. 4th Int. Conf. on Language Resources and Evaluation (LREC) , volume 5 , pages 1603 -- 1606 , May 2004 . S. M\u00f6ller, J. Krebber, A. Raake, P. Smeele, M. Rajman, M. Melichar, V. Pallotta, G. Tsakou, B. Kladis, A. Vovos, J. Hoonhout, D. Schuchardt, N. Fakotakis, T. Ganchev, and I. Potamitis. INSPIRE: Evaluation of a Smart-Home System for Infotainment Management and Device Control. In Proc. 4th Int. Conf. on Language Resources and Evaluation (LREC), volume 5, pages 1603--1606, May 2004."},{"key":"e_1_3_2_1_22_1","first-page":"8","volume-title":"Proc. of the International Workshop on Meaningful Measures: Valid Useful User Experience Measurement (VUUM)","author":"Naumann A.","year":"2008","unstructured":"A. Naumann and I. Wechsung . Developing usability methods for multimodal systems: The use of subjective and objective measures. In E. L.-C. Law, N. Bevan, G. Christou, M. Springett, and M. Larusdottir, editors , Proc. of the International Workshop on Meaningful Measures: Valid Useful User Experience Measurement (VUUM) , pages 8 -- 12 . Institute of Research in Informatics of Toulouse (IRIT) , 2008 . A. Naumann and I. Wechsung. Developing usability methods for multimodal systems: The use of subjective and objective measures. In E. L.-C. Law, N. Bevan, G. Christou, M. Springett, and M. Larusdottir, editors, Proc. of the International Workshop on Meaningful Measures: Valid Useful User Experience Measurement (VUUM), pages 8--12. Institute of Research in Informatics of Toulouse (IRIT), 2008."},{"key":"e_1_3_2_1_23_1","volume-title":"Proc. of the 23rd British Computer Society Human Computer Interaction Workshop and Conference (HCI 2009)","author":"Naumann A.","year":"2009","unstructured":"A. Naumann , I. Wechsung , and J. Hurtienne . Multimodality, inclusive design, and intuitive use . In Proc. of the 23rd British Computer Society Human Computer Interaction Workshop and Conference (HCI 2009) , 2009 . A. Naumann, I. Wechsung, and J. Hurtienne. Multimodality, inclusive design, and intuitive use. In Proc. of the 23rd British Computer Society Human Computer Interaction Workshop and Conference (HCI 2009), 2009."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/175276.175282"},{"key":"e_1_3_2_1_25_1","first-page":"409","volume-title":"Proc. of Gesture-Based Communication in Human-Computer Interaction, 5th International Gesture Workshop","author":"Nielsen M.","year":"2003","unstructured":"M. Nielsen , T. Moeslund , M. St\u00f6rring , and E. Granum . A procedure for developing intuitive and ergonomic gesture interfaces for HCI . In Proc. of Gesture-Based Communication in Human-Computer Interaction, 5th International Gesture Workshop , pages 409 -- 420 . Springer , 2003 . M. Nielsen, T. Moeslund, M. St\u00f6rring, and E. Granum. A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In Proc. of Gesture-Based Communication in Human-Computer Interaction, 5th International Gesture Workshop, pages 409--420. Springer, 2003."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/169059.169143"},{"key":"e_1_3_2_1_27_1","unstructured":"S. Oviatt. Multimodal interfaces. pages 286--304 2003.   S. Oviatt. Multimodal interfaces. pages 286--304 2003."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.3115\/1118078.1118092"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/962081.962106"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1054972.1055028"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/258549.258626"},{"volume-title":"Proc","author":"Tullis T.","key":"e_1_3_2_1_32_1","unstructured":"T. Tullis and J. Stetson . A comparison of questionnaires for assessing website usability . In Proc . of Usability Professionals Association (UPA) . T. Tullis and J. Stetson. A comparison of questionnaires for assessing website usability. In Proc. of Usability Professionals Association (UPA)."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"crossref","first-page":"2567","DOI":"10.21437\/Interspeech.2009-676","volume-title":"Proc. of Interspeech","author":"Turunen M.","year":"2009","unstructured":"M. Turunen , J. Hakulinen , A. Melto , T. Heimonen , T. Laivo , and J. Hella . Suxes -- user experience evaluation method for spoken and multimodal interaction . In Proc. of Interspeech , pages 2567 -- 2570 , 2009 . M. Turunen, J. Hakulinen, A. Melto, T. Heimonen, T. Laivo, and J. Hella. Suxes -- user experience evaluation method for spoken and multimodal interaction. In Proc. of Interspeech, pages 2567--2570, 2009."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.3115\/976909.979652"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-69369-7_32"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518866"}],"event":{"name":"NordiCHI '10: 6th Nordic Conference on Human-Computer Interaction","sponsor":["Reykjavik University","University of Iceland","SIGCHI ACM Special Interest Group on Computer-Human Interaction"],"location":"Reykjavik Iceland","acronym":"NordiCHI '10"},"container-title":["Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1868914.1868949","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1868914.1868949","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T12:08:30Z","timestamp":1750248510000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1868914.1868949"}},"subtitle":["a comparison of established questionnaires and interaction parameters"],"short-title":[],"issued":{"date-parts":[[2010,10,16]]},"references-count":36,"alternative-id":["10.1145\/1868914.1868949","10.1145\/1868914"],"URL":"https:\/\/doi.org\/10.1145\/1868914.1868949","relation":{},"subject":[],"published":{"date-parts":[[2010,10,16]]},"assertion":[{"value":"2010-10-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}