{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2023,9,13]],"date-time":"2023-09-13T19:45:23Z","timestamp":1694634323167},"reference-count":28,"publisher":"Cambridge University Press (CUP)","issue":"1","license":[{"start":{"date-parts":[[2009,1,1]],"date-time":"2009-01-01T00:00:00Z","timestamp":1230768000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2009,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>One of the basic topics of question answering (QA) dialogue systems is how follow-up questions should be interpreted by a QA system. In this paper, we shall discuss our experience with the IMIX and Ritel systems, for both of which a follow-up question handling scheme has been developed, and corpora have been collected. These two systems are each other's opposites in many respects: IMIX is multimodal, non-factoid, black-box QA, while Ritel is speech, factoid, keyword-based QA. Nevertheless, we will show that they are quite comparable, and that it is fruitful to examine the similarities and differences. We shall look at how the systems are composed, and how real, non-expert, users interact with the systems. We shall also provide comparisons with systems from the literature where possible, and indicate where open issues lie and in what areas existing systems may be improved. We conclude that most systems have a common architecture with a set of common subtasks, in particular detecting follow-up questions and finding referents for them. We characterise these tasks using the typical techniques used for performing them, and data from our corpora. We also identify a special type of follow-up question, the discourse question, which is asked when the user is trying to understand an answer, and propose some basic methods for handling it.<\/jats:p>","DOI":"10.1017\/s1351324908004920","type":"journal-article","created":{"date-parts":[[2008,10,24]],"date-time":"2008-10-24T10:50:56Z","timestamp":1224845456000},"page":"97-118","source":"Crossref","is-referenced-by-count":8,"title":["Follow-up question handling in the IMIX and Ritel systems: A comparative study"],"prefix":"10.1017","volume":"15","author":[{"given":"B. W.","family":"VAN SCHOOTEN","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"R.","family":"OP DEN AKKER","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"S.","family":"ROSSET","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"O.","family":"GALIBERT","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"A.","family":"MAX","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"G.","family":"ILLOUZ","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2009,1,1]]},"reference":[{"key":"S1351324908004920_ref28","doi-asserted-by":"crossref","unstructured":"Yang F. , Feng J. , and Di Fabbrizio G. 2006. A data driven approach to relevancy recognition for contextual question answering. In Workshop on Interactive Question Answering, HLT-NAACL 06, New York, USA.","DOI":"10.3115\/1641579.1641584"},{"key":"S1351324908004920_ref27","unstructured":"Willems D. J. M. , Rossignol S. Y. P. , and Vuurpijl L. G. 2005. Features for mode detection in natural online pen input. In BIGS 2005: Proceedings of the Twelfth Biennial Conference of the International Graphonomics Society, Salerno, Italy, pp. 113\u2013117, Civitella: Editrice Zona."},{"key":"S1351324908004920_ref26","doi-asserted-by":"crossref","unstructured":"Voorhees E. 2005. Overview of the TREC 2005 question answering track. Technical report, NIST.","DOI":"10.1145\/1147197.1147205"},{"key":"S1351324908004920_ref25","unstructured":"van Schooten B. , Rosset S. , Galibert O. , Max A. , op den Akker R. , and Illouz G. 2007. Handling speech input in the Ritel QA dialogue system. In Interspeech 2007, pp. 181\u2013206."},{"key":"S1351324908004920_ref23","unstructured":"van Schooten B. , and op den Akker R. 2005. Follow-up proceedings of ntcir-5 workshop meeting, December 6\u20139, 2005, Tokyo, Japanutterances in QA dialogue. Traitement Automatique des Langues, 46(3): 181\u2013206."},{"key":"S1351324908004920_ref22","unstructured":"Theune M. , Krahmer E. , van Schooten B. , op den Akker R. , van Hooijdonk C. , Marsi E. , Bosma W. , Hofs D. , and Nijholt A. 2007. Questions, pictures, answers: introducing pictures in question-answering systems. In Tenth international symposium on social communication, Universidad de Oriente Santiago de Cuba, pp. 469\u2013474."},{"key":"S1351324908004920_ref19","unstructured":"Rosset S. , and Petel S. 2006. The Ritel corpus \u2013 an annotated human\u2013machine open-domain question answering spoken dialog corpus. In International Conference on Language Resources and Evaluation, Genoa, Italy."},{"key":"S1351324908004920_ref15","doi-asserted-by":"publisher","DOI":"10.1016\/j.sigpro.2006.02.040"},{"key":"S1351324908004920_ref13","volume-title":"TREC 10","author":"Lin","year":"2001"},{"key":"S1351324908004920_ref2","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2005.1521737"},{"key":"S1351324908004920_ref16","doi-asserted-by":"crossref","unstructured":"Moore J. D. 1989. Responding to \u2018Huh?\u2019: answering vaguely-articulated follow-up questions. In Proceedings of the Conference on Human Factors in Computing Systems, Austin, TX, USA.","DOI":"10.1145\/67449.67469"},{"key":"S1351324908004920_ref24","unstructured":"van Schooten B. , and op den Akker R. 2007. Multimodal follow-up questions to multimodal answers in a QA system. In Tenth International Symposium on Social Communication, Universidad de Oriente Santiago de Cuba, pp. 469\u2013474."},{"key":"S1351324908004920_ref7","doi-asserted-by":"crossref","first-page":"909","DOI":"10.21437\/Interspeech.2005-405","volume-title":"Interspeech 2005","author":"Galibert","year":"2005"},{"key":"S1351324908004920_ref21","doi-asserted-by":"crossref","unstructured":"Small S. , Liu T. , Shimizu N. , and Strzalkowski T. 2003. HITIQA: an interactive question answering system: a preliminary report. In Proceedings of the ACL 2003 Workshop on Multilingual Summarization and Question Answering, Sapporo, Japan.","DOI":"10.3115\/1119312.1119318"},{"key":"S1351324908004920_ref20","unstructured":"Russell B. C. , Torralba A. , Murphy K. P. , and Freeman W. T. 2005. LabelMe: a database and web-based tool for image annotation. Technical report, MIT, MIT AI Lab Memo AIM-2005-025."},{"key":"S1351324908004920_ref12","first-page":"535","article-title":"An algorithm for pronominal anaphora resolution","volume":"20","author":"Lappin","year":"1994","journal-title":"Computational Linguistics"},{"key":"S1351324908004920_ref18","doi-asserted-by":"crossref","unstructured":"Reithinger N. , Bergweiler S. , Engel R. , Herzog G. , Pfleger N. , Romanelli M. , and Sonntag D. 2005. A look under the hood: design and development of the first smartweb system demonstrator. In ICMI '05: Proceedings of the Seventh International Conference on Multimodal Interfaces, pp. 159\u2013166. New York, NY: ACM Press.","DOI":"10.1145\/1088463.1088492"},{"key":"S1351324908004920_ref14","doi-asserted-by":"crossref","unstructured":"Lin J. , Quan D. , Sinha V. , Bakshi K. , Huynh D. , Katz B. , and Karger D. R. 2003. What makes a good answer? The role of context in question answering. In Proceedings of the Ninth IFIP TC13 International Conference on Human\u2013Computer Interaction (INTERACT-2003), Zurich, Switzerland.","DOI":"10.1145\/766117.766119"},{"key":"S1351324908004920_ref10","unstructured":"Kato T. , Fukumoto J. , and Masui F. 2004. Question answering challenge for information access dialogue \u2013 overview of NTCIR4 QAC2 subtask 3. In Working notes of the Fourth NTCIR Workshop Meeting, Tokyo, Japan."},{"key":"S1351324908004920_ref4","volume-title":"Human\u2013Computer Interaction","author":"Dix","year":"2004"},{"key":"S1351324908004920_ref9","unstructured":"Inui K. , Yamashita A. , and Matsumoto Y. 2003. Dialogue management for language-based information seeking. In Proceedings of the First International Workshop on Language Understanding and Agents for Real World Interaction, Sapporo, Japan, pp. 32\u201338."},{"key":"S1351324908004920_ref11","doi-asserted-by":"crossref","unstructured":"Kato T. , Fukumoto J. , Masui F. , and Kando N. 2006. Woz simulation of interactive question answering. In Workshop on Interactive Question Answering, HLT-NAACL 06, New York, USA.","DOI":"10.3115\/1641579.1641581"},{"key":"S1351324908004920_ref17","volume-title":"TREC","author":"Oh","year":"2001"},{"key":"S1351324908004920_ref1","doi-asserted-by":"crossref","unstructured":"Bertomeu N. , Uszkoreit H. , Frank A. , Krieger H.-U. , and J\u00f6rg B. 2006. Contextual phenomena and thematic relations in database qa dialogues: results from a wizard-of-oz experiment. In Workshop on Interactive Question Answering, HLT-NAACL 06, New York, USA.","DOI":"10.3115\/1641579.1641580"},{"key":"S1351324908004920_ref3","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324905003682"},{"key":"S1351324908004920_ref5","unstructured":"Fukumoto J. , Niwa T. , Itoigawa M. , and Matsuda M. 2004. RitsQA: list answer detection and context task with ellipses handling. In Working notes of the Fourth NTCIR Workshop Meeting, Tokyo, Japan, pp. 310\u2013314."},{"key":"S1351324908004920_ref6","unstructured":"Futrelle R. P. , and Rumshisky A. 2001. Discourse structure of text-graphics documents. In First International Symposium on Smart Graphics, New York, USA, pp. 31\u201338, Heidelberg: Springer."},{"key":"S1351324908004920_ref8","volume-title":"ACL 2006","author":"Hickl","year":"2006"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324908004920","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,19]],"date-time":"2021-09-19T09:33:51Z","timestamp":1632044031000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324908004920\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,1]]},"references-count":28,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2009,1]]}},"alternative-id":["S1351324908004920"],"URL":"https:\/\/doi.org\/10.1017\/s1351324908004920","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"value":"1351-3249","type":"print"},{"value":"1469-8110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2009,1]]}}}