{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:42:05Z","timestamp":1750308125291,"version":"3.41.0"},"reference-count":22,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2005,9,1]],"date-time":"2005-09-01T00:00:00Z","timestamp":1125532800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Transactions on Asian Language Information Processing"],"published-print":{"date-parts":[[2005,9]]},"abstract":"<jats:p>There are strong expectations for the use of question answering technologies in information access dialogues, such as for information gathering and browsing. In this paper, we empirically examine what kinds of abilities are needed for question answering systems in such situations, and propose a challenge for evaluating those abilities objectively and quantitatively. We also show that existing technologies have the potential to address this challenge. From the empirical study, we found that questions that have values and names as answers account for a majority in realistic information-gathering situations and that those sequences of questions contain a wide range of reference expressions and are sometimes complicated by the inclusion of subdialogues and focus shifts. The challenge proposed is not only novel as an evaluation of the handling of information access dialogues, but also includes several valuable ideas such as categorization and characterization of information access dialogues, and introduces three measures to evaluate various aspects in addressing list-type questions and reference test sets for evaluating context-processing ability in isolation.<\/jats:p>","DOI":"10.1145\/1111667.1111669","type":"journal-article","created":{"date-parts":[[2006,2,6]],"date-time":"2006-02-06T15:07:09Z","timestamp":1139238429000},"page":"243-262","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Are open-domain question answering technologies useful for information access dialogues?---an empirical study and a proposal of a novel challenge"],"prefix":"10.1145","volume":"4","author":[{"given":"Tsuneaki","family":"Kato","sequence":"first","affiliation":[{"name":"The University of Tokyo, Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jun'ichi","family":"Fukumoto","sequence":"additional","affiliation":[{"name":"Ritsumeikan University, Shiga-ken, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fumito","family":"Masui","sequence":"additional","affiliation":[{"name":"Mie University, Mie-ken, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Noriko","family":"Kando","sequence":"additional","affiliation":[{"name":"National Institute of Informatics, Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2005,9]]},"reference":[{"volume-title":"Working Notes of the Fourth NTCIR Workshop Meeting. 297--303","author":"Akiba T.","key":"e_1_2_1_1_1","unstructured":"Akiba , T. , Itou , K. , and Fujii , A. 2004. Question answering using \u201ccommon sense\u201d and utility maximization principle . Working Notes of the Fourth NTCIR Workshop Meeting. 297--303 . Akiba, T., Itou, K., and Fujii, A. 2004. Question answering using \u201ccommon sense\u201d and utility maximization principle. Working Notes of the Fourth NTCIR Workshop Meeting. 297--303."},{"key":"e_1_2_1_2_1","volume-title":"Advanced Question & Answering for Intelligence","author":"Home Page","year":"2003","unstructured":"AQUAINT Home Page : Advanced Question & Answering for Intelligence 2003 . http:\/\/www.ic-arda.org\/InfoExploit\/aquaint\/. AQUAINT Home Page: Advanced Question & Answering for Intelligence 2003. http:\/\/www.ic-arda.org\/InfoExploit\/aquaint\/."},{"key":"e_1_2_1_3_1","unstructured":"Burger J. Cardie C. Chaudhri V. 2001. Issues Tasks and Program Structures to Roadmap Research in Question & Answering (Q&A) http:\/\/www-nlpir.nist.gov\/projrcts\/duc\/roadmpping.html.  Burger J. Cardie C. Chaudhri V. 2001. Issues Tasks and Program Structures to Roadmap Research in Question & Answering (Q&A) http:\/\/www-nlpir.nist.gov\/projrcts\/duc\/roadmpping.html."},{"volume-title":"Proceedings of HLT-NAACL2004","author":"Chai J. Y.","key":"e_1_2_1_4_1","unstructured":"Chai , J. Y. and Jin , R . 2004. Discource structure for context question answering . Proceedings of HLT-NAACL2004 , Workshop on Pragmatics of Question Answering. 23--30. Chai, J. Y. and Jin, R. 2004. Discource structure for context question answering. Proceedings of HLT-NAACL2004, Workshop on Pragmatics of Question Answering. 23--30."},{"volume-title":"AAAI 2003 Spring Symposium New Directions in Question Answering. 122--133","author":"Fukumoto J.","key":"e_1_2_1_5_1","unstructured":"Fukumoto , J. , Kato , T. , and Masui , F . 2003. Question Answering Challenge (QAC-1) An Evaluation of question answering tasks at the NTCIR workshop 3 . AAAI 2003 Spring Symposium New Directions in Question Answering. 122--133 . Fukumoto, J., Kato, T., and Masui, F. 2003. Question Answering Challenge (QAC-1) An Evaluation of question answering tasks at the NTCIR workshop 3. AAAI 2003 Spring Symposium New Directions in Question Answering. 122--133."},{"volume-title":"Working Notes of the Fourth NTCIR Workshop Meeting. 310--314","author":"Fukumoto J.","key":"e_1_2_1_6_1","unstructured":"Fukumoto , J. , Niwa , T. , Itoigawa , M. , and Matuda , M . 2004. Rits-QA: List answer detection and context task with ellipses handling . Working Notes of the Fourth NTCIR Workshop Meeting. 310--314 . Fukumoto, J., Niwa, T., Itoigawa, M., and Matuda, M. 2004. Rits-QA: List answer detection and context task with ellipses handling. Working Notes of the Fourth NTCIR Workshop Meeting. 310--314."},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of TREC","author":"Harabagiu S.","year":"2001","unstructured":"Harabagiu , S. , Moldovan , D. , Pa\u015fca , M. 2001 . Answering complex, list and context questions with LCC's Question-Answering Server . Proceedings of TREC 2001. Harabagiu, S., Moldovan, D., Pa\u015fca, M. 2001. Answering complex, list and context questions with LCC's Question-Answering Server. Proceedings of TREC 2001."},{"volume-title":"Proceedings of HLT-NAACL2004 Workshop on Pragmatics of Question Answering. 60--69","author":"Hickl A.","key":"e_1_2_1_8_1","unstructured":"Hickl , A. , Lehmann , J. , Williams , J. , and Harabagiu , S . 2004. Experiments with interactive question answering in complex scenarios . Proceedings of HLT-NAACL2004 Workshop on Pragmatics of Question Answering. 60--69 . Hickl, A., Lehmann, J., Williams, J., and Harabagiu, S. 2004. Experiments with interactive question answering in complex scenarios. Proceedings of HLT-NAACL2004 Workshop on Pragmatics of Question Answering. 60--69."},{"volume-title":"Working Notes of the Fourth NTCIR Workshop Meeting. 315--319","author":"Hidaka N.","key":"e_1_2_1_9_1","unstructured":"Hidaka , N. , Masui , F. , and Tosaki , K . 2004. MAIQA: Mie Univ. Participated System at NTCIR4 QAC2 . Working Notes of the Fourth NTCIR Workshop Meeting. 315--319 . Hidaka, N., Masui, F., and Tosaki, K. 2004. MAIQA: Mie Univ. Participated System at NTCIR4 QAC2. Working Notes of the Fourth NTCIR Workshop Meeting. 315--319."},{"key":"e_1_2_1_10_1","unstructured":"Hovy E. 2001. http:\/\/www-nlpir.nist.gov\/projects\/duc\/pubs\/2001papers\/isi_hovy_duc.pdf.  Hovy E. 2001. http:\/\/www-nlpir.nist.gov\/projects\/duc\/pubs\/2001papers\/isi_hovy_duc.pdf."},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the Fifth International Joint Conference on Artificial Intelligence. 158--164","author":"Lehnert W. G.","year":"1977","unstructured":"Lehnert , W. G. 1977 . A conceptual theory of question answering . Proceedings of the Fifth International Joint Conference on Artificial Intelligence. 158--164 . Lehnert, W. G. 1977. A conceptual theory of question answering. Proceedings of the Fifth International Joint Conference on Artificial Intelligence. 158--164."},{"key":"e_1_2_1_12_1","volume-title":"LREC Workshop Proceedings on Question Answering \u00b7 Strategy and Resources. 5--8.","author":"Liddy E. D.","year":"2002","unstructured":"Liddy , E. D. 2002 . Why are people asking these question?: A call for bringing situation into question-answering system evaluation . LREC Workshop Proceedings on Question Answering \u00b7 Strategy and Resources. 5--8. Liddy, E. D. 2002. Why are people asking these question?: A call for bringing situation into question-answering system evaluation. LREC Workshop Proceedings on Question Answering \u00b7 Strategy and Resources. 5--8."},{"key":"e_1_2_1_13_1","unstructured":"Liddy E. D. 2003. Preparing to explore a new paradigm in information access: A scenario approach to question-answering. http:\/\/nrrc.mitre.org\/NRRC\/workshop03\/Scenario_BaseQAWriteup.htm.  Liddy E. D. 2003. Preparing to explore a new paradigm in information access: A scenario approach to question-answering. http:\/\/nrrc.mitre.org\/NRRC\/workshop03\/Scenario_BaseQAWriteup.htm."},{"key":"e_1_2_1_14_1","volume-title":"Technical Report MTR98W0000138","author":"Mani I.","year":"1998","unstructured":"Mani , I. , House , D. , Klein, G. 1998 . The TIPSER SUMMAC text summarization evaluation final report. Technical Report MTR98W0000138 , The MITRE Corporation . Mani, I., House, D., Klein, G. 1998. The TIPSER SUMMAC text summarization evaluation final report. Technical Report MTR98W0000138, The MITRE Corporation."},{"key":"e_1_2_1_15_1","unstructured":"NTCIR (NII-NACSIS Test Collection for IR Systems) Project Home Page. 2003. http:\/\/research.nii.ac.jp\/ntcir\/index-en.html.  NTCIR (NII-NACSIS Test Collection for IR Systems) Project Home Page. 2003. http:\/\/research.nii.ac.jp\/ntcir\/index-en.html."},{"volume-title":"A theory of questions and question asking","author":"Ram A.","key":"e_1_2_1_16_1","unstructured":"Ram , A. 1999. A theory of questions and question asking . Understanding Language Understanding eds. Ashwin Ram and Kenneth Moorman, MIT Press, Cambridge, MA. 253--298. Ram, A. 1999. A theory of questions and question asking. Understanding Language Understanding eds. Ashwin Ram and Kenneth Moorman, MIT Press, Cambridge, MA. 253--298."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.3115\/1119312.1119318"},{"key":"e_1_2_1_18_1","volume-title":"Working Notes of the Fourth NTCIR Workshop Meeting. 402--405","author":"Takaki T.","year":"2004","unstructured":"Takaki , T. 2004 . NTT DATA Question-answering experiment at the NTCIR-4 QAC2 . Working Notes of the Fourth NTCIR Workshop Meeting. 402--405 . Takaki, T. 2004. NTT DATA Question-answering experiment at the NTCIR-4 QAC2. Working Notes of the Fourth NTCIR Workshop Meeting. 402--405."},{"key":"e_1_2_1_19_1","unstructured":"TREC Home Page. 2003. http:\/\/trec.nist.gov\/.  TREC Home Page. 2003. http:\/\/trec.nist.gov\/."},{"key":"e_1_2_1_20_1","unstructured":"Text Summarization Challenge Home Page. 2003. http:\/\/lr-www.pi.titech.ac.jp\/tsc\/index-en.html.  Text Summarization Challenge Home Page. 2003. http:\/\/lr-www.pi.titech.ac.jp\/tsc\/index-en.html."},{"volume-title":"Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 200--207","author":"Voorhees E. M.","key":"e_1_2_1_21_1","unstructured":"Voorhees , E. M. and Tice , D. M . 2000. Building a question answering test collection . In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 200--207 . 10.1145\/345508.345577 Voorhees, E. M. and Tice, D. M. 2000. Building a question answering test collection. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 200--207. 10.1145\/345508.345577"},{"key":"e_1_2_1_22_1","volume-title":"Overview of the TREC 2001 question answering track. Proceedings of TREC 2001. 10","author":"Voorhees E. M.","year":"2001","unstructured":"Voorhees , E. M. 2001 . Overview of the TREC 2001 question answering track. Proceedings of TREC 2001. 10 .1017\/S1351324901002789 Voorhees, E. M. 2001. Overview of the TREC 2001 question answering track. Proceedings of TREC 2001. 10.1017\/S1351324901002789"}],"container-title":["ACM Transactions on Asian Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1111667.1111669","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1111667.1111669","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T16:08:24Z","timestamp":1750262904000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1111667.1111669"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2005,9]]},"references-count":22,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2005,9]]}},"alternative-id":["10.1145\/1111667.1111669"],"URL":"https:\/\/doi.org\/10.1145\/1111667.1111669","relation":{},"ISSN":["1530-0226","1558-3430"],"issn-type":[{"type":"print","value":"1530-0226"},{"type":"electronic","value":"1558-3430"}],"subject":[],"published":{"date-parts":[[2005,9]]},"assertion":[{"value":"2005-09-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}