{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T09:11:54Z","timestamp":1770541914831,"version":"3.49.0"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,12,31]],"date-time":"2018-12-31T00:00:00Z","timestamp":1546214400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Microsoft Azure for Research award"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Manage. Inf. Syst."],"published-print":{"date-parts":[[2018,12,31]]},"abstract":"<jats:p>Traditional information retrieval (such as that offered by web search engines) impedes users with information overload from extensive result pages and the need to manually locate the desired information therein. Conversely, question-answering systems change how humans interact with information systems: users can now ask specific questions and obtain a tailored answer\u2014both conveniently in natural language. Despite obvious benefits, their use is often limited to an academic context, largely because of expensive domain customizations, which means that the performance in domain-specific applications often fails to meet expectations. This article proposes cost-efficient remedies: (i) we leverage metadata through a filtering mechanism, which increases the precision of document retrieval, and (ii) we develop a novel fuse-and-oversample approach for transfer learning to improve the performance of answer extraction. Here, knowledge is inductively transferred from related, yet different, tasks to the domain-specific application, while accounting for potential differences in the sample sizes across both tasks. The resulting performance is demonstrated with actual use cases from a finance company and the film industry, where fewer than 400 question-answer pairs had to be annotated to yield significant performance gains. As a direct implication to management, this presents a promising path to better leveraging of knowledge stored in information systems.<\/jats:p>","DOI":"10.1145\/3309706","type":"journal-article","created":{"date-parts":[[2019,2,27]],"date-time":"2019-02-27T15:00:28Z","timestamp":1551279628000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Putting Question-Answering Systems into Practice"],"prefix":"10.1145","volume":"9","author":[{"given":"Bernhard","family":"Kratzwald","sequence":"first","affiliation":[{"name":"ETH Zurich, Weinbergstrasse, Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stefan","family":"Feuerriegel","sequence":"additional","affiliation":[{"name":"ETH Zurich, Weinbergstrasse, Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,2,27]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of theInternational Conference on Learning Representations.","author":"Bahdanau Dzmitry","year":"2015"},{"key":"e_1_2_1_2_1","first-page":"55","article-title":"Interaction with texts: Information retrieval as information-seeking behavior","volume":"93","author":"Belkin Nicholas","year":"1993","journal-title":"Inform. Retriev."},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1533--1544","author":"Berant Jonathan","year":"2013"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1367497.1367561"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.3115\/1118693.1118726"},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the Text REtrieval Conference. 285--298","author":"Buckley Chris","year":"1997"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/996350.996398"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2011.01.004"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2007.07.006"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1171"},{"key":"e_1_2_1_11_1","doi-asserted-by":"crossref","unstructured":"Abdessamad Echihabi Ulf Hermjakob Eduard Hovy Daniel Marcu Eric Melz and Deepak Ravichandran. 2008. How to select an answer string? In Advances in Open Domain Question Answering T. Strzalkowski and S. M. Harabagi (Eds.). Text Speech and Language Technology Vol 32. Springer Dordrecht 383--406.  Abdessamad Echihabi Ulf Hermjakob Eduard Hovy Daniel Marcu Eric Melz and Deepak Ravichandran. 2008. How to select an answer string? In Advances in Open Domain Question Answering T. Strzalkowski and S. M. Harabagi (Eds.). Text Speech and Language Technology Vol 32. Springer Dordrecht 383--406.","DOI":"10.1007\/978-1-4020-4746-6_12"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2008.09.001"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2012.2184356"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2015.02.005"},{"key":"e_1_2_1_15_1","unstructured":"Ian Goodfellow Yoshua Bengio and Aaron Courville. 2016. Deep Learning. MIT Press Cambridge MA.   Ian Goodfellow Yoshua Bengio and Aaron Courville. 2016. Deep Learning. MIT Press Cambridge MA."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.1090.0249"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 2000 Text REtrieval Conference. 479--488","author":"Harabagiu Sanda","year":"2000"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.3233\/IDA-2002-6504"},{"key":"e_1_2_1_19_1","unstructured":"Daniel S. Jurafsky and James H. Martin. 2009. Speech and Language Processing: An Introduction to Natural Language Processing Computational Linguistics and Speech Recognition (2nd ed.). Pearson Upper Saddle River NJ.   Daniel S. Jurafsky and James H. Martin. 2009. Speech and Language Processing: An Introduction to Natural Language Processing Computational Linguistics and Speech Recognition (2nd ed.). Pearson Upper Saddle River NJ."},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the Text REtrieval Conference.","author":"Kaisser Michael","year":"2004"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1055"},{"key":"e_1_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Bernhard Kratzwald Suzana Ili\u0107 Mathias Kraus Stefan Feuerriegel and Helmut Prendinger. 2018. Deep learning for affective computing: Text-based emotion recognition in decision support. Dec. Supp. Syst. 115 (2018) 24--35.  Bernhard Kratzwald Suzana Ili\u0107 Mathias Kraus Stefan Feuerriegel and Helmut Prendinger. 2018. Deep learning for affective computing: Text-based emotion recognition in decision support. Dec. Supp. Syst. 115 (2018) 24--35.","DOI":"10.1016\/j.dss.2018.09.002"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2017.10.001"},{"key":"e_1_2_1_24_1","unstructured":"M. Kraus S. Feuerriegel and A. Oztekin. 2018. Deep learning in business analytics and operations research: Models applications and managerial implications. arXiv1806.10897 (2018).  M. Kraus S. Feuerriegel and A. Oztekin. 2018. Deep learning in business analytics and operations research: Models applications and managerial implications. arXiv1806.10897 (2018)."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2407740.2407741"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1229179.1229180"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1015"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2008.2007853"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2007.03.003"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/5254.920602"},{"key":"e_1_2_1_31_1","unstructured":"Christopher D. Manning and Hinrich Sch\u00fctze. 1999. Foundations of Statistical Natural Language Processing. MIT Press Cambridge MA.   Christopher D. Manning and Hinrich Sch\u00fctze. 1999. Foundations of Statistical Natural Language Processing. MIT Press Cambridge MA."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1147"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/763693.763694"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli.2007.33.1.41"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1046"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2009.191"},{"key":"e_1_2_1_37_1","unstructured":"Marius Pasca. 2005. Open-domain Question Answering from Large Text Collections. CSLI Studies in Computational Linguistics Stanford CA.  Marius Pasca. 2005. Open-domain Question Answering from Large Text Collections. CSLI Studies in Computational Linguistics Stanford CA."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/544220.544228"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/1059467.1059471"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1264"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.3115\/1073083.1073092"},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing. 193--203","author":"Richardson Matthew","year":"2013"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000019"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2006.04.006"},{"key":"e_1_2_1_46_1","unstructured":"Gerard Salton. 1971. The SMART Retrieval System\u2014Experiments in Automatic Document Processing. Prentice Hall Upper Saddle River NJ.   Gerard Salton. 1971. The SMART Retrieval System\u2014Experiments in Automatic Document Processing. Prentice Hall Upper Saddle River NJ."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2014.09.003"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2006.04.007"},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of theInternational Conference on Learning Representations.","author":"Seo Minjoon","year":"2017"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.3115\/1220175.1220287"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2904--2909","author":"Siddhant Aditya"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/363707.363732"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/243199.243206"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb026526"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187836.2187923"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1007\/11431053_31"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.1100.0324"},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the Text REtrieval Conference. 71--80","author":"Voorhees Ellen M.","year":"2001"},{"key":"e_1_2_1_59_1","volume-title":"Proceedings of the Conference on Artificial Intelligence.","author":"Wang Shuohang","year":"2018"},{"key":"e_1_2_1_60_1","unstructured":"Shuohang Wang Mo Yu Jing Jiang Wei Zhang Xiaoxiao Guo Shiyu Chang Zhiguo Wang Tim Klinger Gerald Tesauro and Murray Campbell. 2017a. Evidence aggregation for answer re-ranking in open-domain question answering. arXiv preprint arXiv:1711.05116 (2017).  Shuohang Wang Mo Yu Jing Jiang Wei Zhang Xiaoxiao Guo Shiyu Chang Zhiguo Wang Tim Klinger Gerald Tesauro and Murray Campbell. 2017a. Evidence aggregation for answer re-ranking in open-domain question answering. arXiv preprint arXiv:1711.05116 (2017)."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1018"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553516"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1220"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2012.08.010"}],"container-title":["ACM Transactions on Management Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3309706","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3309706","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:53:35Z","timestamp":1750204415000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3309706"}},"subtitle":["Transfer Learning for Efficient Domain Customization"],"short-title":[],"issued":{"date-parts":[[2018,12,31]]},"references-count":64,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,12,31]]}},"alternative-id":["10.1145\/3309706"],"URL":"https:\/\/doi.org\/10.1145\/3309706","relation":{},"ISSN":["2158-656X","2158-6578"],"issn-type":[{"value":"2158-656X","type":"print"},{"value":"2158-6578","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,12,31]]},"assertion":[{"value":"2018-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-02-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}