{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T06:17:51Z","timestamp":1771481871371,"version":"3.50.1"},"reference-count":56,"publisher":"MIT Press - Journals","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Transactions of the Association for Computational Linguistics"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:p> For many NLP applications, such as question answering and summarization, the goal is to select the best solution from a large space of candidates to meet a particular user\u2019s needs. To address the lack of user or task-specific training data, we propose an interactive text ranking approach that actively selects pairs of candidates, from which the user selects the best. Unlike previous strategies, which attempt to learn a ranking across the whole candidate space, our method uses Bayesian optimization to focus the user\u2019s labeling effort on high quality candidates and integrate prior knowledge to cope better with small data scenarios. We apply our method to community question answering (cQA) and extractive multidocument summarization, finding that it significantly outperforms existing interactive approaches. We also show that the ranking function learned by our method is an effective reward function for reinforcement learning, which improves the state of the art for interactive summarization. <\/jats:p>","DOI":"10.1162\/tacl_a_00344","type":"journal-article","created":{"date-parts":[[2020,12,22]],"date-time":"2020-12-22T19:56:37Z","timestamp":1608666997000},"page":"759-775","source":"Crossref","is-referenced-by-count":4,"title":["Interactive Text Ranking with Bayesian Optimization: A Case Study on Community QA and Summarization"],"prefix":"10.1162","volume":"8","author":[{"given":"Edwin","family":"Simpson","sequence":"first","affiliation":[{"name":"Ubiquitous Knowledge Processing Lab, Technische Universita\u1e97 Darmstadt"},{"name":"Dept. of Computer Science, University of Bristol."}]},{"given":"Yang","family":"Gao","sequence":"additional","affiliation":[{"name":"Ubiquitous Knowledge Processing Lab, Technische Universita\u1e97 Darmstadt"},{"name":"Dept. of Computer Science, Royal Holloway, University of London."}]},{"given":"Iryna","family":"Gurevych","sequence":"additional","affiliation":[{"name":"Ubiquitous Knowledge Processing Lab, Technische Universita\u1e97 Darmstadt"}]}],"member":"281","reference":[{"key":"bib1","first-page":"9","volume-title":"Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing","author":"Baldridge Jason","year":"2004"},{"key":"bib2","doi-asserted-by":"crossref","first-page":"4220","DOI":"10.18653\/v1\/D18-1454","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Bauer Lisa","year":"2018"},{"key":"bib3","first-page":"1798","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing","author":"Beck Daniel","year":"2014"},{"issue":"3","key":"bib4","first-page":"324","volume":"39","author":"Bradley Ralph Allan","year":"1952","journal-title":"Biometrika"},{"key":"bib5","author":"Brochu Eric","year":"2010","journal-title":"arXiv preprint arXiv:1012.2599"},{"key":"bib6","first-page":"409","volume-title":"Advances in Neural Information Processing Systems","author":"Brochu Eric","year":"2008"},{"key":"bib7","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1145\/1102351.1102369","volume-title":"Proceedings of the 22nd International Conference on Machine Learning","author":"Chu Wei","year":"2005"},{"key":"bib8","first-page":"32","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics","volume":"1","author":"Cohn Trevor","year":"2013"},{"key":"bib9","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin Jacob","year":"2019"},{"key":"bib10","first-page":"178","volume-title":"Handbook of Choice Modelling","author":"Flynn Terry N.","year":"2014"},{"key":"bib11","first-page":"4120","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Gao Yang","year":"2018"},{"key":"bib12","first-page":"1","author":"Gao Yang","year":"2019","journal-title":"Information Retrieval Journal"},{"key":"bib13","first-page":"1347","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Gao Yang","year":"2020"},{"key":"bib14","first-page":"1282","volume-title":"Proceedings of the 34th International Conference on Machine Learning","author":"Gonz\u00e1lez Javier","year":"2017"},{"key":"bib15","first-page":"1303","volume":"14","author":"Hoffman Matthew D.","year":"2013","journal-title":"Journal of Machine Learning Research"},{"key":"bib16","author":"Houlsby Neil","year":"2011","journal-title":"arXiv preprint arXiv:1112.5745"},{"key":"bib17","doi-asserted-by":"crossref","first-page":"570","DOI":"10.18653\/v1\/D18-1054","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Indurthi Sathish Reddy","year":"2018"},{"issue":"4","key":"bib18","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1145\/582415.582418","volume":"20","author":"J\u00e4rvelin Kalervo","year":"2002","journal-title":"ACM Transactions on Information Systems (TOIS)"},{"key":"bib19","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511790423","volume-title":"Probability Theory: The Logic of Science","author":"Jaynes Edwin T.","year":"2003"},{"key":"bib20","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1145\/775047.775067","volume-title":"Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Joachims Thorsten","year":"2002"},{"key":"bib21","volume-title":"Rank Correlation Methods","author":"Kendall Maurice George","year":"1948"},{"issue":"3","key":"bib22","doi-asserted-by":"crossref","first-page":"530","DOI":"10.3368\/le.86.3.530","volume":"86","author":"Kingsley David C.","year":"2010","journal-title":"Land Economics"},{"key":"bib23","doi-asserted-by":"crossref","first-page":"1820","DOI":"10.18653\/v1\/P18-1169","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Lawrence Carolin","year":"2018"},{"key":"bib24","author":"Lin Xiao","year":"2017","journal-title":"arXiv preprint arXiv:1711.01732"},{"key":"bib25","first-page":"97","volume-title":"Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers","author":"Liu Yandong","year":"2008"},{"key":"bib26","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1007\/978-3-642-04174-7_3","volume-title":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","author":"Lopes Manuel","year":"2009"},{"key":"bib27","first-page":"198","volume-title":"International Conference on Theory and Practice of Digital Libraries","author":"Ma\u00f1a L\u00f3pez Manuel J.","year":"1999"},{"issue":"2","key":"bib28","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1037\/h0043178","volume":"66","author":"Duncan Luce R.","year":"1959","journal-title":"Psychological Review"},{"key":"bib29","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1992.4.4.590"},{"key":"bib30","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1007\/978-3-662-38527-2_55","volume-title":"Optimization Techniques IFIP Technical Conference","author":"Mo\u010dkus Jonas","year":"1975"},{"issue":"1","key":"bib31","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/BF02313422","volume":"16","author":"Mosteller Frederick","year":"1951","journal-title":"Psychometrika"},{"key":"bib32","first-page":"660","volume":"2640","author":"Nguyen Tri","year":"2016","journal-title":"choice"},{"key":"bib33","doi-asserted-by":"crossref","first-page":"151","DOI":"10.18653\/v1\/K18-1015","volume-title":"Proceedings of the 22nd Conference on Computational Natural Language Learning","author":"Peris \u00c1lvaro","year":"2018"},{"issue":"2","key":"bib34","first-page":"193","volume":"24","author":"Plackett R. L.","year":"1975","journal-title":"Journal of the Royal Statistical Society, Series C (Applied Statistics)"},{"key":"bib35","first-page":"1353","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Avinesh P.V.S","year":"2017"},{"key":"bib36","first-page":"5381","volume-title":"Advances in Neural Information Processing Systems","author":"Qin Chao","year":"2017"},{"key":"bib37","doi-asserted-by":"crossref","first-page":"235","DOI":"10.18653\/v1\/W19-4328","volume-title":"Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)","author":"Raunak Vikas","year":"2019"},{"key":"bib38","doi-asserted-by":"crossref","first-page":"681","DOI":"10.3115\/v1\/D14-1075","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Rioux Cody","year":"2014"},{"key":"bib39","first-page":"372","volume-title":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","author":"Ruder Sebastian","year":"2017"},{"key":"bib40","first-page":"256","volume-title":"Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning","author":"Ryang Seonggi","year":"2012"},{"key":"bib41","volume-title":"Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence","author":"R\u00fcckl\u00e9 Andreas","year":"2019"},{"key":"bib42","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.ins.2013.07.030","volume":"255","author":"Senge Robin","year":"2014","journal-title":"Information Sciences"},{"issue":"1","key":"bib43","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2200\/S00429ED1V01Y201207AIM018","volume":"6","author":"Settles Burr","year":"2012","journal-title":"Synthesis Lectures on Artificial Intelligence and Machine Learning"},{"key":"bib44","doi-asserted-by":"crossref","first-page":"2904","DOI":"10.18653\/v1\/D18-1318","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Siddhant Aditya","year":"2018"},{"key":"bib45","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00026"},{"issue":"4","key":"bib46","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1007\/s10994-019-05867-2","volume":"109","author":"Simpson Edwin","year":"2020","journal-title":"Machine Learning"},{"key":"bib47","author":"Singh Avi","year":"2019","journal-title":"arXiv preprint arXiv:1904.07854"},{"key":"bib48","first-page":"2951","volume-title":"Advances in Neural Information Processing Systems","author":"Snoek Jasper","year":"2012"},{"key":"bib49","first-page":"1489","volume-title":"Advances in Neural Information Processing Systems","author":"Sokolov Artem","year":"2016"},{"issue":"3","key":"bib50","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1093\/biomet\/25.3-4.285","volume":"25","author":"Thompson William R.","year":"1933","journal-title":"Biometrika"},{"issue":"4","key":"bib51","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1037\/h0070288","volume":"34","author":"Thurstone Louis L.","year":"1927","journal-title":"Psychological Review"},{"key":"bib52","first-page":"2352","volume-title":"Advances in Neural Information Processing Systems","author":"Viappiani Paolo","year":"2010"},{"issue":"1","key":"bib53","first-page":"4945","volume":"18","author":"Wirth Christian","year":"2017","journal-title":"The Journal of Machine Learning Research"},{"key":"bib54","author":"Peng Xu","year":"2019","journal-title":"arXiv preprint arXiv:1905.05910"},{"key":"bib55","author":"Yang Jie","year":"2018","journal-title":"arXiv preprint arXiv:1805. 01867"},{"issue":"4","key":"bib56","doi-asserted-by":"crossref","first-page":"762","DOI":"10.1109\/TASL.2010.2064164","volume":"19","author":"Yang Yi-Hsuan","year":"2011","journal-title":"IEEE Transactions on Audio, Speech, and Language Processing"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/tacl_a_00344","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:39:47Z","timestamp":1615585187000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/96467"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12]]},"references-count":56,"alternative-id":["10.1162\/tacl_a_00344"],"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00344","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12]]}}}