{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,5,23]],"date-time":"2024-05-23T05:52:51Z","timestamp":1716443571184},"reference-count":61,"publisher":"Cambridge University Press (CUP)","issue":"1","license":[{"start":{"date-parts":[[2011,4,7]],"date-time":"2011-04-07T00:00:00Z","timestamp":1302134400000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2012,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In this paper, we apply different supervised learning techniques to build query-focused multi-document summarization systems, where the task is to produce automatic summaries in response to a given query or specific information request stated by the user. A huge amount of labeled data is a prerequisite for supervised training. It is expensive and time-consuming when humans perform the labeling task manually. Automatic labeling can be a good remedy to this problem. We employ five different automatic annotation techniques to build extracts from human abstracts using ROUGE, Basic Element overlap, syntactic similarity measure, semantic similarity measure, and Extended String Subsequence Kernel. The supervised methods we use are Support Vector Machines, Conditional Random Fields, Hidden Markov Models, Maximum Entropy, and two ensemble-based approaches. During different experiments, we analyze the impact of automatic labeling methods on the performance of the applied supervised methods. To our knowledge, no other study has deeply investigated and compared the effects of using different automatic annotation techniques on different supervised learning approaches in the domain of query-focused multi-document summarization.<\/jats:p>","DOI":"10.1017\/s1351324911000167","type":"journal-article","created":{"date-parts":[[2011,4,7]],"date-time":"2011-04-07T12:23:03Z","timestamp":1302178983000},"page":"109-145","source":"Crossref","is-referenced-by-count":20,"title":["Query-focused multi-document summarization: automatic data annotations and supervised learning approaches"],"prefix":"10.1017","volume":"18","author":[{"given":"YLLIAS","family":"CHALI","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"SADID A.","family":"HASAN","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2011,4,7]]},"reference":[{"key":"S1351324911000167_ref61","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1145\/860435.860443","volume-title":"Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval","author":"Zhang","year":"2003"},{"key":"S1351324911000167_ref60","first-page":"985","volume-title":"Proceedings of the 22nd International Conference on Computational Linguistics \u2013 Vol. 1","author":"Wong","year":"2008"},{"key":"S1351324911000167_ref58","first-page":"1586","volume-title":"Proceedings of the 21st International Joint Conference on Artifical Intelligence","author":"Wan","year":"2009"},{"key":"S1351324911000167_ref57","unstructured":"Wallach H. 2002. Efficient Training of Conditional Random Fields. MSc thesis, Division of Informatics, University of Edinburgh."},{"key":"S1351324911000167_ref56","volume-title":"Proceedings of the Document Understanding Conference","author":"Toutanova","year":"2007"},{"key":"S1351324911000167_ref53","unstructured":"Sekine S. 2002. Proteus project-OAK system (English sentence analyzer). http:\/\/nlp.nyu.edu\/oak."},{"key":"S1351324911000167_ref52","volume-title":"Proceedings of the 17th International Florida Artificial Intelligence Research Society Conference (FLAIRS)","author":"Rooney","year":"2004"},{"key":"S1351324911000167_ref51","volume-title":"Proceedings of the International Conference on Intelligent Systems and Knowledge Engineering (ISKE)","author":"Qi","year":"2007"},{"key":"S1351324911000167_ref49","first-page":"882","volume-title":"Advances in Neural Information Processing Systems","author":"Parmanto","year":"1996"},{"key":"S1351324911000167_ref55","first-page":"2862","volume-title":"Proceedings of the 20th International Joint Conference on Artificial Intelligence","author":"Shen","year":"2007"},{"key":"S1351324911000167_ref48","volume-title":"First World Congress of the International Federation for Systems Research (IFSR'05)","author":"Nguyen","year":"2005"},{"key":"S1351324911000167_ref47","first-page":"763","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing","author":"Nastase","year":"2008"},{"key":"S1351324911000167_ref42","volume-title":"Advances in Automatic Text Summarization","author":"Mani","year":"1999"},{"key":"S1351324911000167_ref27","first-page":"446","volume-title":"Proceedings of the 20th International Conference on Computational Linguistics","author":"Hirao","year":"2004"},{"key":"S1351324911000167_ref46","first-page":"776","volume-title":"Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics","author":"Moschitti","year":"2007"},{"key":"S1351324911000167_ref34","volume-title":"Speech and Language Processing: An Introduction to Natural Language Processing, Speech Recognition, and Computational Linguistics","author":"Jurafsky","year":"2009"},{"key":"S1351324911000167_ref59","first-page":"2903","volume-title":"Proceedings of the 20th International Joint Conference on Artifical Intelligence","author":"Wan","year":"2007"},{"key":"S1351324911000167_ref11","first-page":"132","volume-title":"Proceedings of the 1st North American Chapter of the Association for Computational Linguistics Conference","author":"Charniak","year":"2000"},{"key":"S1351324911000167_ref24","first-page":"1","volume-title":"Proceedings of the 19th International Conference on Computational Linguistics \u2013 Vol. 1","author":"Hirao","year":"2002"},{"key":"S1351324911000167_ref31","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1145\/312624.312666","volume-title":"Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Jing","year":"1999"},{"key":"S1351324911000167_ref23","first-page":"226","volume-title":"Proceedings of the Conference of Recent Advances in Natural Language Processing","author":"Harnly","year":"2005"},{"key":"S1351324911000167_ref16","first-page":"305","volume-title":"Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics","author":"Daum\u00e9","year":"2006"},{"key":"S1351324911000167_ref29","volume-title":"Proceedings of the 5th Conference on Language Resources and Evaluation","author":"Hovy","year":"2006"},{"key":"S1351324911000167_ref30","volume-title":"A Practical Guide to Support Vector Classification","author":"Hsu","year":"2008"},{"key":"S1351324911000167_ref41","doi-asserted-by":"publisher","DOI":"10.1075\/nlp.3"},{"key":"S1351324911000167_ref20","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/7287.001.0001","volume-title":"WordNet-An Electronic Lexical Database","author":"Fellbaum","year":"1998"},{"key":"S1351324911000167_ref50","first-page":"38","volume-title":"Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics and 10th Conference of the European Chapter Workshop on Open-Domain Question Answering","author":"Pasca","year":"2001"},{"key":"S1351324911000167_ref54","volume-title":"Proceedings of the Document Understanding Conference","author":"Sekine","year":"2001"},{"key":"S1351324911000167_ref6","first-page":"199","volume-title":"Proceedings of the 22nd Canadian Conference on Artificial Intelligence","author":"Chali","year":"2009"},{"key":"S1351324911000167_ref7","first-page":"329","volume-title":"Proceedings of the Joint conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing","author":"Chali","year":"2009"},{"key":"S1351324911000167_ref19","doi-asserted-by":"crossref","DOI":"10.1201\/9780429246593","volume-title":"An Introduction to the Bootstrap","author":"Efron","year":"1994"},{"key":"S1351324911000167_ref18","doi-asserted-by":"publisher","DOI":"10.1145\/321510.321519"},{"key":"S1351324911000167_ref3","first-page":"39","article-title":"A maximum entropy approach to natural language processing","volume":"22","author":"Berger","year":"1996","journal-title":"Computational Linguistics"},{"key":"S1351324911000167_ref35","first-page":"1989","volume-title":"Proceedings of the International Conference on Language Resources and Evaluation","author":"Kingsbury","year":"2002"},{"key":"S1351324911000167_ref25","first-page":"104","volume-title":"Proceedings of the Document Understanding Conference","author":"Hirao","year":"2002"},{"key":"S1351324911000167_ref21","unstructured":"Ferrier L. 2001. A Maximum Entropy Approach to Text Summarization. MSc thesis, School of Artificial Intelligence, Division of Informatics, University of Edinburgh."},{"key":"S1351324911000167_ref39","first-page":"74","volume-title":"Proceedings of Workshop on Text Summarization Branches Out, Post-Conference Workshop of Association for Computational Linguistics","author":"Lin","year":"2004"},{"key":"S1351324911000167_ref40","first-page":"419","article-title":"Text classification using string kernels","volume":"2","author":"Lodhi","year":"2002","journal-title":"Journal of Machine Learning Research"},{"key":"S1351324911000167_ref2","first-page":"25","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Barzilay","year":"2003"},{"key":"S1351324911000167_ref14","doi-asserted-by":"publisher","DOI":"10.3115\/1273073.1273093"},{"key":"S1351324911000167_ref33","first-page":"169","volume-title":"Making Large-Scale Support Vector Machine Learning Practical","author":"Joachims","year":"1999"},{"key":"S1351324911000167_ref1","volume-title":"Proceedings of the 4th Meeting of the Pacific Association for Computational Linguistics","author":"Banko","year":"1999"},{"key":"S1351324911000167_ref4","first-page":"1059","article-title":"Word sequence kernels","volume":"3","author":"Cancedda","year":"2003","journal-title":"Journal of Machine Learning Research"},{"key":"S1351324911000167_ref15","doi-asserted-by":"publisher","DOI":"10.1007\/BF00994018"},{"key":"S1351324911000167_ref5","first-page":"12","volume-title":"Proceedings of the 15th International Joint Conference on Artificial Intelligence, Workshop: AI in Digital Libraries","author":"Carbonell","year":"1997"},{"key":"S1351324911000167_ref8","first-page":"476","volume-title":"Proceedings of the 4th International Conference on Semantic Evaluations","author":"Chali","year":"2007"},{"key":"S1351324911000167_ref9","doi-asserted-by":"publisher","DOI":"10.3115\/1613715.1613757"},{"key":"S1351324911000167_ref10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1613\/jair.2784","article-title":"Complex question answering: unsupervised learning approaches and experiments","volume":"35","author":"Chali","year":"2009","journal-title":"Journal of Artificial Intelligence Research"},{"key":"S1351324911000167_ref12","first-page":"625","volume-title":"Proceedings of Advances in Neural Information Processing Systems 14","author":"Collins","year":"2001"},{"key":"S1351324911000167_ref13","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1145\/383952.384042","volume-title":"Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Conroy","year":"2001"},{"key":"S1351324911000167_ref17","first-page":"1","volume-title":"Proceedings of the First International Workshop on Multiple Classifier Systems","author":"Dietterich","year":"2000"},{"key":"S1351324911000167_ref22","unstructured":"Hacioglu K. , Pradhan S. , Ward W. , Martin J. H. , and Jurafsky D. 2003. Shallow semantic parsing using support vector machines. Technical Report TR-CSLR-2003-1, Center for Spoken Language Research, Boulder, CO, USA."},{"key":"S1351324911000167_ref26","volume-title":"Proceedings of the Document Understanding Conference","author":"Hirao","year":"2003"},{"key":"S1351324911000167_ref28","volume-title":"Proceedings of the Document Understanding Conference","author":"Hovy","year":"2005"},{"key":"S1351324911000167_ref32","first-page":"137","volume-title":"Proceedings of the 10th European Conference on Machine Learning","author":"Joachims","year":"1998"},{"key":"S1351324911000167_ref36","first-page":"192","volume-title":"Proceedings of the 2nd Meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies","author":"Kudo","year":"2001"},{"key":"S1351324911000167_ref37","first-page":"68","volume-title":"Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Kupiec","year":"1995"},{"key":"S1351324911000167_ref38","first-page":"282","volume-title":"Proceedings of the 18th International Conference on Machine Learning","author":"Lafferty","year":"2001"},{"key":"S1351324911000167_ref43","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1145\/312624.312668","volume-title":"Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Marcu","year":"1999"},{"key":"S1351324911000167_ref44","unstructured":"McCallum A. K. 2002. MALLET: a machine learning for language toolkit. http:\/\/mallet.cs.umass.edu."},{"key":"S1351324911000167_ref45","first-page":"1510","volume-title":"Proceedings of the 5th International Conference on Language Resources and Evaluation","author":"Moschitti","year":"2006"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324911000167","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,11,21]],"date-time":"2021-11-21T08:05:19Z","timestamp":1637481919000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324911000167\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,4,7]]},"references-count":61,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2012,1]]}},"alternative-id":["S1351324911000167"],"URL":"https:\/\/doi.org\/10.1017\/s1351324911000167","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"value":"1351-3249","type":"print"},{"value":"1469-8110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,4,7]]}}}