{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T03:56:06Z","timestamp":1777694166789,"version":"3.51.4"},"reference-count":30,"publisher":"SAGE Publications","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ICA"],"published-print":{"date-parts":[[2022,6,21]]},"abstract":"<jats:p>Extractive summarization is an important natural language processing approach used for document compression, improved reading comprehension, key phrase extraction, indexing, query set generation, and other analytics approaches. Extractive summarization has specific advantages over abstractive summarization in that it preserves style, specific text elements, and compound phrases that might be more directly associated with the text. In this article, the relative effectiveness of extractive summarization is considered on two widely different corpora: (1) a set of works of fiction (100 total, mainly novels) available from Project Gutenberg, and (2) a large set of news articles (3000) for which a ground truthed summarization (gold standard) is provided by the authors of the news articles. Both sets were evaluated using 5 different Python Sumy algorithms and compared to randomly-generated summarizations quantitatively. Two functional approaches to assessing the efficacy of summarization using a query set on both the original documents and their summaries, and using document classification on a 12-class set to compare among different summarization approaches, are introduced. The results, unsurprisingly, show considerable differences consistent with the different nature of these two data sets. The LSA and Luhn summarization approaches were most effective on the database of fiction, while all five summarization approaches were similarly effective on the database of articles. Overall, the Luhn approach was deemed the most generally relevant among those tested.<\/jats:p>","DOI":"10.3233\/ica-220680","type":"journal-article","created":{"date-parts":[[2022,5,10]],"date-time":"2022-05-10T12:15:17Z","timestamp":1652184917000},"page":"227-239","source":"Crossref","is-referenced-by-count":4,"title":["Summarization assessment methodology for multiple corpora using queries and classification for functional evaluation"],"prefix":"10.1177","volume":"29","author":[{"given":"Sam","family":"Wolyn","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Steven J.","family":"Simske","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"key":"10.3233\/ICA-220680_ref1","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.eswa.2018.12.011","article-title":"Abstractive summarization: An overview of the state of the art","volume":"121","author":"Gupta","year":"2019","journal-title":"Expert Systems with Applications."},{"issue":"2","key":"10.3233\/ICA-220680_ref2","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1147\/rd.22.0159","article-title":"The automatic creation of literature abstracts","volume":"2","author":"Luhn","year":"1958","journal-title":"IBM Journal of Research and Development."},{"issue":"5","key":"10.3233\/ICA-220680_ref3","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1145\/366532.366545","article-title":"Automatic abstracting and indexing\u00a0\u2013 survey and recommendations","volume":"4","author":"Edmundson","year":"1961","journal-title":"Communications of the ACM."},{"issue":"2","key":"10.3233\/ICA-220680_ref4","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1145\/321510.321519","article-title":"New methods in automatic extracting","volume":"16","author":"Edmundson","year":"1969","journal-title":"Journal of the ACM (JACM)."},{"issue":"4","key":"10.3233\/ICA-220680_ref5","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1002\/asi.4630220405","article-title":"Automatic abstracting and indexing. II. Production of indicative abstracts by application of contextual inference and syntactic coherence criteria","volume":"22","author":"Rush","year":"1971","journal-title":"Journal of the American Society for Information Science."},{"key":"10.3233\/ICA-220680_ref6","first-page":"9","article-title":"What might be in a summary","volume":"93","author":"Jones","year":"1993","journal-title":"Information Retrieval."},{"key":"10.3233\/ICA-220680_ref7","doi-asserted-by":"crossref","unstructured":"Kupiec J, Pedersen J, Chen F. A trainable document summarizer. In Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 1995; 68-73. ACM.","DOI":"10.1145\/215206.215333"},{"key":"10.3233\/ICA-220680_ref8","doi-asserted-by":"crossref","first-page":"178","DOI":"10.3844\/jcssp.2016.178.190","article-title":"A review on automatic text summarization approaches","volume":"12","author":"Basiron","year":"2016","journal-title":"Journal of Computer Science."},{"key":"10.3233\/ICA-220680_ref9","doi-asserted-by":"crossref","unstructured":"Wong KF, Wu M, Li W. Extractive summarization using supervised and semi-supervised learning. In Proceedings of the 22nd international conference on computational linguistics. 2008; 22: 985-992.","DOI":"10.3115\/1599081.1599205"},{"key":"10.3233\/ICA-220680_ref10","doi-asserted-by":"crossref","unstructured":"Kaikhah K. Automatic text summarization with neural networks. In 2nd International IEEE Conference on \u2018Intelligent Systems\u2019. Proceedings. 2004; 1: 40-44.","DOI":"10.1109\/IS.2004.1344634"},{"key":"10.3233\/ICA-220680_ref11","unstructured":"Svore K, Vanderwende L, Burges C. Enhancing single-document summarization by combining RankNet and third-party sources. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. 2007, pp. 448-457."},{"key":"10.3233\/ICA-220680_ref12","doi-asserted-by":"crossref","unstructured":"Lins RD, Oliveira H, Cabral L, Batista J, Tenorio B, Ferreira R, Lima R, xde Fran\u00e7a Pereira Silva G, Simske SJ. The cnn-corpus: A large textual corpus for single-document extractive summarization. In Proceedings of the ACM Symposium on Document Engineering. 2019, pp. 1-10.","DOI":"10.1145\/3342558.3345388"},{"key":"10.3233\/ICA-220680_ref14","doi-asserted-by":"crossref","first-page":"156043","DOI":"10.1109\/ACCESS.2021.3129786","article-title":"A survey of automatic text summarization: Progress, process and challenges","volume":"9","author":"Mridha","year":"2021","journal-title":"In IEEE Access."},{"issue":"4","key":"10.3233\/ICA-220680_ref15","first-page":"787","article-title":"Extractive summarization: Limits, compression, generalized model and heuristics","volume":"21","author":"Verma","year":"2017","journal-title":"Computaci\u00f3n y Sistemas."},{"key":"10.3233\/ICA-220680_ref16","unstructured":"Simske SJ, Vans M. Functional Applications of Text Analytics Systems. River Publishers, 2021."},{"key":"10.3233\/ICA-220680_ref17","doi-asserted-by":"crossref","unstructured":"See A, Liu PJ, Manning CD. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv1704. 04368, 2017.","DOI":"10.18653\/v1\/P17-1099"},{"key":"10.3233\/ICA-220680_ref18","doi-asserted-by":"crossref","unstructured":"Nallapati R, Zhou B, Gulcehre C, Xiang B. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv1602.06023, 2016.","DOI":"10.18653\/v1\/K16-1028"},{"key":"10.3233\/ICA-220680_ref19","doi-asserted-by":"crossref","unstructured":"Rush AM, Chopra S, Weston JA. Neural attention model for abstractive sentence summarization. arXiv preprint arXiv1509. 00685, 2015.","DOI":"10.18653\/v1\/D15-1044"},{"issue":"1","key":"10.3233\/ICA-220680_ref20","doi-asserted-by":"crossref","first-page":"126","DOI":"10.3390\/e22010126","article-title":"A standardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguistics","volume":"22","author":"Gerlach","year":"2020","journal-title":"Entropy."},{"issue":"1","key":"10.3233\/ICA-220680_ref21","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/973381.973384","article-title":"Literary freedom: Project Gutenberg","volume":"10","author":"Stroube","year":"2003","journal-title":"XRDS: Crossroads, The ACM Magazine for Students."},{"key":"10.3233\/ICA-220680_ref22","doi-asserted-by":"crossref","unstructured":"Vans AM, Simske SJ. Identifying top performing TF* IDF classifiers using the CNN corpus. In Archiving Conference. 2017(1): 105-115.","DOI":"10.2352\/issn.2168-3204.2017.1.0.105"},{"issue":"14","key":"10.3233\/ICA-220680_ref23","doi-asserted-by":"crossref","first-page":"5755","DOI":"10.1016\/j.eswa.2013.04.023","article-title":"Assessing sentence scoring techniques for extractive text summarization","volume":"40","author":"Ferreira","year":"2013","journal-title":"Expert Systems with Applications."},{"key":"10.3233\/ICA-220680_ref24","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.eswa.2016.08.030","article-title":"Assessing shallow sentence scoring techniques and combinations for single and multi-document summarization","volume":"65","author":"Oliveira","year":"2016","journal-title":"Expert Systems with Applications."},{"issue":"1","key":"10.3233\/ICA-220680_ref25","first-page":"4","article-title":"A review of machine learning algorithms for text-documents classification","volume":"1","author":"Khan","year":"2010","journal-title":"Journal of Advances in Information Technology."},{"issue":"2","key":"10.3233\/ICA-220680_ref26","doi-asserted-by":"crossref","first-page":"85","DOI":"10.5121\/ijaia.2012.3208","article-title":"Text classification and classifiers: A survey","volume":"3","author":"Korde","year":"2012","journal-title":"International Journal of Artificial Intelligence & Applications."},{"key":"10.3233\/ICA-220680_ref27","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.ins.2018.10.006","article-title":"Multi-co-training for document classification using various document representations: TF-IDF, LDA, and Doc2Vec","volume":"477","author":"Kim","year":"2019","journal-title":"Information Sciences."},{"issue":"3","key":"10.3233\/ICA-220680_ref28","doi-asserted-by":"crossref","first-page":"197","DOI":"10.3233\/ICA-2010-0345","article-title":"Enhanced probabilistic neural network with local decision circles: A robust classifier","volume":"17","author":"Ahmadlou","year":"2010","journal-title":"Integrated Computer-Aided Engineering."},{"issue":"12","key":"10.3233\/ICA-220680_ref29","doi-asserted-by":"crossref","first-page":"3074","DOI":"10.1109\/TNNLS.2017.2682102","article-title":"A new neural dynamic classification algorithm","volume":"28","author":"Rafiei","year":"2017","journal-title":"IEEE Transactions on Neural Networks and Learning Systems."},{"issue":"10","key":"10.3233\/ICA-220680_ref30","doi-asserted-by":"crossref","first-page":"6393","DOI":"10.1007\/s00521-019-04146-4","article-title":"FEMa: A finite element machine for fast learning","volume":"32","author":"Pereira","year":"2020","journal-title":"Neural Computing and Applications."},{"issue":"12","key":"10.3233\/ICA-220680_ref31","doi-asserted-by":"crossref","first-page":"8675","DOI":"10.1007\/s00521-019-04359-7","article-title":"A dynamic ensemble learning algorithm for neural networks","volume":"32","author":"Alam","year":"2020","journal-title":"Neural Computing and Applications."}],"container-title":["Integrated Computer-Aided Engineering"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/ICA-220680","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:14:35Z","timestamp":1777454075000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/ICA-220680"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,21]]},"references-count":30,"journal-issue":{"issue":"3"},"URL":"https:\/\/doi.org\/10.3233\/ica-220680","relation":{},"ISSN":["1069-2509","1875-8835"],"issn-type":[{"value":"1069-2509","type":"print"},{"value":"1875-8835","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,21]]}}}