{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,8]],"date-time":"2026-03-08T08:34:15Z","timestamp":1772958855761,"version":"3.50.1"},"reference-count":76,"publisher":"Cambridge University Press (CUP)","issue":"4","license":[{"start":{"date-parts":[[2021,6,10]],"date-time":"2021-06-10T00:00:00Z","timestamp":1623283200000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>With growing amounts of available textual data, development of algorithms capable of automatic analysis, categorization, and summarization of these data has become a necessity. In this research, we present a novel algorithm for keyword identification, that is, an extraction of one or multiword phrases representing key aspects of a given document, called Transformer-Based Neural Tagger for Keyword IDentification (TNT-KID). By adapting the transformer architecture for a specific task at hand and leveraging language model pretraining on a domain-specific corpus, the model is capable of overcoming deficiencies of both supervised and unsupervised state-of-the-art approaches to keyword extraction by offering competitive and robust performance on a variety of different datasets while requiring only a fraction of manually labeled data required by the best-performing systems. This study also offers thorough error analysis with valuable insights into the inner workings of the model and an ablation study measuring the influence of specific components of the keyword identification workflow on the overall performance.<\/jats:p>","DOI":"10.1017\/s1351324921000127","type":"journal-article","created":{"date-parts":[[2021,6,10]],"date-time":"2021-06-10T05:59:37Z","timestamp":1623304777000},"page":"409-448","update-policy":"https:\/\/doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":36,"title":["TNT-KID: Transformer-based neural tagger for keyword identification"],"prefix":"10.1017","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7384-8112","authenticated-orcid":false,"given":"Matej","family":"Martinc","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bla\u017e","family":"\u0160krlj","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Senja","family":"Pollak","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2021,6,10]]},"reference":[{"key":"S1351324921000127_ref20","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1937.10503522"},{"key":"S1351324921000127_ref11","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1439"},{"key":"S1351324921000127_ref38","unstructured":"Liu, Y. , Ott, M. , Goyal, N. , Du, J. , Joshi, M. , Chen, D. , Levy, O. , Lewis, M. , Zettlemoyer, L. and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692."},{"key":"S1351324921000127_ref57","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-019-09477-1"},{"key":"S1351324921000127_ref25","doi-asserted-by":"crossref","unstructured":"Gollapalli, S.D. , Li, X.-L. and Yang, P. (2017). Incorporating expert knowledge into keyphrase extraction. In Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, California, USA. Association for Computing Machinery, pp. 3180\u20133187.","DOI":"10.1609\/aaai.v31i1.10986"},{"key":"S1351324921000127_ref21","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-8617"},{"key":"S1351324921000127_ref71","doi-asserted-by":"publisher","DOI":"10.1145\/313238.313437"},{"key":"S1351324921000127_ref65","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-96133-0_16"},{"key":"S1351324921000127_ref39","unstructured":"Liu, Z. , Chen, X. , Zheng, Y. and Sun, M. (2011). Automatic keyphrase extraction by bridging vocabulary gap. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning, Portland, Oregon, USA. Association for Computational Linguistics, pp. 135\u2013144."},{"key":"S1351324921000127_ref47","unstructured":"Mihalcea, R. and Tarau, P. (2004). TextRank: Bringing order into text. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, Barcelona, Spain. Association for Computational Linguistics, pp. 404\u2013411."},{"key":"S1351324921000127_ref64","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2010.5716982"},{"key":"S1351324921000127_ref35","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-2012"},{"key":"S1351324921000127_ref36","unstructured":"Lafferty, J.D. , McCallum, A. and Pereira, F.C.N. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML\u201901, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc., pp. 282\u2013289."},{"key":"S1351324921000127_ref58","doi-asserted-by":"publisher","DOI":"10.1002\/9780470689646.ch1"},{"key":"S1351324921000127_ref1","unstructured":"Baevski, A. and Auli, M. (2019). Adaptive input representations for neural language modeling. In 7th International Conference on Learning Representations (ICLR), New Orleans, LA, USA. OpenReview.net."},{"key":"S1351324921000127_ref60","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-31372-2_26"},{"key":"S1351324921000127_ref17","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324919000457"},{"key":"S1351324921000127_ref28","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1154"},{"key":"S1351324921000127_ref30","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1031"},{"key":"S1351324921000127_ref19","doi-asserted-by":"publisher","DOI":"10.1109\/PROC.1973.9030"},{"key":"S1351324921000127_ref29","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-1119"},{"key":"S1351324921000127_ref74","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1447"},{"key":"S1351324921000127_ref63","unstructured":"Vaswani, A. , Shazeer, N. , Parmar, N. , Uszkoreit, J. , Jones, L. , Gomez, A.N. , Kaiser, \u0141. and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems, Vancouver, Canada. Curran Associates, Inc., pp. 5998\u20136008."},{"key":"S1351324921000127_ref18","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1102"},{"key":"S1351324921000127_ref37","doi-asserted-by":"publisher","DOI":"10.1080\/15427951.2004.10129091"},{"key":"S1351324921000127_ref10","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1208"},{"key":"S1351324921000127_ref42","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-2100"},{"key":"S1351324921000127_ref7","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2019.09.013"},{"key":"S1351324921000127_ref45","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1054"},{"key":"S1351324921000127_ref50","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1049"},{"key":"S1351324921000127_ref41","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1279"},{"key":"S1351324921000127_ref5","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"S1351324921000127_ref59","doi-asserted-by":"crossref","unstructured":"Sahrawat, D. , Mahata, D. , Kulkarni, M. , Zhang, H. , Gosangi, R. , Stent, A. , Sharma, A. , Kumar, Y. , Shah, R.R. and Zimmermann, R. (2020). Keyphrase extraction from scholarly articles as sequence labeling using contextualized embeddings. In Proceedings of European Conference on Information Retrieval (ECIR 2020), Lisbon, Portugal. Springer, pp. 328\u2013335.","DOI":"10.1007\/978-3-030-45442-5_41"},{"key":"S1351324921000127_ref70","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2018.00075"},{"key":"S1351324921000127_ref22","doi-asserted-by":"publisher","DOI":"10.1145\/3383583.3398517"},{"key":"S1351324921000127_ref14","first-page":"1","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","author":"Dem\u0161ar","year":"2006","journal-title":"Journal of Machine Learning Research"},{"key":"S1351324921000127_ref9","doi-asserted-by":"publisher","DOI":"10.23919\/FRUCT48121.2019.8981519"},{"key":"S1351324921000127_ref33","unstructured":"Kim, S.N. , Medelyan, O. , Kan, M.-Y. and Baldwin, T. (2010). Semeval-2010 task 5: Automatic keyphrase extraction from scientific articles. In Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval\u201910, Stroudsburg, PA, USA. Association for Computational Linguistics, pp. 21\u201326."},{"key":"S1351324921000127_ref34","unstructured":"Krapivin, M. , Autaeu, A. and Marchese, M. (2009). Large dataset for keyphrases extraction. Technical report, University of Trento."},{"key":"S1351324921000127_ref69","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-19548-3_21"},{"key":"S1351324921000127_ref54","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"S1351324921000127_ref75","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.710"},{"key":"S1351324921000127_ref3","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K18-1022"},{"key":"S1351324921000127_ref31","doi-asserted-by":"publisher","DOI":"10.3115\/1119355.1119383"},{"key":"S1351324921000127_ref40","doi-asserted-by":"publisher","DOI":"10.3115\/1699510.1699544"},{"key":"S1351324921000127_ref73","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2016.10.052"},{"key":"S1351324921000127_ref52","first-page":"e1339","article-title":"A review of keyphrase extraction","volume":"10","author":"Papagiannopoulou","year":"2020","journal-title":"Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery"},{"key":"S1351324921000127_ref72","doi-asserted-by":"publisher","DOI":"10.29007\/mmk4"},{"key":"S1351324921000127_ref15","unstructured":"Devlin, J. , Chang, M.-W. , Lee, K. and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota. Association for Computational Linguistics, pp. 4171\u20134186."},{"key":"S1351324921000127_ref27","doi-asserted-by":"publisher","DOI":"10.1145\/1526709.1526798"},{"key":"S1351324921000127_ref48","unstructured":"Nemenyi, P. (1963). Distribution-Free Multiple Comparisons. PhD Thesis, Princeton University."},{"key":"S1351324921000127_ref16","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2008.05.002"},{"key":"S1351324921000127_ref24","doi-asserted-by":"crossref","unstructured":"Gollapalli, S.D. and Caragea, C. (2014). Extracting keyphrases from research papers using citation networks. In Twenty-Eighth AAAI Conference on Artificial Intelligence, Qu\u00e9bec City, Qu\u00e9bec, Canada. Association for Computing Machinery, pp. 1629\u20131635.","DOI":"10.1609\/aaai.v28i1.8946"},{"key":"S1351324921000127_ref32","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1356"},{"key":"S1351324921000127_ref12","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1285"},{"key":"S1351324921000127_ref62","doi-asserted-by":"publisher","DOI":"10.3115\/1119282.1119287"},{"key":"S1351324921000127_ref23","unstructured":"Goldberg, Y. and Orwant, J. (2013). A dataset of syntactic-ngrams over time from a very large corpus of English books. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity, Atlanta, Georgia, USA. Association for Computational Linguistics, pp. 241\u2013247."},{"key":"S1351324921000127_ref68","unstructured":"Wang, R. , Liu, W. and McDonald, C. (2015a). Corpus-independent generic keyphrase extraction using word embedding vectors. In Proceedings of the Workshop on Deep Learning for Web Search and Data Mining (DL-WSDM), Shanghai, China. Association for Computing Machinery, pp. 39\u201346."},{"key":"S1351324921000127_ref43","doi-asserted-by":"publisher","DOI":"10.3115\/1699648.1699678"},{"key":"S1351324921000127_ref53","first-page":"8024","article-title":"Pytorch: An imperative style, high-performance deep learning library","volume":"32","author":"Paszke","year":"2019","journal-title":"In Advances in Neural Information Processing Systems"},{"key":"S1351324921000127_ref56","unstructured":"Radford, A. , Wu, J. , Child, R. , Luan, D. , Amodei, D. and Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog 1(8)."},{"key":"S1351324921000127_ref6","unstructured":"Bougouin, A. , Boudin, F. and Daille, B. (2013). TopicRank: Graph-based topic ranking for keyphrase extraction. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, Nagoya, Japan. Asian Federation of Natural Language Processing, pp. 543\u2013551."},{"key":"S1351324921000127_ref4","doi-asserted-by":"publisher","DOI":"10.3115\/1219044.1219075"},{"key":"S1351324921000127_ref67","doi-asserted-by":"publisher","DOI":"10.1007\/11739685_66"},{"key":"S1351324921000127_ref44","unstructured":"Meng, R. , Yuan, X. , Wang, T. , Brusilovsky, P. , Trischler, A. and He, D. (2019). Does order matter? an empirical study on generating multiple keyphrases as a sequence. arXiv preprint arXiv:1909.03590."},{"key":"S1351324921000127_ref66","unstructured":"Wan, X. and Xiao, J. (2008). Single document keyphrase extraction using neighborhood knowledge. In Proceedings of the 23rd National Conference on Artificial Intelligence - Volume 2, Chicago, Illinois, USA. AAAI Press, pp. 855\u2013860."},{"key":"S1351324921000127_ref46","doi-asserted-by":"publisher","DOI":"10.1007\/s10844-019-00558-9"},{"key":"S1351324921000127_ref61","doi-asserted-by":"publisher","DOI":"10.1145\/2740908.2742730"},{"key":"S1351324921000127_ref49","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-77094-7_41"},{"key":"S1351324921000127_ref13","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S15-1013"},{"key":"S1351324921000127_ref26","unstructured":"Grave, E. , Joulin, A. , Ciss\u00e9, M. , Grangier, D. and J\u00e9gou, H. (2017). Efficient softmax approximation for gpus. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, Sydney, Australia. Association for Computing Machinery, pp. 1302\u20131310."},{"key":"S1351324921000127_ref76","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1139"},{"key":"S1351324921000127_ref51","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2018.06.004"},{"key":"S1351324921000127_ref2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1371"},{"key":"S1351324921000127_ref8","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-76941-7_80"},{"key":"S1351324921000127_ref55","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324921000127","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,30]],"date-time":"2022-12-30T14:35:46Z","timestamp":1672410946000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324921000127\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,10]]},"references-count":76,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["S1351324921000127"],"URL":"https:\/\/doi.org\/10.1017\/s1351324921000127","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"value":"1351-3249","type":"print"},{"value":"1469-8110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,10]]},"assertion":[{"value":"\u00a9 The Author(s), 2021. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http:\/\/creativecommons.org\/licenses\/by\/4.0\/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.","name":"license","label":"License","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This content has been made available to all.","name":"free","label":"Free to read"}]}}