{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T15:22:21Z","timestamp":1770996141989,"version":"3.50.1"},"reference-count":65,"publisher":"Association for Computing Machinery (ACM)","issue":"3","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,3,31]]},"abstract":"<jats:p>Part-of-Speech (POS) tags are natural attributes of words in natural languages, and they are fundamental for natural language analysis. Many automated approaches have been proposed to tag natural language texts. Identifiers in source code have POS tags as well, which are useful for various source code analysis tasks, like code search, code comment generation, and code completion. Currently, state-of-the-art POS taggers originally designed for natural languages are often employed to tag source code identifiers. However, identifiers in source code are significantly different from natural languages. Consequently, POS taggers designed for natural languages could be less accurate in source code identifiers. Recently, several identifier-specific taggers have been proposed within the field of software engineering, but their adoption in practical software engineering tasks remains limited. This raises the question of why these taggers have not been more widely utilized in such tasks.<\/jats:p>\n                  <jats:p>In this article, we investigate the performance of natural language POS taggers on source code identifiers, specifically method names, parameter names, and class names. To do so, we manually annotated identifiers from open source projects in Java, C, and Python, creating a large dataset IDData for evaluation. We then evaluated six widely used natural language POS taggers: NLTK, CoreNLP, OpenNLP, spaCy, Flair, and Stanza, alongside three identifier-specific taggers: SWUM, POSSE, and Ensemble Tagger. Our evaluation reveals that while natural language-oriented POS taggers outperform identifier-specific taggers, their performance on identifiers is still significantly lower compared to their performance on natural language sentences. To understand the underlying reasons for this, we conducted an in-depth analysis, examining factors such as identifier length, POS distribution, syntactic structures, and special tags, which differentiate identifiers from natural language sentences. To further improve POS tagging performance on identifiers, we created a large-scale method name dataset MNTrain with manually labeled tags and retrained the natural language taggers on this new dataset. The results show substantial improvements in method name POS tagging performance, with taggers achieving performance comparable to their results on natural language sentences. Finally, we discuss the significance and practical implications of our findings, offering insights for future research.<\/jats:p>","DOI":"10.1145\/3744919","type":"journal-article","created":{"date-parts":[[2025,6,23]],"date-time":"2025-06-23T08:13:01Z","timestamp":1750666381000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["POS Tagging on Code Identifiers: How Far Are We?"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0306-1637","authenticated-orcid":false,"given":"Hanlin","family":"Tang","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6404-9143","authenticated-orcid":false,"given":"Yanjie","family":"Jiang","sequence":"additional","affiliation":[{"name":"College of Intelligence and Computing, Tianjin University, Tianjin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9371-5931","authenticated-orcid":false,"given":"Yuxia","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5566-2368","authenticated-orcid":false,"given":"Nan","family":"Niu","sequence":"additional","affiliation":[{"name":"EECS, University of Cincinnati, Cincinnati, Ohio, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3267-6801","authenticated-orcid":false,"given":"Hui","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,2,13]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"496","volume-title":"Proceedings of the 2017 IEEE 25th International Requirements Engineering Conference (RE)","author":"Abad Zahra Shakeri Hossein","year":"2017","unstructured":"Zahra Shakeri Hossein Abad, Oliver Karras, Parisa Ghazi, Martin Glinz, Guenther Ruhe, and Kurt Schneider. 2017. What works better? A study of classifying requirements. In Proceedings of the 2017 IEEE 25th International Requirements Engineering Conference (RE). IEEE, 496\u2013501."},{"key":"e_1_3_1_3_2","first-page":"54","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)","author":"Akbik Alan","year":"2019","unstructured":"Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019. FLAIR: An easy-to-use framework for state-of-the-art Nlp. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), 54\u201359."},{"key":"e_1_3_1_4_2","first-page":"2091","volume-title":"Proceedings of the International Conference on Machine Learning (ICML)","author":"Allamanis Miltiadis","year":"2016","unstructured":"Miltiadis Allamanis, Hao Peng, and Charles Sutton. 2016. A convolutional attention network for extreme summarization of source code. In Proceedings of the International Conference on Machine Learning (ICML), 2091\u20132100."},{"key":"e_1_3_1_5_2","first-page":"1","volume-title":"Proceedings of the 2015 IEEE 5th Workshop on Mining Unstructured Data (MUD)","author":"Alsuhaibani Reem S.","year":"2015","unstructured":"Reem S. Alsuhaibani, Christian D. Newman, Michael L. Collard, and Jonathan I. Maletic. 2015. Heuristic-based part-of-speech tagging of source code identifiers and comments. In Proceedings of the 2015 IEEE 5th Workshop on Mining Unstructured Data (MUD). IEEE, 1\u20136."},{"key":"e_1_3_1_6_2","volume-title":"PC-KIMMO: A Two-Level Processor for Morphological Analysis","author":"Antworth Evan L.","year":"1990","unstructured":"Evan L. Antworth. 1990. PC-KIMMO: A Two-Level Processor for Morphological Analysis. Summer Institute of Linguistics."},{"key":"e_1_3_1_7_2","unstructured":"Apache. 2024. Apache OpenNLP. Retrieved from https:\/\/opennlp.apache.org\/"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.3115\/1220355.1220435"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-015-0226-2"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11749-016-0481-7"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/1985441.1985471"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.3115\/1225403.1225421"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.3115\/1075527.1075553"},{"key":"e_1_3_1_14_2","unstructured":"Eric Brill. 1994. Some advances in transformation-based part of speech tagging. arXiv:cmp-lg\/9406010. Retrieved from https:\/\/arxiv.org\/abs\/cmp-lg\/9406010"},{"issue":"4","key":"e_1_3_1_15_2","first-page":", 543","article-title":"Transformation-based error-driven learning and natural language processing: A case study in part-of-speech tagging","volume":"21","author":"Brill Eric","year":"1995","unstructured":"Eric Brill. 1995. Transformation-based error-driven learning and natural language processing: A case study in part-of-speech tagging. Computational Linguistics 21, 4 (1995), 543\u2013565.","journal-title":"Computational Linguistics"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","first-page":"516","DOI":"10.1109\/ICSM.2013.85","volume-title":"Proceedings of the 2013 IEEE International Conference on Software Maintenance","author":"Collard Michael L.","year":"2013","unstructured":"Michael L. Collard, Michael John Decker, and Jonathan I. Maletic. 2013. srcML: An infrastructure for the exploration, analysis, and manipulation of source code: A tool demonstration. In Proceedings of the 2013 IEEE International Conference on Software Maintenance. IEEE, 516\u2013519."},{"key":"e_1_3_1_17_2","first-page":"133","volume-title":"Proceedings of the 3rd Conference on Applied Natural Language Processing","author":"Cutting Douglass","unstructured":"Douglass Cutting, Julian Kupiec, Jan Pedersen, and Penelope Sibun. 1992. A practical part-of-speech tagger. In Proceedings of the 3rd Conference on Applied Natural Language Processing, 133\u2013140."},{"issue":"2","key":"e_1_3_1_18_2","first-page":"255","article-title":"Universal dependencies","volume":"47","author":"Marneffe Marie-Catherine De","year":"2021","unstructured":"Marie-Catherine De Marneffe, Christopher D. Manning, Joakim Nivre, and Daniel Zeman. 2021. Universal dependencies. Computational Linguistics 47, 2 (2021), 255\u2013308.","journal-title":"Computational Linguistics"},{"key":"e_1_3_1_19_2","unstructured":"Explosion. 2023. spaCy: Industrial-Strength NLP. Retrieved from https:\/\/spacy.io\/"},{"key":"e_1_3_1_20_2","unstructured":"The Python Software Foundation. 2019. ast\u2014Abstract Syntax Trees. Retrieved from https:\/\/docs.python.org\/3\/library\/ast.html"},{"key":"e_1_3_1_21_2","first-page":"744","volume-title":"Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics","author":"Goldwater Sharon","year":"2007","unstructured":"Sharon Goldwater and Tom Griffiths. 2007. A fully Bayesian approach to unsupervised part-of-speech tagging. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, 744\u2013751."},{"key":"e_1_3_1_22_2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/ICPC.2013.6613828","volume-title":"Proceedings of the 2013 21st International Conference on Program Comprehension (ICPC)","author":"Gupta Samir","year":"2013","unstructured":"Samir Gupta, Sana Malik, Lori Pollock, and K. Vijay-Shanker. 2013. Part-of-speech tagging of program identifiers for improved text-based software engineering tools. In Proceedings of the 2013 21st International Conference on Program Comprehension (ICPC). IEEE, 3\u201312."},{"key":"e_1_3_1_23_2","unstructured":"Zhiyong He Zanbo Wang Wei Wei Shanshan Feng Xianling Mao and Sheng Jiang. 2020. A survey on recent advances in sequence labeling from deep learning models. arXiv:2011.06727. Retrieved from https:\/\/arxiv.org\/abs\/2011.06727"},{"key":"e_1_3_1_24_2","volume-title":"Integrating Natural Language and Program Structure Information to Improve Software Search and Exploration","author":"Hill Emily","year":"2010","unstructured":"Emily Hill. 2010. Integrating Natural Language and Program Structure Information to Improve Software Search and Exploration. University of Delaware."},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00653"},{"key":"e_1_3_1_26_2","unstructured":"JavaParser. 2019. JavaParser: Analyse Transform and Generate Your Java Codebase. Retrieved from https:\/\/javaparser.org\/"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-76626-8_35"},{"key":"e_1_3_1_28_2","first-page":"1306","volume-title":"Proceedings of the 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Jin Shuo","year":"2021","unstructured":"Shuo Jin, Songqiang Chen, and Xiaoyuan Xie. 2021. Property-based test for part-of-speech tagging tool. In Proceedings of the 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE). IEEE, 1306\u20131311."},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2022.3188107"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.5555\/555733"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1249"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/0885-2308(92)90019-Z"},{"issue":"2","key":"e_1_3_1_33_2","first-page":"1","article-title":"A survey on renamings of software entities","volume":"53","author":"Li Guangjie","year":"2020","unstructured":"Guangjie Li, Hui Liu, and Ally S. Nyamawe. 2020. A survey on renamings of software entities. ACM Computing Surveys 53, 2 (2020), 1\u201338.","journal-title":"ACM Computing Surveys"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-4421"},{"key":"e_1_3_1_35_2","doi-asserted-by":"crossref","unstructured":"Wang Ling Tiago Lu\u00eds Lu\u00eds Marujo Ram\u00f3n Fernandez Astudillo Silvio Amir Chris Dyer Alan W. Black and Isabel Trancoso. 2015. Finding function in form: Compositional character models for open vocabulary word representation. arXiv:1508.02096. Retrieved from https:\/\/arxiv.org\/abs\/1508.02096","DOI":"10.18653\/v1\/D15-1176"},{"key":"e_1_3_1_36_2","first-page":"5253","volume-title":"Proceedings of the 32rd AAAI Conference on Artificial Intelligence","author":"Liu Liyuan","year":"2018","unstructured":"Liyuan Liu, Jingbo Shang, Xiang Ren, Frank Xu, Huan Gui, Jian Peng, and Jiawei Han. 2018. Empower sequence labeling with task-aware neural language model. In Proceedings of the 32rd AAAI Conference on Artificial Intelligence, 5253\u20135260."},{"key":"e_1_3_1_37_2","first-page":"545","volume-title":"Proceedings of the 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER)","author":"Lu Meili","year":"2015","unstructured":"Meili Lu, Xiaobing Sun, Shaowei Wang, David Lo, and Yucong Duan. 2015. Query expansion via wordnet for effective code search. In Proceedings of the 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER). IEEE, 545\u2013549."},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.5555\/311445"},{"key":"e_1_3_1_39_2","first-page":"171","volume-title":"International Conference on Intelligent Text Processing and Computational Linguistics","author":"Manning Christopher D.","year":"2011","unstructured":"Christopher D. Manning. 2011. Part-of-speech tagging from 97% to 100%: Is it time for some linguistics? In International Conference on Intelligent Text Processing and Computational Linguistics. Springer, 171\u2013189."},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-5010"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.5555\/972470.972475"},{"issue":"1","key":"e_1_3_1_42_2","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1023\/A:1007673816718","article-title":"A machine learning approach to POS tagging","volume":"39","author":"Marquez Lluis","year":"2000","unstructured":"Lluis Marquez, Lluis Padro, and Horacio Rodriguez. 2000. A machine learning approach to POS tagging. Machine Learning 39, 1 (2000), 59\u201391.","journal-title":"Machine Learning"},{"issue":"1","key":"e_1_3_1_43_2","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1002\/wics.195","article-title":"Part-of-speech tagging","volume":"4","author":"Martinez Angel R.","year":"2012","unstructured":"Angel R. Martinez. 2012. Part-of-speech tagging. Wiley Interdisciplinary Reviews: Computational Statistics 4, 1 (2012), 107\u2013113.","journal-title":"Wiley Interdisciplinary Reviews: Computational Statistics"},{"key":"e_1_3_1_44_2","first-page":"303","volume-title":"Proceedings of the 12th International Conference on Information and Knowledge Management","author":"Mayfield James","year":"2003","unstructured":"James Mayfield, Paul McNamee, Christine Piatko, and Claudia Pearce. 2003. Lattice-based tagging using support vector machines. In Proceedings of the 12th International Conference on Information and Knowledge Management, 303\u2013308."},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/219717.219748"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1002\/cem.873"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110740"},{"issue":"9","key":"e_1_3_1_48_2","doi-asserted-by":"crossref","first-page":"3506","DOI":"10.1109\/TSE.2021.3098242","article-title":"An ensemble approach for annotating source code identifiers with part-of-speech tags","volume":"48","author":"Newman Christian D.","year":"2021","unstructured":"Christian D. Newman, Michael J. Decker, Reem S. Alsuhaibani, Anthony Peruma, Mohamed Wiem Mkaouer, Satyajit Mohapatra, Tejal Vishnoi, Marcos Zampieri, Timothy J. Sheldon, and Emily Hill. 2021. An ensemble approach for annotating source code identifiers with part-of-speech tags. IEEE Transactions on Software Engineering 48, 9 (2021), 3506\u20133522.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_1_49_2","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1109\/ICSME.2016.80","volume-title":"Proceedings of the 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME)","author":"Olney Wyatt","year":"2016","unstructured":"Wyatt Olney, Emily Hill, Chris Thurber, and Bezalem Lemma. 2016. Part of speech tagging Java method names. In Proceedings of the 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 483\u2013487."},{"key":"e_1_3_1_50_2","first-page":"1278","volume-title":"Proceedings of the 5th International Conference on Language Resources and Evaluation","author":"Popovic Maja","year":"2006","unstructured":"Maja Popovic and Hermann Ney. 2006. POS-based word reorderings for statistical machine translation. In Proceedings of the 5th International Conference on Language Resources and Evaluation, 1278\u20131283."},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-demos.14"},{"key":"e_1_3_1_52_2","first-page":"133","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing","author":"Ratnaparkhi Adwait","year":"1996","unstructured":"Adwait Ratnaparkhi. 1996. A maximum entropy model for part-of-speech tagging. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 133\u2013142."},{"key":"e_1_3_1_53_2","unstructured":"Beatrice Santorini. 1990. Part-Of-Speech Tagging Guidelines for the Penn Treebank Project. (3rd Revision). Technical Reports (CIS) 570."},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-022-07872-4"},{"key":"e_1_3_1_55_2","volume-title":"Concise Oxford English Dictionary","author":"Soanes Catherine","year":"2004","unstructured":"Catherine Soanes and Angus Stevenson. 2004. Concise Oxford English Dictionary. Vol. 11. Oxford University Press, Oxford."},{"key":"e_1_3_1_56_2","unstructured":"Hanlin Tang. 2025. TaggerEva. Retrieved from https:\/\/github.com\/hanlintang\/TaggerEva"},{"key":"e_1_3_1_57_2","first-page":"570","volume-title":"Proceedings of the 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER)","author":"Tian Yuan","year":"2015","unstructured":"Yuan Tian and David Lo. 2015. A comparative study on the effectiveness of part-of-speech tagging techniques on bug reports. In Proceedings of the 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER). IEEE, 570\u2013574."},{"key":"e_1_3_1_58_2","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1145\/2875913.2875919","volume-title":"Proceedings of the 7th Asia-Pacific Symposium on Internetware","author":"Tong Yanxiang","year":"2015","unstructured":"Yanxiang Tong, Yu Zhou, Lisheng Fang, and Taolue Chen. 2015. Towards a novel approach for defect localization based on part-of-speech and invocation. In Proceedings of the 7th Asia-Pacific Symposium on Internetware, 52\u201361."},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.3115\/1073445.1073478"},{"key":"e_1_3_1_60_2","first-page":"63","volume-title":"Proceedings of the 2000 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora","author":"Toutanvoa Kristina","year":"2000","unstructured":"Kristina Toutanvoa and Christopher D. Manning. 2000. Enriching the knowledge sources used in a maximum entropy part-of-speech tagger. In Proceedings of the 2000 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, 63\u201370."},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884800"},{"key":"e_1_3_1_62_2","first-page":"90","volume-title":"Proceedings of the 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER)","author":"Ye Deheng","year":"2016","unstructured":"Deheng Ye, Zhenchang Xing, Chee Yong Foo, Zi Qun Ang, Jing Li, and Nachiket Kapre. 2016. Software-specific named entity recognition in software engineering social content. In Proceedings of the 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER). IEEE, 90\u2013101."},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-016-9343-x"},{"key":"e_1_3_1_64_2","doi-asserted-by":"crossref","first-page":"964","DOI":"10.1145\/3368089.3409674","volume-title":"Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering","author":"Zhao Yutong","year":"2020","unstructured":"Yutong Zhao, Lu Xiao, Pouria Babvey, Lei Sun, Sunny Wong, Angel A. Martinez, and Xiao Wang. 2020. Automatically identifying performance issue reports with heuristic linguistic patterns. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 964\u2013975."},{"issue":"6","key":"e_1_3_1_65_2","doi-asserted-by":"crossref","first-page":"925","DOI":"10.1142\/S0218194017500346","article-title":"Augmenting bug localization with part-of-speech and invocation","volume":"27","author":"Zhou Yu","year":"2017","unstructured":"Yu Zhou, Yanxiang Tong, Taolue Chen, and Jin Han. 2017. Augmenting bug localization with part-of-speech and invocation. International Journal of Software Engineering and Knowledge Engineering 27, 6 (2017), 925\u2013949.","journal-title":"International Journal of Software Engineering and Knowledge Engineering"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.07.087"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3744919","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T14:38:49Z","timestamp":1770993529000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3744919"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,13]]},"references-count":65,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,3,31]]}},"alternative-id":["10.1145\/3744919"],"URL":"https:\/\/doi.org\/10.1145\/3744919","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,13]]},"assertion":[{"value":"2024-06-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-06-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}