{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T19:34:18Z","timestamp":1774121658348,"version":"3.50.1"},"reference-count":126,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,3,14]],"date-time":"2024-03-14T00:00:00Z","timestamp":1710374400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61932012, 62141215, 62372228"],"award-info":[{"award-number":["61932012, 62141215, 62372228"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Program B for Outstanding PhD Candidate of Nanjing University","award":["202201B054"],"award-info":[{"award-number":["202201B054"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,3,31]]},"abstract":"<jats:p>\n            (Source) Code summarization aims to automatically generate summaries\/comments for given code snippets in the form of natural language. Such summaries play a key role in helping developers understand and maintain source code. Existing code summarization techniques can be categorized into\n            <jats:italic>extractive methods<\/jats:italic>\n            and\n            <jats:italic>abstractive methods<\/jats:italic>\n            . The\n            <jats:italic>extractive methods<\/jats:italic>\n            extract a subset of important statements and keywords from the code snippet using retrieval techniques and generate a summary that preserves factual details in important statements and keywords. However, such a subset may miss identifier or entity naming, and consequently, the naturalness of the generated summary is usually poor. The\n            <jats:italic>abstractive methods<\/jats:italic>\n            can generate human-written-like summaries leveraging encoder-decoder models. However, the generated summaries often miss important factual details.\n          <\/jats:p>\n          <jats:p>To generate human-written-like summaries with preserved factual details, we propose a novel extractive-and-abstractive framework. The extractive module in the framework performs the task of extractive code summarization, which takes in the code snippet and predicts important statements containing key factual details. The abstractive module in the framework performs the task of abstractive code summarization, which takes in the code snippet and important statements in parallel and generates a succinct and human-written-like natural language summary. We evaluate the effectiveness of our technique, called EACS, by conducting extensive experiments on three datasets involving six programming languages. Experimental results show that EACS significantly outperforms state-of-the-art techniques for all three widely used metrics, including BLEU, METEOR, and ROUGH-L. In addition, the human evaluation demonstrates that the summaries generated by EACS have higher naturalness and informativeness and are more relevant to given code snippets.<\/jats:p>","DOI":"10.1145\/3632742","type":"journal-article","created":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T11:49:08Z","timestamp":1699876148000},"page":"1-39","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":31,"title":["An Extractive-and-Abstractive Framework for Source Code Summarization"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9236-8264","authenticated-orcid":false,"given":"Weisong","family":"Sun","sequence":"first","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9930-7111","authenticated-orcid":false,"given":"Chunrong","family":"Fang","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3380-5564","authenticated-orcid":false,"given":"Yuchen","family":"Chen","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2495-3805","authenticated-orcid":false,"given":"Quanjun","family":"Zhang","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4701-1327","authenticated-orcid":false,"given":"Guanhong","family":"Tao","sequence":"additional","affiliation":[{"name":"Purdue University, West Lafayette, Indiana, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-2193-3696","authenticated-orcid":false,"given":"Yudu","family":"You","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1821-611X","authenticated-orcid":false,"given":"Tingxu","family":"Han","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0957-854X","authenticated-orcid":false,"given":"Yifei","family":"Ge","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-0168-8842","authenticated-orcid":false,"given":"Yuling","family":"Hu","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9036-0063","authenticated-orcid":false,"given":"Bin","family":"Luo","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9592-7022","authenticated-orcid":false,"given":"Zhenyu","family":"Chen","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,3,14]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3314111.3319834"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.449"},{"issue":"4","key":"e_1_3_1_4_2","first-page":"81:1\u201381:37","article-title":"A survey of machine learning for big code and naturalness","volume":"51","author":"Allamanis Miltiadis","year":"2018","unstructured":"Miltiadis Allamanis, Earl T. Barr, Premkumar T. Devanbu, and Charles Sutton. 2018. A survey of machine learning for big code and naturalness. Comput. Surveys 51, 4 (2018), 81:1\u201381:37.","journal-title":"Comput. Surveys"},{"key":"e_1_3_1_5_2","article-title":"Code2seq: Generating sequences from structured representations of code","volume":"1808","author":"Alon Uri","year":"2018","unstructured":"Uri Alon, Omer Levy, and Eran Yahav. 2018. Code2seq: Generating sequences from structured representations of code. CoRR abs\/1808.01400 (2018).","journal-title":"CoRR"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-019-00727-y"},{"key":"e_1_3_1_7_2","first-page":"1","volume-title":"Proceedings of the 3rd International Conference on Learning Representations","author":"Bahdanau Dzmitry","year":"2015","unstructured":"Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations. OpenReview.net, San Diego, CA, USA, 1\u201315."},{"key":"e_1_3_1_8_2","first-page":"65","volume-title":"Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization","author":"Banerjee Satanjeev","year":"2005","unstructured":"Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization. Association for Computational Linguistics, Ann Arbor, Michigan, USA, 65\u201372."},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC52881.2021.00032"},{"key":"e_1_3_1_10_2","first-page":"314","volume-title":"Proceedings of the 8th International Joint Conference on Natural Language Processing","author":"Barone Antonio Valerio Miceli","year":"2017","unstructured":"Antonio Valerio Miceli Barone and Rico Sennrich. 2017. A parallel corpus of Python functions and documentation strings for automated code documentation and code generation. In Proceedings of the 8th International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, Taipei, Taiwan, 314\u2013319."},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3434280"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1063"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1046"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-4012"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-1012"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/1085313.1085331"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1080\/00401706.1964.10490181"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2013.6613829"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2020.113679"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1523"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"e_1_3_1_23_2","article-title":"Structured neural summarization","volume":"1811","author":"Fernandes Patrick","year":"2018","unstructured":"Patrick Fernandes, Miltiadis Allamanis, and Marc Brockschmidt. 2018. Structured neural summarization. CoRR abs\/1811.01824 (2018).","journal-title":"CoRR"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1037\/a0024338"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-016-9475-9"},{"key":"e_1_3_1_26_2","first-page":"340","volume-title":"Proceedings of the 23rd International Conference on Computational Linguistics","author":"Ganesan Kavita","year":"2010","unstructured":"Kavita Ganesan, ChengXiang Zhai, and Jiawei Han. 2010. Opinosis: A graph based approach to abstractive summarization of highly redundant opinions. In Proceedings of the 23rd International Conference on Computational Linguistics. Tsinghua University Press, Beijing, China, 340\u2013348."},{"key":"e_1_3_1_27_2","article-title":"M2TS: Multi-scale multi-modal approach based on transformer for source code summarization","volume":"2203","author":"Gao Yuexiu","year":"2022","unstructured":"Yuexiu Gao and Chen Lyu. 2022. M2TS: Multi-scale multi-modal approach based on transformer for source code summarization. CoRR abs\/2203.09707 (2022).","journal-title":"CoRR"},{"key":"e_1_3_1_28_2","first-page":"354","volume-title":"Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics","author":"Genest Pierre-Etienne","year":"2012","unstructured":"Pierre-Etienne Genest and Guy Lapalme. 2012. Fully abstractive approach to guided summarization. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics. The Association for Computer Linguistics, Jeju Island, Korea, 354\u2013358."},{"key":"e_1_3_1_29_2","article-title":"GitHub","author":"GitHub Inc.","year":"2008","unstructured":"Inc. GitHub. 2008. GitHub. site: https:\/\/github.comAccessed: 2022.","journal-title":"site: https:\/\/github.com"},{"key":"e_1_3_1_30_2","article-title":"Source code summarization with structural relative position guided transformer","volume":"2202","author":"Gong Zi","year":"2022","unstructured":"Zi Gong, Cuiyun Gao, Yasheng Wang, Wenchao Gu, Yun Peng, and Zenglin Xu. 2022. Source code summarization with structural relative position guided transformer. CoRR abs\/2202.06521 (2022).","journal-title":"CoRR"},{"key":"e_1_3_1_31_2","first-page":"746","volume-title":"Proceedings of the 35th International Conference on Automated Software Engineering","author":"Gros David","year":"2020","unstructured":"David Gros, Hariharan Sezhiyan, Prem Devanbu, and Zhou Yu. 2020. Code to comment \u201ctranslation\u201d: Data, metrics, baselining & evaluation. In Proceedings of the 35th International Conference on Automated Software Engineering. IEEE, Melbourne, Australia, 746\u2013757."},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2018.12.011"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/1810295.1810335"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/WCRE.2010.13"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER50967.2021.00038"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3527909"},{"key":"e_1_3_1_37_2","first-page":"138","volume-title":"Proceedings of the 3rd Conference of the Centre for Advanced Studies on Collaborative Research","author":"Hartzman Carl S.","year":"1993","unstructured":"Carl S. Hartzman and Charles F. Austin. 1993. Maintenance productivity: Observations based on an experience in a large system environment. In Proceedings of the 3rd Conference of the Centre for Advanced Studies on Collaborative Research. IBM, Toronto, Ontario, Canada, 138\u2013170."},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1013"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196321.3196334"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-019-09730-9"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.5555\/3304889.3304975"},{"key":"e_1_3_1_43_2","article-title":"CodeSearchNet challenge: Evaluating the state of semantic code search","volume":"1909","author":"Husain Hamel","year":"2019","unstructured":"Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. 2019. CodeSearchNet challenge: Evaluating the state of semantic code search. CoRR abs\/1909.09436 (2019).","journal-title":"CoRR"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1195"},{"key":"e_1_3_1_45_2","first-page":"1","volume-title":"Proceedings of the 24th International Conference on Knowledge Discovery and Data Mining","author":"Reddy Peddamail Jayavardhan","year":"2018","unstructured":"Peddamail Jayavardhan Reddy, Ziyu Yao, Zhen Wang, and Huan Sun. 2018. A comprehensive study of StaQC for deep code summarization. In Proceedings of the 24th International Conference on Knowledge Discovery and Data Mining. ACM, London, UK, 1\u20138."},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1023\/B:LIDA.0000048322.42751.ca"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2015.01.070"},{"key":"e_1_3_1_48_2","first-page":"1","volume-title":"Proceedings of the 3rd International Conference on Learning Representations \u2013 Poster","author":"Kingma Diederik P.","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations \u2013 Poster. OpenReview.net, San Diego, CA, USA, 1\u201315."},{"key":"e_1_3_1_49_2","first-page":"1","volume-title":"Proceedings of the 1st Text Analysis Conference","author":"Kireyev Kirill","year":"2008","unstructured":"Kirill Kireyev. 2008. Using latent semantic analysis for extractive summarization. In Proceedings of the 1st Text Analysis Conference. NIST, Gaithersburg, Maryland, USA, 1\u20134."},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2006.02.040"},{"key":"e_1_3_1_51_2","volume-title":"Proceedings of the 8th International Joint Conference on Natural Language Processing","author":"Kurisinkel Litton J.","year":"2017","unstructured":"Litton J. Kurisinkel, Yue Zhang, and Vasudeva Varma. 2017. Abstractive multi-document summarization by partial tree extraction, recombination and linearization. In Proceedings of the 8th International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, Taipei, Taiwan."},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387904.3389268"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00087"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1394"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678724"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC52881.2021.00026"},{"key":"e_1_3_1_58_2","first-page":"74","volume-title":"Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics \u2013 Workshop on Text Summarization Branches Out","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics \u2013 Workshop on Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74\u201381."},{"key":"e_1_3_1_59_2","article-title":"Automatic code summarization via multi-dimensional semantic fusing in GNN","volume":"2006","author":"Liu Shangqing","year":"2020","unstructured":"Shangqing Liu, Yu Chen, Xiaofei Xie, Jing Kai Siow, and Yang Liu. 2020. Automatic code summarization via multi-dimensional semantic fusing in GNN. CoRR abs\/2006.05405 (2020).","journal-title":"CoRR"},{"key":"e_1_3_1_60_2","article-title":"Fine-tune BERT for extractive summarization","volume":"1903","author":"Liu Yang","year":"2019","unstructured":"Yang Liu. 2019. Fine-tune BERT for extractive summarization. CoRR abs\/1903.10318 (2019).","journal-title":"CoRR"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1387"},{"key":"e_1_3_1_62_2","first-page":"1","article-title":"RoBERTa: A robustly optimized BERT pretraining approach","volume":"1907","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. CoRR abs\/1907.11692 (2019), 1\u201313.","journal-title":"CoRR"},{"key":"e_1_3_1_63_2","first-page":"1","volume-title":"Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks","author":"Lu Shuai","year":"2021","unstructured":"Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin B. Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, Shao Kun Deng, Shengyu Fu, and Shujie Liu. 2021. CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. virtual, 1\u201314."},{"key":"e_1_3_1_64_2","volume-title":"Clean Code: A Handbook of Agile Software Craftsmanship","author":"Martin Robert C.","year":"2009","unstructured":"Robert C. Martin. 2009. Clean Code: A Handbook of Agile Software Craftsmanship. Pearson Education."},{"key":"e_1_3_1_65_2","volume-title":"Advances in Automatic Text Summarization","author":"Maybury Mani","year":"1999","unstructured":"Mani Maybury. 1999. Advances in Automatic Text Summarization. MIT Press."},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2015.2465386"},{"key":"e_1_3_1_67_2","article-title":"CodeBERT.","year":"2021","unstructured":"Microsoft. 2021. CodeBERT. site: https:\/\/github.com\/microsoft\/CodeBERT. Accessed August, 2022.","journal-title":"site: https:\/\/github.com\/microsoft\/CodeBERT"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273576"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.5220\/0007678100150026"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2013.6613830"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.10958"},{"key":"e_1_3_1_72_2","article-title":"Classify or select: Neural architectures for extractive document summarization","volume":"1611","author":"Nallapati Ramesh","year":"2016","unstructured":"Ramesh Nallapati, Bowen Zhou, and Mingbo Ma. 2016. Classify or select: Neural architectures for extractive document summarization. CoRR abs\/1611.04244 (2016).","journal-title":"CoRR"},{"key":"e_1_3_1_73_2","article-title":"ChatGPT.","year":"2022","unstructured":"OpenAI. 2022. ChatGPT. site: https:\/\/chat-gpt.org\/Accessed March, 2023.","journal-title":"site: https:\/\/chat-gpt.org\/"},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-4407"},{"key":"e_1_3_1_75_2","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. ACL, Philadelphia, PA, USA, 311\u2013318."},{"key":"e_1_3_1_76_2","article-title":"CoTexT: Multi-task learning with code-text transformer","volume":"2105","author":"Phan Long N.","year":"2021","unstructured":"Long N. Phan, Hieu Tran, Daniel Le, Hieu Nguyen, James T. Anibal, Alec Peltekian, and Yanfang Ye. 2021. CoTexT: Multi-task learning with code-text transformer. CoRR abs\/2105.08645 (2021).","journal-title":"CoRR"},{"key":"e_1_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.748"},{"key":"e_1_3_1_78_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W15-3049"},{"issue":"140","key":"e_1_3_1_79_2","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21, 140 (2020), 1\u201367.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_1_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/1810295.1810464"},{"key":"e_1_3_1_81_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1410"},{"key":"e_1_3_1_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568247"},{"key":"e_1_3_1_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468588"},{"key":"e_1_3_1_84_2","article-title":"CodeT5.","year":"2021","unstructured":"Salesforce. 2021. CodeT5. site: https:\/\/github.com\/salesforce\/CodeT5Accessed August, 2022.","journal-title":"site: https:\/\/github.com\/salesforce\/CodeT5"},{"key":"e_1_3_1_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/361219.361220"},{"key":"e_1_3_1_86_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1099"},{"key":"e_1_3_1_87_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC52881.2021.00049"},{"key":"e_1_3_1_88_2","volume-title":"CoRR","author":"Shi Ensheng","year":"2021","unstructured":"Ensheng Shi, Yanlin Wang, Lun Du, Junjie Chen, Shi Han, Hongyu Zhang, Dongmei Zhang, and Hongbin Sun. 2021. Neural code summarization: How far are we?. In CoRR, Vol. abs\/2107.07112."},{"key":"e_1_3_1_89_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510060"},{"key":"e_1_3_1_90_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.332"},{"issue":"1","key":"e_1_3_1_91_2","first-page":"1:1\u20131:37","article-title":"Neural abstractive text summarization with sequence-to-sequence models","volume":"2","author":"Shi Tian","year":"2021","unstructured":"Tian Shi, Yaser Keneshloo, Naren Ramakrishnan, and Chandan K. Reddy. 2021. Neural abstractive text summarization with sequence-to-sequence models. Transactions on Data Science 2, 1 (2021), 1:1\u20131:37.","journal-title":"Transactions on Data Science"},{"key":"e_1_3_1_92_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2019.8851751"},{"key":"e_1_3_1_93_2","unstructured":"GraphPad Software. 1995. Friedman\u2019s test. site: https:\/\/www.graphpad.com\/guides\/prism\/latest\/statistics\/how_the_friedman_test_works.htm Accessed March 2023."},{"key":"e_1_3_1_94_2","unstructured":"GraphPad Software. 1995. Which multiple comparisons tests does Prism offer? https:\/\/www.graphpad.com\/guides\/prism\/latest\/statistics\/stat_summary_of_multiple_comparison.htm. Accessed March 2023"},{"key":"e_1_3_1_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/1858996.1859006"},{"key":"e_1_3_1_96_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387904.3389258"},{"key":"e_1_3_1_97_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04208-9_14"},{"key":"e_1_3_1_98_2","first-page":"1","volume-title":"Proceedings of the 7th International Conference ISIM","author":"Steinberger Josef","year":"2004","unstructured":"Josef Steinberger and Karel Jezek. 2004. Using latent semantic analysis in text summarization and summary evaluation. In Proceedings of the 7th International Conference ISIM. 1\u20138."},{"key":"e_1_3_1_99_2","unstructured":"Weisong Sun Chunrong Fang Yuchen Chen Quanjun Zhang Guanhong Tao Yudu You Tingxu Han Yifei Ge Yuling Hu Bin Luo and Zhenyu Chen. 2022. EACS. https:\/\/github.com\/wssun\/EACS"},{"key":"e_1_3_1_100_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510140"},{"key":"e_1_3_1_101_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.6171"},{"issue":"1","key":"e_1_3_1_102_2","first-page":"19","article-title":"The need to report effect size estimates revisited. An overview of some recommended measures of effect size.","volume":"21","author":"Tomczak Maciej","year":"2014","unstructured":"Maciej Tomczak and Ewa Tomczak. 2014. The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Trends in Sport Sciences 21, 1 (2014), 19\u201325.","journal-title":"Trends in Sport Sciences"},{"key":"e_1_3_1_103_2","first-page":"5998","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems. Curran Associates Inc., Long Beach, CA, USA, 5998\u20136008."},{"key":"e_1_3_1_104_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238206"},{"key":"e_1_3_1_105_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigDataCongress.2017.46"},{"key":"e_1_3_1_106_2","first-page":"1","article-title":"Reinforcement-learning-guided source code summarization using hierarchical attention","author":"Wang Wenhua","year":"2020","unstructured":"Wenhua Wang, Yuqun Zhang, Yulei Sui, Yao Wan, Zhou Zhao, Jian Wu, Philip Yu, and Guandong Xu. 2020. Reinforcement-learning-guided source code summarization using hierarchical attention. IEEE Transactions on Software Engineering (Early Access) (2020), 1\u201319.","journal-title":"IEEE Transactions on Software Engineering (Early Access)"},{"key":"e_1_3_1_107_2","first-page":"1","article-title":"CoCoSum: Contextual code summarization with multi-relational graph neural network","volume":"2107","author":"Wang Yanlin","year":"2021","unstructured":"Yanlin Wang, Ensheng Shi, Lun Du, Xiaodi Yang, Yuxuan Hu, Shi Han, Hongyu Zhang, and Dongmei Zhang. 2021. CoCoSum: Contextual code summarization with multi-relational graph neural network. CoRR abs\/2107.01933 (2021), 1\u201324.","journal-title":"CoRR"},{"key":"e_1_3_1_108_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.685"},{"key":"e_1_3_1_109_2","first-page":"6559","volume-title":"Proceedings of the 33rd Annual Conference on Neural Information Processing Systems (NeurIPS 2019)","author":"Wei Bolin","year":"2019","unstructured":"Bolin Wei, Ge Li, Xin Xia, Zhiyi Fu, and Zhi Jin. 2019. Code generation as a dual task of code summarization. In Proceedings of the 33rd Annual Conference on Neural Information Processing Systems (NeurIPS 2019). Vancouver, BC, Canada, 6559\u20136569."},{"key":"e_1_3_1_110_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416578"},{"key":"e_1_3_1_111_2","volume-title":"Critical Values and Probability Levels for the Wilcoxon Rank Sum Test and the Wilcoxon Signed Rank Test","author":"Wilcoxon Frank","year":"1963","unstructured":"Frank Wilcoxon, S. K. Katti, and Roberta A. Wilcox. 1963. Critical Values and Probability Levels for the Wilcoxon Rank Sum Test and the Wilcoxon Signed Rank Test. American Cyanamid Company."},{"key":"e_1_3_1_112_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-29044-2"},{"key":"e_1_3_1_113_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2015.7081848"},{"key":"e_1_3_1_114_2","doi-asserted-by":"publisher","DOI":"10.5555\/800078.802534"},{"key":"e_1_3_1_115_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-acl.93"},{"key":"e_1_3_1_116_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11987"},{"key":"e_1_3_1_117_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2734091"},{"key":"e_1_3_1_118_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC52881.2021.00010"},{"key":"e_1_3_1_119_2","first-page":"1","article-title":"Efficient summarization with read-again and copy mechanism","volume":"1611","author":"Zeng Wenyuan","year":"2016","unstructured":"Wenyuan Zeng, Wenjie Luo, Sanja Fidler, and Raquel Urtasun. 2016. Efficient summarization with read-again and copy mechanism. CoRR abs\/1611.03382 (2016), 1\u201311.","journal-title":"CoRR"},{"key":"e_1_3_1_120_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380427"},{"key":"e_1_3_1_121_2","article-title":"Learning to generate code comments from class hierarchies","volume":"2103","author":"Zhang Jiyang","year":"2021","unstructured":"Jiyang Zhang, Sheena Panthaplackel, Pengyu Nie, Raymond J. Mooney, Junyi Jessy Li, and Milos Gligoric. 2021. Learning to generate code comments from class hierarchies. CoRR abs\/2103.13426 (2021).","journal-title":"CoRR"},{"key":"e_1_3_1_122_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380383"},{"key":"e_1_3_1_123_2","first-page":"11328","volume-title":"Proceedings of the 37th International Conference on Machine Learning","author":"Zhang Jingqing","year":"2020","unstructured":"Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2020. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the 37th International Conference on Machine Learning. PMLR, Virtual Event, 11328\u201311339."},{"key":"e_1_3_1_124_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1061"},{"issue":"1","key":"e_1_3_1_125_2","first-page":"1","article-title":"Adversarial robustness of deep code comment generation","volume":"1","author":"Zhou Yu","year":"2021","unstructured":"Yu Zhou, Xiaoqing Zhang, Juanjuan Shen, Tingting Han, Taolue Chen, and Harald C. Gall. 2021. Adversarial robustness of deep code comment generation. ACM Transactions on Software Engineering and Methodology 1, 1 (2021), 1\u201330.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_1_126_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416530"},{"key":"e_1_3_1_127_2","article-title":"Automatic code summarization: A systematic literature review","volume":"1909","author":"Zhu Yuxiang","year":"2019","unstructured":"Yuxiang Zhu and Minxue Pan. 2019. Automatic code summarization: A systematic literature review. CoRR abs\/1909.04352 (2019).","journal-title":"CoRR"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3632742","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3632742","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:51:04Z","timestamp":1750287064000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3632742"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,14]]},"references-count":126,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,3,31]]}},"alternative-id":["10.1145\/3632742"],"URL":"https:\/\/doi.org\/10.1145\/3632742","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,14]]},"assertion":[{"value":"2022-08-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-03","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}