{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T11:52:22Z","timestamp":1791460342701,"version":"4.3.4"},"reference-count":70,"publisher":"Association for Computing Machinery (ACM)","issue":"7","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U24A20337, 61932012, 62372228"],"award-info":[{"award-number":["U24A20337, 61932012, 62372228"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Shenzhen-Hong Kong-Macau Technology Research Programme","award":["SGDX20230821091559018"],"award-info":[{"award-number":["SGDX20230821091559018"]}]},{"name":"CCF-Huawei Populus Grove","award":["CCF-HuaweiSE202304, CCF-HuaweiSY202306"],"award-info":[{"award-number":["CCF-HuaweiSE202304, CCF-HuaweiSY202306"]}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"crossref","award":["14380029"],"award-info":[{"award-number":["14380029"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,9,30]]},"abstract":"<jats:p>Unit testing validates the correctness of the units of the software system under test and serves as the cornerstone in improving software quality and reliability. To reduce manual efforts in writing unit tests, some techniques have been proposed to generate test assertions automatically, including Deep Learning (DL)-based, retrieval-based, and integration-based ones. Among them, recent integration-based approaches inherit from both DL-based and retrieval-based approaches and are considered state-of-the-art. Despite being promising, such integration-based approaches suffer from inherent limitations, such as retrieving assertions with lexical matching while ignoring meaningful code semantics and generating assertions with a limited training corpus.<\/jats:p>\n                  <jats:p>\n                    In this article, we propose a novel Retrieval-Augmented Deep Assertion Generation (RetriGen) approach based on a hybrid assertion retriever and a Pre-Trained Language Model (PLM)-based assertion generator. Given a focal-test, RetriGen first builds a hybrid assertion retriever to search for the most relevant test\u2013assert pair from external codebases. The retrieval process takes both lexical similarity and semantical similarity into account via a token-based and an embedding-based retriever, respectively. RetriGen then treats assertion generation as a sequence-to-sequence task and designs a PLM-based assertion generator to predict a correct assertion with historical test\u2013assert pairs and the retrieved external assertion. Although our concept is general and can be adapted to various off-the-shelf encoder\u2013decoder PLMs, we implement RetriGen to facilitate assertion generation based on the recent CodeT5 model. We conduct extensive experiments to evaluate RetriGen against six state-of-the-art approaches across two large-scale datasets and two metrics. The experimental results demonstrate that RetriGen achieves 57.66% and 73.24% in terms of accuracy and CodeBLEU, outperforming all baselines with an average improvement of 50.66% and 14.14%, respectively. Furthermore, RetriGen generates 1,598 and 1,818 unique correct assertions that all baselines fail to produce, 3.71X and 4.58X more than the most recent approach\n                    <jats:sc>EditAS<\/jats:sc>\n                    . We also demonstrate that adopting other PLMs can provide substantial advancement, e.g., four additionally utilized PLMs outperform\n                    <jats:sc>EditAS<\/jats:sc>\n                    by 7.91%\u201312.70% accuracy improvement, indicating the generalizability of RetriGen. Overall, our study highlights the promising future of fine-tuning off-the-shelf PLMs to generate accurate assertions by incorporating external knowledge sources.\n                  <\/jats:p>","DOI":"10.1145\/3721128","type":"journal-article","created":{"date-parts":[[2025,2,28]],"date-time":"2025-02-28T10:31:29Z","timestamp":1740738689000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-Trained Language Models"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2495-3805","authenticated-orcid":false,"given":"Quanjun","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Computing Technologies, Swinburne University of Technology, Melbourne, Australia and State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9930-7111","authenticated-orcid":false,"given":"Chunrong","family":"Fang","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-2040-8496","authenticated-orcid":false,"given":"Yi","family":"Zheng","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-6311-5184","authenticated-orcid":false,"given":"Yaxin","family":"Zhang","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1980-6277","authenticated-orcid":false,"given":"Yuan","family":"Zhao","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1769-6126","authenticated-orcid":false,"given":"Rubing","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Macau University of Science and Technology, Taipa, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4867-5416","authenticated-orcid":false,"given":"Jianyi","family":"Zhou","sequence":"additional","affiliation":[{"name":"Huawei Cloud Computing Technologies Co., Ltd., Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7868-5471","authenticated-orcid":false,"given":"Yun","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Computing Technologies, Swinburne University of Technology, Melbourne, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-3736-4604","authenticated-orcid":false,"given":"Tao","family":"Zheng","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China and Shenzhen Research Institute of Nanjing University, Shenzhen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9592-7022","authenticated-orcid":false,"given":"Zhenyu","family":"Chen","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China and Shenzhen Research Institute of Nanjing University, Shenzhen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,8,14]]},"reference":[{"key":"e_1_3_3_2_2","unstructured":"Hugging Face. 2024. Retrieved from https:\/\/huggingface.co\/"},{"key":"e_1_3_3_3_2","unstructured":"OpenDDAL. 2024. Retrieved from https:\/\/github.com\/neradb\/openddal"},{"key":"e_1_3_3_4_2","unstructured":"OWLAPI. 2024. Retrieved from https:\/\/github.com\/owlcs\/owlapi"},{"key":"e_1_3_3_5_2","unstructured":"PyTorch. 2024. Retrieved from https:\/\/pytorch.org\/"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2017.27"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293455"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635898"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3663529.3663801"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2014.11"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2017.29"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510141"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.988497"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-FoSE59343.2023.00008"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025179"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549098"},{"key":"e_1_3_3_18_2","unstructured":"Siqi Gu Chunrong Fang Quanjun Zhang Fangyuan Tian and Zhenyu Chen. 2024. TestART: Improving LLM-based unit test via co-evolution of automated generation and repair iteration. arXiv:2408.03095. Retrieved from https:\/\/arxiv.org\/abs\/2408.03095"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.499"},{"key":"e_1_3_3_20_2","first-page":"1","volume-title":"Proceedings of the 9th International Conference on Learning Representations","author":"Guo Daya","year":"2021","unstructured":"Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al. 2021. GraphCodeBERT: Pre-training code representations with data flow. In Proceedings of the 9th International Conference on Learning Representations, 1\u201318."},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/566171.566207"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3660785"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616265"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2628055"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.550"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00085"},{"key":"e_1_3_3_27_2","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","volume":"33","author":"Lewis Patrick S. H.","year":"2020","unstructured":"Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\u00fcttler, Mike Lewis, Wen-tau Yih, Tim Rockt\u00e4schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems, Vol. 33, 9459\u20139474.","journal-title":"Advances in Neural Information Processing Systems, Vol"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678724"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549081"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598080"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409760"},{"key":"e_1_3_3_32_2","first-page":"1","volume-title":"Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks","author":"Lu Shuai","year":"2021","unstructured":"Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin B. Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. 2021. CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, 1\u201316."},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3183297"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00041"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00205"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/1297846.1297902"},{"key":"e_1_3_3_37_2","doi-asserted-by":"crossref","unstructured":"Rongqi Pan Taher Ahmed Ghaleb and Lionel C. Briand. 2024. LTM: Scalable and black-box similarity-based test suite minimization based on language models. IEEE Transactions on Software Engineering 50 11 (2024) 3053\u20133070.","DOI":"10.1109\/TSE.2024.3469582"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-emnlp.232"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395363.3397383"},{"key":"e_1_3_3_40_2","volume-title":"The Economic Impacts of Inadequate Infrastructure for Software Testing","author":"Strategic Planning","year":"2002","unstructured":"Strategic Planning. 2002. The Economic Impacts of Inadequate Infrastructure for Software Testing. National Institute of Standards and Technology, 1."},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.5555\/3455716.3455856"},{"key":"e_1_3_3_42_2","unstructured":"Baptiste Rozi\u00e8re Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat Xiaoqing Ellen Tan Yossi Adi Jingyu Liu Tal Remez J\u00e9r\u00e9my Rapin et al. 2023. Code Llama: Open foundation models for code. arXiv:2308.12950. Retrieved from https:\/\/arxiv.org\/abs\/2308.12950"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643769"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2023.3334955"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644884"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00090"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3382365"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-acl.247"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524481.3527220"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510621"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3368208"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616256"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.685"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3691620.3695501"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485275"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380429"},{"key":"e_1_3_3_57_2","first-page":"1482","volume-title":"Proceedings of the 45th International Conference on Software Engineering","author":"Steven Xia Chunqiu","year":"2023","unstructured":"Chunqiu Steven Xia, Yuxiang Wei, and Lingming Zhang. 2023. Automated program repair in the era of large pre-trained language models. In Proceedings of the 45th International Conference on Software Engineering, 1482\u20131494."},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3691620.3695529"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3505243"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510149"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534219"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3660783"},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3631974"},{"key":"e_1_3_3_64_2","unstructured":"Quanjun Zhang Chunrong Fang Yang Xie Yuxiang Ma Weisong Sun Yun Yang and Zhenyu Chen. 2024. A systematic literature review on large language models for automated program repair. arXiv:2405.01466. Retrieved from https:\/\/arxiv.org\/abs\/2405.01466"},{"key":"e_1_3_3_65_2","unstructured":"Quanjun Zhang Chunrong Fang Yang Xie Yaxin Zhang Yun Yang Weisong Sun Shengcheng Yu and Zhenyu Chen. 2023. A survey on large language models for software engineering. arXiv:2312.15223. Retrieved from https:\/\/arxiv.org\/abs\/2405.01466"},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2023.3308897"},{"key":"e_1_3_3_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00063"},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","unstructured":"Quanjun Zhang Weifeng Sun Chunrong Fang Bowen Yu Hongyan Li Meng Yan Jianyi Zhou and Zhenyu Chen. 2024. Exploring automated assertion generation via large language models. ACM Transactions on Software Engineering and Methodology 34 3 (Oct. 2024) 1\u201325. DOI: 10.1145\/3699598","DOI":"10.1145\/3699598"},{"key":"e_1_3_3_69_2","unstructured":"Quanjun Zhang Tongke Zhang Juan Zhai Chunrong Fang Bowen Yu Weisong Sun and Zhenyu Chen. 2023. A critical review of large language model on software engineering: An example from ChatGPT and automated program repair. arXiv:2310.08879. Retrieved from https:\/\/arxiv.org\/abs\/2310.08879"},{"key":"e_1_3_3_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639222"},{"key":"e_1_3_3_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639125"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3721128","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,14]],"date-time":"2025-08-14T15:00:57Z","timestamp":1755183657000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3721128"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,14]]},"references-count":70,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,9,30]]}},"alternative-id":["10.1145\/3721128"],"URL":"https:\/\/doi.org\/10.1145\/3721128","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,14]]},"assertion":[{"value":"2024-10-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-14","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-14","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}