{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T20:50:05Z","timestamp":1774126205030,"version":"3.50.1"},"reference-count":72,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T00:00:00Z","timestamp":1739836800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T00:00:00Z","timestamp":1739836800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2025,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Context<\/jats:title>\n                    <jats:p>As Software Engineering (SE) practices evolve due to extensive increases in software size and complexity, the importance of tools to analyze and understand source code grows significantly.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aims to evaluate the abilities of Large Language Models (LLMs) in identifying DPs in source code, which can facilitate the development of better Design Pattern Recognition (DPR) tools. We compare the effectiveness of different LLMs in capturing semantic information relevant to the DPR task.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We studied Gang of Four (GoF) DPs from the P-MARt repository of curated Java projects. State-of-the-art language models, including Code2Vec, CodeBERT, CodeGPT, CodeT5, and RoBERTa, are used to generate embeddings from source code. These embeddings are then used for DPR via a k-nearest neighbors prediction. Precision, recall, and F1-score metrics are computed to evaluate performance.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>RoBERTa is the top performer, followed by CodeGPT and CodeBERT, which showed mean F1 Scores of 0.91, 0.79, and 0.77, respectively. The results show that LLMs without explicit pre-training can effectively store semantics and syntactic information, which can be used in building better DPR tools.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>The performance of LLMs in DPR is comparable to existing state-of-the-art methods but with less effort in identifying pattern-specific rules and pre-training. Factors influencing prediction performance in Java files\/programs are analyzed. These findings can advance software engineering practices and show the importance and abilities of LLMs for effective DPR in source code.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s10664-025-10625-1","type":"journal-article","created":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T07:07:56Z","timestamp":1739862476000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Design pattern recognition: a study of large language models"],"prefix":"10.1007","volume":"30","author":[{"given":"Sushant Kumar","family":"Pandey","sequence":"first","affiliation":[]},{"given":"Sivajeet","family":"Chand","sequence":"additional","affiliation":[]},{"given":"Jennifer","family":"Horkoff","sequence":"additional","affiliation":[]},{"given":"Miroslaw","family":"Staron","sequence":"additional","affiliation":[]},{"given":"Miroslaw","family":"Ochodek","sequence":"additional","affiliation":[]},{"given":"Darko","family":"Durisic","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,2,18]]},"reference":[{"key":"10625_CR1","doi-asserted-by":"crossref","unstructured":"Allamanis M, Sutton C (2013) Mining source code repositories at massive scale using language modeling. In: 2013 10th working conference on mining software repositories (MSR), pp 207\u2013216","DOI":"10.1109\/MSR.2013.6624029"},{"key":"10625_CR2","doi-asserted-by":"crossref","unstructured":"Almadi SH (2022) Toward investigating the violations roles of pattern grime occurrence in software design patterns: Violations roles of pattern grime. In: Proceedings of the 26th international conference on evaluation and assessment in software engineering, pp 336\u2013341","DOI":"10.1145\/3530019.3535347"},{"key":"10625_CR3","doi-asserted-by":"publisher","unstructured":"Alon U, Zilberstein M, Levy O, Yahav E (2019) Code2vec: Learning distributed representations of code. Proc ACM Program Lang 3(POPL): 40:1\u201340:29. https:\/\/doi.org\/10.1145\/3290353","DOI":"10.1145\/3290353"},{"issue":"3","key":"10625_CR4","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1080\/00031305.1992.10475879","volume":"46","author":"NS Altman","year":"1992","unstructured":"Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Am Stat 46(3):175\u2013185","journal-title":"Am Stat"},{"issue":"4","key":"10625_CR5","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1016\/j.infsof.2011.10.006","volume":"54","author":"A Ampatzoglou","year":"2012","unstructured":"Ampatzoglou A, Frantzeskou G, Stamelos I (2012) A methodology to assess the impact of design patterns on software quality. Inf Softw Technol 54(4):331\u2013346","journal-title":"Inf Softw Technol"},{"issue":"7","key":"10625_CR6","doi-asserted-by":"publisher","first-page":"1945","DOI":"10.1016\/j.jss.2013.03.063","volume":"86","author":"A Ampatzoglou","year":"2013","unstructured":"Ampatzoglou A, Charalampidou S, Stamelos I (2013) Research state of the art on gof design patterns: a mapping study. J Syst Softw 86(7):1945\u20131964","journal-title":"J Syst Softw"},{"key":"10625_CR7","doi-asserted-by":"crossref","unstructured":"Arshad S, Abid S, Shamail S (2022) Codebert for code clone detection: a replication study. In: 2022 IEEE 16th International Workshop on Software Clones (IWSC), pp 39\u201345","DOI":"10.1109\/IWSC55060.2022.00015"},{"key":"10625_CR8","doi-asserted-by":"publisher","first-page":"110919","DOI":"10.1016\/j.jss.2021.110919","volume":"175","author":"R Barbudo","year":"2021","unstructured":"Barbudo R, Ram\u00edrez A, Servant F, Romero JR (2021) Geml: A grammar-based evolutionary machine learning approach for design-pattern detection. J Syst Softw 175:110919","journal-title":"J Syst Softw"},{"issue":"12","key":"10625_CR9","first-page":"1233","volume":"26","author":"ML Bernardi","year":"2014","unstructured":"Bernardi ML, Cimitile M, Di Lucca G (2014) Design pattern detection using a dsl-driven graph matching approach. J Softw: Evol Process 26(12):1233\u20131266","journal-title":"J Softw: Evol Process"},{"key":"10625_CR10","doi-asserted-by":"crossref","unstructured":"Bui ND, Yu Y, Jiang L (2021) Self-supervised contrastive learning for code retrieval and summarization via semantic-preserving transformations. Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pp 511\u2013521","DOI":"10.1145\/3404835.3462840"},{"key":"10625_CR11","doi-asserted-by":"crossref","unstructured":"Chand S, Pandey SK, Horkoff J, Staron M, Ochodek M, Durisic D (2023) Comparing word-based and ast-based models for design pattern recognition. In: Proceedings of the 19th international conference on predictive models and data analytics in software engineering, pp 44\u201348","DOI":"10.1145\/3617555.3617873"},{"key":"10625_CR12","unstructured":"Chen M, Tworek J, Jun H, Yuan Q, Pinto HPdO, Kaplan J, Edwards H, Burda Y, Joseph N, Brockman G et al (2021) Evaluating large language models trained on code. arXiv:2107.03374"},{"issue":"11","key":"10625_CR13","doi-asserted-by":"publisher","first-page":"1202","DOI":"10.1016\/j.infsof.2012.05.004","volume":"54","author":"A Christopoulou","year":"2012","unstructured":"Christopoulou A, Giakoumakis EA, Zafeiris VE, Soukara V (2012) Automated refactoring to the strategy design pattern. Inf Softw Technol 54(11):1202\u20131214","journal-title":"Inf Softw Technol"},{"issue":"12","key":"10625_CR14","first-page":"4818","volume":"48","author":"M Ciniselli","year":"2021","unstructured":"Ciniselli M, Cooper N, Pascarella L, Mastropaolo A, Aghajani E, Poshyvanyk D, Di Penta M, Bavota G (2021) An empirical study on the usage of transformer models for code completion. IEEE Trans Software Eng 48(12):4818\u20134837","journal-title":"IEEE Trans Software Eng"},{"issue":"1","key":"10625_CR15","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1109\/TIT.1967.1053964","volume":"13","author":"T Cover","year":"1967","unstructured":"Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21\u201327","journal-title":"IEEE Trans Inf Theory"},{"key":"10625_CR16","doi-asserted-by":"crossref","unstructured":"Feng Z, Guo D, Tang D, Duan N, Feng X, Gong M, Shou L, Qin B, Liu T, Jiang D et al (2020) Codebert: A pre-trained model for programming and natural languages. arXiv:2002.08155","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"10625_CR17","unstructured":"Gamma E, Helm R, Johnson R, Johnson RE, Vlissides J (1995) Design patterns: elements of reusable object-oriented software. Pearson Deutschland GmbH"},{"key":"10625_CR18","doi-asserted-by":"crossref","unstructured":"Garc\u00eda S, Luengo J, Herrera F (2015) Data Preprocessing in Data Mining, Springer","DOI":"10.1007\/978-3-319-10247-4"},{"issue":"5","key":"10625_CR19","doi-asserted-by":"publisher","first-page":"667","DOI":"10.1109\/TSE.2008.48","volume":"34","author":"YG Gu\u00e9h\u00e9neuc","year":"2008","unstructured":"Gu\u00e9h\u00e9neuc YG, Antoniol G (2008) Demima: A multilayered approach for design pattern identification. IEEE Trans Software Eng 34(5):667\u2013684","journal-title":"IEEE Trans Software Eng"},{"key":"10625_CR20","doi-asserted-by":"crossref","unstructured":"Haque S, Eberhart Z, Bansal A, McMillan C (2022) Semantic similarity metrics for evaluating source code summarization. Proceedings of the 30th IEEE\/ACM international conference on program comprehension, pp 36\u201347","DOI":"10.1145\/3524610.3527909"},{"key":"10625_CR21","doi-asserted-by":"crossref","unstructured":"Heuzeroth D, Mandel S, Lowe W (2003) Generating design pattern detectors from pattern specifications. In: 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings, pp 245\u2013248","DOI":"10.1109\/ASE.2003.1240313"},{"issue":"5","key":"10625_CR22","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1145\/2902362","volume":"59","author":"A Hindle","year":"2016","unstructured":"Hindle A, Barr ET, Gabel M, Su Z, Devanbu P (2016) On the naturalness of software. Commun ACM 59(5):122\u2013131","journal-title":"Commun ACM"},{"key":"10625_CR23","doi-asserted-by":"crossref","unstructured":"Kova\u010devi\u0107 A, Luburi\u0107 N, Slivka J, Proki\u0107 S, Gruji\u0107 KG, Vidakovi\u0107 D, Sladi\u0107 G (2023) Automatic detection of code smells using metrics and codet5 embeddings: a case study in c. Authorea Preprints","DOI":"10.36227\/techrxiv.19682754.v1"},{"key":"10625_CR24","doi-asserted-by":"crossref","unstructured":"Kuchana P (2004) Software architecture design patterns in Java. CRC Press","DOI":"10.1201\/9780203496213"},{"issue":"6624","key":"10625_CR25","doi-asserted-by":"publisher","first-page":"1092","DOI":"10.1126\/science.abq1158","volume":"378","author":"Y Li","year":"2022","unstructured":"Li Y, Choi D, Chung J, Kushman N, Schrittwieser J, Leblond R, Eccles T, Keeling J, Gimeno F, Dal Lago A et al (2022) Competition-level code generation with alphacode. Science 378(6624):1092\u20131097","journal-title":"Science"},{"key":"10625_CR26","unstructured":"Liu Y (2019) Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 364"},{"key":"10625_CR27","first-page":"32568","volume":"35","author":"Z Liu","year":"2022","unstructured":"Liu Z, Xu Y, Xu Y, Qian Q, Li H, Ji X, Chan A, Jin R (2022) Improved fine-tuning by better leveraging pre-training data. Adv Neural Inf Process Syst 35:32568\u201332581","journal-title":"Adv Neural Inf Process Syst"},{"key":"10625_CR28","doi-asserted-by":"crossref","unstructured":"Li Y, Wang S, Nguyen TN, Van Nguyen S (2019) Improving bug detection via context-based code representation learning and attention-based neural networks. In: Proceedings of the ACM on Programming Languages 3(OOPSLA), pp 1\u201330","DOI":"10.1145\/3360588"},{"key":"10625_CR29","doi-asserted-by":"crossref","unstructured":"Li Y, Zhang T, Luo X, Cai H, Fang S, Yuan D (2023) Do pre-trained language models indeed understand software engineering tasks? IEEE Trans Softw Eng","DOI":"10.1109\/TSE.2023.3308952"},{"key":"10625_CR30","unstructured":"Lu S, Guo D, Ren S, Huang J, Svyatkovskiy A, Blanco A, Clement C, Drain D, Jiang D, Tang D et al (2021) Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv:2102.04664"},{"key":"10625_CR31","doi-asserted-by":"crossref","unstructured":"Mashhadi E, Hemmati H (2021) Applying codebert for automated program repair of java simple bugs. In: 2021 IEEE\/ACM 18th International Conference on Mining Software Repositories (MSR), pp 505\u2013509","DOI":"10.1109\/MSR52588.2021.00063"},{"issue":"12","key":"10625_CR32","doi-asserted-by":"publisher","first-page":"1650","DOI":"10.1109\/TPAMI.2002.1114856","volume":"24","author":"U Maulik","year":"2002","unstructured":"Maulik U, Bandyopadhyay S (2002) Performance evaluation of some clustering algorithms and validity indices. IEEE Trans Pattern Anal Mach Intell 24(12):1650\u20131654","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"10625_CR33","unstructured":"Mehra L (2021) Software Design Patterns for Java Developers: Expert-led Approaches to Build Re-usable Software and Enterprise Applications (English Edition) (BPB Publications)"},{"key":"10625_CR34","unstructured":"Moreira R, Fernandes E, Figueiredo E (2022) based comparison of design pattern detection tools. In: 29th International Conference on Pattern Languages of Programs (PLoP), pp 1\u201316"},{"key":"10625_CR35","doi-asserted-by":"crossref","unstructured":"Mucherino A, Papajorgji PJ, Pardalos PM, Mucherino A, Papajorgji PJ, Pardalos PM (2009) K-nearest neighbor classification. Data Mining in Agriculture:83\u2013106","DOI":"10.1007\/978-0-387-88615-2_4"},{"issue":"4","key":"10625_CR36","first-page":"1091","volume":"53","author":"A Naghdipour","year":"2023","unstructured":"Naghdipour A, Hasheminejad SMH, Barmaki RL (2023) Software design pattern selection approaches: a systematic literature review. Softw: Pract Exp 53(4):1091\u20131122","journal-title":"Softw: Pract Exp"},{"key":"10625_CR37","doi-asserted-by":"publisher","first-page":"111179","DOI":"10.1016\/j.jss.2021.111179","volume":"185","author":"N Nazar","year":"2022","unstructured":"Nazar N, Aleti A, Zheng Y (2022) Feature-based software design pattern detection. J Syst Softw 185:111179","journal-title":"J Syst Softw"},{"key":"10625_CR38","doi-asserted-by":"crossref","unstructured":"Pandey SK, Chand S, Horkoff J, Staron M (2023) Design patterns understanding and use in the automotive industry: An interview study. In: International conference on product-focused software process improvement, pp 301\u2013319","DOI":"10.1007\/978-3-031-49266-2_21"},{"key":"10625_CR39","doi-asserted-by":"publisher","unstructured":"Pandey SK, Staron M, Horkoff J, Ochodek M, Durisic D (2024) In: 2024 IEEE 21st International Conference on Software Architecture Companion (ICSA-C), pp 183\u2013190. https:\/\/doi.org\/10.1109\/ICSA-C63560.2024.00041","DOI":"10.1109\/ICSA-C63560.2024.00041"},{"key":"10625_CR40","doi-asserted-by":"crossref","unstructured":"Pandey SK, Staron M, Horkoff J, Ochodek M, Mucci N, Durisic D (2023) Transdpr: Design pattern recognition using programming language models. In: 2023 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) pp 1\u20137","DOI":"10.1109\/ESEM56168.2023.10304862"},{"key":"10625_CR41","doi-asserted-by":"crossref","unstructured":"Parthasarathy D, Ekelin C, Karri A, Sun J, Moraitis P (2022) Measuring design compliance using neural language models: an automotive case study. In: Proceedings of the 18th international conference on predictive models and data analytics in software engineering, pp. 12\u201321","DOI":"10.1145\/3558489.3559067"},{"key":"10625_CR42","doi-asserted-by":"crossref","unstructured":"Parvez MR, Ahmad WU, Chakraborty S, Ray B, Chang KW (2021) Retrieval augmented code generation and summarization. arXiv:2108.11601","DOI":"10.18653\/v1\/2021.findings-emnlp.232"},{"issue":"4","key":"10625_CR43","doi-asserted-by":"publisher","first-page":"575","DOI":"10.1109\/TSE.2009.92","volume":"36","author":"N Pettersson","year":"2010","unstructured":"Pettersson N, L\u00f6we W, Nivre J (2010) Evaluation of accuracy in design pattern occurrence detection. IEEE Trans Software Eng 36(4):575\u2013590","journal-title":"IEEE Trans Software Eng"},{"key":"10625_CR44","doi-asserted-by":"crossref","unstructured":"Qian G, Sural S, Gu Y, Pramanik S (2004) Similarity between euclidean and cosine angle distance for nearest neighbor queries. In: Proceedings of the 2004 ACM symposium on applied computing, pp. 1232\u20131237","DOI":"10.1145\/967900.968151"},{"key":"10625_CR45","unstructured":"Radiya-Dixit E, Wang X (2020) In: International Conference on Artificial Intelligence and Statistics (PMLR), pp 2435\u20132443"},{"key":"10625_CR46","doi-asserted-by":"crossref","unstructured":"Rahman M, Chy MSH, Saha S (2023) A systematic review on software design patterns in today\u2019s perspective. In: 2023 IEEE 11th International Conference on Serious Games and Applications for Health (SeGAH), pp 1\u20138","DOI":"10.1109\/SeGAH57547.2023.10253758"},{"key":"10625_CR47","doi-asserted-by":"publisher","first-page":"8851","DOI":"10.1007\/s13369-014-1449-0","volume":"39","author":"G Rasool","year":"2014","unstructured":"Rasool G, M\u00e4der P (2014) A customizable approach to design patterns recognition based on feature types. Arab J Sci Eng 39:8851\u20138873","journal-title":"Arab J Sci Eng"},{"key":"10625_CR48","doi-asserted-by":"crossref","unstructured":"Rasool G, M\u00e4der P (2011) Flexible design pattern detection based on feature types. In: 2011 26th IEEE\/ACM International Conference on Automated Software Engineering (ASE 2011), pp 243\u2013252","DOI":"10.1109\/ASE.2011.6100060"},{"key":"10625_CR49","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1016\/0377-0427(87)90125-7","volume":"20","author":"PJ Rousseeuw","year":"1987","unstructured":"Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53\u201365","journal-title":"J Comput Appl Math"},{"issue":"3","key":"10625_CR50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2699696","volume":"24","author":"G Scanniello","year":"2015","unstructured":"Scanniello G, Gravino C, Risi M, Tortora G, Dodero G (2015) Documenting design-pattern instances: a family of experiments on source-code comprehensibility. ACM Transactions on Software Engineering and Methodology (TOSEM) 24(3):1\u201335","journal-title":"ACM Transactions on Software Engineering and Methodology (TOSEM)"},{"issue":"12","key":"10625_CR51","doi-asserted-by":"publisher","first-page":"2786","DOI":"10.1109\/TSE.2020.2968072","volume":"47","author":"D Silva","year":"2020","unstructured":"Silva D, da Silva JP, Santos G, Terra R, Valente MT (2020) Refdiff 2.0: A multi-language refactoring detection tool. IEEE Trans Softw Eng 47(12):2786\u20132802","journal-title":"IEEE Trans Softw Eng"},{"key":"10625_CR52","doi-asserted-by":"crossref","unstructured":"Svyatkovskiy A, Deng SK, Fu S, Sundaresan N (2020) Intellicode compose: Code generation using transformer. In: Proceedings of the 28th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 1433\u20131443","DOI":"10.1145\/3368089.3417058"},{"key":"10625_CR53","doi-asserted-by":"crossref","unstructured":"Tian H, Liu K, Kabor\u00e9 AK, Koyuncu A, Li L, Klein J, Bissyand\u00e9 TF (2020) Evaluating representation learning of code changes for predicting patch correctness in program repair. In: Proceedings of the 35th IEEE\/ACM international conference on automated software engineering, pp 981\u2013992","DOI":"10.1145\/3324884.3416532"},{"key":"10625_CR54","unstructured":"Tipirneni S, Zhu M, Reddy CK (2022) Structcoder: Structure-aware transformer for code generation. arXiv:2206.05239"},{"issue":"11","key":"10625_CR55","doi-asserted-by":"publisher","first-page":"896","DOI":"10.1109\/TSE.2006.112","volume":"32","author":"N Tsantalis","year":"2006","unstructured":"Tsantalis N, Chatzigeorgiou A, Stephanides G, Halkidis ST (2006) Design pattern detection using similarity scoring. IEEE Trans Software Eng 32(11):896\u2013909","journal-title":"IEEE Trans Software Eng"},{"issue":"4","key":"10625_CR56","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3340544","volume":"28","author":"M Tufano","year":"2019","unstructured":"Tufano M, Watson C, Bavota G, Penta MD, White M, Poshyvanyk D (2019) An empirical study on learning bug-fixing patches in the wild via neural machine translation. ACM Transactions on Software Engineering and Methodology (TOSEM) 28(4):1\u201329","journal-title":"ACM Transactions on Software Engineering and Methodology (TOSEM)"},{"key":"10625_CR57","unstructured":"Unger B, Tichy WF (2000) In: Proceedings of the international workshop on empirical studies of software maintenance"},{"key":"10625_CR58","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141, Polosukhin I (2017) Attention is all you need. Advan Neural Inform Process Syst 30"},{"key":"10625_CR59","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1016\/j.infsof.2016.02.003","volume":"74","author":"B Walter","year":"2016","unstructured":"Walter B, Alkhaeir T (2016) The relationship between design patterns and code smells: an exploratory study. Inf Softw Technol 74:127\u2013142","journal-title":"Inf Softw Technol"},{"key":"10625_CR60","doi-asserted-by":"crossref","unstructured":"Wang S, Geng M, Lin B, Sun Z, Wen M, Liu Y, Li L, Bissyand\u00e9 TF, Mao X (2023) Natural language to code: How far are we? In: Proceedings of the 31st ACM joint European software engineering conference and symposium on the foundations of software engineering, pp. 375\u2013387","DOI":"10.1145\/3611643.3616323"},{"key":"10625_CR61","doi-asserted-by":"crossref","unstructured":"Wang D, Jia Z, Li S, Yu Y, Xiong Y, Dong W, Liao X (2022) Bridging pre-trained models and downstream tasks for source code understanding. In: Proceedings of the 44th international conference on software engineering, pp 287\u2013298","DOI":"10.1145\/3510003.3510062"},{"key":"10625_CR62","doi-asserted-by":"crossref","unstructured":"Wang Y, Wang W, Joty S, Hoi SC (2021) Codet 5: identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. arXiv:2109.00859","DOI":"10.18653\/v1\/2021.emnlp-main.685"},{"issue":"1","key":"10625_CR63","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1049\/iet-sen.2018.5446","volume":"14","author":"F Wedyan","year":"2020","unstructured":"Wedyan F, Abufakher S (2020) Impact of design patterns on software quality: a systematic literature review. IET Software 14(1):1\u201317","journal-title":"IET Software"},{"key":"10625_CR64","doi-asserted-by":"crossref","unstructured":"Wohlin C, Runeson P, H\u00f6st M, Ohlsson MC, Regnell B, Wessl\u00e9n A et al (2012) Experimentation in software engineering, Springer, vol 236","DOI":"10.1007\/978-3-642-29044-2"},{"key":"10625_CR65","unstructured":"Xiao Y, Zuo X, Xue L, Wang K, Dong JS, Beschastnikh I (2023) Empirical study on transformer-based techniques for software engineering. arXiv:2310.00399"},{"key":"10625_CR66","doi-asserted-by":"crossref","unstructured":"Xiong R, Li B (2019) Accurate design pattern detection based on idiomatic implementation matching in java language context. In: 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER), pp 163\u2013174","DOI":"10.1109\/SANER.2019.8668031"},{"key":"10625_CR67","doi-asserted-by":"crossref","unstructured":"Yuan D, Fang S, Zhang T, Xu Z, Luo X (2022) Java code clone detection by exploiting semantic and syntax information from intermediate code-based graph. IEEE Trans Reliab","DOI":"10.1109\/TR.2022.3176922"},{"key":"10625_CR68","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1016\/j.jss.2015.01.037","volume":"103","author":"M Zanoni","year":"2015","unstructured":"Zanoni M, Fontana FA, Stella F (2015) On applying machine learning techniques for design pattern detection. J Syst Softw 103:102\u2013117","journal-title":"J Syst Softw"},{"key":"10625_CR69","doi-asserted-by":"crossref","unstructured":"Zeng Z, Tan H, Zhang H, Li J, Zhang Y, Zhang L (2022) An extensive study on pre-trained models for program understanding and generation. In: Proceedings of the 31st ACM SIGSOFT international symposium on software testing and analysis, pp 39\u201351","DOI":"10.1145\/3533767.3534390"},{"issue":"5","key":"10625_CR70","doi-asserted-by":"publisher","first-page":"1213","DOI":"10.1109\/TSE.2011.79","volume":"38","author":"C Zhang","year":"2011","unstructured":"Zhang C, Budgen D (2011) What do we know about the effectiveness of software design patterns? IEEE Trans Software Eng 38(5):1213\u20131231","journal-title":"IEEE Trans Software Eng"},{"key":"10625_CR71","doi-asserted-by":"crossref","unstructured":"Zhang D, Lu G (2003) Evaluation of similarity measurement for image retrieval. In: International conference on neural networks and signal processing, 2003. Proceedings of the 2003, vol 2, pp 928\u2013931","DOI":"10.1109\/ICNNSP.2003.1280752"},{"key":"10625_CR72","unstructured":"Zhang K, Schiele B, Yeung DY (2004) In: Proceedings of the 21st International Conference on Machine Learning (ICML), pp 941\u2013948"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-025-10625-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-025-10625-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-025-10625-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,20]],"date-time":"2025-11-20T13:27:20Z","timestamp":1763645240000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-025-10625-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,18]]},"references-count":72,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,5]]}},"alternative-id":["10625"],"URL":"https:\/\/doi.org\/10.1007\/s10664-025-10625-1","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,18]]},"assertion":[{"value":"5 February 2025","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 February 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study did not involve human participants or animals, and therefore, ethical approval was not required.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"We confirm that no known Conflict of interest are associated with this publication, and there has been no significant financial support for this work that could have influenced its outcome.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable, as the study did not involve human participants.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed Consent"}}],"article-number":"69"}}