{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T06:52:19Z","timestamp":1778482339791,"version":"3.51.4"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,4,30]]},"abstract":"<jats:p>Model-Driven Engineering (MDE) simplifies software development through abstraction, yet challenges such as time constraints, incomplete domain understanding, and adherence to syntactic constraints hinder the design process. This article presents a study to evaluate the usefulness of a novel approach utilizing Large Language Models (LLMs) and few-shot prompt learning to assist in domain modeling. The aim of this approach is to overcome the need for extensive training of traditional AI-based completion algorithms on domain-specific datasets and to offer versatile support for various modeling activities, providing valuable recommendations to software modelers. To support this approach, we developed MAGDA, a user-friendly tool, through which we conduct a user study and assess the real-world applicability of our approach in the context of domain modeling, offering valuable insights into its usability and effectiveness.<\/jats:p>","DOI":"10.1145\/3744920","type":"journal-article","created":{"date-parts":[[2025,6,20]],"date-time":"2025-06-20T11:54:43Z","timestamp":1750420483000},"page":"1-38","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["On the Utility of Domain Modeling Assistance with Large Language Models"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8133-0199","authenticated-orcid":false,"given":"Meriem Ben","family":"Chaaben","sequence":"first","affiliation":[{"name":"Universit\u00e9 de Montr\u00e9al, Montreal, Quebec, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7779-8810","authenticated-orcid":false,"given":"Lola","family":"Burgue\u00f1o","sequence":"additional","affiliation":[{"name":"ITIS Software, University of Malaga, M\u00e1laga, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4870-8433","authenticated-orcid":false,"given":"Istvan","family":"David","sequence":"additional","affiliation":[{"name":"McMaster University, Hamilton, Ontario, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6304-9926","authenticated-orcid":false,"given":"Houari","family":"Sahraoui","sequence":"additional","affiliation":[{"name":"Universit\u00e9 de Montr\u00e9al, Montreal, Quebec, Canada"}]}],"member":"320","published-online":{"date-parts":[[2026,3,12]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.5220\/0006555700710082"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3593434.3593468"},{"key":"e_1_3_2_4_2","unstructured":"AlgoWriting. 2020. GPT-3 Temperature Setting 101. Retrieved March 2 2024 from https:\/\/algowriting.medium.com\/gpt-3-temperature-setting-101-41200ff0d0be"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-023-00619-6"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/2976767.2976769"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2005.05.003"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.5555\/3103551"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-79382-1_6"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/S10270-023-01105-5"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.5381\/jot.2022.21.3.a4"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-NIER58687.2023.00008"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/MODELS58315.2023.00037"},{"key":"e_1_3_2_14_2","unstructured":"Oleg Chursin. 2017. A Brief Introduction to Domain Modeling. Retrieved March 4 2024 from https:\/\/olegchursin.medium.com\/a-brief-introduction-to-domain-modeling-862a30b38353"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/MODELS58315.2023.00020"},{"key":"e_1_3_2_16_2","volume-title":"Practical Nonparametric Statistics","author":"Conover W. J.","year":"1999","unstructured":"W. J. Conover. 1999. Practical Nonparametric Statistics (3rd ed.). Wiley, New York.","edition":"3"},{"key":"e_1_3_2_17_2","volume-title":"Engineering Design Methods: Strategies for Product Design","author":"Cross Nigel","year":"2021","unstructured":"Nigel Cross. 2021. Engineering Design Methods: Strategies for Product Design. John Wiley & Sons."},{"key":"e_1_3_2_18_2","volume-title":"Prompt Engineering Guide","author":"Dair A. I.","year":"2023","unstructured":"A. I. Dair. 2023. Prompt Engineering Guide. Democratizing Artificial Intelligence Research, Education, and Technologies. Retrieved February 11, 2024 from https:\/\/www.promptingguide.ai\/"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/s007790170019"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/MODELS50736.2021.00016"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEFM.2003.1236202"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/AICCSA.2016.7945659"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2014.03.003"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3624032.3624035"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/7287.003.0011"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2012.6227135"},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","unstructured":"Xinyi Hou Yanjie Zhao Yue Liu Zhou Yang Kailong Wang Li Li Xiapu Luo David Lo John Grundy and Haoyu Wang. 2024. Large language models for software engineering: A systematic literature review. ACM Transactions on Software Engineering and Methodology 33 8 (2024) 1\u201379.","DOI":"10.1145\/3695988"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCRD.2010.71"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.5555\/256664.256823"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.5555\/1203720"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2642937.2642949"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2006.51"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-823941-4.00013-5"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2014.930311"},{"key":"e_1_3_2_35_2","first-page":"11","volume-title":"Proceedings of the 18th Annual Workshop of the Psychology of Programming Interest Group (PPIG","author":"Liblit Ben","year":"2006","unstructured":"Ben Liblit, Andrew Begel, and Eve Sweetser. 2006. Cognitive perspectives on the role of naming in computer programs. In Proceedings of the 18th Annual Workshop of the Psychology of Programming Interest Group (PPIG \u201906). Psychology of Programming Interest Group, Brighton, UK, 11."},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/52.268955"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3560815"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501825"},{"key":"e_1_3_2_39_2","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692. Retrieved from https:\/\/arxiv.org\/abs\/1907.11692"},{"key":"e_1_3_2_40_2","first-page":"57","volume-title":"Proceedings of the 23rd ACM\/IEEE International Conference on Model Driven Engineering Languages and Systems","author":"Antonio Jos\u00e9","year":"2020","unstructured":"Jos\u00e9 Antonio, Hern\u00e1ndez L\u00f3pez, and Jes\u00fas S\u00e1nchez Cuadrado. 2020. MAR: A structure-based search engine for models. In Proceedings of the 23rd ACM\/IEEE International Conference on Model Driven Engineering Languages and Systems. ACM, Canada, 57\u201367."},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-84628-856-2_5"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-020-00814-5"},{"key":"e_1_3_2_43_2","unstructured":"OpenAI. 2024. What Are Tokens and How to Count Them? Retrieved March 2 2024 from https:\/\/help.openai.com\/en\/articles\/4936856-what-are-tokens-and-how-to-count-them"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2023.3248401"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.5555\/574344"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA60479.2023.00061"},{"key":"e_1_3_2_47_2","unstructured":"Baptiste Rozi\u00e8re Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat Xiaoqing Ellen Tan Yossi Adi Jingyu Liu Romain Sauvestre Tal Remez et al. 2024. Code Llama: Open foundation models for code. arXiv:2308.12950. Retrieved from https:\/\/arxiv.org\/abs\/2308.12950"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-008-9102-8"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1002\/9781118634646"},{"key":"e_1_3_2_50_2","first-page":"1","article-title":"Understanding the need for assistance in software modeling: Interviews with experts","volume":"23","author":"Savary-Leblanc Maxime","year":"2023","unstructured":"Maxime Savary-Leblanc, Xavier Le Pallec, and S\u00e9bastien G\u00e9rard. 2023. Understanding the need for assistance in software modeling: Interviews with experts. Software and Systems Modeling 23 (2023), 1\u201333.","journal-title":"Software and Systems Modeling"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2006.58"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/238218.238281"},{"key":"e_1_3_2_53_2","unstructured":"Eugene Syriani Istvan David and Gauransh Kumar. 2023. Assessing the ability of ChatGPT to screen articles for systematic reviews. arXiv:2307.06464. Retrieved from https:\/\/arxiv.org\/abs\/2307.06464"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cola.2024.101287"},{"key":"e_1_3_2_55_2","unstructured":"Christof Tinnes Alisa Welter and Sven Apel. 2024. Leveraging large language models for software model completion: Results from industrial and public datasets. arXiv:2406.17651. Retrieved from https:\/\/arxiv.org\/abs\/2406.17651"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2015.2467378"},{"key":"e_1_3_2_57_2","doi-asserted-by":"crossref","unstructured":"Beian Wang Chong Wang Peng Liang Bing Li and Cheng Zeng. 2024. How LLMs aid in UML modeling: An exploratory study with novice analysts. arXiv:2404.17739. Retrieved from https:\/\/arxiv.org\/abs\/2404.17739","DOI":"10.1109\/SSE62657.2024.00046"},{"key":"e_1_3_2_58_2","first-page":"1","volume-title":"Proceedings of the ACM on Human-Computer Interaction","author":"Wessel Mairieli","year":"2021","unstructured":"Mairieli Wessel, Igor Wiese, Igor Steinmacher, and Marco Aurelio Gerosa. 2021. Don\u2019t disturb me: Challenges of interacting with software bots on open source software projects. Proceedings of the ACM on Human-Computer Interaction. CSCW2, 1\u201321."},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-022-00975-5"},{"key":"e_1_3_2_60_2","unstructured":"Jules White Quchen Fu Sam Hays Michael Sandborn Carlos Olea Henry Gilbert Ashraf Elnashar Jesse Spencer-Smith and Douglas C. Schmidt. 2023. A prompt pattern catalog to enhance prompt engineering with ChatGPT. arXiv:2302.11382. Retrieved from https:\/\/arxiv.org\/abs\/2302.11382"},{"key":"e_1_3_2_61_2","unstructured":"Shengyu Zhang Linfeng Dong Xiaoya Li Sen Zhang Xiaofei Sun Shuhe Wang Jiwei Li Runyi Hu Tianwei Zhang Fei Wu et al. 2023. Instruction tuning for large language models: A survey. arXiv:2308.10792. Retrieved from https:\/\/arxiv.org\/abs\/2308.10792"},{"key":"e_1_3_2_62_2","unstructured":"Wayne Xin Zhao Kun Zhou Junyi Li Tianyi Tang Xiaolei Wang Yupeng Hou Yingqian Min Beichen Zhang Junjie Zhang Zican Dong et al. 2023. A survey of large language models. arXiv:2303.18223. Retrieved from https:\/\/arxiv.org\/abs\/2303.18223"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3744920","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T15:07:35Z","timestamp":1773328055000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3744920"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,12]]},"references-count":61,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,4,30]]}},"alternative-id":["10.1145\/3744920"],"URL":"https:\/\/doi.org\/10.1145\/3744920","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,12]]},"assertion":[{"value":"2024-10-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-06-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}