{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T23:06:48Z","timestamp":1780960008701,"version":"3.54.1"},"publisher-location":"Cham","reference-count":30,"publisher":"Springer Nature Switzerland","isbn-type":[{"value":"9783031822247","type":"print"},{"value":"9783031822254","type":"electronic"}],"license":[{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,3,28]],"date-time":"2025-03-28T00:00:00Z","timestamp":1743120000000},"content-version":"vor","delay-in-days":86,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Large language models (LLMs) have become a promising tool for automating complex tasks such as process model generation from text. In order to evaluate the capabilities of LLMs in generating process models, it is crucial to provide means to assess the output quality. A few studies have already provided key performance indicators for assessing aspects such as completeness of the models in a quantitative way. In this paper, we focus on the qualitative assessment of generated process models generated by LLMs based on a user survey. By analyzing user preferences, we aim to determine whether LLM-generated process models meet the needs and expectations of experts. Our analysis reveals that 60% of users, regardless of their modeling experience, prefer LLM-generated models over human-created ground truth models.<\/jats:p>","DOI":"10.1007\/978-3-031-82225-4_47","type":"book-chapter","created":{"date-parts":[[2025,3,30]],"date-time":"2025-03-30T03:00:50Z","timestamp":1743303650000},"page":"637-649","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Process Modeler vs. Chatbot: Is Generative AI Taking over Process Modeling?"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-9010-2855","authenticated-orcid":false,"given":"Nataliia","family":"Klievtsova","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3979-400X","authenticated-orcid":false,"given":"Janik-Vasily","family":"Benzin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6332-5801","authenticated-orcid":false,"given":"Juergen","family":"Mangler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6458-2252","authenticated-orcid":false,"given":"Timotheus","family":"Kampik","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5656-6108","authenticated-orcid":false,"given":"Stefanie","family":"Rinderle-Ma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,3,28]]},"reference":[{"key":"47_CR1","doi-asserted-by":"crossref","unstructured":"Ackermann, L., Neuberger, J., Jablonski, S.: Data-driven annotation of textual process descriptions based on formal meaning representations. In: Advanced Information Systems Engineering, pp. 75\u201390 (2021)","DOI":"10.1007\/978-3-030-79382-1_5"},{"issue":"1","key":"47_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1108\/BPMJ-10-2019-0407","volume":"27","author":"DT Avila","year":"2021","unstructured":"Avila, D.T., dos Santos, R.I., Mendling, J., Thom, L.H.: A systematic literature review of process modeling guidelines and their empirical support. Bus. Process. Manag. J. 27(1), 1\u201323 (2021). https:\/\/doi.org\/10.1108\/BPMJ-10-2019-0407","journal-title":"Bus. Process. Manag. J."},{"key":"47_CR3","doi-asserted-by":"crossref","unstructured":"Beerepoot, I., Ciccio, C.D., Reijers, H.A., et\u00a0al.: The biggest business process management problems to solve before we die. Comput. Ind. 146, 103837 (2023)","DOI":"10.1016\/j.compind.2022.103837"},{"key":"47_CR4","doi-asserted-by":"crossref","unstructured":"Bellan, P., van\u00a0der Aa, H., Dragoni, M., Ghidini, C., Ponzetto, S.P.: PET: an annotated dataset for process extraction from natural language text tasks. In: Business Process Management Workshops, pp. 315\u2013321 (2022)","DOI":"10.1007\/978-3-031-25383-6_23"},{"key":"47_CR5","unstructured":"Bellan, P., Dragoni, M., Ghidini, C.: A qualitative analysis of the state of the art in process extraction from text. In: Italian Association for Artificial Intelligence, pp. 19\u201330. CEUR Workshop Proceedings (2020)"},{"key":"47_CR6","doi-asserted-by":"crossref","unstructured":"Bellan, P., Dragoni, M., Ghidini, C.: Extracting business process entities and relations from text using pre-trained language models and in-context learning. In: Enterprise Design, Operations, and Computing, pp. 182\u2013199 (2022)","DOI":"10.1007\/978-3-031-17604-3_11"},{"issue":"19","key":"47_CR7","doi-asserted-by":"publisher","first-page":"3661","DOI":"10.1002\/sim.2832","volume":"26","author":"I Campbell","year":"2007","unstructured":"Campbell, I.: Chi-squared and Fisher-Irwin tests of two-by-two tables with small sample recommendations. Stat. Med. 26(19), 3661\u20133675 (2007)","journal-title":"Stat. Med."},{"key":"47_CR8","doi-asserted-by":"crossref","unstructured":"Casciani, A., Bernardi, M.L., Cimitile, M., Marrella, A.: Conversational systems for AI-augmented business process management. In: Research Challenges in Information Science, pp. 183\u2013200 (2024)","DOI":"10.1007\/978-3-031-59465-6_12"},{"key":"47_CR9","doi-asserted-by":"crossref","unstructured":"Dalianis, H.: A method for validating a conceptual model by natural language discourse generation. In: Advanced Information Systems Engineering, pp. 425\u2013444 (1992)","DOI":"10.1007\/BFb0035146"},{"issue":"3","key":"47_CR10","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1142\/S0218843004000973","volume":"13","author":"J Dehnert","year":"2004","unstructured":"Dehnert, J., van der Aalst, W.M.P.: Bridging the gap between business models and workflow specifications. Int. J. Coop. Inf. Syst. 13(3), 289\u2013332 (2004)","journal-title":"Int. J. Coop. Inf. Syst."},{"key":"47_CR11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3576047","volume":"14","author":"M Dumas","year":"2022","unstructured":"Dumas, M., et al.: AI-augmented business process management systems: a research manifesto. ACM Trans. Manage. Inf. Syst. 14, 1\u201319 (2022)","journal-title":"ACM Trans. Manage. Inf. Syst."},{"key":"47_CR12","unstructured":"Fowler\u00a0Jr, F.J.: Survey Research Methods. Sage Publications (2013)"},{"key":"47_CR13","unstructured":"Friedrich, F.: Automated generation of business process models from natural language input. Master\u2019s thesis, Humbold-University zu Berlin (2010)"},{"key":"47_CR14","doi-asserted-by":"crossref","unstructured":"Ghose, A., Koliadis, G., Chueng, A.: Process discovery from model and text artefacts. In: Services Computing Workshops, pp. 167\u2013174 (2007)","DOI":"10.1109\/SERVICES.2007.52"},{"key":"47_CR15","doi-asserted-by":"crossref","unstructured":"Grohs, M., Abb, L., Elsayed, N., Rehse, J.: Large language models can accomplish business process management tasks. CoRR abs\/2307.09923 (2023)","DOI":"10.1007\/978-3-031-50974-2_34"},{"key":"47_CR16","doi-asserted-by":"crossref","unstructured":"Honkisz, K., Kluza, K., Wisniewski, P.: A concept for generating business process models from natural language description. In: Knowledge Science, Engineering and Management, pp. 91\u2013103 (2018)","DOI":"10.1007\/978-3-319-99365-2_8"},{"key":"47_CR17","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1016\/j.procs.2018.07.207","volume":"126","author":"F Kahloun","year":"2018","unstructured":"Kahloun, F., Ghannouchi, S.A.: Improvement of quality for business process modeling driven by guidelines. Procedia Comput. Sci. 126, 39\u201348 (2018)","journal-title":"Procedia Comput. Sci."},{"key":"47_CR18","unstructured":"Kannengiesser, U., Oppl, S.: Business processes to touch: Engaging domain experts in process modelling, vol.\u00a01418 (2015)"},{"key":"47_CR19","unstructured":"Kesari, M., Chang, S., Seddon, P.B.: A content-analytic study of the advantages and disadvantages of process modelling. In: Australasian Conference on Information Systems, ACIS 2003, Perth, Australia, 26\u201328 November 2003 (2003)"},{"key":"47_CR20","doi-asserted-by":"crossref","unstructured":"Klievtsova, N., Benzin, J., Kampik, T., Mangler, J., Rinderle-Ma, S.: Conversational process modelling: state of the art, applications, and implications in practice. In: Business Process Management Forum, pp. 319\u2013336 (2023)","DOI":"10.1007\/978-3-031-41623-1_19"},{"key":"47_CR21","unstructured":"Klievtsova, N., Mangler, J., Kampik, T., Benzin, J.V., Rinderle-Ma, S.: How can generative AI empower domain experts in creating process models? In: Wirtschaftsinformatik (2024)"},{"key":"47_CR22","doi-asserted-by":"crossref","unstructured":"Koepke, J., Safan, A.: Efficient LLM-based conversational process modeling. In: Business Process Management Workshops (2024)","DOI":"10.1007\/978-3-031-78666-2_20"},{"key":"47_CR23","unstructured":"Kourani, H., Berti, A., Schuster, D., van\u00a0der Aalst, W.M.P.: ProMoAI: process modeling with generative AI. CoRR abs\/2403.04327 (2024)"},{"key":"47_CR24","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1016\/j.infsof.2014.07.011","volume":"58","author":"IMM de Oca","year":"2015","unstructured":"de Oca, I.M.M., Snoeck, M., Reijers, H.A., Rodr\u00edguez-Morffi, A.: A systematic literature review of studies on business process modeling quality. Inf. Softw. Technol. 58, 187\u2013205 (2015)","journal-title":"Inf. Softw. Technol."},{"key":"47_CR25","doi-asserted-by":"crossref","unstructured":"Odeh, Y.: BPMN in engineering software requirements: an introductory brief guide. In: International Conference on Information Management and Engineering (2017)","DOI":"10.1145\/3149572.3149584"},{"key":"47_CR26","doi-asserted-by":"crossref","unstructured":"Polyvyanyy, A., Smirnov, S., Weske, M.: Business process model abstraction. In: Handbook on Business Process Management. Introduction, Methods, and Information Systems. International Handbooks on Information Systems, pp. 147\u2013165. Springer (2015)","DOI":"10.1007\/978-3-642-45100-3_7"},{"key":"47_CR27","doi-asserted-by":"crossref","unstructured":"Qian, C., et al.: An approach for process model extraction by multi-grained text classification. In: Advanced Information Systems Engineering, pp. 268\u2013282 (2020)","DOI":"10.1007\/978-3-030-49435-3_17"},{"key":"47_CR28","doi-asserted-by":"publisher","unstructured":"Sallou, J., Durieux, T., Panichella, A.: Breaking the silence: the threats of using LLMs in software engineering. In: Proceedings of the 2024 ACM\/IEEE 44th International Conference on Software Engineering, NIER@ICSE 2024, Lisbon, Portugal, 14\u201320 April 2024, pp. 102\u2013106. ACM (2024). https:\/\/doi.org\/10.1145\/3639476.3639764","DOI":"10.1145\/3639476.3639764"},{"issue":"2","key":"47_CR29","doi-asserted-by":"publisher","first-page":"43","DOI":"10.20982\/tqmp.03.2.p043","volume":"3","author":"CW VanVoorhis","year":"2007","unstructured":"VanVoorhis, C.W., Morgan, B.L., et al.: Understanding power and rules of thumb for determining sample sizes. Tutor. Quant. Methods Psychol. 3(2), 43\u201350 (2007)","journal-title":"Tutor. Quant. Methods Psychol."},{"key":"47_CR30","doi-asserted-by":"crossref","unstructured":"Yue, T., Briand, L.C., Labiche, Y.: An automated approach to transform use cases into activity diagrams. In: Modelling Foundations and Applications, pp. 337\u2013353 (2010)","DOI":"10.1007\/978-3-642-13595-8_26"}],"container-title":["Lecture Notes in Business Information Processing","Process Mining Workshops"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-82225-4_47","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,19]],"date-time":"2025-10-19T16:04:50Z","timestamp":1760889890000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-82225-4_47"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025]]},"ISBN":["9783031822247","9783031822254"],"references-count":30,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-82225-4_47","relation":{},"ISSN":["1865-1348","1865-1356"],"issn-type":[{"value":"1865-1348","type":"print"},{"value":"1865-1356","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025]]},"assertion":[{"value":"28 March 2025","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"ICPM","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Conference on Process Mining","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Lyngby","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Denmark","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2024","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"14 October 2024","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"18 October 2024","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"6","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"icpm2024","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/icpmconference.org\/2024\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}}]}}