{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T10:41:58Z","timestamp":1783075318887,"version":"3.54.6"},"reference-count":90,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,12,26]],"date-time":"2024-12-26T00:00:00Z","timestamp":1735171200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,1,31]]},"abstract":"<jats:p>Process mining aims to derive insights into business processes from event logs recorded from information systems. Process discovery algorithms construct process models that describe the executed process. With the increasing availability of large-scale event logs, process discovery has shifted towards a data-oriented research discipline, aiming to design algorithms that are applicable and useful in practice. This shift has revealed a fundamental problem in process discovery research: Currently, contributions can only be considered in isolation. Researchers conduct experiments to show that they move the field forward, but due to a lack of reliability and validity, the individual contributions are hard to generalize. In this article, we argue that one reason for these problems is the lack of conventions or standards for experimental design in process discovery. Hence, we propose \u201cprocess discovery engineering\u201d: a research methodology for process discovery, consisting of a shared terminology and a checklist for conducting experiments. We demonstrate its applicability by means of an example experimental evaluation of process discovery algorithms and discuss the implications of the methodology on the field. This article is not meant to be prescriptive but to invite and encourage the community to contribute to this discussion to advance the field as a whole.<\/jats:p>","DOI":"10.1145\/3672447","type":"journal-article","created":{"date-parts":[[2024,11,13]],"date-time":"2024-11-13T15:10:21Z","timestamp":1731510621000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["On Process Discovery Experimentation: Addressing the Need for Research Methodology in Process Discovery"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5707-6944","authenticated-orcid":false,"given":"Jana-Rebecca","family":"Rehse","sequence":"first","affiliation":[{"name":"University of Mannheim, Mannheim, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5201-7125","authenticated-orcid":false,"given":"Sander J. J.","family":"Leemans","sequence":"additional","affiliation":[{"name":"RWTH Aachen, Aachen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0624-4431","authenticated-orcid":false,"given":"Peter","family":"Fettke","sequence":"additional","affiliation":[{"name":"Saarland University, Saarbr\u00fccken, Germany and German Research Center for Artificial Intelligence (DFKI), Saarbr\u00fccken, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7264-381X","authenticated-orcid":false,"given":"Jan Martijn E. M.","family":"van der Werf","sequence":"additional","affiliation":[{"name":"Utrecht University, Utrecht, the Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,12,26]]},"reference":[{"issue":"2","key":"e_1_3_2_2_2","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/MCI.2017.2670420","article-title":"IEEE 1849: The XES standard: The second IEEE standard sponsored by IEEE computational intelligence society [society briefs]","volume":"12","author":"Acampora Giovanni","year":"2017","unstructured":"Giovanni Acampora, Autilia Vitiello, Bruno Di Stefano, Wil van der Aalst, Christian G\u00fcnther, and Eric Verbeek. 2017. IEEE 1849: The XES standard: The second IEEE standard sponsored by IEEE computational intelligence society [society briefs]. IEEE Computational Intelligence Magazine 12, 2 (2017), 4\u20138.","journal-title":"IEEE Computational Intelligence Magazine"},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1007\/978-3-642-36285-9_15","volume-title":"Proceedings of the International Conference on Business Process Management Workshops","author":"Adriansyah Arya","year":"2013","unstructured":"Arya Adriansyah, Jorge Munoz-Gama, Josep Carmona, Boudewijn van Dongen, and Wil van der Aalst. 2013. Alignment based precision checking. In Proceedings of the International Conference on Business Process Management Workshops. Springer, 137\u2013149."},{"issue":"1","key":"e_1_3_2_4_2","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1007\/s10257-014-0234-7","article-title":"Measuring precision of modeled behavior","volume":"13","author":"Adriansyah Arya","year":"2015","unstructured":"Arya Adriansyah, Jorge Munoz-Gama, Josep Carmona, Boudewijn van Dongen, and Wil van der Aalst. 2015. Measuring precision of modeled behavior. Information systems and e-Business Management 13, 1 (2015), 37\u201367.","journal-title":"Information systems and e-Business Management"},{"key":"e_1_3_2_5_2","first-page":"55","volume-title":"Proceedings of the 2011 IEEE 15th International Enterprise Distributed Object Computing","author":"Adriansyah Arya","year":"2011","unstructured":"Arya Adriansyah, Boudewijn van Dongen, and Wil van der Aalst. 2011. Conformance checking using cost-based fitness analysis. In Proceedings of the 2011 IEEE 15th International Enterprise Distributed Object Computing. IEEE, 55\u201364."},{"issue":"4","key":"e_1_3_2_6_2","first-page":"1870","article-title":"Measuring fitness and precision of automatically discovered process models: A principled and scalable approach","volume":"34","author":"Augusto Adriano","year":"2022","unstructured":"Adriano Augusto, Raffaele Conforti, Abel Armas-Cervantes, Marlon Dumas, and Marcello La Rosa. 2022. Measuring fitness and precision of automatically discovered process models: A principled and scalable approach. IEEE Transactions on Knowledge and Data Engineering 34, 4 (2022), 1870\u20131888.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"issue":"4","key":"e_1_3_2_7_2","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1109\/TKDE.2018.2841877","article-title":"Automated discovery of process models from event logs: Review and benchmark","volume":"31","author":"Augusto Adriano","year":"2018","unstructured":"Adriano Augusto, Raffaele Conforti, Marlon Dumas, Marcello La Rosa, Fabrizio Maria Maggi, Andrea Marrella, Massimo Mecella, and Allar Soo. 2018. Automated discovery of process models from event logs: Review and benchmark. IEEE Transactions on Knowledge and Data Engineering 31, 4 (2018), 686\u2013705.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"issue":"2","key":"e_1_3_2_8_2","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1007\/s10115-018-1214-x","article-title":"Split miner: Automated discovery of accurate and simple business process models from event logs","volume":"59","author":"Augusto Adriano","year":"2019","unstructured":"Adriano Augusto, Raffaele Conforti, Marlon Dumas, Marcello La Rosa, and Artem Polyvyanyy. 2019. Split miner: Automated discovery of accurate and simple business process models from event logs. Knowledge and Information Systems 59, 2 (2019), 251\u2013284.","journal-title":"Knowledge and Information Systems"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","first-page":"101666","DOI":"10.1016\/j.is.2020.101666","article-title":"Sampling and approximation techniques for efficient process conformance checking","volume":"104","author":"Bauer Martin","year":"2022","unstructured":"Martin Bauer, Han van der Aa, and Matthias Weidlich. 2022. Sampling and approximation techniques for efficient process conformance checking. Information Systems 104 (2022), 101666.","journal-title":"Information Systems"},{"issue":"1","key":"e_1_3_2_10_2","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: A practical and powerful approach to multiple testing","volume":"57","author":"Benjamini Yoav","year":"1995","unstructured":"Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B, Methodological 57, 1 (1995), 289\u2013300.","journal-title":"Journal of the Royal Statistical Society. Series B, Methodological"},{"key":"e_1_3_2_11_2","unstructured":"Alessandro Berti Sebastiaan van Zelst and Wil van der Aalst. 2019. Process mining for python (PM4Py): Bridging the gap between process- and data science. arXiv:1905.06169. Retrieved from https:\/\/arxiv.org\/abs\/1905.06169"},{"key":"e_1_3_2_12_2","first-page":"22","volume-title":"Proceedings of the International Conference on Information, Process, and Knowledge Management","author":"Bozkaya Melike","year":"2009","unstructured":"Melike Bozkaya, Joost Gabriels, and Jan Martijn van der Werf. 2009. Process diagnostics: A method based on process mining. In Proceedings of the International Conference on Information, Process, and Knowledge Management, 22\u201327."},{"key":"e_1_3_2_13_2","first-page":"305","volume-title":"Proceedings of the Conference on the Move to Meaningful Internet Systems","author":"Buijs Joos","year":"2012","unstructured":"Joos Buijs, Boudewijn van Dongen, and Wil van der Aalst. 2012. On the role of fitness, precision, generalization and simplicity in process discovery. In Proceedings of the Conference on the Move to Meaningful Internet Systems, 305\u2013322."},{"issue":"1","key":"e_1_3_2_14_2","doi-asserted-by":"crossref","first-page":"1440001","DOI":"10.1142\/S0218843014400012","article-title":"Quality dimensions in process discovery: The importance of fitness, precision, generalization and simplicity","volume":"23","author":"Buijs Joos","year":"2014","unstructured":"Joos Buijs, Boudewijn van Dongen, and Wil van der Aalst. 2014. Quality dimensions in process discovery: The importance of fitness, precision, generalization and simplicity. International Journal of Cooperative Information Systems 23, 1 (2014), 1440001.","journal-title":"International Journal of Cooperative Information Systems"},{"key":"e_1_3_2_15_2","volume-title":"Conformance Checking","author":"Carmona Josep","year":"2018","unstructured":"Josep Carmona, Boudewijn van Dongen, Andreas Solti, and Matthias Weidlich. 2018. Conformance Checking. Springer."},{"issue":"2","key":"e_1_3_2_16_2","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1109\/TKDE.2016.2614680","article-title":"Filtering out infrequent behavior from business process event logs","volume":"29","author":"Conforti Raffaele","year":"2016","unstructured":"Raffaele Conforti, Marcello La Rosa, and Arthur ter Hofstede. 2016. Filtering out infrequent behavior from business process event logs. IEEE Transactions on Knowledge and Data Engineering 29, 2 (2016), 300\u2013314.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"issue":"3","key":"e_1_3_2_17_2","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1145\/287000.287001","article-title":"Discovering models of software processes from event-based data","volume":"7","author":"Cook Jonathan","year":"1998","unstructured":"Jonathan Cook and Alexander Wolf. 1998. Discovering models of software processes from event-based data. ACM Transactions on Software Engineering and Methodology 7, 3 (1998), 215\u2013249.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"issue":"2","key":"e_1_3_2_18_2","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1145\/304399.304401","article-title":"Software process validation: Quantitatively measuring the correspondence of a process to a model","volume":"8","author":"Cook Jonathan","year":"1999","unstructured":"Jonathan Cook and Alexander Wolf. 1999. Software process validation: Quantitatively measuring the correspondence of a process to a model. ACM Transactions on Software Engineering and Methodology 8, 2 (1999), 147\u2013176.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_19_2","volume-title":"Introduction to Algorithms","author":"Cormen Thomas","year":"2009","unstructured":"Thomas Cormen, Charles Leiserson, Ronald Rivest, and Clifford Stein. 2009. Introduction to Algorithms. MIT Press."},{"key":"e_1_3_2_20_2","volume-title":"Research Design: Qualitative, Quantitative, and Mixed Methods Approaches","author":"Creswell John","year":"2009","unstructured":"John Creswell. 2009. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (3rd ed.). Sage Publications.","edition":"3"},{"key":"e_1_3_2_21_2","unstructured":"Massimiliano de Leoni and Felix Mannhardt. 2015. Road Traffic Fine Management Process. Retrieved from https:\/\/data.4tu.nl\/articles\/dataset\/Road_Traffic_Fine_Management_Process\/12683249\/1"},{"issue":"7","key":"e_1_3_2_22_2","doi-asserted-by":"crossref","first-page":"654","DOI":"10.1016\/j.is.2012.02.004","article-title":"A multi-dimensional quality assessment of state-of-the-art process discovery algorithms using real-life event logs","volume":"37","author":"De Weerdt Jochen","year":"2012","unstructured":"Jochen De Weerdt, Manu De Backer, Jan Vanthienen, and Bart Baesens. 2012. A multi-dimensional quality assessment of state-of-the-art process discovery algorithms using real-life event logs. Information Systems 37, 7 (2012), 654\u2013676.","journal-title":"Information Systems"},{"key":"e_1_3_2_23_2","first-page":"48","article-title":"Algorithm engineering","volume":"79","author":"Demetrescu Camil","year":"2003","unstructured":"Camil Demetrescu, Irene Finocchi, and Giuseppe Italiano. 2003. Algorithm engineering. Bulletin of the EATCS 79 (2003), 48\u201363.","journal-title":"Bulletin of the EATCS"},{"key":"e_1_3_2_24_2","first-page":"127","volume-title":"Proceedings of the Conference on the Move to Meaningful Internet Systems","author":"Emamjome Fahame","year":"2019","unstructured":"Fahame Emamjome, Robert Andrews, and Arthur ter Hofstede. 2019. A case study lens on process mining in practice. In Proceedings of the Conference on the Move to Meaningful Internet Systems, 127\u2013145."},{"issue":"6408","key":"e_1_3_2_25_2","doi-asserted-by":"crossref","first-page":"1178","DOI":"10.1126\/science.361.6408.1178","article-title":"Research on research","volume":"361","author":"Enserink Martin","year":"2018","unstructured":"Martin Enserink. 2018. Research on research. Science 361, 6408 (2018), 1178\u20131179.","journal-title":"Science"},{"issue":"6","key":"e_1_3_2_26_2","doi-asserted-by":"crossref","first-page":"1085","DOI":"10.1007\/s00607-021-00910-4","article-title":"The impact of biased sampling of event logs on the performance of process discovery","volume":"103","author":"Sani Mohammadreza Fani","year":"2021","unstructured":"Mohammadreza Fani Sani, Sebastiaan van Zelst, and Wil van der Aalst. 2021. The impact of biased sampling of event logs on the performance of process discovery. Computing 103, 6 (2021), 1085\u20131104.","journal-title":"Computing"},{"key":"e_1_3_2_27_2","volume-title":"Towards a Pluralistic Conception of Research Methods in Information Systems Research","author":"Frank Ulrich","year":"2006","unstructured":"Ulrich Frank. 2006. Towards a Pluralistic Conception of Research Methods in Information Systems Research. Technical Report 7. ICB-Research Report, Universit\u00e4t Duisburg-Essen."},{"issue":"8","key":"e_1_3_2_28_2","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1016\/j.infsof.2005.07.001","article-title":"Towards a consistent terminology for software measurement","volume":"48","author":"Garc\u00eda F.","year":"2006","unstructured":"F. Garc\u00eda, M. F. Bertoa, C. Calero, A. Vallecillo, F. Ru\u00edz, M. Piattini, and M. Genero. 2006. Towards a consistent terminology for software measurement. Information and Software Technology 48, 8 (2006), 631\u2013644.","journal-title":"Information and Software Technology"},{"key":"e_1_3_2_29_2","volume-title":"\u00dcber die Vollst\u00e4ndigkeit des Logikkalk\u00fcls","author":"G\u00f6del Kurt","year":"1930","unstructured":"Kurt G\u00f6del. 1930. \u00dcber die Vollst\u00e4ndigkeit des Logikkalk\u00fcls. Ph.D. Dissertation. University of Vienna."},{"key":"e_1_3_2_30_2","first-page":"1305","article-title":"Robust process discovery with artificial negative events","volume":"10","author":"Goedertier Stijn","year":"2009","unstructured":"Stijn Goedertier, David Martens, Jan Vanthienen, and Bart Baesens. 2009. Robust process discovery with artificial negative events. Journal of Machine Learning Research 10 (2009), 1305\u20131340.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0019-9958(67)91165-5"},{"issue":"2","key":"e_1_3_2_32_2","first-page":"1","article-title":"The limits of replicability","volume":"10","author":"Guttinger Stephan","year":"2020","unstructured":"Stephan Guttinger. 2020. The limits of replicability. European Journal for Philosophy of Science 10, 2 (2020), 1\u201317.","journal-title":"European Journal for Philosophy of Science"},{"issue":"2","key":"e_1_3_2_33_2","first-page":"4","article-title":"A three cycle view of design science research","volume":"19","author":"Hevner Alan","year":"2007","unstructured":"Alan Hevner. 2007. A three cycle view of design science research. Scandinavian Journal of Information Systems 19, 2 (2007), 4.","journal-title":"Scandinavian Journal of Information Systems"},{"issue":"1","key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"75","DOI":"10.2307\/25148625","article-title":"Design science in information systems research","volume":"28","author":"Hevner Alan","year":"2004","unstructured":"Alan Hevner, Salvatore March, Jinsoo Park, and Sudha Ram. 2004. Design science in information systems research. MIS Quarterly 28, 1 (2004), 75\u2013105.","journal-title":"MIS Quarterly"},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1007\/BF02430364","article-title":"Testing heuristics: We have it all wrong","author":"Hooker John","year":"1995","unstructured":"John Hooker. 1995. Testing heuristics: We have it all wrong. Journal of Heuristics (1995), 33\u201342.","journal-title":"Journal of Heuristics"},{"key":"e_1_3_2_36_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.is.2017.06.002","article-title":"A comparative study of existing quality measures for process discovery","volume":"71","author":"Janssenswillen Gert","year":"2017","unstructured":"Gert Janssenswillen, Niels Donders, Toon Jouck, and Beno\u00eet Depaire. 2017. A comparative study of existing quality measures for process discovery. Information Systems 71 (2017), 1\u201315.","journal-title":"Information Systems"},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1007\/978-3-319-45348-4_5","volume-title":"Proceedings of the 14th International Conference on Business Process Management","author":"Janssenswillen Gert","year":"2016","unstructured":"Gert Janssenswillen, Toon Jouck, Mathijs Creemers, and Beno\u00eet Depaire. 2016. Measuring the quality of models with respect to the underlying system: An empirical study. In M. La Rosa, P. Loos, O. Pastor (Eds.), Proceedings of the 14th International Conference on Business Process Management, 73\u201389."},{"key":"e_1_3_2_38_2","unstructured":"Toon Jouck Alfredo Bolt Beno\u00eet Depaire Massimiliano de Leoni and Wil van der Aalst. 2018. An integrated framework for process discovery algorithm evaluation. arXiv:1806.07222."},{"key":"e_1_3_2_39_2","first-page":"64","volume-title":"Proceedings of the International Conference on Process Mining","author":"Kabierski Martin","year":"2021","unstructured":"Martin Kabierski, Hoang Lam Nguyen, Lars Grunske, and Matthias Weidlich. 2021. Sampling what matters: Relevance-guided sampling of event logs. In Proceedings of the International Conference on Process Mining. IEEE, 64\u201371."},{"key":"e_1_3_2_40_2","first-page":"36","volume-title":"Proceedings of the BPM Demo Sessions","volume":"1295","author":"Kalenkova Anna","year":"2014","unstructured":"Anna Kalenkova, Massimiliano de Leoni, and Wil van der Aalst. 2014. Discovering, analyzing and enhancing BPMN models using ProM. In L. Limonad and B. Weber (Eds.), Proceedings of the BPM Demo Sessions. Vol. 1295, CEUR-WS.org., 36\u201341."},{"key":"e_1_3_2_41_2","first-page":"97","volume-title":"Proceedings of the International Conference on Process Mining","author":"Knols Bram","year":"2019","unstructured":"Bram Knols and Jan Martijn van der Werf. 2019. Measuring the behavioral quality of log sampling. In Proceedings of the International Conference on Process Mining, 97\u2013104."},{"key":"e_1_3_2_42_2","first-page":"120","volume-title":"Proceedings of the International Conference on Process Mining","author":"Koorn Jelmer","year":"2021","unstructured":"Jelmer Koorn, Iris Beerepoot, Vinicius Stein Dani, Xixi Lu, Inge van de Weerd, Henrik Leopold, and Hajo Reijers. 2021. Bringing rigor to the qualitative evaluation of process mining findings: An analysis and a proposal. In Proceedings of the International Conference on Process Mining, 120\u2013127."},{"key":"e_1_3_2_43_2","volume-title":"The Structure of Scientific Revolutions","author":"Kuhn Thomas S.","year":"1962","unstructured":"Thomas S. Kuhn. 1962. The Structure of Scientific Revolutions (50th ed.). University of Chicago Press.","edition":"50"},{"key":"e_1_3_2_44_2","volume-title":"Women, Fire, and Dangerous Things","author":"Lakoff George","year":"1985","unstructured":"George Lakoff. 1985. Women, Fire, and Dangerous Things. University of Chicago Press, Chicago."},{"issue":"3","key":"e_1_3_2_45_2","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1287\/isre.14.3.221.16560","article-title":"Generalizing generalizability in information systems research","volume":"14","author":"Lee Allen","year":"2003","unstructured":"Allen Lee and Richard Baskerville. 2003. Generalizing generalizability in information systems research. Information Systems Research 14, 3 (2003), 221\u2013243.","journal-title":"Information Systems Research"},{"key":"e_1_3_2_46_2","series-title":"Lecture Notes in Computer Science, Vol. 7927","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1007\/978-3-642-38697-8_17","volume-title":"Proceedings of the International Conference on Application and Theory of Petri Nets and Concurrency (PETRI NETS \u201913)","author":"Leemans Sander","year":"2013","unstructured":"Sander Leemans, Dirk Fahland, and Wil van der Aalst. 2013. Discovering block-structured process models from event logs \u2013 A constructive approach. In Proceedings of the International Conference on Application and Theory of Petri Nets and Concurrency (PETRI NETS \u201913). J. M. Colom and J. Desel (Eds.), Lecture Notes in Computer Science, Vol. 7927, Springer, Berlin, 311\u2013329."},{"key":"e_1_3_2_47_2","series-title":"Lecture Notes in Computer Science, Vol. 8489","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1007\/978-3-319-07734-5_6","volume-title":"Proceedings of the International Conference on Application and Theory of Petri Nets and Concurrency (PETRI NETS \u201914)","author":"Leemans Sander","year":"2014","unstructured":"Sander Leemans, Dirk Fahland, and Wil van der Aalst. 2014. Discovering block-structured process models from incomplete event logs. In Proceedings of the International Conference on Application and Theory of Petri Nets and Concurrency (PETRI NETS \u201914). G. Ciardo and E. Kindler (Eds.), Lecture Notes in Computer Science, Vol. 8489. Springer, Cham. 91\u2013110."},{"key":"e_1_3_2_48_2","first-page":"25","volume-title":"Proceedings of the International Conference on Process Mining","author":"Leemans Sander","year":"2019","unstructured":"Sander Leemans, Erik Poppe, and Moe Wynn. 2019. Directly follows-based process mining: exploration & a case study. In Proceedings of the International Conference on Process Mining. 25\u201332."},{"key":"e_1_3_2_49_2","unstructured":"Felix Mannhardt. 2016. Sepsis Cases - Event Log. Retrieved from https:\/\/data.4tu.nl\/articles\/dataset\/Sepsis_Cases_Event_Log\/12707639\/1"},{"issue":"1","key":"e_1_3_2_50_2","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1007\/s10618-005-0029-z","article-title":"A rule-based approach for process discovery: Dealing with noise and imbalance in process logs","volume":"13","author":"Maruster Laura","year":"2006","unstructured":"Laura Maruster, Ton Weijters, Wil van der Aalst, and Antal van den Bosch. 2006. A rule-based approach for process discovery: Dealing with noise and imbalance in process logs. Data Mining and Knowledge Discovery 13, 1 (2006), 67\u201387.","journal-title":"Data Mining and Knowledge Discovery"},{"key":"e_1_3_2_51_2","volume-title":"Detection and Prediction of Errors in EPC Business Process Models","author":"Mendling Jan","year":"2007","unstructured":"Jan Mendling. 2007. Detection and Prediction of Errors in EPC Business Process Models. Ph.D. Dissertation. Wirtschaftsuniversit\u00e4t Wien."},{"key":"e_1_3_2_52_2","unstructured":"Jan Mendling Beno\u00eet Depaire and Henrik Leopold. 2021. Theory and practice of algorithm engineering. arXiv:2107.10675. Retrieved from https:\/\/arxiv.org\/abs\/2107.10675"},{"key":"e_1_3_2_53_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-49451-7","volume-title":"Conformance Checking and Diagnosis in Process Mining","author":"Munoz-Gama Jorge","year":"2016","unstructured":"Jorge Munoz-Gama. 2016. Conformance Checking and Diagnosis in Process Mining. Springer."},{"key":"e_1_3_2_54_2","series-title":"Lecture Notes in Computer Science, Vol. 8094","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1007\/978-3-642-40176-3_11","volume-title":"Proceedings of the International Conference on Business Process Management","author":"Munoz-Gama Jorge","year":"2013","unstructured":"Jorge Munoz-Gama, Josep Carmona, and Wil van der Aalst. 2013. Conformance checking in the large: Partitioning and topology. In Proceedings of the International Conference on Business Process Management. F. Daniel, J. Wang, and B. Weber (Eds.), Lecture Notes in Computer Science, Vol. 8094. Springer, Berlin, 130\u2013145."},{"key":"e_1_3_2_55_2","unstructured":"Object Management Group. 2011. Notation BPMN version 2.0."},{"issue":"3","key":"e_1_3_2_56_2","doi-asserted-by":"crossref","first-page":"45","DOI":"10.2753\/MIS0742-1222240302","article-title":"A design science research methodology for information systems research","volume":"24","author":"Peffers Ken","year":"2007","unstructured":"Ken Peffers, Tuure Tuunanen, Marcus Rothenberger, and Samir Chatterjee. 2007. A design science research methodology for information systems research. Journal of Management Information Systems 24, 3 (2007), 45\u201377.","journal-title":"Journal of Management Information Systems"},{"issue":"6060","key":"e_1_3_2_57_2","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1126\/science.1213847","article-title":"Reproducible research in computational science","volume":"334","author":"Peng Roger","year":"2011","unstructured":"Roger Peng. 2011. Reproducible research in computational science. Science 334, 6060 (2011), 1226\u20131227.","journal-title":"Science"},{"issue":"3","key":"e_1_3_2_58_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3387909","article-title":"Monotone precision and recall measures for comparing executions and specifications of dynamic systems","volume":"29","author":"Polyvyanyy Artem","year":"2020","unstructured":"Artem Polyvyanyy, Andreas Solti, Matthias Weidlich, Claudio Di Ciccio, and Jan Mendling. 2020. Monotone precision and recall measures for comparing executions and specifications of dynamic systems. ACM Transactions on Software Engineering and Methodology 29, 3 (2020), 1\u201341.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_59_2","author":"Popper Karl","year":"1935","unstructured":"Karl Popper. 1935. Logik der Forschung \u2013 Zur Erkenntnistheorie der modernen Naturwissenschaften.","journal-title":"Logik der Forschung \u2013 Zur Erkenntnistheorie der modernen Naturwissenschaften"},{"key":"e_1_3_2_60_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-85436-2","volume-title":"Scientific Research in Information Systems","author":"Recker Jan","year":"2021","unstructured":"Jan Recker. 2021. Scientific Research in Information Systems (2nd ed.). Springer.","edition":"2"},{"key":"e_1_3_2_61_2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-3-319-98651-7_1","volume-title":"Proceedings of the International Conference on Business Process Management Forum","author":"Rehse Jana-Rebecca","year":"2018","unstructured":"Jana-Rebecca Rehse and Peter Fettke. 2018. Process mining crimes \u2013 A threat to the validity of process discovery evaluations. In Proceedings of the International Conference on Business Process Management Forum, 3\u201319."},{"key":"e_1_3_2_62_2","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1007\/978-3-319-74030-0_19","volume-title":"Proceedings of the International Conference on Business Process Management Workshops","author":"Rehse Jana-Rebecca","year":"2018","unstructured":"Jana-Rebecca Rehse, Peter Fettke, and Peter Loos. 2018. Process mining and the black swan: An empirical analysis of the influence of unobserved behavior on the quality of mined process models. In Proceedings of the International Conference on Business Process Management Workshops, 256\u2013268."},{"key":"e_1_3_2_63_2","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1007\/978-3-319-10172-9_5","volume-title":"Proceedings of the International Conference on Business Process Management","author":"Ribeiro Joel","year":"2014","unstructured":"Joel Ribeiro, Josep Carmona, Mustafa Mi\u0307si\u0307r, and Michele Sebag. 2014. A recommender system for process discovery. In Proceedings of the International Conference on Business Process Management, 67\u201383."},{"key":"e_1_3_2_64_2","author":"Rozinat Anne","year":"2007","unstructured":"Anne Rozinat, Ana Karla Alves de Medeiros, Christian G\u00fcnther, Ton Weijters, and Wil van der Aalst. 2007. Towards an Evaluation Framework for Process Mining Algorithms. BPM Center Report BPM-07-06.","journal-title":"Towards an Evaluation Framework for Process Mining Algorithms"},{"key":"e_1_3_2_65_2","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1007\/978-3-540-78238-4_10","volume-title":"Proceedings of the International Conference on Business Process Management Workshops","author":"Rozinat Anne","year":"2008","unstructured":"Anne Rozinat, Ana Karla Alves de Medeiros, Christian G\u00fcnther, Ton Weijters, and Wil van der Aalst. 2008. The need for a process mining evaluation framework in research and practice. In Proceedings of the International Conference on Business Process Management Workshops, 84\u201389."},{"issue":"2","key":"e_1_3_2_66_2","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1007\/s10664-008-9102-8","article-title":"Guidelines for conducting and reporting case study research in software engineering","volume":"14","author":"Runeson Per","year":"2009","unstructured":"Per Runeson and Martin H\u00f6st. 2009. Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering 14, 2 (2009), 131\u2013164.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_2_67_2","series-title":"Lecture Notes in Computer Science, Vol. 5760","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1007\/978-3-642-03456-5_22","volume-title":"Proceedings of the International Conference on Efficient Algorithms","author":"Sanders Peter","year":"2009","unstructured":"Peter Sanders. 2009. Algorithm engineering \u2013 An attempt at a definition. In Proceedings of the International Conference on Efficient Algorithms. S. Albers, H. Alt, and S. N\u00e4her (Eds.), Lecture Notes in Computer Science, Vol. 5760, Springer, Berlin, 321\u2013340."},{"key":"e_1_3_2_68_2","volume-title":"Experimental and Quasi-Experimental Designs for Generalized Causal Inference","author":"Shadish William","year":"2002","unstructured":"William Shadish, Thomas Cook, and Donald Campbell. 2002. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Wadsworth Cengage Learning."},{"issue":"1","key":"e_1_3_2_69_2","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1007\/BF00139833","article-title":"A common base for quality control criteria in quantitative and qualitative research","volume":"30","author":"Swanborn Peter","year":"1996","unstructured":"Peter Swanborn. 1996. A common base for quality control criteria in quantitative and qualitative research. Quality & Quantity 30, 1 (1996), 19\u201335.","journal-title":"Quality & Quantity"},{"key":"e_1_3_2_70_2","series-title":"Lecture Notes in Computer Science, Vol. 11790","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1007\/978-3-662-60651-3_8","volume-title":"Proceedings of the Transactions on Petri Nets and Other Models of Concurrency XIV","author":"Syring Anja","year":"2019","unstructured":"Anja Syring, Niek Tax, and Wil van der Aalst. 2019. Evaluating conformance measures in process mining using conformance propositions. In Proceedings of the Transactions on Petri Nets and Other Models of Concurrency XIV. Maciej Koutny, Lucia Pomello, and Lars Michael Kristensen (Eds.), Lecture Notes in Computer Science, Vol. 11790, Springer, 192\u2013221."},{"key":"e_1_3_2_71_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ipl.2018.01.013","article-title":"The imprecisions of precision measures in process mining","volume":"135","author":"Tax Niek","year":"2018","unstructured":"Niek Tax, Xixi Lu, Natalia Sidorova, Dirk Fahland, and Wil van der Aalst. 2018. The imprecisions of precision measures in process mining. Information Processing Letters 135 (2018), 1\u20138.","journal-title":"Information Processing Letters"},{"key":"e_1_3_2_72_2","first-page":"407","volume-title":"Proceedings of the International Conference on Application and Theory of Petri NetsLecture Notes in Computer Science, Vol. 1248","author":"van der Aalst Wil","year":"1997","unstructured":"Wil van der Aalst. 1997. Verification of workflow nets. In Proceedings of the International Conference on Application and Theory of Petri Nets. P. Az\u00e9ma and G. Balbo (Eds.), Lecture Notes in Computer Science, Vol. 1248. Springer, Berlin, 407\u2013426."},{"key":"e_1_3_2_73_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-662-49851-4","volume-title":"Process Mining - Data Science in Action","author":"van der Aalst Wil","year":"2016","unstructured":"Wil van der Aalst. 2016. Process Mining - Data Science in Action (2nd ed.). Springer.","edition":"2"},{"issue":"2","key":"e_1_3_2_74_2","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1002\/widm.1045","article-title":"Replaying history on process models for conformance checking and performance analysis","volume":"2","author":"van der Aalst Wil","year":"2012","unstructured":"Wil van der Aalst, Arya Adriansyah, and Boudewijn van Dongen. 2012. Replaying history on process models for conformance checking and performance analysis. Data Mining and Knowledge Discovery 2, 2 (2012), 182\u2013192.","journal-title":"Data Mining and Knowledge Discovery"},{"issue":"3","key":"e_1_3_2_75_2","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1007\/s00165-010-0161-4","article-title":"Soundness of workflow nets: classification, decidability, and analysis","volume":"23","author":"van der Aalst Wil","year":"2011","unstructured":"Wil van der Aalst, Kees van Hee, Arthur ter Hofstede, Natalia Sidorova, Eric Verbeek, Marc Voorhoeve, and Moe Wynn. 2011. Soundness of workflow nets: classification, decidability, and analysis. Formal Aspects of Computing 23, 3 (2011), 333\u2013363.","journal-title":"Formal Aspects of Computing"},{"issue":"9","key":"e_1_3_2_76_2","doi-asserted-by":"crossref","first-page":"1128","DOI":"10.1109\/TKDE.2004.47","article-title":"Workflow mining: Discovering process models from event logs","volume":"16","author":"van der Aalst Wil","year":"2004","unstructured":"Wil van der Aalst, Ton Weijters, and Laura Maruster. 2004. Workflow mining: Discovering process models from event logs. Knowledge & Data Engineering 16, 9 (2004), 1128\u20131142.","journal-title":"Knowledge & Data Engineering"},{"key":"e_1_3_2_77_2","first-page":"141","volume-title":"Proceedings of the 33rd International Conference on Advanced Information Systems Engineering","author":"van der Werf Jan Martijn","year":"2021","unstructured":"Jan Martijn van der Werf, Artem Polyvyanyy, Bart van Wensveen, Matthieu Brinkhuis, and Hajo Reijers. 2021. All that glitters is not gold: Towards process discovery techniques with guarantees. In Proceedings of the 33rd International Conference on Advanced Information Systems Engineering, 141\u2013157."},{"key":"e_1_3_2_78_2","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1007\/978-3-540-68746-7_24","volume-title":"Proceedings of the International Conference on Applications and Theory of Petri Nets","author":"van der Werf Jan Martijn","year":"2008","unstructured":"Jan Martijn van der Werf, Boudewijn van Dongen, Cor Hurkens, and Alexander Serebrenik. 2008. Process discovery using integer linear programming. In Proceedings of the International Conference on Applications and Theory of Petri Nets, 368\u2013387."},{"key":"e_1_3_2_79_2","unstructured":"Boudewijn van Dongen. 2024. BPI Challenge. Retrieved from https:\/\/data.4tu.nl\/"},{"key":"e_1_3_2_80_2","first-page":"444","volume-title":"Proceedings of the International Conference on Applications and Theory of Petri Nets","author":"van Dongen Boudewijn","year":"2005","unstructured":"Boudewijn van Dongen, Ana Karla Alves de Medeiros, Eric Verbeek, Ton Weijters, and Wil van der Aalst. 2005. The ProM framework: A new era in process mining tool support. In Proceedings of the International Conference on Applications and Theory of Petri Nets, 444\u2013454."},{"key":"e_1_3_2_81_2","first-page":"297","article-title":"PM \\({}^{2}\\) : A process mining project methodology","author":"van Eck Maikel","year":"2015","unstructured":"Maikel van Eck, Xixi Lu, Sander Leemans, and Wil van der Aalst. 2015. PM \\({}^{2}\\) : A process mining project methodology. In Proceedings of the International Conference on Advanced Information Systems Engineering, 297\u2013313.","journal-title":"Proceedings of the International Conference on Advanced Information Systems Engineering"},{"issue":"5","key":"e_1_3_2_82_2","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1007\/s00607-017-0582-5","article-title":"Discovering workflow nets using integer linear programming","volume":"100","author":"van Zelst Sebastiaan","year":"2018","unstructured":"Sebastiaan van Zelst, Boudewijn van Dongen, Wil van der Aalst, and Eric Verbeek. 2018. Discovering workflow nets using integer linear programming. Computing 100, 5 (2018), 529\u2013556.","journal-title":"Computing"},{"key":"e_1_3_2_83_2","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.dss.2017.04.005","article-title":"Fodina: A robust and flexible heuristic process discovery technique","volume":"100","author":"Broucke Seppe vanden","year":"2017","unstructured":"Seppe vanden Broucke and Jochen De Weerdt. 2017. Fodina: A robust and flexible heuristic process discovery technique. Decision Support Systems 100 (2017), 109\u2013118.","journal-title":"Decision Support Systems"},{"key":"e_1_3_2_84_2","first-page":"254","volume-title":"Proceedings of the Symposium on Computational Intelligence and Data Mining","author":"Broucke Seppe vanden","year":"2013","unstructured":"Seppe vanden Broucke, Jochen De Weerdt, Jan Vanthienen, and Bart Baesens. 2013. A comprehensive benchmarking framework (CoBeFra) for conformance analysis between procedural process models and event logs in ProM. In Proceedings of the Symposium on Computational Intelligence and Data Mining. IEEE, 254\u2013261."},{"issue":"8","key":"e_1_3_2_85_2","doi-asserted-by":"crossref","first-page":"1877","DOI":"10.1109\/TKDE.2013.130","article-title":"Determining process model precision and generalization with weighted artificial negative events","volume":"26","author":"Broucke Seppe vanden","year":"2014","unstructured":"Seppe vanden Broucke, Jochen De Weerdt, Jan Vanthienen, and Bart Baesens. 2014. Determining process model precision and generalization with weighted artificial negative events. IEEE Transactions on Knowledge and Data Engineering 26, 8 (2014), 1877\u20131889.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"e_1_3_2_86_2","first-page":"625","volume-title":"Proceedings of the GCC Conference and Exhibition","author":"Weber Philip","year":"2011","unstructured":"Philip Weber, Behzad Bordbar, Peter Ti\u0148o, and Basim Majeed. 2011. A framework for comparing process mining algorithms. In Proceedings of the GCC Conference and Exhibition. IEEE, 625\u2013628."},{"key":"e_1_3_2_87_2","first-page":"310","volume-title":"Proceedings of the Symposium on Computational Intelligence and Data Mining","author":"Weijters Ton","year":"2011","unstructured":"Ton Weijters and Joel Ribeiro. 2011. Flexible heuristics miner (FHM). In Proceedings of the Symposium on Computational Intelligence and Data Mining, 310\u2013317."},{"key":"e_1_3_2_88_2","volume-title":"Process Mining with the Heuristics Miner-Algorithm","author":"Weijters Ton","year":"2006","unstructured":"Ton Weijters, Wil van der Aalst, and Ana Karla Alves de Medeiros. 2006. Process Mining with the Heuristics Miner-Algorithm. Technical Report 166. Technische Universiteit Eindhoven."},{"key":"e_1_3_2_89_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-662-59432-2","volume-title":"Business Process Management: Concepts, Languages, Architectures","author":"Weske Mathias","year":"2019","unstructured":"Mathias Weske. 2019. Business Process Management: Concepts, Languages, Architectures (3rd ed.). Springer.","edition":"3"},{"key":"e_1_3_2_90_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-662-43839-8","volume-title":"Design Science Methodology for Information Systems and Software Engineering","author":"Wieringa Roel","year":"2014","unstructured":"Roel Wieringa. 2014. Design Science Methodology for Information Systems and Software Engineering. Springer."},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco_a_01362"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3672447","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3672447","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:58:01Z","timestamp":1750294681000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3672447"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,26]]},"references-count":90,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1,31]]}},"alternative-id":["10.1145\/3672447"],"URL":"https:\/\/doi.org\/10.1145\/3672447","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,26]]},"assertion":[{"value":"2022-11-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-12","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}