{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,25]],"date-time":"2025-12-25T07:27:28Z","timestamp":1766647648293,"version":"3.40.3"},"publisher-location":"Cham","reference-count":26,"publisher":"Springer Nature Switzerland","isbn-type":[{"type":"print","value":"9783031416194"},{"type":"electronic","value":"9783031416200"}],"license":[{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,1]],"date-time":"2023-09-01T00:00:00Z","timestamp":1693526400000},"content-version":"vor","delay-in-days":243,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Business Process Simulation (BPS) is an approach to analyze the performance of business processes under different scenarios. For example, BPS allows us to estimate what would be the cycle time of a process if one or more resources became unavailable. The starting point of BPS is a process model annotated with simulation parameters (a BPS model). BPS models may be manually designed, based on information collected from stakeholders and empirical observations, or automatically discovered from execution data. Regardless of its origin, a key question when using a BPS model is how to assess its quality. In this paper, we propose a collection of measures to evaluate the quality of a BPS model w.r.t. its ability to replicate the observed behavior of the process. We advocate an approach whereby different measures tackle different process perspectives. We evaluate the ability of the proposed measures to discern the impact of modifications to a BPS model, and their ability to uncover the relative strengths and weaknesses of two approaches for automated discovery of BPS models. The evaluation shows that the measures not only capture how close a BPS model is to the observed behavior, but they also help us to identify sources of discrepancies.<\/jats:p>","DOI":"10.1007\/978-3-031-41620-0_2","type":"book-chapter","created":{"date-parts":[[2023,8,31]],"date-time":"2023-08-31T23:03:46Z","timestamp":1693523026000},"page":"20-37","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Can I Trust My Simulation Model? Measuring the\u00a0Quality of\u00a0Business Process Simulation Models"],"prefix":"10.1007","author":[{"given":"David","family":"Chapela-Campa","sequence":"first","affiliation":[]},{"given":"Ismail","family":"Benchekroun","sequence":"additional","affiliation":[]},{"given":"Opher","family":"Baron","sequence":"additional","affiliation":[]},{"given":"Marlon","family":"Dumas","sequence":"additional","affiliation":[]},{"given":"Dmitry","family":"Krass","sequence":"additional","affiliation":[]},{"given":"Arik","family":"Senderovich","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,9,1]]},"reference":[{"key":"2_CR1","unstructured":"Abel, M.: Lightning fast business process simulator. Master\u2019s thesis. Institute of Computer Science, University of Tartu (2011)"},{"issue":"4","key":"2_CR2","first-page":"1870","volume":"34","author":"A Augusto","year":"2022","unstructured":"Augusto, A., Armas-Cervantes, A., Conforti, R., Dumas, M., Rosa, M.L.: Measuring fitness and precision of automatically discovered process models: a principled and scalable approach. IEEE Trans. Knowl. Data Eng. 34(4), 1870\u20131888 (2022)","journal-title":"IEEE Trans. Knowl. Data Eng."},{"issue":"2","key":"2_CR3","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1007\/s10115-018-1214-x","volume":"59","author":"A Augusto","year":"2019","unstructured":"Augusto, A., Conforti, R., Dumas, M., Rosa, M.L., Polyvyanyy, A.: Split miner: automated discovery of accurate and simple business process models from event logs. Knowl. Inf. Syst. 59(2), 251\u2013284 (2019)","journal-title":"Knowl. Inf. Syst."},{"key":"2_CR4","doi-asserted-by":"publisher","DOI":"10.1007\/b97391","volume-title":"Introduction to Time Series and Forecasting","author":"PJ Brockwell","year":"2002","unstructured":"Brockwell, P.J., Davis, R.A.: Introduction to Time Series and Forecasting. Springer, New York (2002). https:\/\/doi.org\/10.1007\/b97391"},{"key":"2_CR5","doi-asserted-by":"crossref","unstructured":"Burke, A., Leemans, S.J.J., Wynn, M.T., van der Aalst, W.M.P., ter Hofstede, A.H.M.: Stochastic process model-log quality dimensions: an experimental study. In: ICPM 2022, pp. 80\u201387. IEEE (2022)","DOI":"10.1109\/ICPM57379.2022.9980707"},{"key":"2_CR6","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2020.113284","volume":"134","author":"M Camargo","year":"2020","unstructured":"Camargo, M., Dumas, M., Gonz\u00e1lez, O.: Automated discovery of business process simulation models from event logs. Decis. Support Syst. 134, 113284 (2020)","journal-title":"Decis. Support Syst."},{"key":"2_CR7","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.577","volume":"7","author":"M Camargo","year":"2021","unstructured":"Camargo, M., Dumas, M., Rojas, O.G.: Discovering generative models from event logs: data-driven simulation vs deep learning. PeerJ Comput. Sci. 7, e577 (2021)","journal-title":"PeerJ Comput. Sci."},{"key":"2_CR8","doi-asserted-by":"crossref","unstructured":"Chapela-Campa, D., Dumas, M.: Modeling extraneous activity delays in business process simulation. In: ICPM 2022, pp. 72\u201379. IEEE (2022)","DOI":"10.1109\/ICPM57379.2022.9980544"},{"key":"2_CR9","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-56509-4","volume-title":"Fundamentals of Business Process Management","author":"M Dumas","year":"2018","unstructured":"Dumas, M., Rosa, M.L., Mendling, J., Reijers, H.A.: Fundamentals of Business Process Management, 2nd edn. Springer, Heidelberg (2018). https:\/\/doi.org\/10.1007\/978-3-662-56509-4","edition":"2"},{"key":"2_CR10","series-title":"LNCS","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1007\/978-3-031-07472-1_17","volume-title":"CAiSE 2022","author":"C Fracca","year":"2022","unstructured":"Fracca, C., de Leoni, M., Asnicar, F., Turco, A.: Estimating activity start timestamps in the presence of waiting times via process simulation. In: Franch, X., Poels, G., Gailly, F., Snoeck, M. (eds.) CAiSE 2022. LNCS, vol. 13295, pp. 287\u2013303. Springer, Cham (2022). https:\/\/doi.org\/10.1007\/978-3-031-07472-1_17"},{"key":"2_CR11","volume-title":"The Statistical Analysis of Failure Time Data","author":"JD Kalbfleisch","year":"2011","unstructured":"Kalbfleisch, J.D., Prentice, R.L.: The Statistical Analysis of Failure Time Data. Wiley, Hoboken (2011)"},{"key":"2_CR12","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1007\/978-3-030-49435-3_14","volume-title":"Advanced Information Systems Engineering","author":"SJJ Leemans","year":"2020","unstructured":"Leemans, S.J.J., Polyvyanyy, A.: Stochastic-aware conformance checking: an entropy-based approach. In: Dustdar, S., Yu, E., Salinesi, C., Rieu, D., Pant, V. (eds.) CAiSE 2020. LNCS, vol. 12127, pp. 217\u2013233. Springer, Cham (2020). https:\/\/doi.org\/10.1007\/978-3-030-49435-3_14"},{"key":"2_CR13","series-title":"Lecture Notes in Business Information Processing","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1007\/978-3-030-26643-1_8","volume-title":"Business Process Management Forum","author":"SJJ Leemans","year":"2019","unstructured":"Leemans, S.J.J., Syring, A.F., van der Aalst, W.M.P.: Earth movers\u2019 stochastic conformance checking. In: Hildebrandt, T., van Dongen, B.F., R\u00f6glinger, M., Mendling, J. (eds.) BPM 2019. LNBIP, vol. 360, pp. 127\u2013143. Springer, Cham (2019). https:\/\/doi.org\/10.1007\/978-3-030-26643-1_8"},{"key":"2_CR14","doi-asserted-by":"crossref","unstructured":"Levina, E., Bickel, P.J.: The earth mover\u2019s distance is the mallows distance: some insights from statistics. In: ICCV 2001, vol. 2, pp. 251\u2013256. IEEE Computer Society (2001)","DOI":"10.1109\/ICCV.2001.937632"},{"issue":"11","key":"2_CR15","doi-asserted-by":"publisher","first-page":"1857","DOI":"10.1016\/j.patcog.2005.01.025","volume":"38","author":"TW Liao","year":"2005","unstructured":"Liao, T.W.: Clustering of time series data - a survey. Pattern Recognit. 38(11), 1857\u20131874 (2005)","journal-title":"Pattern Recognit."},{"key":"2_CR16","series-title":"LNCS","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1007\/978-3-031-16103-2_24","volume-title":"BPM 2022","author":"O L\u00f3pez-Pintado","year":"2022","unstructured":"L\u00f3pez-Pintado, O., Dumas, M.: Business process simulation with differentiated resources: does it make a difference? In: Di Ciccio, C., Dijkman, R., del R\u00edo Ortega, A., Rinderle-Ma, S. (eds.) BPM 2022. LNCS, vol. 13420, pp. 361\u2013378. Springer, Cham (2022). https:\/\/doi.org\/10.1007\/978-3-031-16103-2_24"},{"issue":"1","key":"2_CR17","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1007\/s12599-015-0410-4","volume":"58","author":"N Martin","year":"2016","unstructured":"Martin, N., Depaire, B., Caris, A.: The use of process mining in business process simulation model construction - structuring the field. Bus. Inf. Syst. Eng. 58(1), 73\u201387 (2016)","journal-title":"Bus. Inf. Syst. Eng."},{"key":"2_CR18","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2020.101642","volume":"95","author":"N Martin","year":"2021","unstructured":"Martin, N., Pufahl, L., Mannhardt, F.: Detection of batch activities from event logs. Inf. Syst. 95, 101642 (2021)","journal-title":"Inf. Syst."},{"issue":"1","key":"2_CR19","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1016\/j.physd.2010.08.005","volume":"240","author":"M Muskulus","year":"2011","unstructured":"Muskulus, M., Verduyn-Lunel, S.: Wasserstein distances in the analysis of time series and dynamical systems. Physica D 240(1), 45\u201358 (2011)","journal-title":"Physica D"},{"key":"2_CR20","doi-asserted-by":"publisher","first-page":"78527","DOI":"10.1109\/ACCESS.2022.3193507","volume":"10","author":"M Pourbafrani","year":"2022","unstructured":"Pourbafrani, M., van der Aalst, W.M.P.: Discovering system dynamics simulation models using process mining. IEEE Access 10, 78527\u201378547 (2022)","journal-title":"IEEE Access"},{"issue":"5","key":"2_CR21","doi-asserted-by":"publisher","first-page":"569","DOI":"10.1007\/s12599-021-00690-3","volume":"63","author":"K Rosenthal","year":"2021","unstructured":"Rosenthal, K., Ternes, B., Strecker, S.: Business process simulation on procedural graphical process models. Bus. Inf. Syst. Eng. 63(5), 569\u2013602 (2021)","journal-title":"Bus. Inf. Syst. Eng."},{"issue":"3","key":"2_CR22","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1016\/j.is.2008.09.002","volume":"34","author":"A Rozinat","year":"2009","unstructured":"Rozinat, A., Mans, R.S., Song, M., van der Aalst, W.M.P.: Discovering simulation models. Inf. Syst. 34(3), 305\u2013327 (2009)","journal-title":"Inf. Syst."},{"key":"2_CR23","doi-asserted-by":"crossref","unstructured":"Senderovich, A., Beck, J.C., Gal, A., Weidlich, M.: Congestion graphs for automated time predictions. In: AAAI 2019, pp. 4854\u20134861. AAAI Press (2019)","DOI":"10.1609\/aaai.v33i01.33014854"},{"key":"2_CR24","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1016\/j.is.2016.01.002","volume":"62","author":"A Senderovich","year":"2016","unstructured":"Senderovich, A., et al.: Conformance checking and performance improvement in scheduled processes: a queueing-network perspective. Inf. Syst. 62, 185\u2013206 (2016)","journal-title":"Inf. Syst."},{"key":"2_CR25","doi-asserted-by":"crossref","unstructured":"Thomas, M.U.: Queueing systems. volume 1: theory (leonard kleinrock). SIAM Rev. 18(3), 512\u2013514 (1976)","DOI":"10.1137\/1018095"},{"key":"2_CR26","doi-asserted-by":"crossref","unstructured":"Zhao, C., Sahni, S.: String correction using the damerau-levenshtein distance. BMC Bioinform. 20-S(11), 277:1\u2013277:28 (2019)","DOI":"10.1186\/s12859-019-2819-0"}],"container-title":["Lecture Notes in Computer Science","Business Process Management"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-41620-0_2","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,27]],"date-time":"2024-10-27T08:53:37Z","timestamp":1730019217000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-41620-0_2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"ISBN":["9783031416194","9783031416200"],"references-count":26,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-41620-0_2","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"type":"print","value":"0302-9743"},{"type":"electronic","value":"1611-3349"}],"subject":[],"published":{"date-parts":[[2023]]},"assertion":[{"value":"1 September 2023","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"BPM","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Conference on Business Process Management","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Utrecht","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"The Netherlands","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2023","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"11 September 2023","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"15 September 2023","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"21","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"bpm2023","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/bpm2023.sites.uu.nl\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Single-blind","order":1,"name":"type","label":"Type","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"EasyChair","order":2,"name":"conference_management_system","label":"Conference Management System","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"151","order":3,"name":"number_of_submissions_sent_for_review","label":"Number of Submissions Sent for Review","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"27","order":4,"name":"number_of_full_papers_accepted","label":"Number of Full Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"0","order":5,"name":"number_of_short_papers_accepted","label":"Number of Short Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"18% - The value is computed by the equation \"Number of Full Papers Accepted \/ Number of Submissions Sent for Review * 100\" and then rounded to a whole number.","order":6,"name":"acceptance_rate_of_full_papers","label":"Acceptance Rate of Full Papers","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3","order":7,"name":"average_number_of_reviews_per_paper","label":"Average Number of Reviews per Paper","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"5","order":8,"name":"average_number_of_papers_per_reviewer","label":"Average Number of Papers per Reviewer","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"Yes","order":9,"name":"external_reviewers_involved","label":"External Reviewers Involved","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"In addition, for the BPM Forum 23 papers were accepted. 18 papers of 42 submissions were accepted for the Blockchain- Educators- and RPA Forum.","order":10,"name":"additional_info_on_review_process","label":"Additional Info on Review Process","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}}]}}