{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T15:29:11Z","timestamp":1791646151256,"version":"4.3.4"},"reference-count":31,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2022,6,8]],"date-time":"2022-06-08T00:00:00Z","timestamp":1654646400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ulm University"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Predictive Process Monitoring (PPM) has been integrated into process mining use cases as a value-adding task. PPM provides useful predictions on the future of the running business processes with respect to different perspectives, such as the upcoming activities to be executed next, the final execution outcome, and performance indicators. In the context of PPM, Machine Learning (ML) techniques are widely employed. In order to gain trust of stakeholders regarding the reliability of PPM predictions, eXplainable Artificial Intelligence (XAI) methods have been increasingly used to compensate for the lack of transparency of most of predictive models. Multiple XAI methods exist providing explanations for almost all types of ML models. However, for the same data, as well as, under the same preprocessing settings or same ML models, generated explanations often vary significantly. Corresponding variations might jeopardize the consistency and robustness of the explanations and, subsequently, the utility of the corresponding model and pipeline settings. This paper introduces a framework that enables the analysis of the impact PPM-related settings and ML-model-related choices may have on the characteristics and expressiveness of the generated explanations. Our framework provides a means to examine explanations generated either for the whole reasoning process of an ML model, or for the predictions made on the future of a certain business process instance. Using well-defined experiments with different settings, we uncover how choices made through a PPM workflow affect and can be reflected through explanations. This framework further provides the means to compare how different characteristics of explainability methods can shape the resulting explanations and reflect on the underlying model reasoning process.<\/jats:p>","DOI":"10.3390\/a15060199","type":"journal-article","created":{"date-parts":[[2022,6,9]],"date-time":"2022-06-09T09:59:38Z","timestamp":1654768778000},"page":"199","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["XAI in the Context of Predictive Process Monitoring: An Empirical Analysis Framework"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7801-7310","authenticated-orcid":false,"given":"Ghada","family":"El-khawaga","sequence":"first","affiliation":[{"name":"Institute of Databases and Information Systems, Ulm University, 89081 Ulm, Germany"},{"name":"Faculty of Computers and Information, Mansoura University, Mansoura 35516, Egypt"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0958-7322","authenticated-orcid":false,"given":"Mervat","family":"Abu-Elkheir","sequence":"additional","affiliation":[{"name":"Faculty of Media Engineering and Technology, German University in Cairo, New Cairo 11511, Egypt"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2536-4153","authenticated-orcid":false,"given":"Manfred","family":"Reichert","sequence":"additional","affiliation":[{"name":"Institute of Databases and Information Systems, Ulm University, 89081 Ulm, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Van der Aalst, W. (2016). Process Mining: Data Science in Action, Springer. [2nd ed.].","DOI":"10.1007\/978-3-662-49851-4"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1145\/3331449","article-title":"Survey and Cross-benchmark Comparison of Remaining Time Prediction Methods in Business Process Monitoring","volume":"10","author":"Verenich","year":"2019","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1145\/3301300","article-title":"Outcome-Oriented Predictive Process Monitoring: Review and Benchmark","volume":"13","author":"Teinemaa","year":"2019","journal-title":"ACM Trans. Knowl. Discov. Data"},{"key":"ref_4","unstructured":"(2022, June 06). Molnar, Christoph: Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. Available online: https:\/\/christophm.github.io\/interpretable-ml-book\/."},{"key":"ref_5","unstructured":"Lundberg, S., and Lee, S. (2017, January 4\u20139). A unified approach to interpreting model predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: A gradient boosting machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann. Statist."},{"key":"ref_7","unstructured":"Shrikumar, A., Greenside, P., and Kundaje, A. (2017, January 6\u201311). Learning important features through propagating activation differences. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_8","first-page":"63","article-title":"Layer-Wise Relevance Propagation for Neural Networks with Local Renormalization Layers","volume":"Volume 9887","author":"Villa","year":"2016","journal-title":"Artificial Neural Networks and Machine Learning\u2014ICANN"},{"key":"ref_9","first-page":"841","article-title":"Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR","volume":"31","author":"Wachter","year":"2018","journal-title":"Harv. J. Law Technol."},{"key":"ref_10","first-page":"267","article-title":"The (Un)reliability of Saliency Methods","volume":"Volume 11700","author":"Samek","year":"2019","journal-title":"Explainable AI: Interpreting, Explaining and Visualizing Deep Learning"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). Why Should I Trust You?. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1111\/rssb.12377","article-title":"Visualizing the effects of predictor variables in black box supervised learning models","volume":"82","author":"Apley","year":"2020","journal-title":"J. R. Stat. Soc."},{"key":"ref_13","unstructured":"Doshi-Velez, F., and Kortz, M. (2022, June 06). Accountability of AI under the Law: The Role of Explanation, Berkman Klein Center Working Group on Explanation and the Law, Berkman Klein Center for Internet & Society Working Paper. Available online: http:\/\/nrs.harvard.edu\/urn-3:HUL.InstRepos:34372584."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3387166","article-title":"A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems","volume":"11","author":"Mohseni","year":"2021","journal-title":"ACM Trans. Interact. Intell. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhou, J., Gandomi, A.H., Chen, F., and Holzinger, A. (2021). Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics. Electronics, 10.","DOI":"10.3390\/electronics10050593"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3236009","article-title":"A Survey of Methods for Explaining Black Box Models","volume":"51","author":"Guidotti","year":"2019","journal-title":"ACM Comput. Surv."},{"key":"ref_18","unstructured":"Nguyen, A., and Mart\u00ednez, M.R. (2022, June 06). On Quantitative Aspects of Model Interpretability. Available online: http:\/\/arxiv.org\/pdf\/2007.07584v1."},{"key":"ref_19","first-page":"1","article-title":"Statistical stability indices for LIME: Obtaining reliable explanations for machine learning models","volume":"12","author":"Visani","year":"2021","journal-title":"J. Oper. Res. Soc."},{"key":"ref_20","unstructured":"(2022, April 26). Outcome-Oriented Predictive Process Monitoring Benchmark- Github. Available online: https:\/\/github.com\/irhete\/predictive-monitoring-benchmark."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Elkhawaga, G., Abuelkheir, M., and Reichert, M. (2022). Explainability of Predictive Process Monitoring Results: Can You See My Data Issues?. arXiv.","DOI":"10.3390\/app12168192"},{"key":"ref_22","unstructured":"(2022, April 26). 4TU Centre for Research Data. Available online: https:\/\/data.4tu.nl\/Eindhoven_University_of_Technology."},{"key":"ref_23","first-page":"2825","article-title":"Scikit-learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_24","unstructured":"(2022, April 26). Alibi Explain. Available online: https:\/\/github.com\/SeldonIO\/alibi."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1007\/978-3-030-66498-5_10","article-title":"XNAP: Making LSTM-Based Next Activity Predictions Explainable by Using LRP","volume":"Volume 397","author":"Ortega","year":"2020","journal-title":"Business Process Management Workshops: International Publishing (Lecture Notes in Business Information Processing)"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Galanti, R., Coma-Puig, B., de Leoni, M., Carmona, J., and Navarin, N. (2020, January 4\u20139). Explainable Predictive Process Monitoring. Proceedings of the 2nd International Conference on Process Mining (ICPM), Padua, Italy.","DOI":"10.1109\/ICPM49681.2020.00012"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1007\/978-3-030-58638-6_9","article-title":"Explainability in Predictive Process Monitoring: When Understanding Helps Improving","volume":"Volume 392","author":"Fahland","year":"2020","journal-title":"Business Process Management Forum: Lecture Notes in Business Information Processing"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1002\/smr.2170","article-title":"Predicting process performance: A white-box approach based on process models","volume":"31","author":"Verenich","year":"2019","journal-title":"J. Softw. Evol. Proc."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1007\/978-3-030-58666-9_15","article-title":"Exploring Interpretable Predictive Models for Business Processes","volume":"Volume 12168","author":"Fahland","year":"2020","journal-title":"Business Process Management, LNCS"},{"key":"ref_30","unstructured":"Jain, S., and Wallace, B.C. (2019, January 3\u20135). Attention is not explanation. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wiegreffe, S., and Pinter, Y. (2019, January 3\u20137). Attention is not not Explanation. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China.","DOI":"10.18653\/v1\/D19-1002"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/6\/199\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:26:11Z","timestamp":1760138771000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/6\/199"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,8]]},"references-count":31,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["a15060199"],"URL":"https:\/\/doi.org\/10.3390\/a15060199","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,8]]}}}