{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T14:24:17Z","timestamp":1760711057225,"version":"build-2065373602"},"reference-count":61,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,10,3]],"date-time":"2022-10-03T00:00:00Z","timestamp":1664755200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Classifying facial expressions is a vital part of developing systems capable of aptly interacting with users. In this field, the use of deep-learning models has become the standard. However, the inner workings of these models are unintelligible, which is an important issue when deploying them to high-stakes environments. Recent efforts to generate explanations for emotion classification systems have been focused on this type of models. In this work, an alternative way of explaining the decisions of a more conventional model based on geometric features is presented. We develop a geometric-features-based deep neural network (DNN) and a convolutional neural network (CNN). Ensuring a sufficient level of predictive accuracy, we analyze explainability using both objective quantitative criteria and a user study. Results indicate that the fidelity and accuracy scores of the explanations approximate the DNN well. From the performed user study, it becomes clear that the explanations increase the understanding of the DNN and that they are preferred over the explanations for the CNN, which are more commonly used. All scripts used in the study are publicly available.<\/jats:p>","DOI":"10.3390\/a15100367","type":"journal-article","created":{"date-parts":[[2022,10,7]],"date-time":"2022-10-07T22:47:27Z","timestamp":1665182847000},"page":"367","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Comparing Approaches for Explaining DNN-Based Facial Expression Classifications"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4630-4897","authenticated-orcid":false,"given":"Kaya","family":"ter Burg","sequence":"first","affiliation":[{"name":"Informatics Institute, University of Amsterdam, 1090 GE Amsterdam, The Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7947-5508","authenticated-orcid":false,"given":"Heysem","family":"Kaya","sequence":"additional","affiliation":[{"name":"Department of Information and Computing Sciences, Utrecht University, 3584 CC Utrecht, The Netherlands"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Tao, J., and Tan, T. (2005, January 22\u201324). Affective computing: A review. Proceedings of the International Conference on Affective computing and Intelligent Interaction, Beijing, China.","DOI":"10.1007\/11573548"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ko, B.C. (2018). A brief review of facial emotion recognition based on visual information. Sensors, 18.","DOI":"10.3390\/s18020401"},{"key":"ref_3","first-page":"040901","article-title":"Development of convolutional neural network and its application in image classification: A survey","volume":"58","author":"Wang","year":"2019","journal-title":"Opt. Eng."},{"key":"ref_4","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1350","DOI":"10.1214\/15-AOAS848","article-title":"Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model","volume":"9","author":"Letham","year":"2015","journal-title":"Ann. Appl. Stat."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Weitz, K., Schiller, D., Schlagowski, R., Huber, T., and Andr\u00e9, E. (2019, January 2\u20135). \u2018Do you trust me?\u2019 Increasing user-trust by integrating virtual agents in explainable AI interaction design. Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents, Paris, France.","DOI":"10.1145\/3308532.3329441"},{"key":"ref_8","unstructured":"Hoffman, R.R., Mueller, S.T., Klein, G., and Litman, J. (2018). Metrics for explainable AI: Challenges and prospects. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"52138","DOI":"10.1109\/ACCESS.2018.2870052","article-title":"Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)","volume":"6","author":"Adadi","year":"2018","journal-title":"IEEE Access"},{"key":"ref_10","unstructured":"Weitz, K., Hassan, T., Schmid, U., and Garbas, J. (2018). Towards explaining deep learning networks to distinguish facial expressions of pain and emotions. Forum Bildverarbeitung, Institut f\u00fcr Industrielle Informationstechnik (IIIT)."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Gund, M., Bharadwaj, A.R., and Nwogu, I. (2021, January 10\u201315). Interpretable Emotion Classification Using Temporal Convolutional Models. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.","DOI":"10.1109\/ICPR48806.2021.9412134"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Escalante, H.J., Guyon, I., Escalera, S., Jacques, J., Madadi, M., Bar\u00f3, X., Ayache, S., Viegas, E., G\u00fc\u00e7l\u00fct\u00fcrk, Y., and G\u00fc\u00e7l\u00fc, U. (2017, January 14\u201319). Design of an explainable machine learning challenge for video interviews. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.","DOI":"10.1109\/IJCNN.2017.7966320"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"894","DOI":"10.1109\/TAFFC.2020.2973984","article-title":"Modeling, Recognizing, and Explaining Apparent Personality from Videos","volume":"13","author":"Escalante","year":"2022","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kaya, H., G\u00fcrpinar, F., Afshar, S., and Salah, A.A. (2015, January 9\u201313). Contrasting and combining least squares based learners for emotion recognition in the wild. Proceedings of the 2015 ACM International Conference on Multimodal Interaction, Seattle, WA, USA.","DOI":"10.1145\/2818346.2830588"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Dresvyanskiy, D., Ryumina, E., Kaya, H., Markitantov, M., Karpov, A., and Minker, W. (2022). End-to-End Modeling and Transfer Learning for Audiovisual Emotion Recognition in-the-Wild. Multimodal Technol. Interact., 6.","DOI":"10.3390\/mti6020011"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1093\/idpl\/ipx022","article-title":"Meaningful information and the right to explanation","volume":"7","author":"Selbst","year":"2017","journal-title":"Int. Data Priv. Law"},{"key":"ref_17","unstructured":"Doshi-Velez, F., and Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3236009","article-title":"A survey of methods for explaining black box models","volume":"51","author":"Guidotti","year":"2018","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Jacovi, A., and Goldberg, Y. (2020). Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), Association for Computational Linguistics.","DOI":"10.18653\/v1\/2020.acl-main.386"},{"key":"ref_21","unstructured":"Nori, H., Jenkins, S., Koch, P., and Caruana, R. (2019). Interpretml: A unified framework for machine learning interpretability. arXiv."},{"key":"ref_22","unstructured":"Lundberg, S.M., and Lee, S.I. (2017, January 4\u20139). A unified approach to interpreting model predictions. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_23","unstructured":"Kuhn, H.W. (1997). A value for n-person games. Class. Game Theory, Princeton University Press."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). \u201cWhy should I trust you?\u201d Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pone.0130140","article-title":"On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation","volume":"10","author":"Bach","year":"2015","journal-title":"PLoS ONE"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ventura, C., Masip, D., and Lapedriza, A. (2017, January 21\u201326). Interpreting CNN models for apparent personality trait regression. Proceedings of the CVPR Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.217"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., and Zurada, J.M. (2020). Explaining Machine Learning Models of Emotion Using the BIRAFFE Dataset. Artificial Intelligence and Soft Computing, Springer International Publishing.","DOI":"10.1007\/978-3-030-61534-5"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"61513","DOI":"10.1109\/ACCESS.2021.3072120","article-title":"Emotion Recognition Using Explainable Genetically Optimized Fuzzy ART Ensembles","volume":"9","author":"Liew","year":"2021","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Prajod, P., Schiller, D., Huber, T., and Andr\u00e9, E. (2021). Do Deep Neural Networks Forget Facial Action Units?\u2013Exploring the Effects of Transfer Learning in Health Related Facial Expression Recognition. arXiv.","DOI":"10.1007\/978-3-030-93080-6_16"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Xiong, X., and De la Torre, F. (2013, January 23\u201328). Supervised Descent Method and Its Application to Face Alignment. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.75"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Cao, Q., Shen, L., Xie, W., Parkhi, O.M., and Zisserman, A. (2018, January 15\u201319). VGGFace2: A Dataset for Recognising Faces across Pose and Age. Proceedings of the 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00020"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1007\/s11263-019-01158-4","article-title":"Deep affect prediction in-the-wild: Aff-wild database and challenge, deep architectures, and beyond","volume":"127","author":"Kollias","year":"2019","journal-title":"Int. J. Comput. Vis."},{"key":"ref_34","unstructured":"Korobov, M., and Lopuhin, K. (2022, May 01). ELI5. Available online: https:\/\/eli5.readthedocs.io\/en\/latest\/."},{"key":"ref_35","first-page":"55","article-title":"A technique for the measurement of attitudes","volume":"140","author":"Likert","year":"1932","journal-title":"Arch. Psychol."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Lundqvist, D., Flykt, A., and \u00d6hman, A. (1998). The Karolinska Directed Emotional Faces-KDEF, Karolinska Institutet. CD ROM from Department of Clinical Neuroscience, Psychology section.","DOI":"10.1037\/t27732-000"},{"key":"ref_37","first-page":"16","article-title":"Basic emotions","volume":"98","author":"Ekman","year":"1999","journal-title":"Handb. Cogn. Emot."},{"key":"ref_38","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference for Learning Representations (ICLR), San Diego, CA, USA."},{"key":"ref_39","unstructured":"Chollet, F. (2022, May 01). Keras. Available online: https:\/\/keras.io."},{"key":"ref_40","first-page":"6765","article-title":"Hyperband: A novel bandit-based approach to hyperparameter optimization","volume":"18","author":"Li","year":"2017","journal-title":"J. Mach. Learn. Res."},{"key":"ref_41","first-page":"281","article-title":"Random search for hyper-parameter optimization","volume":"13","author":"Bergstra","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_42","unstructured":"Ng, A.Y. (2004, January 4\u20138). Feature selection, L 1 vs. L 2 regularization, and rotational invariance. Proceedings of the Twenty-First International Conference on Machine Learning, Banff, AB, Canada."},{"key":"ref_43","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1023\/A:1012487302797","article-title":"Gene selection for cancer classification using support vector machines","volume":"46","author":"Guyon","year":"2002","journal-title":"Mach. Learn."},{"key":"ref_45","first-page":"2825","article-title":"Scikit-learn: Machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Deng, J., Guo, J., Ververas, E., Kotsia, I., and Zafeiriou, S. (2020, January 13\u201319). RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00525"},{"key":"ref_47","unstructured":"Zheng, E. (2022, May 01). Batch Face. Available online: https:\/\/github.com\/elliottzheng\/batch-face."},{"key":"ref_48","first-page":"6","article-title":"SUS: A quick and dirty usability scale","volume":"189","author":"Brooke","year":"1995","journal-title":"Usability Eval. Ind."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Holzinger, A., Carrington, A., and M\u00fcller, H. (2020). Measuring the quality of explanations: The system causability scale (SCS). KI-K\u00fcnstliche Intelligenz, Springer.","DOI":"10.1007\/s13218-020-00636-z"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1217","DOI":"10.1111\/j.1365-2929.2004.02012.x","article-title":"Likert scales: How to (ab)use them?","volume":"38","author":"Jamieson","year":"2004","journal-title":"Med Educ."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1007\/s10459-010-9222-y","article-title":"Likert scales, levels of measurement and the \u201claws\u201d of statistics","volume":"15","author":"Norman","year":"2010","journal-title":"Adv. Health Sci. Educ."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"80","DOI":"10.2307\/3001968","article-title":"Individual Comparisons by Ranking Methods","volume":"1","author":"Wilcoxon","year":"1945","journal-title":"Biom. Bull."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1214\/aoms\/1177730491","article-title":"On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other","volume":"18","author":"Mann","year":"1947","journal-title":"Ann. Math. Stat."},{"key":"ref_54","unstructured":"Burg, K.t. (2021). Explaining DNN Based Facial Expression Classifications. [BSc Thesis, Utrecht University]."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Davis, B., Glenski, M., Sealy, W., and Arendt, D. (2020, January 25). Measure Utility, Gain Trust: Practical Advice for XAI Researchers. Proceedings of the 2020 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX), Salt Lake City, Utah, USA.","DOI":"10.1109\/TREX51495.2020.00005"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1038\/s42256-020-00280-0","article-title":"Estimation of continuous valence and arousal levels from faces in naturalistic conditions","volume":"3","author":"Toisoul","year":"2021","journal-title":"Nat. Mach. Intell."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Mahmud, F., Islam, B., Hossain, A., and Goala, P.B. (2018, January 27\u201328). Facial region segmentation based emotion recognition using K-nearest neighbors. Proceedings of the International Conference on Innovation in Engineering and Technology (ICIET), Dhaka, Bangladesh.","DOI":"10.1109\/CIET.2018.8660900"},{"key":"ref_58","unstructured":"Del Bimbo, A., Cucchiara, R., Sclaroff, S., Farinella, G.M., Mei, T., Bertini, M., Escalante, H.J., and Vezzani, R. (2021). Explainable Model Selection of a Convolutional Neural Network for Driver\u2019s Facial Emotion Identification. ICPR Workshops and Challenges, Springer."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Puthanidam, R.V., and Moh, T.S. (2018, January 5\u20137). A Hybrid approach for facial expression recognition. Proceedings of the 12th International Conference on Ubiquitous Information Management and Communication, Langkawi, Malaysia.","DOI":"10.1145\/3164541.3164593"},{"key":"ref_60","first-page":"I","article-title":"Rapid object detection using a boosted cascade of simple features","volume":"Volume 1","author":"Viola","year":"2001","journal-title":"Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition"},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Yu, Z., and Zhang, C. (2015, January 9\u201313). Image based static facial expression recognition with multiple deep network learning. Proceedings of the 2015 ACM International Conference on International Conference on Multimodal Interaction, Seattle, WA, USA.","DOI":"10.1145\/2818346.2830595"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/10\/367\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:45:52Z","timestamp":1760143552000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/10\/367"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,3]]},"references-count":61,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["a15100367"],"URL":"https:\/\/doi.org\/10.3390\/a15100367","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2022,10,3]]}}}