{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T13:08:06Z","timestamp":1772716086871,"version":"3.50.1"},"reference-count":23,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2022,12,23]],"date-time":"2022-12-23T00:00:00Z","timestamp":1671753600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Project EXPLainable Artificial INtelligence systems for health and well-beING (EXPLAINING)","award":["PID2019-104829RA-I00\/MCIN\/AEI\/10.13039\/501100011033"],"award-info":[{"award-number":["PID2019-104829RA-I00\/MCIN\/AEI\/10.13039\/501100011033"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Recognizing facial expressions has been a persistent goal in the scientific community. Since the rise of artificial intelligence, convolutional neural networks (CNN) have become popular to recognize facial expressions, as images can be directly used as input. Current CNN models can achieve high recognition rates, but they give no clue about their reasoning process. Explainable artificial intelligence (XAI) has been developed as a means to help to interpret the results obtained by machine learning models. When dealing with images, one of the most-used XAI techniques is LIME. LIME highlights the areas of the image that contribute to a classification. As an alternative to LIME, the CEM method appeared, providing explanations in a way that is natural for human classification: besides highlighting what is sufficient to justify a classification, it also identifies what should be absent to maintain it and to distinguish it from another classification. This study presents the results of comparing LIME and CEM applied over complex images such as facial expression images. While CEM could be used to explain the results on images described with a reduced number of features, LIME would be the method of choice when dealing with images described with a huge number of features.<\/jats:p>","DOI":"10.3390\/s23010131","type":"journal-article","created":{"date-parts":[[2022,12,23]],"date-time":"2022-12-23T03:55:21Z","timestamp":1671767721000},"page":"131","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Understanding How CNNs Recognize Facial Expressions: A Case Study with LIME and CEM"],"prefix":"10.3390","volume":"23","author":[{"given":"Guillermo","family":"del Castillo Torres","sequence":"first","affiliation":[{"name":"Department of Mathematics and Computer Science, University of the Balearic Islands, 07122 Palma, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6040-6270","authenticated-orcid":false,"given":"Maria Francesca","family":"Roig-Maim\u00f3","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Computer Science, University of the Balearic Islands, 07122 Palma, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9719-0183","authenticated-orcid":false,"given":"Miquel","family":"Mascar\u00f3-Oliver","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Computer Science, University of the Balearic Islands, 07122 Palma, Spain"}]},{"given":"Esperan\u00e7a","family":"Amengual-Alcover","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Computer Science, University of the Balearic Islands, 07122 Palma, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7559-5163","authenticated-orcid":false,"given":"Ramon","family":"Mas-Sans\u00f3","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Computer Science, University of the Balearic Islands, 07122 Palma, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Shortliffe, E.H. A Rule-Based Computer Program for Advising Physicians Regarding Antimicrobial Therapy Selection. Proceedings of the 1974 Annual ACM Conference\u2014Volume 2.","DOI":"10.1145\/1408800.1408906"},{"key":"ref_2","unstructured":"Clancey, W.J. (1987). Knowledge-Based Tutoring: The GUIDON Program, MIT Press."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Brown, J.S., and Burton, R.R. SOPHIE: A Pragmatic Use of Artificial Intelligence in CAI. Proceedings of the 1974 Annual ACM Conference\u2014Volume 2.","DOI":"10.1145\/1408800.1408855"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1016\/S0020-7373(88)80012-9","article-title":"Protos: An exemplar-based learning apprentice","volume":"29","author":"Bareiss","year":"1988","journal-title":"Int. J. Man-Mach. Stud."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Minsky, M., Kurzweil, R., and Mann, S. (2013, January 27\u201329). The society of intelligent veillance. Proceedings of the 2013 IEEE International Symposium on Technology and Society (ISTAS): Social Implications of Wearable Computing and Augmediated Reality in Everyday Life, Toronto, ON, Canada.","DOI":"10.1109\/ISTAS.2013.6613095"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Bach, S., Binder, A., Montavon, G., Klauschen, F., M\u00fcller, K.R., and Samek, W. (2015). On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0130140"},{"key":"ref_7","unstructured":"Lundberg, S.M., and Lee, S.I. (2017, January 4\u20139). A Unified Approach to Interpreting Model Predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1340","DOI":"10.1093\/bioinformatics\/btq134","article-title":"Permutation importance: A corrected feature importance measure","volume":"26","author":"Altmann","year":"2010","journal-title":"Bioinformatics"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1111\/rssb.12377","article-title":"Visualizing the effects of predictor variables in black box supervised learning models","volume":"82","author":"Apley","year":"2020","journal-title":"J. R. Stat. Soc. Ser. B Stat. Methodol."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Yang, C., Rangarajan, A., and Ranka, S. (2018, January 28\u201330). Global Model Interpretation Via Recursive Partitioning. Proceedings of the 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC\/SmartCity\/DSS), Exeter, UK.","DOI":"10.1109\/HPCC\/SmartCity\/DSS.2018.00256"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). \u201cWhy Should I Trust You?\u201d: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_12","unstructured":"Dhurandhar, A., Chen, P.Y., Luss, R., Tu, C.C., Ting, P., Shanmugam, K., and Das, P. (2018, January 2\u20138). Explanations Based on the Missing: Towards Contrastive Explanations with Pertinent Negatives. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montr\u00e9al, QC, Canada."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2018, January 2\u20133). Anchors: High-Precision Model-Agnostic Explanations. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11491"},{"key":"ref_14","unstructured":"Darias, J.M., D\u00edaz-Agudo, B., and Recio-Garc\u00eda, J.A. (2021, January 13\u201316). A Systematic Review on Model-agnostic XAI Libraries. Proceedings of the ICCBR Workshops, Salamanca, Spain."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Lim, C.P., Chen, Y.W., Vaidya, A., Mahorkar, C., and Jain, L.C. (2022). Advances in XAI: Explanation Interfaces in Healthcare. Handbook of Artificial Intelligence in Healthcare: Volume 2: Practicalities and Prospects, Springer International Publishing.","DOI":"10.1007\/978-3-030-83620-7"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Luss, R., Chen, P.Y., Dhurandhar, A., Sattigeri, P., Zhang, Y., Shanmugam, K., and Tu, C.C. (2021, January 14\u201318). Leveraging Latent Features for Local Explanations. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Singapore.","DOI":"10.1145\/3447548.3467265"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"103404","DOI":"10.1016\/j.artint.2020.103404","article-title":"Evaluating XAI: A comparison of rule-based and example-based explanations","volume":"291","author":"Nieuwburg","year":"2021","journal-title":"Artif. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mascar\u00f3 Oliver, M., and Amengual Alcover, E. (2020). UIBVFED: Virtual facial expression dataset. PLoS ONE, 15.","DOI":"10.1371\/journal.pone.0231266"},{"key":"ref_19","unstructured":"Faigin, G. (2012). The Artist\u2019s Complete Guide to Facial Expression, Watson-Guptill."},{"key":"ref_20","first-page":"5","article-title":"Facial Action Coding System: A Technique for the Measurement of Facial Movement","volume":"3","author":"Ekman","year":"1978","journal-title":"Palo Alto"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Colbois, L., Freitas Pereira, T.d., and Marcel, S. (2021, January 4\u20137). On the use of automatically generated synthetic image datasets for benchmarking face recognition. Proceedings of the 2021 IEEE International Joint Conference on Biometrics (IJCB), Shenzhen, China.","DOI":"10.1109\/IJCB52358.2021.9484363"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"675515","DOI":"10.3389\/fpsyg.2021.675515","article-title":"How Interpersonal Distance Between Avatar and Human Influences Facial Affect Recognition in Immersive Virtual Reality","volume":"12","year":"2021","journal-title":"Front. Psychol."},{"key":"ref_23","unstructured":"LeCun, Y. (2022, October 17). The MNIST Database of Handwritten Digits. Available online: http:\/\/yann.lecun.com\/exdb\/mnist\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/1\/131\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:49:01Z","timestamp":1760147341000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/1\/131"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,23]]},"references-count":23,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["s23010131"],"URL":"https:\/\/doi.org\/10.3390\/s23010131","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,23]]}}}