{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T04:28:44Z","timestamp":1785472124297,"version":"3.56.0"},"reference-count":39,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,12,22]],"date-time":"2021-12-22T00:00:00Z","timestamp":1640131200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Can we really \u201cread the mind in the eyes\u201d? Moreover, can AI assist us in this task? This paper answers these two questions by introducing a machine learning system that predicts personality characteristics of individuals on the basis of their face. It does so by tracking the emotional response of the individual\u2019s face through facial emotion recognition (FER) while watching a series of 15 short videos of different genres. To calibrate the system, we invited 85 people to watch the videos, while their emotional responses were analyzed through their facial expression. At the same time, these individuals also took four well-validated surveys of personality characteristics and moral values: the revised NEO FFI personality inventory, the Haidt moral foundations test, the Schwartz personal value system, and the domain-specific risk-taking scale (DOSPERT). We found that personality characteristics and moral values of an individual can be predicted through their emotional response to the videos as shown in their face, with an accuracy of up to 86% using gradient-boosted trees. We also found that different personality characteristics are better predicted by different videos, in other words, there is no single video that will provide accurate predictions for all personality characteristics, but it is the response to the mix of different videos that allows for accurate prediction.<\/jats:p>","DOI":"10.3390\/fi14010005","type":"journal-article","created":{"date-parts":[[2021,12,23]],"date-time":"2021-12-23T02:01:19Z","timestamp":1640224879000},"page":"5","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Your Face Mirrors Your Deepest Beliefs\u2014Predicting Personality and Morals through Facial Emotion Recognition"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7271-3224","authenticated-orcid":false,"given":"Peter A.","family":"Gloor","sequence":"first","affiliation":[{"name":"MIT Center for Collective Intelligence, Cambridge, MA 02142, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5348-9722","authenticated-orcid":false,"given":"Andrea","family":"Fronzetti Colladon","sequence":"additional","affiliation":[{"name":"Department of Engineering, University of Perugia, 06123 Perugia, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Erkin","family":"Altuntas","sequence":"additional","affiliation":[{"name":"Galaxyadvisors AG, 5000 Aarau, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cengiz","family":"Cetinkaya","sequence":"additional","affiliation":[{"name":"Department of Data Science, Lucerne University of Applied Sciences and Arts, 6002 Lucerne, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9838-4414","authenticated-orcid":false,"given":"Maximilian F.","family":"Kaiser","sequence":"additional","affiliation":[{"name":"Department of Information Systems, University of Cologne, 50923 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lukas","family":"Ripperger","sequence":"additional","affiliation":[{"name":"Department of Information Systems, University of Cologne, 50923 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0490-8636","authenticated-orcid":false,"given":"Tim","family":"Schaefer","sequence":"additional","affiliation":[{"name":"Department of Information Systems, University of Cologne, 50923 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1111\/1469-7610.00715","article-title":"The \u201creading the mind in the eyes\u201d test revised version: A study with normal adults, and adults with asperger syndrome or high-functioning autism","volume":"42","author":"Wheelwright","year":"2001","journal-title":"J. Child Psychol. Psychiatry"},{"key":"ref_2","unstructured":"Barrett, L.F. (2017). How Emotions Are Made: The Secret Life of the Brain, Mariner Books."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1036","DOI":"10.1073\/pnas.1418680112","article-title":"Computer-based personality judgments are more accurate than those made by humans","volume":"112","author":"Youyou","year":"2015","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_4","unstructured":"Hjortsj\u00f6, C.-H. (1969). Man\u2019s Face and Mimic Language, Studentlitteratur."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Biel, J.-I., Teijeiro-Mosquera, L., and Gatica-Perez, D. (2012, January 22\u201326). FaceTube. Proceedings of the 14th ACM International Conference on Multimodal Interaction\u2014ICMI \u201912, Santa Monica, CA, USA.","DOI":"10.1145\/2388676.2388689"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ko, B. (2018). A brief review of facial emotion recognition based on visual information. Sensors, 18.","DOI":"10.3390\/s18020401"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"R\u00f6\u00dfler, J., Sun, J., and Gloor, P. (2021). Reducing videoconferencing fatigue through facial emotion recognition. Future Internet, 13.","DOI":"10.3390\/fi13050126"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Costa, P.T., and McCrae, R.R. (2008). The revised NEO personality inventory (NEO-PI-R). The SAGE Handbook of Personality Theory and Assessment: Volume 2\u2014Personality Measurement and Testing, SAGE Publications.","DOI":"10.4135\/9781849200479.n9"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/B978-0-12-407236-7.00002-4","article-title":"Moral foundations theory","volume":"47","author":"Graham","year":"2013","journal-title":"Adv. Exp. Soc. Psychol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1037\/0022-3514.53.3.550","article-title":"Toward a universal psychological structure of human values","volume":"53","author":"Schwartz","year":"1987","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1017\/S1930297500000334","article-title":"A domain-specific risk-taking (DOSPERT) scale for adult populations","volume":"1","author":"Blais","year":"2006","journal-title":"Judgm. Decis. Mak."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1037\/0022-3514.61.4.598","article-title":"Moral affect: The good, the bad, and the ugly","volume":"61","author":"Tangney","year":"1991","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1146\/annurev.psych.56.091103.070145","article-title":"Moral emotions and moral behavior","volume":"58","author":"Tangney","year":"2007","journal-title":"Annu. Rev. Psychol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1080\/13869790500492466","article-title":"The emotional basis of moral judgments","volume":"9","author":"Prinz","year":"2006","journal-title":"Philos. Explor."},{"key":"ref_15","first-page":"173","article-title":"What do two men kissing and a bucket of maggots have in common? Heterosexual men\u2019s indistinguishable salivary \u03b1-amylase responses to photos of two men kissing and disgusting images","volume":"8","author":"Blair","year":"2017","journal-title":"Psychol. Sex."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1348\/014466606X156546","article-title":"Disgust is a factor in extreme prejudice","volume":"46","author":"Taylor","year":"2007","journal-title":"Br. J. Soc. Psychol."},{"key":"ref_17","unstructured":"Lavater, J.C. (1775). Physiognomische Fragmente, zur Bef\u00f6rderung der Menschenkenntni\u00df und Menschenliebe, Weidmann and Reich."},{"key":"ref_18","first-page":"132","article-title":"Composite portraits, made by combining those of many different persons into a single resultant figure","volume":"8","author":"Galton","year":"1879","journal-title":"J. Anthropol. Inst. G. B. Irel."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Lombroso Ferrero, G. (1911). Criminal Man, According to the Classification of Cesare Lombroso, G P Putnam\u2019s Sons.","DOI":"10.1037\/10912-000"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1111\/bjop.12035","article-title":"Increased facial width-to-height ratio and perceived dominance in the faces of the UK\u2019s leading business leaders","volume":"105","author":"Alrajih","year":"2014","journal-title":"Br. J. Psychol."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Haselhuhn, M.P., Ormiston, M.E., and Wong, E.M. (2015). Men\u2019s facial width-to-height ratio predicts aggression: A meta-analysis. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0122637"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"20130049","DOI":"10.1098\/rsbl.2013.0049","article-title":"Facial morphology predicts male fitness and rank but not survival in second world war finnish soldiers","volume":"9","author":"Loehr","year":"2013","journal-title":"Biol. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"214","DOI":"10.3389\/fpsyg.2018.00214","article-title":"Group facial width-to-height ratio predicts intergroup negotiation outcomes","volume":"9","author":"Yang","year":"2018","journal-title":"Front. Psychol."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Agapito, L., Bronstein, M., and Rother, C. (2015). ChaLearn looking at people challenge 2014: Dataset and results. Computer Vision-ECCV 2014 Workshops, Springer.","DOI":"10.1007\/978-3-319-16199-0"},{"key":"ref_25","unstructured":"Hua, G., and J\u00e9gou, H. (2016). ChaLearn LAP 2016: First round challenge on first impressions-dataset and results. Computer Vision-ECCV 2016 Workshops, Springer."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1109\/TAFFC.2017.2762299","article-title":"Deep bimodal regression of apparent personality traits from short video sequences","volume":"9","author":"Wei","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2702","DOI":"10.1109\/TNSM.2020.3018303","article-title":"Estimation of the quality of experience during video streaming from facial expression and gaze direction","volume":"17","author":"Porcu","year":"2020","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Amour, L., Boulabiar, M.I., Souihi, S., and Mellouk, A. (2018, January 24\u201326). An improved QoE estimation method based on QoS and affective computing. Proceedings of the 2018 International Symposium on Programming and Systems (ISPS), Algiers, Algeria.","DOI":"10.1109\/ISPS.2018.8379009"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1186\/2192-1962-2-7","article-title":"Quality of experience evaluation of voice communication: An affect-based approach","volume":"2","author":"Bhattacharya","year":"2012","journal-title":"Hum.-Centric Comput. Inf. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1037\/0003-066X.48.4.384","article-title":"Facial expression and emotion","volume":"48","author":"Ekman","year":"1993","journal-title":"Am. Psychol."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1037\/h0030377","article-title":"Constants across cultures in the face and emotion","volume":"17","author":"Ekman","year":"1971","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0065-2601(08)60281-6","article-title":"Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries","volume":"25","author":"Schwartz","year":"1992","journal-title":"Adv. Exp. Soc. Psychol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1093\/poq\/nfn035","article-title":"Bringing values back in: The adequacy of the European social survey to measure values in 20 countries","volume":"72","author":"Davidov","year":"2008","journal-title":"Public Opin. Q."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: Reliable large-scale tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1613\/jair.953","article-title":"SMOTE: Synthetic minority over-sampling technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J. Artif. Intell. Res."},{"key":"ref_37","unstructured":"He, H., Bai, Y., Garcia, E.A., and Li, S. (2008, January 1\u20138). ADASYN: Adaptive synthetic sampling approach for imbalanced learning. Proceedings of the 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), Hong Kong, China."},{"key":"ref_38","unstructured":"Lundberg, S.M., Erion, G.G., and Lee, S.I. (2021, December 21). Consistent Feature Attribution for Tree Ensembles. Available online: http:\/\/arxiv.org\/abs\/1802.03888."},{"key":"ref_39","unstructured":"Lundberg, S.M., and Lee, S.I. (2017, January 4\u20139). A unified approach to interpreting model predictions. Proceedings of the 31st Conference on Neural Information Processing System, Long Beach, CA, USA."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/1\/5\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:51:35Z","timestamp":1760169095000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/1\/5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,22]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["fi14010005"],"URL":"https:\/\/doi.org\/10.3390\/fi14010005","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,22]]}}}