{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T08:27:23Z","timestamp":1767860843314,"version":"3.49.0"},"reference-count":44,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2020,6,4]],"date-time":"2020-06-04T00:00:00Z","timestamp":1591228800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Facial thermal imaging is a non-contact technology which can be useful for ubiquitous deceptive anxiety recognition. To date, studies investigating this technology have produced equivocal results in classification accuracy and finding the most correlated regions on the face. This study was conducted using our dataset with 41 subjects using two different protocols and three modalities (thermal, GSR and PPG). We selected and tracked five regions of interest (ROI) on each facial thermal imprint including periorbital, forehead, cheek, perinasal and chin that were mostly used in previous papers. By employing six statistical features, four feature reduction techniques and three classifiers, we attempted to identify the ROIs which are mostly associated with activation of the sympathetic nervous system to increase the final classification accuracy rate. The results of linear classification models show significant improvement of classification accuracy by using ROC feature selection method. We achieved 90.1% and 74.7% accuracy rate for thermal features in mock crime and best friend scenarios, respectively. Our experimental results show that perinasal and cheek areas have greater discriminatory power in comparison with other ROIs on the face.<\/jats:p>","DOI":"10.3390\/mti4020025","type":"journal-article","created":{"date-parts":[[2020,6,4]],"date-time":"2020-06-04T04:36:09Z","timestamp":1591245369000},"page":"25","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Identifying the Optimal Features in Multimodal Deception Detection"],"prefix":"10.3390","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1078-6516","authenticated-orcid":false,"given":"Amin","family":"Derakhshan","sequence":"first","affiliation":[{"name":"Biomedical Engineering Department, Imam Reza International University, Mashhad 9138833186, Iran"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7245-885X","authenticated-orcid":false,"given":"Mohammad","family":"Mikaeili","sequence":"additional","affiliation":[{"name":"Biomedical Engineering Department, Shahed University, Tehran 3319118651, Iran"}]},{"given":"Tom","family":"Gedeon","sequence":"additional","affiliation":[{"name":"Human Centred Computing Research Group, Australian National University, Canberra 0200, Australia"}]},{"given":"Ali Motie","family":"Nasrabadi","sequence":"additional","affiliation":[{"name":"Biomedical Engineering Department, Shahed University, Tehran 3319118651, Iran"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,4]]},"reference":[{"key":"ref_1","unstructured":"Vrij, A. (2000). Detecting Lies and Deceit: The Psychology of Lying and Implications for Professional Practice, Wiley."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"913","DOI":"10.1037\/0003-066X.46.9.913","article-title":"Who can catch a liar?","volume":"46","author":"Ekman","year":"1991","journal-title":"Am. Psychol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1348\/1355325041719356","article-title":"Why professionals fail to catch liars and how they can improve","volume":"9","author":"Vrij","year":"2004","journal-title":"Leg. Criminol. Psychol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1089\/107555301753393841","article-title":"The physiology of mind\u2013body interactions: The stress response and the relaxation response","volume":"7","author":"Jacobs","year":"2001","journal-title":"J. Altern. Complementary Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1418","DOI":"10.1109\/TBME.2007.891930","article-title":"Contact-free measurement of cardiac pulse based on the analysis of thermal imagery","volume":"54","author":"Garbey","year":"2007","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1105","DOI":"10.1109\/TBME.2018.2866878","article-title":"Noncontact monitoring of respiratory rate in newborn infants using thermal imaging","volume":"66","author":"Pereira","year":"2018","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.ijpsycho.2014.06.011","article-title":"A non-contact technique for measuring eccrine sweat gland activity using passive thermal imaging","volume":"94","author":"Krzywicki","year":"2014","journal-title":"Int. J. Psychophysiol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/j.biopsycho.2011.09.018","article-title":"Mother and child in synchrony: Thermal facial imprints of autonomic contagion","volume":"89","author":"Ebisch","year":"2012","journal-title":"Biol. Psychol."},{"key":"ref_9","first-page":"I","article-title":"Estimation of blood flow speed and vessel location from thermal video","volume":"Volume 1","author":"Marc","year":"2004","journal-title":"Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition"},{"key":"ref_10","unstructured":"Pavlidis, I., and Levine, J. (2001, January 25\u201328). Monitoring of periorbital blood flow rate through thermal image analysis and its application to polygraph testing. Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Istanbul, Turkey."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1109\/TBME.2008.2003265","article-title":"Imaging facial signs of neurophysiological responses","volume":"56","author":"Shastri","year":"2008","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.cviu.2006.11.018","article-title":"Interacting with human physiology","volume":"108","author":"Pavlidis","year":"2007","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1586","DOI":"10.3389\/fpsyg.2015.01586","article-title":"Behavioral and facial thermal variations in 3-to 4-month-old infants during the Still-Face Paradigm","volume":"6","author":"Aureli","year":"2015","journal-title":"Front. Psychol."},{"key":"ref_14","first-page":"284","article-title":"Emotion detection through functional infrared imaging: Preliminary results","volume":"48","author":"Merla","year":"2004","journal-title":"Biomed. Tech."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Merla, A., and Romani, G. (2007, January 22\u201326). Thermal signatures of emotional arousal: A functional infrared imaging study. Proceedings of the 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Lyon, France.","DOI":"10.1109\/IEMBS.2007.4352270"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1109\/MEMB.2002.1175139","article-title":"Thermal image analysis for polygraph testing","volume":"21","author":"Pavlidis","year":"2002","journal-title":"IEEE Eng. Med. Biol. Mag."},{"key":"ref_17","first-page":"4","article-title":"Hemifacial skin temperature changes related to deception: Blood flow or thermal capacitance?","volume":"6","author":"Pollina","year":"2015","journal-title":"Int. J. Glob. Res. Comput. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1111\/psyp.12243","article-title":"Thermal infrared imaging in psychophysiology: Potentialities and limits","volume":"51","author":"Ioannou","year":"2014","journal-title":"Psychophysiology"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Abouelenien, M., P\u00e9rez-Rosas, V., Mihalcea, R., and Burzo, M. (2014, January 12\u201316). Deception detection using a multimodal approach. Proceedings of the 16th International Conference on Multimodal Interaction (ICMI \u201914), Istanbul, Turkey.","DOI":"10.1145\/2663204.2663229"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1042","DOI":"10.1109\/TIFS.2016.2639344","article-title":"Detecting deceptive behavior via integration of discriminative features from multiple modalities","volume":"12","author":"Abouelenien","year":"2017","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_21","unstructured":"Derakhshan, A., Mikaeili, M., Khalilzadeh, M.A., and Mohammadian, A. (July, January 30). Preliminary study on facial thermal imaging for stress recognition. Proceedings of the 10th International Conference on Intelligent Environments, Shanghai, China."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Puri, C., Olson, L., Pavlidis, I., Levine, J., and Starren, J. (2005, January 2\u20137). Stresscam: Non-contact measurement of users\u2019 emotional states through thermal imaging. Proceedings of the 2005 ACM Conference on Human Factorsin Computing Systems (CHI), Portland, OR, USA.","DOI":"10.1145\/1056808.1057007"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1038\/srep00305","article-title":"Fast by nature-how stress patterns define human experience and performance in dexterous tasks","volume":"2","author":"Pavlidis","year":"2012","journal-title":"Sci. Rep."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1007\/s11263-006-6106-y","article-title":"Imaging facial physiology for the detection of deceit","volume":"71","author":"Tsiamyrtzis","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1182","DOI":"10.1007\/s10439-006-9143-3","article-title":"Facial skin surface temperature changes during a \u201cconcealed information\u201d test","volume":"34","author":"Pollina","year":"2006","journal-title":"Ann. Biomed. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1007\/s10979-010-9251-3","article-title":"Thermal imaging as a lie detection tool at airports","volume":"35","author":"Warmelink","year":"2011","journal-title":"Law Hum. Behav."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1015","DOI":"10.1109\/TIFS.2014.2317309","article-title":"Thermal facial analysis for deception detection","volume":"9","author":"Rajoub","year":"2014","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Dcosta, M. (2015, January 4\u20138). Perinasal indicators of deceptive behavior. Proceedings of the 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Ljubljana, Sloveniam.","DOI":"10.1109\/FG.2015.7163080"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zhu, Z., Tsiamyrtzis, P., and Pavlidis, I. (2007, January 22\u201326). Forehead thermal signature extraction in lie detection. Proceedings of the 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Lyon, France.","DOI":"10.1109\/IEMBS.2007.4352269"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"35174","DOI":"10.1038\/srep35174","article-title":"Thermal signatures of voluntary deception in ecological conditions","volume":"6","author":"Panasiti","year":"2016","journal-title":"Sci. Rep."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"014002","DOI":"10.1088\/1361-6579\/aaf089","article-title":"Network physiology of \u2018fight or flight\u2019response in facial superficial blood vessels","volume":"40","author":"Derakhshan","year":"2019","journal-title":"Physiol. Meas."},{"key":"ref_32","unstructured":"Raskin, D.C., and Honts, C.R. (2002). The Comparison Question Test. Handbook of Polygraph Testing, Academic Press."},{"key":"ref_33","unstructured":"Hossain, M.Z., Gedeon, T., and Sankaranarayana, R. (2016, January 5\u20137). Observer\u2019s galvanic skin response for discriminating real from fake smiles. Proceedings of the 27th Australasian Conference on Information Systems, Wollongong, Australia."},{"key":"ref_34","unstructured":"Lynch, P.J., and Jaffe, C.C. (2006). Medical Illustrations, Center for Advanced Instructional Media, Yale University School of Medicine."},{"key":"ref_35","unstructured":"Viola, P., and Jones, M. (2001, January 8\u201314). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA."},{"key":"ref_36","unstructured":"Tomasi, C., and Takeo, K. (1991). Detection and Tracking of Point Features, International Journal of Computer Vision."},{"key":"ref_37","unstructured":"Shi, J. (1994, January 21\u201323). Good features to track. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1016\/S0261-3794(99)00028-1","article-title":"Loess: A nonparametric, graphical tool for depicting relationships between variables","volume":"19","author":"Jacoby","year":"2000","journal-title":"Elect. Stud."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Hossain, M.Z., and Gedeon, T. (2017, January 23\u201326). Classifying posed and real smiles from observers\u2019 peripheral physiology. Proceedings of the 11th International Conference on Pervasive Computing Technologies for Healthcare, Barcelona, Spain.","DOI":"10.1145\/3154862.3154893"},{"key":"ref_40","first-page":"11","article-title":"Gene feature extraction using T-test statistics and Kernel partial least squares. Lecture notes in computer science","volume":"4234","author":"Li","year":"2006","journal-title":"Neural Inf. Process."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"3797","DOI":"10.1109\/TIT.2014.2320500","article-title":"R\u00e9nyi divergence and Kullback-Leibler divergence","volume":"60","author":"Harremos","year":"2014","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","article-title":"An introduction to ROC analysis","volume":"27","author":"Fawcett","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Kirk, R. (2007). Statistics: An Introduction, Belmont.","DOI":"10.1007\/978-0-230-21274-9_1"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"984353","DOI":"10.1155\/2015\/984353","article-title":"Thermal infrared imaging-based computational psychophysiology for psychometrics","volume":"2015","author":"Cardone","year":"2015","journal-title":"Comput. Math. Methods Med."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/4\/2\/25\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:35:30Z","timestamp":1760175330000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/4\/2\/25"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,4]]},"references-count":44,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["mti4020025"],"URL":"https:\/\/doi.org\/10.3390\/mti4020025","relation":{},"ISSN":["2414-4088"],"issn-type":[{"value":"2414-4088","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,4]]}}}