{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T11:27:02Z","timestamp":1783078022133,"version":"3.54.6"},"reference-count":11,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2023,4,28]],"date-time":"2023-04-28T00:00:00Z","timestamp":1682640000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,4,28]],"date-time":"2023-04-28T00:00:00Z","timestamp":1682640000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Tokyo University of Science"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Artif Life Robotics"],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Given the ongoing COVID-19 pandemic, remote interviews have become an increasingly popular approach in many fields. For example, a survey by the HR Research Institute (PCR Institute in Survey on hiring activities for graduates of 2021 and 2022. <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/www.hrpro.co.jp\/research_detail.php?r_no=273\">https:\/\/www.hrpro.co.jp\/research_detail.php?r_no=273<\/jats:ext-link>. Accessed 03 Oct 2021) shows that more than 80% of job interviews are conducted remotely, particularly in large companies. However, for some reason, an interviewee might attempt to deceive an interviewer or feel difficult to tell the truth. Although the ability of interviewers to detect deception among interviewees is significant for their company or organization, it still strongly depends on their individual experience and cannot be automated. To address this issue, in this study, we propose a machine learning approach to aid in detecting whether a person is attempting to deceive the interlocutor by associating the features of their facial expressions with those of their pulse rate. We also constructed a more realistic dataset for the task of deception detection by asking subjects not to respond artificially, but rather to improvise natural responses using a web camera and wearable device (smartwatch). The results of an experimental evaluation of the proposed approach with 10-fold cross-validation using random forests classifier show that the accuracy and the F1 value were in the range between 0.75 and 0.8 for each subject, and the highest values were 0.87 and 0.88, respectively. Through the analysis of the importance of the features the trained models, we revealed the crucial features of each subject during deception, which differed among the subjects.<\/jats:p>","DOI":"10.1007\/s10015-023-00869-9","type":"journal-article","created":{"date-parts":[[2023,4,28]],"date-time":"2023-04-28T11:03:09Z","timestamp":1682679789000},"page":"509-519","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Detecting deception using machine learning with facial expressions and pulse rate"],"prefix":"10.1007","volume":"28","author":[{"given":"Kento","family":"Tsuchiya","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryo","family":"Hatano","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hiroyuki","family":"Nishiyama","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,4,28]]},"reference":[{"key":"869_CR1","doi-asserted-by":"crossref","unstructured":"Baltrusaitis T, Zadeh A, Lim YC, Morency LP (2018) Openface 2.0: facial behavior analysis toolkit. In: 2018 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2018), pp 59\u201366","DOI":"10.1109\/FG.2018.00019"},{"issue":"3","key":"869_CR2","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1207\/s15327957pspr1003_2","volume":"10","author":"CF Bond Jr","year":"2006","unstructured":"Bond CF Jr, DePaulo BM (2006) Accuracy of deception judgments. Pers Soc Psychol Rev 10(3):214\u2013234","journal-title":"Pers Soc Psychol Rev"},{"key":"869_CR3","first-page":"1","volume":"6","author":"N Hosomi","year":"2017","unstructured":"Hosomi N, Yoshino K, Nakamura S et al (2017) Deception detection and analysis from dialogue using fasttext. Res Rep Spok Lang Process 6:1\u20135 (in Japanese)","journal-title":"Res Rep Spok Lang Process"},{"key":"869_CR4","unstructured":"Institute PCR Survey on Hiring Activities for Graduates of 2021 and 2022 (2021). https:\/\/www.hrpro.co.jp\/research_detail.php?r_no=273. Accessed 03 Oct 2021 (in Japanese)"},{"key":"869_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2020.114341","volume":"169","author":"W Khan","year":"2021","unstructured":"Khan W, Crockett K, O\u2019Shea J, Hussain A, Khan BM (2021) Deception in the eyes of deceiver: a computer vision and machine learning based automated deception detection. Expert Syst Appl 169:114341","journal-title":"Expert Syst Appl"},{"key":"869_CR6","doi-asserted-by":"crossref","unstructured":"Mathur L, Matari\u0107 MJ (2020) Introducing representations of facial affect in automated multimodal deception detection. In: Proceedings of the 2020 International Conference on Multimodal Interaction, pp 305\u2013314","DOI":"10.1145\/3382507.3418864"},{"key":"869_CR7","unstructured":"Sakaue S, Nomiya H, Bauchin T, et\u00a0al (2016) Estimation of facial expression intensity using features from changes in position of facial feature points. In: Proceedings of the 2016 Annual Conference of the IPSJ, Kansai Branch (in Japanese)"},{"key":"869_CR8","unstructured":"Shushoku Journal Editorial Department YA Is it Ok to Lie or Embellish Your Story in Job Hunting? An Honest Survey of 300 Hiring Managers (2021). https:\/\/journal.rikunabi.com\/p\/advice\/25761.html. Accessed 03 Oct 2021 (in Japanese)"},{"key":"869_CR9","unstructured":"Watanabe K (2016) Relationship between emotional change and gravitational perturbation caused by lying. Report on the Research Results of the Independent Research Encouragement Project by Undergraduate Students in 2015 (in Japanese)"},{"issue":"1","key":"869_CR10","first-page":"2018","volume":"32","author":"Z Wu","year":"2018","unstructured":"Wu Z, Singh B, Davis L, Subrahmanian V (2018) Deception detection in videos. Proc AAAI Conf Artif Intell 32(1):2018","journal-title":"Proc AAAI Conf Artif Intell"},{"issue":"9","key":"869_CR11","first-page":"1","volume":"2018","author":"M Yasumaru","year":"2018","unstructured":"Yasumaru M, Takeuchi M, Kanai K, Kotto J et al (2018) Human emotion estimation using pulse wave measurement for facial movie. Tech Rep Spec Interest Groups Audio Vis Multimed Inf Process (AVM) 2018(9):1\u20134","journal-title":"Tech Rep Spec Interest Groups Audio Vis Multimed Inf Process (AVM)"}],"updated-by":[{"DOI":"10.1007\/s10015-023-00877-9","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2023,6,13]],"date-time":"2023-06-13T00:00:00Z","timestamp":1686614400000}}],"container-title":["Artificial Life and Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10015-023-00869-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10015-023-00869-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10015-023-00869-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,20]],"date-time":"2023-07-20T06:03:52Z","timestamp":1689833032000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10015-023-00869-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,28]]},"references-count":11,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["869"],"URL":"https:\/\/doi.org\/10.1007\/s10015-023-00869-9","relation":{"correction":[{"id-type":"doi","id":"10.1007\/s10015-023-00877-9","asserted-by":"object"}]},"ISSN":["1433-5298","1614-7456"],"issn-type":[{"value":"1433-5298","type":"print"},{"value":"1614-7456","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,28]]},"assertion":[{"value":"17 March 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 March 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 April 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 June 2023","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Correction","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A Correction to this paper has been published:","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"https:\/\/doi.org\/10.1007\/s10015-023-00877-9","URL":"https:\/\/doi.org\/10.1007\/s10015-023-00877-9","order":7,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}}]}}