{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,26]],"date-time":"2026-01-26T00:51:21Z","timestamp":1769388681672,"version":"3.49.0"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2017,12,13]],"date-time":"2017-12-13T00:00:00Z","timestamp":1513123200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"JST CREST","award":["JPMJCR14E1"],"award-info":[{"award-number":["JPMJCR14E1"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2017,12,31]]},"abstract":"<jats:p>This article presents a novel smart eyewear that recognizes the wearer\u2019s facial expressions in daily scenarios. Our device uses embedded photo-reflective sensors and machine learning to recognize the wearer\u2019s facial expressions. Our approach focuses on skin deformations around the eyes that occur when the wearer changes his or her facial expressions. With small photo-reflective sensors, we measure the distances between the skin surface on the face and the 17 sensors embedded in the eyewear frame. A Support Vector Machine (SVM) algorithm is then applied to the information collected by the sensors. The sensors can cover various facial muscle movements. In addition, they are small and light enough to be integrated into daily-use glasses. Our evaluation of the device shows the robustness to the noises from the wearer\u2019s facial direction changes and the slight changes in the glasses\u2019 position, as well as the reliability of the device\u2019s recognition capacity.<\/jats:p><jats:p>The main contributions of our work are as follows: (1) We evaluated the recognition accuracy in daily scenes, showing 92.8% accuracy regardless of facial direction and removal\/remount. Our device can recognize facial expressions with 78.1% accuracy for repeatability and 87.7% accuracy in case of its positional drift. (2) We designed and implemented the device by taking usability and social acceptability into account. The device looks like a conventional eyewear so that users can wear it anytime, anywhere. (3) Initial field trials in a daily life setting were undertaken to test the usability of the device.<\/jats:p><jats:p>Our work is one of the first attempts to recognize and evaluate a variety of facial expressions with an unobtrusive wearable device.<\/jats:p>","DOI":"10.1145\/3012941","type":"journal-article","created":{"date-parts":[[2017,12,13]],"date-time":"2017-12-13T14:50:37Z","timestamp":1513176637000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Evaluation of Facial Expression Recognition by a Smart Eyewear for Facial Direction Changes, Repeatability, and Positional Drift"],"prefix":"10.1145","volume":"7","author":[{"given":"Katsutoshi","family":"Masai","sequence":"first","affiliation":[{"name":"Keio University, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai","family":"Kunze","sequence":"additional","affiliation":[{"name":"Keio University, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuta","family":"Sugiura","sequence":"additional","affiliation":[{"name":"Keio University, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Masa","family":"Ogata","sequence":"additional","affiliation":[{"name":"National Institute of Advanced Industrial Science Technology, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Masahiko","family":"Inami","sequence":"additional","affiliation":[{"name":"The University of Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maki","family":"Sugimoto","sequence":"additional","affiliation":[{"name":"Keio University, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,12,13]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.69.2.280"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2010.1"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1322355111"},{"key":"e_1_2_1_4_1","unstructured":"P. Ekman. 1989. The argument and evidence about universals in facial expressions. In Handbook of Social Psychophysiology. 143--164. P. Ekman. 1989. The argument and evidence about universals in facial expressions. In Handbook of Social Psychophysiology. 143--164."},{"key":"e_1_2_1_5_1","doi-asserted-by":"crossref","unstructured":"P. Ekman and W. V. Friesen. 1977. Facial action coding system. Consulting Psychologists Press Stanford University Palo Alto. P. Ekman and W. V. Friesen. 1977. Facial action coding system. Consulting Psychologists Press Stanford University Palo Alto.","DOI":"10.1037\/t27734-000"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2459236.2459273"},{"key":"e_1_2_1_7_1","volume-title":"2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG\u201915)","volume":"1","author":"Girard J. M.","unstructured":"J. M. Girard , J. F. Cohn , L. A. Jeni , S. Lucey , and F. D. Torre . 2015. How much training data for facial action unit detection? In 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG\u201915) , Vol. 1 . 1--8. J. M. Girard, J. F. Cohn, L. A. Jeni, S. Lucey, and F. D. Torre. 2015. How much training data for facial action unit detection? In 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG\u201915), Vol. 1. 1--8."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1037\/1089-2680.2.3.271"},{"key":"e_1_2_1_9_1","volume-title":"Handbook of Emotion Regulation","author":"Gross J. J.","unstructured":"J. J. Gross . 2013. Handbook of Emotion Regulation . Guilford Publications . J. J. Gross. 2013. Handbook of Emotion Regulation. Guilford Publications."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2014.2313557"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/795661.796155"},{"key":"e_1_2_1_12_1","doi-asserted-by":"crossref","unstructured":"D. Keltner P. Ekman G. C. Gonzaga and J. Beer. 2003. Expression of emotion. In Handbook of Affective Sciences. 411--414. D. Keltner P. Ekman G. C. Gonzaga and J. Beer. 2003. Expression of emotion. In Handbook of Affective Sciences. 411--414.","DOI":"10.1093\/oso\/9780195126013.003.0021"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2493988.2494330"},{"key":"e_1_2_1_14_1","unstructured":"M. L. Knapp J. A. Hall and T. G. Horgan. 2013. Nonverbal Communication in Human Interaction. Cengage Learning. M. L. Knapp J. A. Hall and T. G. Horgan. 2013. Nonverbal Communication in Human Interaction. Cengage Learning."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766939"},{"key":"e_1_2_1_16_1","unstructured":"G. Littlewort I. Fasel M. S. Bartlett and J. R. Movellan. 2002. Fully Automatic Coding of Basic Expressions from Video. University of California San Diego CA 92093. G. Littlewort I. Fasel M. S. Bartlett and J. R. Movellan. 2002. Fully Automatic Coding of Basic Expressions from Video. University of California San Diego CA 92093."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2856767.2856770"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2782782.2792495"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992972"},{"key":"e_1_2_1_20_1","volume-title":"Nonverbal Communication","author":"Mehrabian A.","unstructured":"A. Mehrabian . 1972. Nonverbal Communication . Transaction Publishers . A. Mehrabian. 1972. Nonverbal Communication. Transaction Publishers."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1785455.1785475"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2501988.2502039"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380135"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01682026"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.115.1.102"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/632716.632878"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.908962"},{"key":"e_1_2_1_29_1","doi-asserted-by":"crossref","unstructured":"Y. Tian T. Kanade and J. F. Cohn. 2011. Facial expression recognition. In Handbook of Face Recognition S. Z. Li and A. K. Jain (Eds.). Springer London 487--519. Y. Tian T. Kanade and J. F. Cohn. 2011. Facial expression recognition. In Handbook of Face Recognition S. Z. Li and A. K. Jain (Eds.). Springer London 487--519.","DOI":"10.1007\/978-0-85729-932-1_19"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2008.52"}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3012941","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3012941","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T19:05:39Z","timestamp":1750273539000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3012941"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,12,13]]},"references-count":29,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2017,12,31]]}},"alternative-id":["10.1145\/3012941"],"URL":"https:\/\/doi.org\/10.1145\/3012941","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,12,13]]},"assertion":[{"value":"2016-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-12-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}