{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:45:13Z","timestamp":1760244313376,"version":"build-2065373602"},"reference-count":27,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,9,12]],"date-time":"2022-09-12T00:00:00Z","timestamp":1662940800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Here we describe a proof-of-concept case study focusing on the design and development of a novel computer interface that uses facial muscles to control interactivity within a virtual environment. We have developed a system comprised of skin-mounted electrodes that detect underlying muscle activity through electromyography. The signals from the electrodes are filtered and smoothed, then used as input data to an application that displays a virtual environment with a 3D animated avatar. The user\u2019s expressions control the facial movements of the avatar, thus conveying user emotions through real-time animation of a representative face in a virtual scenario. To achieve this, we collaborated with our Public and Patient Involvement focus group to discuss concepts and design appropriate interactions, while simultaneously developing a prototype system. Programmers and 3D artists worked together to create a system whereby individual user facial muscles are connected to 3D animated models of the same muscle features represented in an avatar, providing the user with an option to receive visual and numerical feedback on the extent of their muscle control. Using the prototype system, people can communicate facial expressions virtually with each other, without the need for a camera. This research is part of an on-going project to develop a facial muscle rehabilitation system that can be optimized to help patients with conditions such as hypomimia.<\/jats:p>","DOI":"10.3390\/mti6090078","type":"journal-article","created":{"date-parts":[[2022,9,13]],"date-time":"2022-09-13T01:44:03Z","timestamp":1663033443000},"page":"78","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Expressive Interaction Design Using Facial Muscles as Controllers"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4226-6889","authenticated-orcid":false,"given":"Fiona","family":"French","sequence":"first","affiliation":[{"name":"Interaction Design Research Group, School of Computing and Digital Media, London Metropolitan University, London N7 8DB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8611-8821","authenticated-orcid":false,"given":"Cassandra","family":"Terry","sequence":"additional","affiliation":[{"name":"Molecular Systems for Health Research Group, School of Human Sciences, London Metropolitan University, London N7 8DB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9973-8555","authenticated-orcid":false,"given":"Saif","family":"Huq","sequence":"additional","affiliation":[{"name":"Interaction Design Research Group, School of Computing and Digital Media, London Metropolitan University, London N7 8DB, UK"}]},{"given":"Isaac","family":"Furieri","sequence":"additional","affiliation":[{"name":"School of Computing and Digital Media, London Metropolitan University, London N7 8DB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5626-5196","authenticated-orcid":false,"given":"Mateusz","family":"Jarzembinski","sequence":"additional","affiliation":[{"name":"School of Computing and Digital Media, London Metropolitan University, London N7 8DB, UK"}]},{"given":"Simas","family":"Pauliukenas","sequence":"additional","affiliation":[{"name":"School of Computing and Digital Media, London Metropolitan University, London N7 8DB, UK"}]},{"given":"Neil","family":"Morrison","sequence":"additional","affiliation":[{"name":"No affiliation"}]},{"given":"Karen","family":"Shepherd","sequence":"additional","affiliation":[{"name":"No affiliation"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"17602","DOI":"10.1038\/s41598-018-35905-3","article-title":"Facial expression as a potential measure of both intent and emotion","volume":"8","author":"Camerlink","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1550","DOI":"10.1037\/emo0001015","article-title":"Do emotions result in their predicted facial expressions? A meta-analysis of studies on the co-occurrence of expression and emotion","volume":"21","year":"2021","journal-title":"Emotion"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"678","DOI":"10.61872\/sdj-2018-09-442","article-title":"The Face\u2014A Musculoskeletal Perspective. A literature review","volume":"128","author":"Nakashima","year":"2018","journal-title":"Swiss Dent. J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1080\/11745398.2017.1358098","article-title":"Digital nomads\u2014A quest for holistic freedom in work and leisure","volume":"21","author":"Reichenberger","year":"2018","journal-title":"Ann. Leis. Res."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Rainoldi, M., Ladkin, A., and Buhalis, D. (2022). Blending work and leisure: A future digital worker hybrid lifestyle perspective. Ann. Leis. Res.","DOI":"10.1080\/11745398.2022.2070513"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1016\/j.jneumeth.2007.09.030","article-title":"Automated video-based facial expression analysis of neuropsychiatric disorders","volume":"168","author":"Wang","year":"2008","journal-title":"J. Neurosci. Methods"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Darmawanti, I. (2019, January 24). Interpreting Facial Expression: A Challenging Study Using Existing Video. Proceedings of the 3rd International Conference on Education Innovation (ICEI 2019), Surabaya, Indonesia.","DOI":"10.2991\/icei-19.2019.85"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"20696","DOI":"10.1038\/s41598-021-99998-z","article-title":"A deep learning model for classifying human facial expressions from infrared thermal images","volume":"11","author":"Bhattacharyya","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Earnshaw, R.A., Guedj, R.A., Dam, A.v., and Vince, J.A. (2001). Perceptual User Interfaces. Frontiers of Human-Centered Computing, Online Communities and Virtual Environments, Springer.","DOI":"10.1007\/978-1-4471-0259-5"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"112105","DOI":"10.1016\/j.sna.2020.112105","article-title":"Progress and challenges in fabrication of wearable sensors for health monitoring","volume":"312","author":"Nasiri","year":"2020","journal-title":"Sens. Actuators A Phys."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"222","DOI":"10.3389\/fneur.2020.00222","article-title":"A Systematic Review of Physical Rehabilitation of Facial Palsy","volume":"11","author":"Vaughan","year":"2020","journal-title":"Front. Neurol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/j.neubiorev.2016.11.005","article-title":"Action observation and motor imagery for rehabilitation in Parkinson\u2019s disease: A systematic review and an integrative hypothesis","volume":"72","author":"Caligiore","year":"2017","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40945-015-0013-x","article-title":"Action observation training to improve motor function recovery: A systematic review","volume":"5","author":"Sarasso","year":"2015","journal-title":"Arch. Physiother."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1111\/j.1467-9450.1995.tb00974.x","article-title":"Facial EMG reactions to facial expressions: A case of facial emotional contagion?","volume":"36","author":"Lundqvist","year":"1995","journal-title":"Scand. J. Psychol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1109\/TAFFC.2016.2601101","article-title":"Virtual Character Facial Expressions Influence Human Brain and Facial EMG Activity in a Decision-Making Game","volume":"9","author":"Ravaja","year":"2016","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Cross, M.P., Acevedo, A.M., Leger, K.A., and Pressman, S.D. (2022). How and why could smiling influence physical health? A conceptual review. Health Psychol. Rev., 1\u201323.","DOI":"10.1080\/17437199.2022.2052740"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1007\/s00415-016-8309-7","article-title":"Brain plasticity in Parkinson\u2019s disease with freezing of gait induced by action observation training","volume":"264","author":"Agosta","year":"2017","journal-title":"J. Neurol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/j.parkreldis.2018.11.001","article-title":"Combined action observation and motor imagery influences hand movement amplitude in Parkinson\u2019s disease","volume":"61","author":"Bek","year":"2019","journal-title":"Park. Relat. Disord."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"723","DOI":"10.3389\/fneur.2017.00723","article-title":"Action Observation Plus Sonification. A Novel Therapeutic Protocol for Parkinson\u2019s Patient with Freezing of Gait","volume":"8","author":"Mezzarobba","year":"2018","journal-title":"Front. Neurol."},{"key":"ref_20","unstructured":"Rea, P. (2020). Virtual Reality Design for Stroke Rehabilitation. Biomedical Visualisation, Springer. Advances in Experimental Medicine and Biology, 1235;."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"110192","DOI":"10.1016\/j.chaos.2020.110192","article-title":"Effectiveness of gamification for the rehabilitation of neurodegenerative disorders","volume":"140","author":"Adlakha","year":"2020","journal-title":"Chaos Solitons Fractals"},{"key":"ref_22","first-page":"1179547621994579","article-title":"Virtual Reality Augmented Feedback Rehabilitation Associated to Action Observation Therapy in Buccofacial Apraxia: Case Report","volume":"14","author":"Emedoli","year":"2021","journal-title":"Clin. Med. Insights: Case Rep."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Iwanaga, J., Hur, M.-S., Kikuta, S., Ibaragi, S., Watanabe, K., and Tubbs, R.S. (2022). Anatomical contribution of the orbicularis oculi to the zygomaticus major: An improved understanding of the smile with consideration for facial cosmetic procedures. PLoS ONE, 17.","DOI":"10.1371\/journal.pone.0272060"},{"key":"ref_24","unstructured":"Amara, K., Ramzan, N., Zenati, N., Djekoune, O., Larbes, C., Guerroudji, M.A., and Aouam, D. A method for Facial emotion recognition. CEUR Workshop Proceedings, Proceedings of the ICCSA\u201921: The 2nd International Conference on Complex Systems and their Applications, Oum El Bouaghi, Algeria, 25\u221226 May 2021, Available online: http:\/\/ceur-ws.org\/Vol-2904\/51.pdf."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1007\/s10055-021-00575-6","article-title":"Performance enhancement of facial electromyogram-based facial-expression recognition for social virtual reality applications using linear discriminant analysis adaptation","volume":"26","author":"Cha","year":"2022","journal-title":"Virtual Real."},{"key":"ref_26","first-page":"1","article-title":"AffectiveHMD: Facial expression recognition in head mounted display using embedded photo reflective sensors","volume":"Volume 7","author":"Murakami","year":"2019","journal-title":"ACM SIGGRAPH 2019 Emerging Technologies (SIGGRAPH \u201819)"},{"key":"ref_27","unstructured":"Jamal, M.Z. (2012). Signal Acquisition Using Surface EMG and Circuit Design Considerations for Robotic Prosthesis. Computational Intelligence in Electromyography Analysis\u2014A Perspective on Current Applications and Future Challenges, IntechOpen."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/6\/9\/78\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:29:59Z","timestamp":1760142599000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/6\/9\/78"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,12]]},"references-count":27,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["mti6090078"],"URL":"https:\/\/doi.org\/10.3390\/mti6090078","relation":{},"ISSN":["2414-4088"],"issn-type":[{"type":"electronic","value":"2414-4088"}],"subject":[],"published":{"date-parts":[[2022,9,12]]}}}