{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T01:38:49Z","timestamp":1769737129302,"version":"3.49.0"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"3s","license":[{"start":{"date-parts":[[2021,10,26]],"date-time":"2021-10-26T00:00:00Z","timestamp":1635206400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2017YFB1400603"],"award-info":[{"award-number":["2017YFB1400603"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61825205, 61772459"],"award-info":[{"award-number":["61825205, 61772459"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Science and Technology Major Project of China","award":["50-D36B02-9002-16\/19"],"award-info":[{"award-number":["50-D36B02-9002-16\/19"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2021,10,31]]},"abstract":"<jats:p>\n            Parkinson\u2019s disease is the second most common neurodegenerative disorder, commonly affecting elderly people over the age of 65. As the cardinal manifestation, hypomimia, referred to as impairments in normal facial expressions, stays covert. Even some experienced doctors may miss these subtle changes, especially in a mild stage of this disease. The existing methods for hypomimia recognition are mainly dominated by statistical variable-based methods with the help of traditional machine learning algorithms. Despite the success of recognizing hypomimia, they show a limited accuracy and lack the capability of performing semantic analysis. Therefore, developing a computer-aided diagnostic method for semantically recognizing hypomimia is appealing. In this article, we propose a\n            <jats:bold>Semantic Feature based Hypomimia Recognition network<\/jats:bold>\n            , named\n            <jats:bold>SFHR-NET<\/jats:bold>\n            , to recognize hypomimia based on facial videos. First, a\n            <jats:bold>Semantic Feature Classifier (SF-C)<\/jats:bold>\n            is proposed to adaptively adjust feature maps salient to hypomimia, which leads the encoder and classifier to focus more on areas of hypomimia-interest. In SF-C, the\n            <jats:bold>progressive confidence strategy (PCS)<\/jats:bold>\n            ensures more reliable semantic features. Then, a two-stream framework is introduced to fuse the spatial data stream and temporal optical stream, which allows the encoder to semantically and progressively characterize the rigid process of hypomimia. Finally, to improve the interpretability of the model,\n            <jats:bold>Gradient-weighted Class Activation Mapping (Grad-CAM)<\/jats:bold>\n            is integrated to generate attention maps that cast our engineered features into hypomimia-interest regions. These highlighted regions provide visual explanations for decisions of our network. Experimental results based on real-world data demonstrate the effectiveness of our method in detecting hypomimia.\n          <\/jats:p>","DOI":"10.1145\/3476778","type":"journal-article","created":{"date-parts":[[2021,10,26]],"date-time":"2021-10-26T15:46:12Z","timestamp":1635263172000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":17,"title":["Hypomimia Recognition in Parkinson\u2019s Disease With Semantic Features"],"prefix":"10.1145","volume":"17","author":[{"given":"Ge","family":"Su","sequence":"first","affiliation":[{"name":"College of Computer Science, Zhejiang University, Hangzhou, Zhejiang Province, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bo","family":"Lin","sequence":"additional","affiliation":[{"name":"College of Computer Science, Zhejiang University, Hangzhou, Zhejiang Province, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Luo","sequence":"additional","affiliation":[{"name":"Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianwei","family":"Yin","sequence":"additional","affiliation":[{"name":"College of Computer Science, Zhejiang University, Hangzhou, Zhejiang Province, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuiguang","family":"Deng","sequence":"additional","affiliation":[{"name":"College of Computer Science, Zhejiang University, Hangzhou, Zhejiang Province, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Honghao","family":"Gao","sequence":"additional","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Renjun","family":"Xu","sequence":"additional","affiliation":[{"name":"Center for Data Science, Zhejiang University, Hangzhou, Zhejiang Province, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,10,26]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.3758\/s13423-015-0974-5"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jneumeth.2017.02.006"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.50"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00371-011-0611-x"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1136\/jnnp-2012-303993"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCE-Berlin47944.2019.8966224"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(03)00568-3"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00107"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1002\/mds.21198"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3316782.3322756"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1037\/a0018104"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-61310-w"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(81)90024-2"},{"key":"e_1_3_1_16_2","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Jakab Tomas","year":"2020","unstructured":"Tomas Jakab, Ankush Gupta, Hakan Bilen, and Andrea Vedaldi. 2020. Self-supervised learning of interpretable keypoints from unlabelled videos. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.59"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.2196\/18697"},{"key":"e_1_3_1_19_2","article-title":"Parkinson\u2019s disease: Addressing health care practitioners\u2019 automatic responses to hypomimia.","author":"Lilly M. L.","year":"2020","unstructured":"M. L. Lilly, Melinda Hermanns, Deborah L. Dallam, and Amal Soomro. 2020. Parkinson\u2019s disease: Addressing health care practitioners\u2019 automatic responses to hypomimia. Journal of the American Association of Nurse Practitioners (2020).","journal-title":"Journal of the American Association of Nurse Practitioners"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0169110"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3381086"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-016-0911-8"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2019.00112"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3311747"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ncl.2014.09.010"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377882"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCE.2019.8901033"},{"key":"e_1_3_1_28_2","first-page":"1","volume-title":"2018 10th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT)","author":"Rajnoha M.","year":"2018","unstructured":"M. Rajnoha, J. Mekyska, R. Burget, I. Eliasova, M. Kostalova, and I. Rektorova. 2018. Towards identification of hypomimia in Parkinson\u2019s disease based on face recognition methods. In 2018 10th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT). 1\u20134. https:\/\/doi.org\/10.1109\/ICUMT.2018.8631249"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.lfs.2019.03.057"},{"key":"e_1_3_1_30_2","article-title":"Rehabilitation of hypomimia in Parkinson\u2019s disease: A feasibility study of two different approaches","volume":"37","author":"Ricciardi Lucia","year":"2015","unstructured":"Lucia Ricciardi, Paola Baggio, Diego Ricciardi, Bruno Morabito, Massimiliano Pomponi, Anna Bentivoglio, Roberto Bernabei, Roberto Maestri, Giuseppe Frazzitta, and Daniele Volpe. 2015. Rehabilitation of hypomimia in Parkinson\u2019s disease: A feasibility study of two different approaches. Neurological Sciences 37 (11 2015). https:\/\/doi.org\/10.1007\/s10072-015-2421-9","journal-title":"Neurological Sciences"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1111\/ene.14452"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0160329"},{"key":"e_1_3_1_33_2","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Sariyanidi Evangelos","year":"2020","unstructured":"Evangelos Sariyanidi, Casey J. Zampella, Robert T. Schultz, and Birkan Tunc. 2020. Can facial pose and expression be separated with weak perspective camera?. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)."},{"issue":"15","key":"e_1_3_1_34_2","article-title":"Hypomimia detection with a smartphone camera as a possible self-screening tool for Parkinson disease (P3.047)","volume":"90","author":"Seliverstov Yury","year":"2018","unstructured":"Yury Seliverstov, Dmitrii Diagovchenko, Michael Kravchenko, Mikhail Babin, Ekaterina Fedotova, and Mikhail Belyaev. 2018. Hypomimia detection with a smartphone camera as a possible self-screening tool for Parkinson disease (P3.047). Neurology 90, 15 Supplement (2018). arXiv: https:\/\/n.neurology.org\/content https:\/\/n.neurology.org\/content\/90\/15_Supplement\/P3.047.","journal-title":"Neurology"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.74"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01230"},{"key":"e_1_3_1_37_2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).","journal-title":"arXiv preprint arXiv:1409.1556"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00972"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuropsychologia.2012.04.018"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-32270-4_7"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.3034233"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2765830"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00887"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3476778","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3476778","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T17:49:20Z","timestamp":1750268960000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3476778"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,26]]},"references-count":42,"journal-issue":{"issue":"3s","published-print":{"date-parts":[[2021,10,31]]}},"alternative-id":["10.1145\/3476778"],"URL":"https:\/\/doi.org\/10.1145\/3476778","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,26]]},"assertion":[{"value":"2020-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-10-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}