{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T14:21:08Z","timestamp":1760710868829,"version":"3.41.0"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2022,2,16]],"date-time":"2022-02-16T00:00:00Z","timestamp":1644969600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62076262, 61673402, 61273270, 60802069"],"award-info":[{"award-number":["62076262, 61673402, 61273270, 60802069"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003453","name":"Natural Science Foundation of Guangdong Province","doi-asserted-by":"crossref","award":["2017A030311029"],"award-info":[{"award-number":["2017A030311029"]}],"id":[{"id":"10.13039\/501100003453","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Science and Technology Program of Huizhou of China","award":["2020SC0702002"],"award-info":[{"award-number":["2020SC0702002"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2022,5,31]]},"abstract":"<jats:p>\n            Recently, great progress has been achieved on facial landmark detection based on convolutional neural network, while it is still challenging due to partial occlusion and extreme head pose. In this paper, we propose a\n            <jats:bold>Cascaded Structure-Learning Network (CSLN)<\/jats:bold>\n            with using adversarial training to improve the performance of 2D facial landmark detection by taking the structure of facial landmarks into account. In the first stage, we improve the original stacked hourglass network, which applies a multi-branch module to capture different scales of features, a progressive convolution structure to compensate for the missing structural features in hourglass networks, and a pyramid inception structure to expand the receptive field. Specially, by introducing a discriminator, we use the adversarial training strategy to urge the improved hourglass network for generating more accurate heatmaps. The second stage, which is based on attention mechanism, optimizes the spatial correlations between different facial landmarks by reusing the structural features. Moreover, we propose a novel region loss, which can adaptively allocate proper weights to different regions. In this way, the network can focus more on those occluded landmarks. The experimental results on several datasets,\n            <jats:italic>i.e.<\/jats:italic>\n            300W, COFW, and AFLW, show that our proposed method achieves superior performance compared with the state-of-the-art methods.\n          <\/jats:p>","DOI":"10.1145\/3474595","type":"journal-article","created":{"date-parts":[[2022,2,16]],"date-time":"2022-02-16T17:56:32Z","timestamp":1645034192000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Cascaded Structure-Learning Network with Using Adversarial Training for Robust Facial Landmark Detection"],"prefix":"10.1145","volume":"18","author":[{"given":"Shenming","family":"Feng","sequence":"first","affiliation":[{"name":"School of Electronics and Information Technology, Sun Yat-Sen University, Guangzhou, 510006, Guangdong, People\u2019s Republic of China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xingzhong","family":"Nong","sequence":"additional","affiliation":[{"name":"Guangzhou Metro Design &amp; Research Institute Co., Ltd, Guangzhou, Guangdong, People\u2019s Republic of China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haifeng","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Electronics and Information Technology, Sun Yat-Sen University, Guangzhou, 510006, Guangdong, People\u2019s Republic of China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,2,16]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.23"},{"key":"e_1_3_1_3_2","article-title":"BEGAN: Boundary equilibrium generative adversarial networks","volume":"1703","author":"Berthelot David","year":"2017","unstructured":"David Berthelot, Tom Schumm, and Luke Metz. 2017. BEGAN: Boundary equilibrium generative adversarial networks. CoRR abs\/1703.10717 (2017).","journal-title":"CoRR"},{"key":"e_1_3_1_4_2","first-page":"3726","volume-title":"ICCV","author":"Bulat Adrian","year":"2017","unstructured":"Adrian Bulat and Georgios Tzimiropoulos. 2017. Binarized convolutional landmark localizers for human pose estimation and face alignment with limited resources. In ICCV. 3726\u20133734."},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.191"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-013-0667-3"},{"key":"e_1_3_1_7_2","first-page":"109","volume-title":"ECCV","author":"Chen Dong","year":"2014","unstructured":"Dong Chen, Shaoqing Ren, Yichen Wei, Xudong Cao, and Jian Sun. 2014. Joint cascade face detection and alignment. In ECCV. 109\u2013122."},{"key":"e_1_3_1_8_2","first-page":"196","volume-title":"ICIP","author":"Chen Weiliang","year":"2018","unstructured":"Weiliang Chen, Qiang Zhou, and Roland Hu. 2018. Face alignment by combining residual features in cascaded hourglass network. In ICIP. 196\u2013200."},{"key":"e_1_3_1_9_2","first-page":"1221","volume-title":"ICCV","author":"Chen Yu","year":"2017","unstructured":"Yu Chen, Chunhua Shen, Xiu-Shen Wei, Lingqiao Liu, and Jian Yang. 2017. Adversarial PoseNet: A structure-aware convolutional network for human pose estimation. In ICCV. 1221\u20131230."},{"key":"e_1_3_1_10_2","first-page":"7103","volume-title":"CVPR","author":"Chen Yilun","year":"2018","unstructured":"Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, and Jian Sun. 2018. Cascaded pyramid network for multi-person pose estimation. In CVPR. 7103\u20137112."},{"key":"e_1_3_1_11_2","first-page":"17","volume-title":"APSIPA","author":"Chou Chia-Jung","year":"2018","unstructured":"Chia-Jung Chou, Jui-Ting Chien, and Hwann-Tzong Chen. 2018. Self adversarial training for human pose estimation. In APSIPA. 17\u201330."},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/34.927467"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1006\/cviu.1995.1004"},{"key":"e_1_3_1_14_2","article-title":"Joint multi-view face alignment in the wild","volume":"1708","author":"Deng Jiankang","year":"2017","unstructured":"Jiankang Deng, George Trigeorgis, Yuxiang Zhou, and Stefanos Zafeiriou. 2017. Joint multi-view face alignment in the wild. CoRR abs\/1708.06023 (2017).","journal-title":"CoRR"},{"key":"e_1_3_1_15_2","first-page":"1078","volume-title":"CVPR","author":"Doll\u00e1r Piotr","year":"2010","unstructured":"Piotr Doll\u00e1r, Peter Welinder, and Pietro Perona. 2010. Cascaded pose regression. In CVPR. 1078\u20131085."},{"key":"e_1_3_1_16_2","first-page":"379","volume-title":"CVPR","author":"Dong Xuanyi","year":"2018","unstructured":"Xuanyi Dong, Yan Yan, Wanli Ouyang, and Yi Yang. 2018. Style aggregated network for facial landmark detection. In CVPR. 379\u2013388."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.306"},{"key":"e_1_3_1_18_2","article-title":"Occlusion coherence: Detecting and localizing occluded faces","volume":"1506","author":"Ghiasi Golnaz","year":"2015","unstructured":"Golnaz Ghiasi and Charless C. Fowlkes. 2015. Occlusion coherence: Detecting and localizing occluded faces. CoRR abs\/1506.08347 (2015).","journal-title":"CoRR"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.5555\/2969033.2969125"},{"key":"e_1_3_1_20_2","first-page":"7297","volume-title":"CVPR","author":"G\u00fcler Riza Alp","year":"2018","unstructured":"Riza Alp G\u00fcler, Natalia Neverova, and Iasonas Kokkinos. 2018. DensePose: Dense human pose estimation in the wild. In CVPR. 7297\u20137306."},{"key":"e_1_3_1_21_2","first-page":"4295","volume-title":"CVPR","author":"Hassner Tal","year":"2015","unstructured":"Tal Hassner, Shai Harel, Eran Paz, and Roee Enbar. 2015. Effective face frontalization in unconstrained images. In CVPR. 4295\u20134304."},{"key":"e_1_3_1_22_2","first-page":"1546","volume-title":"CVPR","author":"Honari Sina","year":"2018","unstructured":"Sina Honari, Pavlo Molchanov, Stephen Tyree, Pascal Vincent, Christopher J. Pal, and Jan Kautz. 2018. Improving landmark localization with semi-supervised learning. In CVPR. 1546\u20131555."},{"key":"e_1_3_1_23_2","article-title":"Coarse-to-fine face alignment with multi-scale local patch regression","volume":"1511","author":"Huang Zhiao","year":"2015","unstructured":"Zhiao Huang, Erjin Zhou, and Zhimin Cao. 2015. Coarse-to-fine face alignment with multi-scale local patch regression. CoRR abs\/1511.04901 (2015).","journal-title":"CoRR"},{"key":"e_1_3_1_24_2","article-title":"Learning human pose estimation features with convolutional networks","volume":"1312","author":"Jain Arjun","year":"2013","unstructured":"Arjun Jain, Jonathan Tompson, Mykhaylo Andriluka, Graham W. Taylor, and Christoph Bregler. 2013. Learning human pose estimation features with convolutional networks. CoRR abs\/1312.7302 (2013).","journal-title":"CoRR"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.241"},{"key":"e_1_3_1_26_2","first-page":"731","volume-title":"ECCV","author":"Ke Lipeng","year":"2018","unstructured":"Lipeng Ke, Ming-Ching Chang, Honggang Qi, and Siwei Lyu. 2018. Multi-scale structure-aware network for human pose estimation. In ECCV. 731\u2013746."},{"key":"e_1_3_1_27_2","first-page":"185","volume-title":"AMDO","author":"Kittler Josef","year":"2016","unstructured":"Josef Kittler, Patrik Huber, Zhen-Hua Feng, Guosheng Hu, and William J. Christmas. 2016. 3D morphable face models and their applications. In AMDO. 185\u2013206."},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2011.6130513"},{"key":"e_1_3_1_29_2","first-page":"430","volume-title":"CVPR","author":"Kumar Amit","year":"2018","unstructured":"Amit Kumar and Rama Chellappa. 2018. Disentangling 3D pose in a dendritic CNN for unconstrained 2D face alignment. In CVPR. 430\u2013439."},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33712-3_49"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2019.2897243"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2016.2633939"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2017.190"},{"key":"e_1_3_1_35_2","first-page":"3691","volume-title":"CVPR","author":"Lv Jiang-Jing","year":"2017","unstructured":"Jiang-Jing Lv, Xiaohu Shao, Junliang Xing, Cheng Cheng, and Xi Zhou. 2017. A deep regression architecture with two-stage re-initialization for high performance facial landmark detection. In CVPR. 3691\u20133700."},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2017.107"},{"key":"e_1_3_1_37_2","first-page":"5040","volume-title":"CVPR","author":"Miao Xin","year":"2018","unstructured":"Xin Miao, Xiantong Zhen, Xianglong Liu, Cheng Deng, Vassilis Athitsos, and Heng Huang. 2018. Direct shape regression networks for end-to-end face alignment. In CVPR. 5040\u20135049."},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-88693-8_37"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.5555\/3327546.3327642"},{"key":"e_1_3_1_40_2","first-page":"483","volume-title":"ECCV","author":"Newell Alejandro","year":"2016","unstructured":"Alejandro Newell, Kaiyu Yang, and Jia Deng. 2016. Stacked hourglass networks for human pose estimation. In ECCV. 483\u2013499."},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2781233"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2016.2518867"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2636829"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2013.59"},{"key":"e_1_3_1_45_2","first-page":"1","volume-title":"ICCV","author":"Saragih Jason M.","year":"2007","unstructured":"Jason M. Saragih and Roland G\u00f6cke. 2007. A nonlinear discriminative approach to AAM fitting. In ICCV. 1\u20138."},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.5555\/2969033.2969049"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.446"},{"key":"e_1_3_1_48_2","first-page":"348","volume-title":"ECCV","author":"Tang Zhiqiang","year":"2018","unstructured":"Zhiqiang Tang, Xi Peng, Shijie Geng, Lingfei Wu, Shaoting Zhang, and Dimitris N. Metaxas. 2018. Quantized densely connected u-nets for efficient landmark localization. In ECCV. 348\u2013364."},{"key":"e_1_3_1_49_2","first-page":"4177","volume-title":"CVPR","author":"Trigeorgis George","year":"2016","unstructured":"George Trigeorgis, Patrick Snape, Mihalis A. Nicolaou, Epameinondas Antonakos, and Stefanos Zafeiriou. 2016. Mnemonic descent method: A recurrent process applied for end-to-end face alignment. In CVPR. 4177\u20134187."},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.79"},{"key":"e_1_3_1_51_2","first-page":"609","volume-title":"ECCV","author":"Valle Roberto","year":"2018","unstructured":"Roberto Valle, Jos\u00e9 Miguel Buenaposada, Antonio Vald\u00e9s, and Luis Baumela. 2018. A deeply-initialized coarse-to-fine ensemble of regression trees for face alignment. In ECCV. 609\u2013624."},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2018.2810881"},{"key":"e_1_3_1_53_2","first-page":"4724","volume-title":"CVPR","author":"Wei Shih-En","year":"2016","unstructured":"Shih-En Wei, Varun Ramakrishna, Takeo Kanade, and Yaser Sheikh. 2016. Convolutional pose machines. In CVPR. 4724\u20134732."},{"key":"e_1_3_1_54_2","first-page":"2129","volume-title":"CVPR","author":"Wu Wayne","year":"2018","unstructured":"Wayne Wu, Chen Qian, Shuo Yang, Quan Wang, Yici Cai, and Qiang Zhou. 2018. Look at boundary: A boundary-aware face alignment algorithm. In CVPR. 2129\u20132138."},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2017.261"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2787130"},{"key":"e_1_3_1_57_2","first-page":"57","volume-title":"ECCV","author":"Xiao Shengtao","year":"2016","unstructured":"Shengtao Xiao, Jiashi Feng, Junliang Xing, Hanjiang Lai, Shuicheng Yan, and Ashraf A. Kassim. 2016. Robust facial landmark detection via recurrent attentive-refinement networks. In ECCV. 57\u201372."},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.75"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2017.253"},{"key":"e_1_3_1_60_2","first-page":"5216","volume-title":"CVPR","author":"Yang Jiaolong","year":"2017","unstructured":"Jiaolong Yang, Peiran Ren, Dongqing Zhang, Dong Chen, Fang Wen, Hongdong Li, and Gang Hua. 2017. Neural aggregation network for video face recognition. In CVPR. 5216\u20135225."},{"key":"e_1_3_1_61_2","first-page":"2116","volume-title":"2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2017, Honolulu, HI, USA, July 21\u201326, 2017","author":"Zafeiriou Stefanos","year":"2017","unstructured":"Stefanos Zafeiriou, George Trigeorgis, Grigorios Chrysos, Jiankang Deng, and Jie Shen. 2017. The MMenpo facial landmark localisation challenge: A step towards the solution. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2017, Honolulu, HI, USA, July 21\u201326, 2017. 2116\u20132125."},{"key":"e_1_3_1_62_2","first-page":"1","volume-title":"ECCV","author":"Zhang Jie","year":"2014","unstructured":"Jie Zhang, Shiguang Shan, Meina Kan, and Xilin Chen. 2014. Coarse-to-ffine auto-encoder networks (CFAN) for real-time face alignment. In ECCV. 1\u201316."},{"key":"e_1_3_1_63_2","article-title":"Joint face detection and alignment using multi-task cascaded convolutional networks","volume":"1604","author":"Zhang Kaipeng","year":"2016","unstructured":"Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016. Joint face detection and alignment using multi-task cascaded convolutional networks. CoRR abs\/1604.02878 (2016).","journal-title":"CoRR"},{"key":"e_1_3_1_64_2","first-page":"94","volume-title":"ECCV","author":"Zhang Zhanpeng","year":"2014","unstructured":"Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. 2014. Facial landmark detection by deep multi-task learning. In ECCV. 94\u2013108."},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00360"},{"key":"e_1_3_1_66_2","first-page":"4998","volume-title":"CVPR","author":"Zhu Shizhan","year":"2015","unstructured":"Shizhan Zhu, Cheng Li, Chen Change Loy, and Xiaoou Tang. 2015. Face alignment by coarse-to-fine shape searching. In CVPR. 4998\u20135006."},{"key":"e_1_3_1_67_2","first-page":"3409","volume-title":"CVPR","author":"Zhu Shizhan","year":"2016","unstructured":"Shizhan Zhu, Cheng Li, Chen Change Loy, and Xiaoou Tang. 2016. Unconstrained face alignment via cascaded compositional learning. In CVPR. 3409\u20133417."},{"key":"e_1_3_1_68_2","first-page":"146","volume-title":"CVPR","author":"Zhu Xiangyu","year":"2016","unstructured":"Xiangyu Zhu, Zhen Lei, Xiaoming Liu, Hailin Shi, and Stan Z. Li. 2016. Face alignment across large poses: A 3D solution. In CVPR. 146\u2013155."},{"key":"e_1_3_1_69_2","first-page":"2879","volume-title":"2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, June 16\u201321, 2012","author":"Zhu Xiangxin","year":"2012","unstructured":"Xiangxin Zhu and Deva Ramanan. 2012. Face detection, pose estimation, and landmark localization in the wild. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, June 16\u201321, 2012. 2879\u20132886."},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00023"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3474595","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3474595","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:50Z","timestamp":1750191530000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3474595"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,16]]},"references-count":69,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,5,31]]}},"alternative-id":["10.1145\/3474595"],"URL":"https:\/\/doi.org\/10.1145\/3474595","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2022,2,16]]},"assertion":[{"value":"2020-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-02-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}