{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T22:13:13Z","timestamp":1770329593321,"version":"3.49.0"},"reference-count":44,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,2,1]],"date-time":"2023-02-01T00:00:00Z","timestamp":1675209600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Minister of Science and Technology, Taiwan","award":["MOST 111-2221-E-002-134-MY3"],"award-info":[{"award-number":["MOST 111-2221-E-002-134-MY3"]}]},{"name":"National Center for High-performance Computing in Taiwan","award":["MOST 111-2221-E-002-134-MY3"],"award-info":[{"award-number":["MOST 111-2221-E-002-134-MY3"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Privacy protection data processing has been critical in recent years when pervasively equipped mobile devices could easily capture high-resolution personal images and videos that may disclose personal information. We propose a new controllable and reversible privacy protection system to address the concern in this work. The proposed scheme can automatically and stably anonymize and de-anonymize face images with one neural network and provide strong security protection with multi-factor identification solutions. Furthermore, users can include other attributes as identification factors, such as passwords and specific facial attributes. Our solution lies in a modified conditional-GAN-based training framework, the Multi-factor Modifier (MfM), to simultaneously accomplish the function of multi-factor facial anonymization and de-anonymization. It can successfully anonymize face images while generating realistic faces satisfying the conditions specified by the multi-factor features, such as gender, hair colors, and facial appearance. Furthermore, MfM can also de-anonymize de-identified faces to their corresponding original ones. One crucial part of our work is design of physically meaningful information-theory-based loss functions, which include mutual information between authentic and de-identification images and mutual information between original and re-identification images. Moreover, extensive experiments and analyses show that, with the correct multi-factor feature information, the MfM can effectively achieve nearly perfect reconstruction and generate high-fidelity and diverse anonymized faces to defend attacks from hackers better than other methods with compatible functionalities. Finally, we justify the advantages of this work through perceptual quality comparison experiments. Our experiments show that the resulting LPIPS (with a value of 0.35), FID (with a value of 28), and SSIM (with a value of 0.95) of MfM demonstrate significantly better de-identification effects than state-of-the-art works. Additionally, the MfM we designed can achieve re-identification, which improves real-world practicability.<\/jats:p>","DOI":"10.3390\/e25020272","type":"journal-article","created":{"date-parts":[[2023,2,2]],"date-time":"2023-02-02T02:21:12Z","timestamp":1675304472000},"page":"272","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Towards a Controllable and Reversible Privacy Protection System for Facial Images through Enhanced Multi-Factor Modifier Networks"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6493-7607","authenticated-orcid":false,"given":"Yi-Lun","family":"Pan","sequence":"first","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Taiwan University, Taipei 116, Taiwan"},{"name":"National Center for High-Performance Computing, Hsinchu 300, Taiwan"}]},{"given":"Jun-Cheng","family":"Chen","sequence":"additional","affiliation":[{"name":"Research Center for Information Technology Innovation, Academia Sinica, Taipei 115, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3631-1551","authenticated-orcid":false,"given":"Ja-Ling","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Taiwan University, Taipei 116, Taiwan"},{"name":"Graduate Institute of Networking and Multimedia, National Taiwan University, Taipei 106, Taiwan"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Pan, Y.L., Chen, J.C., and Wu, J.L. (2021, January 5\u20139). A Multi-Factor Combinations Enhanced Reversible Privacy Protection System for Facial Images. Proceedings of the 2021 IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, China.","DOI":"10.1109\/ICME51207.2021.9428264"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ng, H.W., and Winkler, S. (2014, January 27\u201330). A Data-driven Approach to Cleaning Large Face Datasets. Proceedings of the IEEE international conference on image processing (ICIP), Paris, France.","DOI":"10.1109\/ICIP.2014.7025068"},{"key":"ref_3","unstructured":"Yi, D., Lei, Z., Liao, S., and Li, S.Z. (2014). Learning Face Representation from Scratch. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Liu, Z., Luo, P., Wang, X., and Tang, X. (2015, January 7\u201313). Deep Learning Face Attributes in the Wild. Proceedings of the International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.425"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ren, Z., Yong, J.L., and Ryoo, M.S. (2018, January 8\u201314). Learning to anonymize faces for privacy preserving action detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01246-5_38"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Boyle, M., Edwards, C., and Greenberg, S. (2000, January 2\u20136). The Effects of Filtered Video on Awareness and Privacy. Proceedings of the ACM Conference on Computer Supported Cooperative Work, Philadelphia, PA, USA.","DOI":"10.1145\/358916.358935"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1143518.1143519","article-title":"Blur Filtration Fails to Preserve Privacy for Home-Based Video Conferencing","volume":"13","author":"Neustaedter","year":"2006","journal-title":"ACM Trans. Comput. Hum. Interact."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kanade, T., Jain, A., and Ratha, N. (2005). Audio- and Video-Based Biometric Person Authentication, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-540-31638-1"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Seo, J., Hwang, S., and Suh, Y.-H. (2008, January 14\u201316). A Reversible Face De-Identification Method based on Robust Hashing. Proceedings of the International Conference on Consumer Electronics, Algarve, Portugal.","DOI":"10.1109\/ICCE.2008.4587904"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Senior, A. (2009). Protecting Privacy in Video Surveillance, Springer.","DOI":"10.1007\/978-1-84882-301-3"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1109\/TKDE.2005.32","article-title":"Preserving privacy by de-identifying face images","volume":"17","author":"Newton","year":"2005","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Jourabloo, A., Yin, X., and Liu, X. (2015, January 19\u201322). Attribute Preserved Face De-identification. Proceedings of the 2015 International Conference on Biometrics (ICB), Phuket, Thailand.","DOI":"10.1109\/ICB.2015.7139096"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"4177","DOI":"10.1016\/j.eswa.2015.01.041","article-title":"Visual Privacy Protection Methods: A Survey","volume":"42","author":"Chaaraoui","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1142\/S0218488502001648","article-title":"k-anonymity: A Model for Protecting Privacy","volume":"10","author":"Sweeney","year":"2002","journal-title":"Int. J. Uncertain. Fuzziness Knowl. -Based Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Meden, B., \u017diga Emer\u0161ic, Z., \u0160truc, V.V., and Peer, P. (2018). K-same-net: K-anonymity with Generative Deep Neural Networks for Face Deidentification. Entropy, 20.","DOI":"10.3390\/e20010060"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Pan, Y.-L., Haung, M.-J., Ding, K.-T., Wu, J.-L., and Jang, J.-S.R. (2019, January 18\u201321). K-Same-Siamese-GAN: K-Same Algorithm with Generative Adversarial Network for Facial Image De-identification with Hyperparameter Tuning and Mixed Precision Training. Proceedings of the 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Taipei, Taiwan.","DOI":"10.1109\/AVSS.2019.8909866"},{"key":"ref_17","unstructured":"Jeong, Y., Choi, J., Kim, S., Ro, Y., Oh, T.-H., Kim, D., Ha, H., and Yoon, S. (2021). FICGAN: Facial Identity Controllable GAN for De-identification. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yamac, M., Ahishali, M., Passalis, N., Raitoharju, J., Sankur, B., and Gabbouj, M. (2019, January 2\u20136). Reversible Privacy Preservation using Multi-level Encryption and Compressive Sensing. Proceedings of the 27th European Signal Processing Conference, A Coru\u00f1a, Spain.","DOI":"10.23919\/EUSIPCO.2019.8903056"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Gu, X., Luo, W., Ryooand, M., and Lee, Y. (2020). Password-conditioned Anonymization and Deanonymization with Face Identity Transformers. arXiv.","DOI":"10.1007\/978-3-030-58592-1_43"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Im, D.H., and Seo, Y.S. (2022, January 19\u201321). FaceBERT: Face De-Identification Using VQGAN and BERT. Proceedings of the 2022 13th International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Republic of Korea.","DOI":"10.1109\/ICTC55196.2022.9952883"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"8417","DOI":"10.1016\/j.aej.2022.02.007","article-title":"Generating De-identification facial images based on the attention models and adversarial examples","volume":"61","author":"Yang","year":"2022","journal-title":"Alex. Eng. J."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhai, L., Guo, Q., Xie, X., Ma, L., Wang, Y.E., and Liu, Y. (2022, January 10\u201314). A3GAN: Attribute-Aware Anonymization Networks for Face De-identification. Proceedings of the 30th ACM International Conference on Multimedia (MM \u203222). Association for Computing Machinery, New York, NY, USA.","DOI":"10.1145\/3503161.3547757"},{"key":"ref_23","unstructured":"Seyyed, K., and Shirin, N. (2022). StyleGAN as a Utility-Preserving Face De-identification Method. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e7554","DOI":"10.1002\/cpe.7554","article-title":"Face image de-identification by feature space adversarial perturbation","volume":"35","author":"Xue","year":"2022","journal-title":"Concurrency Computat Pract Exper."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Bao, J., Chen, D., Wen, F., Li, H., and Hua, G. (2018, January 18\u201323). Towards Open-set Identity Preserving Face Synthesis. Proceedings of the Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00702"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Karras, T., Laine, S., and Aila, T. (2018). A style-based generator architecture for generative adversarial networks. arXiv.","DOI":"10.1109\/CVPR.2019.00453"},{"key":"ref_27","unstructured":"Larsen, A.B.L., S\u00f8nderby, S.K., Larochelle, H., and Winther, O. (2015). Autoencoding beyond pixels using a learned similarity metric. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Shen, W., and Liu, R. (2017, January 21\u201326). Learning Residual Images for Face Attribute Manipulation. Proceedings of the Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.135"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., and Choo, J. (2018, January 18\u201322). Star- GAN: Unified generative adversarial networks for multi-domain Image- to-Image translation. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00916"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1254","DOI":"10.1109\/TPAMI.2019.2950198","article-title":"Exploring Explicit Domain Supervision for Latent Space Disentanglement in Unpaired Image-to-Image Translation","volume":"43","author":"Lin","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"63514","DOI":"10.1109\/ACCESS.2020.2982224","article-title":"A State-of-the-Art Review on Image Synthesis with Generative Adversarial Networks","volume":"8","author":"Wang","year":"2020","journal-title":"Comput. Sci. IEEE Access"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A. (2017, January 21\u201326). Image-to-image Translation with Conditional Adversarial Networks. Proceedings of the Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1016\/j.patcog.2004.04.010","article-title":"Illuminant and device invariant colour using histogram equalisation","volume":"38","author":"Finlayson","year":"2005","journal-title":"Pattern Recognit."},{"key":"ref_34","unstructured":"Cao, X., Wei, Y., Wen, F., and Sun, J. (2012, January 16\u201321). Face alignment by Explicit Shape Regression. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"72063","DOI":"10.1109\/ACCESS.2018.2882070","article-title":"Automatic Image Alignment Using Principal Component Analysis","volume":"6","author":"Rehman","year":"2018","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Shen, Y., and Zhou, B. (2021, January 20\u201325). Closed-Form Factorization of Latent Semantics in GANs. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00158"},{"key":"ref_38","unstructured":"Cheng, P., Hao, W., Dai, S., Liu, J., Gan, Z., and Carin, L. (2020, January 12\u201318). CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information. Proceedings of the 37th International Conference on Machine Learning (ICML 2020) Will be Held in Vienna, Virtual."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Deng, J., and Guo, N. (2019, January 15\u201320). Arcface: Additive angular margin loss for deep face recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00482"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Schroff, F., and Kalenichenko, D. (2015, January 7\u201312). Facenet: A unified embedding for face recognition and clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"ref_41","unstructured":"Xie, X., Jones, M.W., and Tam, G.K.L. Deep Face Recognition. Proceedings of the British Machine Vision Conference (BMVC)."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Choi, Y., Uh, J., and Ha, J. (2020, January 13\u201319). Stargan v2: Diverse image synthesis for multiple domains. Proceedings of the The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00821"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Maximov, M., and Elezi, I. (2020, January 13\u201319). Conditional identity anonymization generative adversarial networks. Proceedings of the The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00549"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Cao, J., Liu, B., Wen, Y., Xie, R., and Song, L. (2021, January 10\u201317). Personalized and Invertible Face De-identification by Disentangled Identity Information Manipulation. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00332"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/2\/272\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:21:24Z","timestamp":1760120484000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/2\/272"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,1]]},"references-count":44,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["e25020272"],"URL":"https:\/\/doi.org\/10.3390\/e25020272","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,1]]}}}