{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T17:51:14Z","timestamp":1772041874584,"version":"3.50.1"},"reference-count":37,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,8,30]],"date-time":"2022-08-30T00:00:00Z","timestamp":1661817600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"JSPS KAKENHI","award":["JP21H01327"],"award-info":[{"award-number":["JP21H01327"]}]},{"name":"Support Center for Advanced Telecommunications Technology Research, Foundation (SCAT)","award":["JP21H01327"],"award-info":[{"award-number":["JP21H01327"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>In this paper, we propose a privacy-preserving semantic segmentation method that uses encrypted images and models with the vision transformer (ViT), called the segmentation transformer (SETR). The combined use of encrypted images and SETR allows us not only to apply images without sensitive visual information to SETR as query images but to also maintain the same accuracy as that of using plain images. Previously, privacy-preserving methods with encrypted images for deep neural networks have focused on image classification tasks. In addition, the conventional methods result in a lower accuracy than models trained with plain images due to the influence of image encryption. To overcome these issues, a novel method for privacy-preserving semantic segmentation is proposed by using an embedding that the ViT structure has for the first time. In experiments, the proposed privacy-preserving semantic segmentation was demonstrated to have the same accuracy as that of using plain images under the use of encrypted images.<\/jats:p>","DOI":"10.3390\/jimaging8090233","type":"journal-article","created":{"date-parts":[[2022,8,31]],"date-time":"2022-08-31T00:13:56Z","timestamp":1661904836000},"page":"233","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Privacy-Preserving Semantic Segmentation Using Vision Transformer"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8061-3090","authenticated-orcid":false,"given":"Hitoshi","family":"Kiya","sequence":"first","affiliation":[{"name":"Department of Computer Science, Tokyo Metropolitan University, 6-6 Asahigaoka, Hino-shi, Tokyo 191-0065, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3002-7012","authenticated-orcid":false,"given":"Teru","family":"Nagamori","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Tokyo Metropolitan University, 6-6 Asahigaoka, Hino-shi, Tokyo 191-0065, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6515-9109","authenticated-orcid":false,"given":"Shoko","family":"Imaizumi","sequence":"additional","affiliation":[{"name":"Graduate School of Engineering, Chiba University, 1-33 Yayoicho, Chiba 263-8522, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2364-068X","authenticated-orcid":false,"given":"Sayaka","family":"Shiota","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Tokyo Metropolitan University, 6-6 Asahigaoka, Hino-shi, Tokyo 191-0065, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Shokri, R., and Shmatikov, V. (October, January 29). Privacy-preserving deep learning. Proceedings of the 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), Montecello, IL, USA.","DOI":"10.1109\/ALLERTON.2015.7447103"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chaudhari, H., Rachuri, R., and Suresh, A. (2020, January 23\u201326). Trident: Efficient 4PC Framework for Privacy Preserving Machine Learning. Proceedings of the 27th Annual Network and Distributed System Security Symposium (NDSS) 2020, San Diego, CA, USA.","DOI":"10.14722\/ndss.2020.23005"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"139021","DOI":"10.1109\/ACCESS.2019.2939410","article-title":"MOBIUS: Model-Oblivious Binarized Neural Networks","volume":"7","author":"Kitai","year":"2019","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3454","DOI":"10.1109\/TIFS.2020.2988575","article-title":"Federated Learning With Differential Privacy: Algorithms and Performance Analysis","volume":"15","author":"Wei","year":"2020","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2059","DOI":"10.1109\/TIFS.2022.3176191","article-title":"PVD-FL: A Privacy-Preserving and Verifiable Decentralized Federated Learning Framework","volume":"17","author":"Zhao","year":"2022","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e11","DOI":"10.1561\/116.00000048","article-title":"An Overview of Compressible and Learnable Image Transformation with Secret Key and its Applications","volume":"11","author":"Kiya","year":"2022","journal-title":"APSIPA Trans. Signal Inf. Process."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Tanaka, M. (2018, January 19\u201321). Learnable Image Encryption. Proceedings of the 2018 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), Taichung, Taiwan.","DOI":"10.1109\/ICCE-China.2018.8448772"},{"key":"ref_8","unstructured":"Madono, K., Tanaka, M., Masaki, O., and Tetsuji, O. (2020, January 7\u20138). Block-wise Scrambled Image Recognition Using Adaptation Network. Proceedings of the Workshop on Artificial Intelligence of Things (AAAI WS), New York, NY, USA."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"177844","DOI":"10.1109\/ACCESS.2019.2959017","article-title":"Pixel-Based Image Encryption Without Key Management for Privacy-Preserving Deep Neural Networks","volume":"7","author":"Sirichotedumrong","year":"2019","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2709","DOI":"10.1109\/TIFS.2021.3062977","article-title":"Block-wise Image Transformation with Secret Key for Adversarially Robust Defense","volume":"16","author":"AprilPyone","year":"2021","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"66345","DOI":"10.1109\/ACCESS.2022.3185206","article-title":"Privacy-Preserving Deep Learning With Learnable Image Encryption on Medical Images","volume":"10","author":"Huang","year":"2022","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1515","DOI":"10.1109\/TIFS.2018.2881677","article-title":"Encryption-Then-Compression Systems Using Grayscale-Based Image Encryption for JPEG Images","volume":"14","author":"Chuman","year":"2019","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"111318","DOI":"10.1016\/j.chaos.2021.111318","article-title":"Double image encryption algorithm based on neural network and chaos","volume":"152","author":"Man","year":"2021","journal-title":"Chaos Solitons Fractals"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sirichotedumrong, W., and Kiya, H. (2021, January 18\u201322). A GAN-Based Image Transformation Scheme for Privacy-Preserving Deep Neural Networks. Proceedings of the 2020 28th European Signal Processing Conference (EUSIPCO), Virtual.","DOI":"10.23919\/Eusipco47968.2020.9287532"},{"key":"ref_15","unstructured":"Ito, H., AprilPyone, M., and Kiya, H. (2021, January 14\u201317). Access Control Using Spatially Invariant Permutation of Feature Maps for Semantic Segmentation Models. Proceedings of the 2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Tokyo, Japan."},{"key":"ref_16","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 3\u20137). An Image is Worth 16\u00d716 Words: Transformers for Image Recognition at Scale. Proceedings of the 9th International Conference on Learning Representations, ICLR 2021, Virtual Event."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., and Torr, P.H. (2021, January 19\u201325). Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual.","DOI":"10.1109\/CVPR46437.2021.00681"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2079","DOI":"10.1587\/transinf.2015INP0020","article-title":"Privacy-preserving logistic regression with distributed data sources via homomorphic encryption","volume":"99","author":"Aono","year":"2016","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_19","first-page":"1333","article-title":"Privacy-preserving deep learning via additively homomorphic encryption","volume":"13","author":"Aono","year":"2017","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3003","DOI":"10.1109\/TIFS.2019.2911169","article-title":"Privacy-preserving deep learning via weight transmission","volume":"14","author":"Phuong","year":"2019","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_21","unstructured":"Dowlin, N., Gilad-Bachrach, R., Laine, K., Lauter, K., Naehrig, M., and Wernsing, J. (2016). CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy, Microsoft Research. Technical Report MSR-TR-2016-3."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Wang, Y., Lin, J., and Wang, Z. (2018, January 27\u201330). An efficient convolution core architecture for privacy-preserving deep learning. Proceedings of the 2018 IEEE International Symposium on Circuits and Systems (ISCAS), Florence, Italy.","DOI":"10.1109\/ISCAS.2018.8350963"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1849","DOI":"10.1587\/transfun.E102.A.1849","article-title":"Privacy-Preserving Support Vector Machine Computing Using Random Unitary Transformation","volume":"102","author":"Maekawa","year":"2019","journal-title":"IEICE Trans. Fundam. Electron. Commun. Comput. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1571","DOI":"10.1587\/transfun.2020SMP0022","article-title":"A Privacy-Preserving Machine Learning Scheme Using EtC Images","volume":"103","author":"Kawamura","year":"2020","journal-title":"IEICE Trans. Fundam. Electron. Commun. Comput. Sci."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1587\/transinf.2015MUP0007","article-title":"Unitary Transform-Based Template Protection and Its Application to l2-norm Minimization Problems","volume":"99","author":"Nakamura","year":"2016","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_26","unstructured":"Chang, A.H., and Case, B.M. (2020). Attacks on Image Encryption Schemes for Privacy-Preserving Deep Neural Networks. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1109\/MMUL.2022.3168441","article-title":"Privacy-Preserving Image Classification Using an Isotropic Network","volume":"29","author":"AprilPyone","year":"2022","journal-title":"IEEE MultiMedia"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Qi, Z., AprilPyone, M., Kinoshita, Y., and Kiya, H. (2022). Privacy-Preserving Image Classification Using Vision Transformer. arXiv.","DOI":"10.23919\/EUSIPCO55093.2022.9909972"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27\u201330). The Cityscapes Dataset for Semantic Urban Scene Understanding. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1007\/s11263-018-1140-0","article-title":"Semantic Understanding of Scenes Through the ADE20K Dataset","volume":"127","author":"Zhou","year":"2018","journal-title":"Int. J. Comput. Vis."},{"key":"ref_31","unstructured":"Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and Jegou, H. (2021, January 18\u201324). Training data-efficient image transformers & distillation through attention. Proceedings of the 38th International Conference on Machine Learning, PMLR, Proceedings of Machine Learning Research, Virtual."},{"key":"ref_32","first-page":"e10","article-title":"A protection method of trained CNN model with a secret key from unauthorized access","volume":"10","author":"AprilPyone","year":"2021","journal-title":"APSIPA Trans. Signal Inf. Process."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"129385","DOI":"10.1109\/ACCESS.2021.3112684","article-title":"SIA-GAN: Scrambling Inversion Attack Using Generative Adversarial Network","volume":"9","author":"Madono","year":"2021","journal-title":"IEEE Access"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"64629","DOI":"10.1109\/ACCESS.2021.3074968","article-title":"Image to Perturbation: An Image Transformation Network for Generating Visually Protected Images for Privacy-Preserving Deep Neural Networks","volume":"9","author":"Ito","year":"2021","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Chuman, T., Kurihara, K., and Kiya, H. (2017, January 5\u20139). On the security of block scrambling-based ETC systems against jigsaw puzzle solver attacks. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.","DOI":"10.1109\/ICASSP.2017.7952538"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1587\/transinf.2017MUP0001","article-title":"On the Security of Block Scrambling-Based EtC Systems against Extended Jigsaw Puzzle Solver Attacks","volume":"101","author":"Chuman","year":"2018","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_37","unstructured":"Schneier, B., and Sutherland, P. (1995). Applied Cryptography: Protocols, Algorithms, and Source Code in C, John Wiley & Sons, Inc.. [2nd ed.]."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/9\/233\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:20:05Z","timestamp":1760142005000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/9\/233"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,30]]},"references-count":37,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["jimaging8090233"],"URL":"https:\/\/doi.org\/10.3390\/jimaging8090233","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,30]]}}}