{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T13:28:20Z","timestamp":1792675700427,"version":"4.5.0"},"reference-count":73,"publisher":"Association for Computing Machinery (ACM)","issue":"10","license":[{"start":{"date-parts":[[2024,10,30]],"date-time":"2024-10-30T00:00:00Z","timestamp":1730246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation","doi-asserted-by":"crossref","award":["62071449 and U20A20184"],"award-info":[{"award-number":["62071449 and U20A20184"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"crossref","award":["E2ET1104"],"award-info":[{"award-number":["E2ET1104"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,10,31]]},"abstract":"<jats:p>Image compression aims to minimize the amount of data in image representation while maintaining a certain visual quality for humans, which is an essential technique for storage and transmission. Recently, along with the development of computer vision, machines have become another primary receiver for images and require compressed images at a certain quality level, which may be different from that of human vision. In many scenarios, compressed images should serve both human and machine vision tasks, but few compression methods are designed for both goals simultaneously. In this article, we propose a unified and scalable deep image compression (USDIC) framework that jointly optimizes the image quality according to human and machine vision in an end-to-end style. For the encoder, we propose an information splitting mechanism (ISM) to separate images into semantic and visual features, which mainly aims at machine analysis and human viewing tasks. For the decoder, we design a scalable decoding architecture. The encoded semantic feature is first decoded for machine analysis tasks, and the image is decoded and reconstructed further by leveraging the decoded semantic features. Herein, to further remove the redundancy between the semantic and visual features of images, we propose a scalable entropy model (SEM) with a joint optimization strategy to reconstruct the image using the two kinds of decoded features. Extensive experimental results show that the proposed USDIC achieves much better performance on the image analysis task while maintaining competitive performance on the traditional image reconstruction task compared with popular image compression methods.<\/jats:p>","DOI":"10.1145\/3678472","type":"journal-article","created":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T15:43:02Z","timestamp":1721230982000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Unified and Scalable Deep Image Compression Framework for Human and Machine"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0418-0689","authenticated-orcid":false,"given":"Gai","family":"Zhang","sequence":"first","affiliation":[{"name":"University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7517-3868","authenticated-orcid":false,"given":"Xinfeng","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7359-1057","authenticated-orcid":false,"given":"Lv","family":"Tang","sequence":"additional","affiliation":[{"name":"University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,10,30]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00031"},{"key":"e_1_3_1_3_2","first-page":"20014","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","volume":"34","author":"Ali Alaaeldin","year":"2021","unstructured":"Alaaeldin Ali, Hugo Touvron, Mathilde Caron, Piotr Bojanowski, Matthijs Douze, Armand Joulin, Ivan Laptev, Natalia Neverova, Gabriel Synnaeve, Jakob Verbeek, and Herv\u00e9 Jegou. 2021. XCiT: Cross-covariance image transformers. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 34, 20014\u201320027."},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/MMUL.2023.3245919"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/VCIP47243.2019.8965808"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i1.19884"},{"key":"e_1_3_1_7_2","first-page":"101","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Ball\u00e9 Johannes","year":"2016","unstructured":"Johannes Ball\u00e9, Valero Laparra, and Eero P. Simoncelli. 2016. End-to-end optimized image compression. In Proceedings of the International Conference on Learning Representations, 101\u2013128."},{"key":"e_1_3_1_8_2","first-page":"220","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Ball\u00e9 Johannes","year":"2018","unstructured":"Johannes Ball\u00e9, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. 2018. Variational image compression with a scale hyperprior. In Proceedings of the International Conference on Learning Representations, 220\u2013243."},{"key":"e_1_3_1_9_2","unstructured":"Fabrice Bellard. 2015. BPG image format. Retrieved July 27 2024 from http:\/\/bellard.org\/bpg\/"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2021.3101953"},{"key":"e_1_3_1_11_2","unstructured":"Jean B\u00e9gaint Fabien Racap\u00e9 Simon Feltman and Akshay Pushparaja. 2020. CompressAI: A PyTorch library and evaluation platform for end-to-end compression research. arXiv:2011.03029. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2011.03029"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2019.00066"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/DCC50243.2021.00024"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.02129"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2941660"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00796"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/MMSP.2018.8547134"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP42928.2021.9506712"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2022.3160602"},{"key":"e_1_3_1_20_2","first-page":"309","author":"Choi J.","year":"2020","unstructured":"J. Choi and B. Han. 2020. Task-Aware Quantization Network for JPEG Image Compression: Computer Vision (ECCV \u201920), 309\u2013324","journal-title":"Task-Aware Quantization Network for JPEG Image Compression: Computer Vision (ECCV \u201920)"},{"key":"e_1_3_1_21_2","first-page":"3743","volume-title":"Proceedings of the 25th IEEE International Conference on Image Processing (ICIP \u201918)","author":"Hyomin Choi","year":"2018","unstructured":"Choi Hyomin and Baji\u0107 Ivan V. 2018. Deep feature compression for collaborative object detection. In Proceedings of the 25th IEEE International Conference on Image Processing (ICIP \u201918). IEEE, 3743\u20133747."},{"key":"e_1_3_1_22_2","unstructured":"U Cisco. 2020. Cisco Annual Internet Report (2018\u20132023) White Paper. Cisco San Jose CA."},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01039"},{"key":"e_1_3_1_24_2","volume-title":"Proceedings of the 9th International Conference on Learning Representations (ICLR \u201921)","author":"Dosovitskiy Alexey","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 202. An image is worth 16x16 words: Transformers for image recognition at scale. In Proceedings of the 9th International Conference on Learning Representations (ICLR \u201921). Retrieved firom https:\/\/openreview.net\/forum?id=YicbFdNTTy"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.3016485"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2015.2500034"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/MMUL.2018.2873844"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-19836-6_29"},{"key":"e_1_3_1_29_2","unstructured":"Wen Gao Shan Liu Xiaozhong Xu Manouchehr Rafie Yuan Zhang and Igor Curcio. 2021. Recent standard development activities on video coding for machines. arXiv:2105.12653. Retrieved from https:\/\/arxiv.org\/abs\/2105.12653"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","unstructured":"Gregory Griffin Alex Holub and Pietro Perona. 2022. Caltech 256. DOI: 10.22002\/D1.20087","DOI":"10.22002\/D1.20087"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00563"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICME46284.2020.9102750"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICME51207.2021.9428228"},{"key":"e_1_3_1_35_2","first-page":"248","volume-title":"Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition","author":"Jia D.","year":"2009","unstructured":"D. Jia, D. Wei, R. Socher, L. J. Li, L. Kai, and F. F. Li. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248\u2013255."},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2017.2734838"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00461"},{"key":"e_1_3_1_38_2","volume-title":"Proceedings of the 3rd International Conference on Learning Representations (ICLR \u201915)","author":"Kingma Diederik P.","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR \u201915). Yoshua Bengio and Yann LeCun (Eds.). Retrieved from http:\/\/arxiv.org\/abs\/1412.6980"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP39728.2021.9414465"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02166"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR56361.2022.9956532"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01383"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/DCC52660.2022.00080"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/DCC50243.2021.00049"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2019.2910119"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2018.2873102"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299155"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00492"},{"key":"e_1_3_1_50_2","first-page":"11913","volume-title":"Proceedings of theAdvances in Neural Information Processing Systems","volume":"33","author":"Mentzer Fabian","year":"2020","unstructured":"Fabian Mentzer, George D Toderici, Michael Tschannen, and Eirikur Agustsson. 2020. High-fidelity generative image compression. In Proceedings of theAdvances in Neural Information Processing Systems, Vol. 33, 11913\u201311924."},{"key":"e_1_3_1_51_2","first-page":"10794","article-title":"Joint autoregressive and hierarchical priors for learned image compression","volume":"31","author":"Minnen David","year":"2018","unstructured":"David Minnen, Johannes Ball\u00e9, and George Toderici. 2018. Joint autoregressive and hierarchical priors for learned image compression. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 31, 10794\u201310803.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_52_2","article-title":"JPEG still image data compression standard","author":"Pennebaker W. B.","year":"1993","unstructured":"W. B. Pennebaker and J. L. Mitchell. 1993. JPEG still image data compression standard. Van Nostrand Reinhold (1993).","journal-title":"Van Nostrand Reinhold"},{"key":"e_1_3_1_53_2","unstructured":"Yichen Qian Ming Lin Xiuyu Sun Zhiyu Tan and Rong Jin. 2022. Entroformer: A transformer-based entropy model for learned image compression. arXiv:2202.05492. Retrieved from https:\/\/arxiv.org\/abs\/2202.05492"},{"key":"e_1_3_1_54_2","first-page":"286","volume-title":"JPEG2000: Image Compression Fundamentals, Standards and Practice","author":"Rabbani Majid","year":"2002","unstructured":"Majid Rabbani. 2002. JPEG2000: Image Compression Fundamentals, Standards and Practice. Vol. 11. International Society for Optics and Photonics, 286 pages."},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01044"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/MMSP.2013.6659301"},{"key":"e_1_3_1_57_2","first-page":"2922","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Rippel Oren","year":"2017","unstructured":"Oren Rippel and Lubomir Bourdev. 2017. Real-time adaptive image compression. In Proceedings of the International Conference on Machine Learning. PMLR, 2922\u20132930."},{"key":"e_1_3_1_58_2","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1109\/PCS.2018.8456298","article-title":"Generative compression","author":"Santurkar Shibani","year":"2018","unstructured":"Shibani Santurkar, David Budden, and Nir Shavit. 2018. Generative compression. In Proceedings of the IEEE Picture Coding Symposium (PCS \u201918). 258\u2013262.","journal-title":"Proceedings of the IEEE Picture Coding Symposium (PCS \u201918)"},{"key":"e_1_3_1_59_2","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556. Retrieved from https:\/\/arxiv.org\/abs\/1409.1556"},{"key":"e_1_3_1_60_2","unstructured":"George Toderici Sean M. O\u2019Malley Sung J. Hwang Damien Vincent David Minnen Shumeet Baluja Michele Covell and Rahul Sukthankar. 2015. Variable rate image compression with recurrent neural networks. arXiv:1511.06085. Retrieved from https:\/\/arxiv.org\/abs\/1511.06085"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.577"},{"key":"e_1_3_1_62_2","unstructured":"Robert Torfason Fabian Mentzer Eirikur Agustsson Michael Tschannen Radu Timofte and Luc Van Gool. 2018. Towards image understanding from deep compression without decoding. arXiv:1803.06131. Retrieved from https:\/\/arxiv.org\/abs\/1803.06131"},{"key":"e_1_3_1_63_2","first-page":"5933","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","volume":"31","author":"Tschannen Michael","year":"2018","unstructured":"Michael Tschannen, Eirikur Agustsson, and Mario Lucic. 2018. Deep generative models for distribution-preserving lossy compression. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 31, 5933\u20135944."},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01686"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP39728.2021.9413506"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2021.3094300"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/OJCAS.2021.3126061"},{"key":"e_1_3_1_68_2","unstructured":"Shurun Wang Wenhan Yang and Shiqi Wang. 2020. End-to-end facial deep learning feature compression with teacher-student enhancement. arXiv:2002.03627. Retrieved from https:\/\/arxiv.org\/abs\/2002.03627"},{"key":"e_1_3_1_69_2","unstructured":"Ross Wightman. 2019. PyTorch Image Models. Retrieved from https:\/\/github.com\/rwightman\/pytorch-image-models"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3474085.3475213"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2021.3121131"},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01505-4"},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2016.2629447"},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2018.2867067"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678472","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3678472","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:04:18Z","timestamp":1750291458000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678472"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,30]]},"references-count":73,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2024,10,31]]}},"alternative-id":["10.1145\/3678472"],"URL":"https:\/\/doi.org\/10.1145\/3678472","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,30]]},"assertion":[{"value":"2024-01-20","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}