{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,14]],"date-time":"2026-06-14T07:11:16Z","timestamp":1781421076463,"version":"3.54.1"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"11","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62202431, 62036009, and 2020YFB1707700"],"award-info":[{"award-number":["62202431, 62036009, and 2020YFB1707700"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2025,11,30]]},"abstract":"<jats:p>\n                    In this article, we delve into a new task known as Small Object Editing (SOE), which focuses on text-based image inpainting within a constrained, small-sized area. Despite the remarkable success have been achieved by current image inpainting approaches, their application to the SOE task generally results in failure cases such as\n                    <jats:italic toggle=\"yes\">Object Missing, Text-Image Mismatch, and Distortion<\/jats:italic>\n                    . These failures stem from the limited use of small-sized objects in training datasets and the down-sampling operations employed by U-Net models, which hinders accurate generation. To overcome these challenges, we introduce a novel training-based approach, SOEDiff, aimed at enhancing the capability of baseline models like StableDiffusion in editing small-sized objects while minimizing training costs. Specifically, our method involves two key components:\n                    <jats:italic toggle=\"yes\">SO-LoRA<\/jats:italic>\n                    , which efficiently fine-tunes low-rank matrices, and\n                    <jats:italic toggle=\"yes\">Cross-scale score distillation<\/jats:italic>\n                    , which leverages high-resolution predictions from the pre-trained teacher diffusion model. Our method presents significant improvements on the test dataset collected from MSCOCO and OpenImage, validating the effectiveness of our proposed method in SOE. In particular, when comparing SOEDiff with SD-I model on the\n                    <jats:italic toggle=\"yes\">OpenImage-small-val<\/jats:italic>\n                    dataset, we observe a 0.99 improvement in CLIP-Score and a reduction of 2.87 in FID.\n                  <\/jats:p>","DOI":"10.1145\/3715915","type":"journal-article","created":{"date-parts":[[2025,1,31]],"date-time":"2025-01-31T10:04:21Z","timestamp":1738317861000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["SOEDiff: Efficient Distillation for Small Object Editing"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9866-669X","authenticated-orcid":false,"given":"Yiming","family":"Wu","sequence":"first","affiliation":[{"name":"College of Computer Science, Zhejiang University of Technology, Hangzhou, China and The University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-4660-5549","authenticated-orcid":false,"given":"Qihe","family":"Pan","sequence":"additional","affiliation":[{"name":"Zhejiang University of Technology, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0796-4078","authenticated-orcid":false,"given":"Zhen","family":"Zhao","sequence":"additional","affiliation":[{"name":"The University of Sydney, Sydney, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8351-0329","authenticated-orcid":false,"given":"Zicheng","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Computer Science, The University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7060-1133","authenticated-orcid":false,"given":"Sifan","family":"Long","sequence":"additional","affiliation":[{"name":"Jilin University, Changchun, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2077-9608","authenticated-orcid":false,"given":"Ronghua","family":"Liang","sequence":"additional","affiliation":[{"name":"Zhejiang University of Technology, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,11,10]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"DreamShaper. 2023. Retrieved from https:\/\/civitai.com\/models\/4384?modelVersionId=131004"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3592450"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01767"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02171"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01197"},{"issue":"3","key":"e_1_3_2_7_2","article-title":"Improving image generation with better captions","volume":"2","author":"Betker James","year":"2023","unstructured":"James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. 2023. Improving image generation with better captions. Computer Science 2, 3 (2023), 8. Retrieved from https:\/\/cdn.openai.com\/papers\/dall-e-3.pdf","journal-title":"Computer Science"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.00846"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01764"},{"key":"e_1_3_2_10_2","doi-asserted-by":"crossref","unstructured":"Goirik Chakrabarty Aditya Chandrasekar Ramya Hebbalaguppe and Prathosh A. P. 2024. LoMOE: Localized multi-object editing via multi-diffusion. arXiv:2403.00437. Retrieved from https:\/\/arxiv.org\/abs\/2403.00437","DOI":"10.1145\/3664647.3681199"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.00630"},{"key":"e_1_3_2_12_2","unstructured":"Guillaume Couairon Jakob Verbeek Holger Schwenk and Matthieu Cord. 2022. DiffEdit: Diffusion-based semantic image editing with mask guidance. arXiv:2210.11427. Retrieved from https:\/\/arxiv.org\/abs\/2210.11427"},{"key":"e_1_3_2_13_2","unstructured":"Patrick Esser Sumith Kulal Andreas Blattmann Rahim Entezari Jonas M\u00fcller Harry Saini Yam Levi Dominik Lorenz Axel Sauer Frederic Boesel et al. 2024. Scaling rectified flow transformers for high-resolution image synthesis. arXiv:2403.03206. Retrieved from https:\/\/arxiv.org\/abs\/2403.03206"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00230"},{"key":"e_1_3_2_15_2","unstructured":"Rohit Gandikota Joanna Materzynska Tingrui Zhou Antonio Torralba and David Bau. 2023. Concept sliders: LoRA adaptors for precise control in diffusion models. arXiv:2311.12092. Retrieved from https:\/\/arxiv.org\/abs\/2311.12092"},{"key":"e_1_3_2_16_2","unstructured":"Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS) Vol. 27."},{"key":"e_1_3_2_17_2","unstructured":"Amir Hertz Ron Mokady Jay Tenenbaum Kfir Aberman Yael Pritch and Daniel Cohen-Or. 2022. Prompt-to-prompt image editing with cross attention control. arXiv:2208.01626. Retrieved from https:\/\/arxiv.org\/abs\/2208.01626"},{"key":"e_1_3_2_18_2","unstructured":"Jonathan Ho Ajay Jain and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS) Vol. 33 6840\u20136851."},{"key":"e_1_3_2_19_2","unstructured":"Edward J. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang and Weizhu Chen. 2021. LoRA: Low-rank adaptation of large language models. arXiv:2106.09685. Retrieved from https:\/\/arxiv.org\/abs\/2106.09685"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00582"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00246"},{"key":"e_1_3_2_22_2","first-page":"12888","volume-title":"Proceedings of the International Conference on Machine Learning (ICML \u201922)","author":"Li Junnan","year":"2022","unstructured":"Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Proceedings of the International Conference on Machine Learning (ICML \u201922). PMLR, 12888\u201312900."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02156"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_3_2_25_2","unstructured":"Simian Luo Yiqin Tan Longbo Huang Jian Li and Hang Zhao. 2023. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv:2310.04378. Retrieved from https:\/\/arxiv.org\/abs\/2310.04378"},{"key":"e_1_3_2_26_2","unstructured":"Simian Luo Yiqin Tan Suraj Patil Daniel Gu Patrick von Platen Apolin\u00e1rio Passos Longbo Huang Jian Li and Hang Zhao. 2023. LCM-LoRA: A universal stable-diffusion acceleration module. arXiv:2311.05556. Retrieved from https:\/\/arxiv.org\/abs\/2311.05556"},{"key":"e_1_3_2_27_2","unstructured":"Hayk Manukyan Andranik Sargsyan Barsegh Atanyan Zhangyang Wang Shant Navasardyan and Humphrey Shi. 2023. HD-Painter: High-resolution and prompt-faithful text-guided image inpainting with diffusion models. arXiv:2312.14091. Retrieved from https:\/\/arxiv.org\/abs\/2312.14091"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00585"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","unstructured":"Chong Mou Xintao Wang Liangbin Xie Yanze Wu Jian Zhang Zhongang Qi and Ying Shan. 2024. T2I-Adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence Vol. 38. 4296\u20134304. 10.1609\/aaai.v38i5.28226","DOI":"10.1609\/aaai.v38i5.28226"},{"key":"e_1_3_2_30_2","unstructured":"Alex Nichol Prafulla Dhariwal Aditya Ramesh Pranav Shyam Pamela Mishkin Bob McGrew Ilya Sutskever and Mark Chen. 2021. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv:2112.10741. Retrieved from https:\/\/arxiv.org\/abs\/2112.10741"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3664647.3680896"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00387"},{"key":"e_1_3_2_33_2","unstructured":"Dustin Podell Zion English Kyle Lacey Andreas Blattmann Tim Dockhorn Jonas M\u00fcller Joe Penna and Robin Rombach. 2023. SDXL: Improving latent diffusion models for high-resolution image synthesis. arXiv:2307.01952. Retrieved from https:\/\/arxiv.org\/abs\/2307.01952"},{"key":"e_1_3_2_34_2","unstructured":"Yujia Qin Xiaozhi Wang Yusheng Su Yankai Lin Ning Ding Jing Yi Weize Chen Zhiyuan Liu Juanzi Li Lei Hou et al. 2021. Exploring universal intrinsic task subspace via prompt tuning. arXiv:2110.07867. Retrieved from https:\/\/arxiv.org\/abs\/2110.07867"},{"key":"e_1_3_2_35_2","first-page":"8748","volume-title":"Proceedings of the International Conference on Machine Learning (ICML \u201921)","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning (ICML \u201921). PMLR, 8748\u20138763."},{"key":"e_1_3_2_36_2","unstructured":"Colin Raffel Noam Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21 140 (2020) 1\u201367."},{"key":"e_1_3_2_37_2","unstructured":"Aditya Ramesh Prafulla Dhariwal Alex Nichol Casey Chu and Mark Chen. 2022. Hierarchical text-conditional image generation with CLIP latents. arXiv:2204.06125. Retrieved from https:\/\/arxiv.org\/abs\/2204.06125"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02155"},{"key":"e_1_3_2_40_2","unstructured":"Chitwan Saharia William Chan Saurabh Saxena Lala Li Jay Whang Emily L. Denton Kamyar Ghasemipour Raphael Gontijo Lopes Burcu Karagol Ayan Tim Salimans et al. 2022. Photorealistic text-to-image diffusion models with deep language understanding. In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS) Vol. 35 36479\u201336494."},{"key":"e_1_3_2_41_2","unstructured":"Tim Salimans and Jonathan Ho. 2022. Progressive distillation for fast sampling of diffusion models. arXiv:2202.00512. Retrieved from https:\/\/arxiv.org\/abs\/2202.00512"},{"key":"e_1_3_2_42_2","unstructured":"Axel Sauer Dominik Lorenz Andreas Blattmann and Robin Rombach. 2023. Adversarial diffusion distillation. arXiv:2311.17042. Retrieved from https:\/\/arxiv.org\/abs\/2311.17042"},{"key":"e_1_3_2_43_2","unstructured":"James Seale Smith Yen-Chang Hsu Lingyu Zhang Ting Hua Zsolt Kira Yilin Shen and Hongxia Jin. 2023. Continual diffusion: Continual customization of text-to-image diffusion with C-LoRA. arXiv:2304.06027. Retrieved from https:\/\/arxiv.org\/abs\/2304.06027"},{"key":"e_1_3_2_44_2","unstructured":"Jiaming Song Chenlin Meng and Stefano Ermon. 2020. Denoising diffusion implicit models. arXiv:2010.02502. Retrieved from https:\/\/arxiv.org\/abs\/2010.02502"},{"key":"e_1_3_2_45_2","unstructured":"Yang Song Prafulla Dhariwal Mark Chen and Ilya Sutskever. 2023. Consistency models. arXiv:2303.01469. Retrieved from https:\/\/arxiv.org\/abs\/2303.01469"},{"key":"e_1_3_2_46_2","volume-title":"Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS)","volume":"32","author":"Song Yang","year":"2019","unstructured":"Yang Song and Stefano Ermon. 2019. Generative modeling by estimating gradients of the data distribution. In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), Vol. 32, 11918\u2013 11930."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01602"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00191"},{"key":"e_1_3_2_49_2","doi-asserted-by":"crossref","unstructured":"Hanzhang Wang Deming Zhai Xiong Zhou Junjun Jiang and Xianming Liu. 2024 Mix-DDPM: Enhancing diffusion models through fitting mixture noise with global stochastic offset. ACM Transactions on Multimedia Computing Communications and Applications 20 9 (2024) 1\u201324.","DOI":"10.1145\/3672080"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00143"},{"key":"e_1_3_2_51_2","unstructured":"Ahmet Burak Yildirim Vedat Baday Erkut Erdem Aykut Erdem and Aysegul Dundar. 2023. Inst-inpaint: Instructing to remove objects with diffusion models. arXiv:2304.03246. Retrieved from https:\/\/arxiv.org\/abs\/2304.03246"},{"key":"e_1_3_2_52_2","unstructured":"Tao Yu Runseng Feng Ruoyu Feng Jinming Liu Xin Jin Wenjun Zeng and Zhibo Chen. 2023. Inpaint anything: Segment anything meets image inpainting. arXiv:2304.06790. Retrieved from https:\/\/arxiv.org\/abs\/2304.06790"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00089"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00355"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00595"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3715915","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,10]],"date-time":"2025-11-10T14:50:58Z","timestamp":1762786258000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715915"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,10]]},"references-count":54,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2025,11,30]]}},"alternative-id":["10.1145\/3715915"],"URL":"https:\/\/doi.org\/10.1145\/3715915","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,10]]},"assertion":[{"value":"2024-06-18","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}