{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T15:46:32Z","timestamp":1782315992503,"version":"3.54.5"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T00:00:00Z","timestamp":1782259200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62261160576, 62203024"],"award-info":[{"award-number":["62261160576, 62203024"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Beijing Natural Science Foundation","award":["4252026"],"award-info":[{"award-number":["4252026"]}]},{"name":"Zhishan Young Scholar Program of SEU, Fundamental Research Funds for the Central Universities of China"},{"name":"Big Data Computing Center of Southeast University"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2026,7,31]]},"abstract":"<jats:p>\n                    We propose a novel task,\n                    <jats:bold>Virtual Object Grounding (VOG)<\/jats:bold>\n                    . It aims to predict plausible locations in an image for inserting virtual objects that align with a given textual description. This VOG task can address the challenge of providing region constraints for object insertion in image editing, thereby ensuring the consistency of irrelevant areas in the image. To support this task, we construct\n                    <jats:bold>Virtual Segmentation (VirtualSeg)<\/jats:bold>\n                    dataset, a dataset of over 92,000 samples automatically generated from VrR-VG via a four-step dataset construction pipeline. This pipeline employs CLIP to automatically filter out low-quality data samples, ensuring the quality of VirtualSeg. Furthermore, we propose the\n                    <jats:italic toggle=\"yes\">VirLLaVA<\/jats:italic>\n                    model, a novel VOG framework built upon LLaVA-7B. By equipping the MLLM backbone with two sequences of learnable tokens and a dual grounding module, and by guiding the model during training to learn step-by-step how to locate virtual objects, our method enables it to reason about their positions from textual and visual inputs. Experiments show that VirLLaVA significantly improves performance in VOG, while also offering a promising direction for consistent and automated image editing. The code and dataset are available at\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/Royxia0818\/MLLM_for_VOG\">https:\/\/github.com\/Royxia0818\/MLLM_for_VOG<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1145\/3796717","type":"journal-article","created":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T14:05:27Z","timestamp":1771855527000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Multimodal Large Language Model for Virtual Object Grounding"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-6052-5412","authenticated-orcid":false,"given":"Ziheng","family":"Xia","sequence":"first","affiliation":[{"name":"Southeast University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3539-4295","authenticated-orcid":false,"given":"Chao","family":"Li","sequence":"additional","affiliation":[{"name":"Alibaba Group, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6597-3725","authenticated-orcid":false,"given":"Ding","family":"Ding","sequence":"additional","affiliation":[{"name":"Southeast University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-7528-5952","authenticated-orcid":false,"given":"Ye","family":"Wang","sequence":"additional","affiliation":[{"name":"Southeast University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3138-505X","authenticated-orcid":false,"given":"Hao","family":"Chen","sequence":"additional","affiliation":[{"name":"Southeast University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,24]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Black Forest Labs. 2024. Flux. Retrieved June 20 2025 from https:\/\/github.com\/black-forest-labs\/flux"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01764"},{"key":"e_1_3_2_4_2","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, Vol. 33, 1877\u20131901.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_5_2","unstructured":"Chaorui Deng Deyao Zhu Kunchang Li Chenhui Gou Feng Li Zeyu Wang Shu Zhong Weihao Yu Xiaonan Nie Ziang Song et al. 2025. Emerging properties in unified multimodal pretraining. arXiv:2505.14683. Retrieved from https:\/\/arxiv.org\/abs\/2505.14683"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01601"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01525"},{"key":"e_1_3_2_8_2","unstructured":"Amir Hertz Ron Mokady Jay Tenenbaum Kfir Aberman Yael Pritch and Daniel Cohen-Or. 2022. Prompt-to-prompt image editing with cross attention control. arXiv:2208.01626. Retrieved from https:\/\/arxiv.org\/abs\/2208.01626"},{"key":"e_1_3_2_9_2","first-page":"6840","article-title":"Denoising diffusion probabilistic models","volume":"33","author":"Ho Jonathan","year":"2020","unstructured":"Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, Vol. 33, 6840\u20136851.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00686"},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","first-page":"57","DOI":"10.52202\/075280-0004","article-title":"Bootstrapping vision-language learning with decoupled language pre-training","volume":"36","author":"Jian Yiren","year":"2023","unstructured":"Yiren Jian, Chongyang Gao, and Soroush Vosoughi. 2023. Bootstrapping vision-language learning with decoupled language pre-training. In Advances in Neural Information Processing Systems, Vol. 36, 57\u201372.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00582"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00371"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-016-0981-7"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.00915"},{"key":"e_1_3_2_16_2","first-page":"19730","volume-title":"International Conference on Machine Learning","author":"Li Junnan","year":"2023","unstructured":"Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning. PMLR, 19730\u201319742."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02156"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/iccv.2019.01050"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3581783.3612260"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.52202\/075280-1516"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01789"},{"key":"e_1_3_2_23_2","unstructured":"Shiyu Liu Yucheng Han Peng Xing Fukun Yin Rui Wang Wei Cheng Jiaqi Liao Yingming Wang Honghao Fu Chunrui Han et al. 2025. Step1X-Edit: A practical framework for general image editing. arXiv:2504.17761. Retrieved from https:\/\/arxiv.org\/abs\/2504.17761"},{"key":"e_1_3_2_24_2","unstructured":"Junyu Lu Dixiang Zhang Songxin Zhang Zejian Xie Zhuoyang Song Cong Lin Jiaxing Zhang Bingyi Jing and Pingjian Zhang. 2023. Lyrics: Boosting fine-grained language-vision alignment and comprehension via semantic-aware visual objects. arXiv:2312.05278. Retrieved from https:\/\/arxiv.org\/abs\/2312.05278"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2016.79"},{"key":"e_1_3_2_26_2","unstructured":"Zhiliang Peng Wenhui Wang Li Dong Yaru Hao Shaohan Huang Shuming Ma and Furu Wei. 2023. Kosmos-2: Grounding multimodal large language models to the world. arXiv:2306.14824. Retrieved from https:\/\/arxiv.org\/abs\/2306.14824"},{"key":"e_1_3_2_27_2","unstructured":"Renjie Pi Jiahui Gao Shizhe Diao Rui Pan Hanze Dong Jipeng Zhang Lewei Yao Jianhua Han Hang Xu Lingpeng Kong et al. 2023. DetGPT: Detect what you need via reasoning. arXiv:2305.14167. Retrieved from https:\/\/arxiv.org\/abs\/2305.14167"},{"key":"e_1_3_2_28_2","first-page":"8748","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning. PMLR, 8748\u20138763."},{"key":"e_1_3_2_29_2","unstructured":"Aditya Ramesh Prafulla Dhariwal Alex Nichol Casey Chu and Mark Chen. 2022. Hierarchical text-conditional image generation with clip latents. arXiv:2204.06125. Retrieved from https:\/\/arxiv.org\/abs\/2204.06125"},{"key":"e_1_3_2_30_2","first-page":"8821","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Ramesh Aditya","year":"2021","unstructured":"Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image generation. In Proceedings of the International Conference on Machine Learning. PMLR, 8821\u20138831."},{"key":"e_1_3_2_31_2","first-page":"26374","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Ren Zhongwei","year":"2024","unstructured":"Zhongwei Ren, Zhicheng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. 2024. PixelLM: Pixel reasoning with large multimodal model. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 26374\u201326383."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.52202\/068431-2643"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV51458.2022.00323"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01761"},{"key":"e_1_3_2_36_2","doi-asserted-by":"crossref","first-page":"61501","DOI":"10.52202\/075280-2688","article-title":"VisionLLM: Large language model is also an open-ended decoder for vision-centric tasks","volume":"36","author":"Wang Wenhai","year":"2023","unstructured":"Wenhai Wang, Zhe Chen, Xiaokang Chen, Jiannan Wu, Xizhou Zhu, Gang Zeng, Ping Luo, Tong Lu, Jie Zhou, Yu Qiao, et al. 2023. VisionLLM: Large language model is also an open-ended decoder for vision-centric tasks. In Advances in Neural Information Processing Systems, Vol. 36, 61501\u201361513.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_37_2","first-page":"18313","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Wasserman Navve","year":"2025","unstructured":"Navve Wasserman, Noam Rotstein, Roy Ganz, and Ron Kimmel. 2025. Paint by inpaint: Learning to add image objects by removing them first. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 18313\u201318324."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.00370"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02148"},{"key":"e_1_3_2_40_2","unstructured":"Ziyang Xu Kangsheng Duan Xiaolei Shen Zhifeng Ding Wenyu Liu Xiaohu Ruan Xiaoxin Chen and Xinggang Wang. 2025. PixelHacker: Image inpainting with structural and semantic consistency. arXiv:2504.20438. Retrieved from https:\/\/arxiv.org\/abs\/2504.20438"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01762"},{"key":"e_1_3_2_42_2","unstructured":"Ahmet Burak Yildirim Vedat Baday Erkut Erdem Aykut Erdem and Aysegul Dundar. 2023. Inst-inpaint: Instructing to remove objects with diffusion models. arXiv:2304.03246. Retrieved from https:\/\/arxiv.org\/abs\/2304.03246"},{"key":"e_1_3_2_43_2","first-page":"69","volume-title":":Proceedings of the 14th European Conference on Computer Vision (ECCV \u201916)","author":"Yu Licheng","year":"2016","unstructured":"Licheng Yu, Patrick Poirson, Shan Yang, Alexander C. Berg, and Tamara L. Berg. 2016. Modeling context in referring expressions. In: Proceedings of the 14th European Conference on Computer Vision (ECCV \u201916). Springer, 69\u201385."},{"key":"e_1_3_2_44_2","first-page":"14164","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Zeng Yu","year":"2021","unstructured":"Yu Zeng, Zhe Lin, Huchuan Lu, and Vishal M. Patel. 2021. CR-Fill: Generative image inpainting with auxiliary contextual reconstruction. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 14164\u201314173."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00355"},{"key":"e_1_3_2_46_2","first-page":"52","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Zhang Shilong","year":"2025","unstructured":"Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Yu Liu, Kai Chen, and Ping Luo. 2025. GPT4RoI: Instruction tuning large language model on region-of-interest. In Proceedings of the European Conference on Computer Vision. Springer, 52\u201370."},{"key":"e_1_3_2_47_2","first-page":"195","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Zhuang Junhao","year":"2024","unstructured":"Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. 2024. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting. In Proceedings of the European Conference on Computer Vision. Springer, 195\u2013211."}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3796717","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T14:47:22Z","timestamp":1782312442000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3796717"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,24]]},"references-count":46,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2026,7,31]]}},"alternative-id":["10.1145\/3796717"],"URL":"https:\/\/doi.org\/10.1145\/3796717","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,24]]},"assertion":[{"value":"2025-06-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-19","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}