{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T15:49:15Z","timestamp":1778255355555,"version":"3.51.4"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,7,19]],"date-time":"2024-07-19T00:00:00Z","timestamp":1721347200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2024,7,19]]},"abstract":"<jats:p>Text-to-image models offer a new level of creative flexibility by allowing users to guide the image generation process through natural language. However, using these models to consistently portray<jats:italic>the same<\/jats:italic>subject across diverse prompts remains challenging. Existing approaches fine-tune the model to teach it new words that describe specific user-provided subjects or add image conditioning to the model. These methods require lengthy persubject optimization or large-scale pre-training. Moreover, they struggle to align generated images with text prompts and face difficulties in portraying multiple subjects. Here, we present<jats:italic>ConsiStory<\/jats:italic>, a<jats:italic>training-free<\/jats:italic>approach that enables consistent subject generation by sharing the internal activations of the pretrained model. We introduce a subject-driven shared attention block and correspondence-based feature injection to promote subject consistency between images. Additionally, we develop strategies to encourage layout diversity while maintaining subject consistency. We compare<jats:italic>ConsiStory<\/jats:italic>to a range of baselines, and demonstrate state-of-the-art performance on subject consistency and text alignment, without requiring a single optimization step. Finally,<jats:italic>ConsiStory<\/jats:italic>can naturally extend to multi-subject scenarios, and even enable training-free<jats:italic>personalization<\/jats:italic>for common objects.<\/jats:p>","DOI":"10.1145\/3658157","type":"journal-article","created":{"date-parts":[[2024,7,19]],"date-time":"2024-07-19T14:47:57Z","timestamp":1721400477000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":68,"title":["Training-Free Consistent Text-to-Image Generation"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-8042-0428","authenticated-orcid":false,"given":"Yoad","family":"Tewel","sequence":"first","affiliation":[{"name":"NVIDIA Research, Tel Aviv, Israel"},{"name":"Tel Aviv University, Tel Aviv, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-2099-6263","authenticated-orcid":false,"given":"Omri","family":"Kaduri","sequence":"additional","affiliation":[{"name":"Independent Scientist, Tel Aviv, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4875-965X","authenticated-orcid":false,"given":"Rinon","family":"Gal","sequence":"additional","affiliation":[{"name":"NVIDIA Research, Tel Aviv, Israel"},{"name":"Tel Aviv University, Tel Aviv, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-9897-0022","authenticated-orcid":false,"given":"Yoni","family":"Kasten","sequence":"additional","affiliation":[{"name":"NVIDIA Research, Tel Aviv, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5578-8892","authenticated-orcid":false,"given":"Lior","family":"Wolf","sequence":"additional","affiliation":[{"name":"Tel Aviv University, Tel Aviv, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9164-5303","authenticated-orcid":false,"given":"Gal","family":"Chechik","sequence":"additional","affiliation":[{"name":"NVIDIA Research, Tel Aviv, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3817-3698","authenticated-orcid":false,"given":"Yuval","family":"Atzmon","sequence":"additional","affiliation":[{"name":"NVIDIA Research, Tel Aviv, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,7,19]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Yuval Alaluf Daniel Garibi Or Patashnik Hadar Averbuch-Elor and Daniel Cohen-Or. 2023. Cross-Image Attention for Zero-Shot Appearance Transfer. arXiv:2311.03335 [cs.CV]","DOI":"10.1145\/3641519.3657423"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3610548.3618154"},{"key":"e_1_2_1_3_1","volume-title":"The Chosen One: Consistent Characters in Text-to-Image Diffusion Models. arXiv preprint arXiv:2311.10093","author":"Avrahami Omri","year":"2023","unstructured":"Omri Avrahami, Amir Hertz, Yael Vinker, Moab Arar, Shlomi Fruchter, Ohad Fried, Daniel Cohen-Or, and Dani Lischinski. 2023b. The Chosen One: Consistent Characters in Text-to-Image Diffusion Models. arXiv preprint arXiv:2311.10093 (2023)."},{"key":"e_1_2_1_4_1","volume-title":"eDiff-I: Text-to-Image Diffusion Models with Ensemble of Expert Denoisers. arXiv preprint arXiv:2211.01324","author":"Balaji Yogesh","year":"2022","unstructured":"Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, Bryan Catanzaro, Tero Karras, and Ming-Yu Liu. 2022. eDiff-I: Text-to-Image Diffusion Models with Ensemble of Expert Denoisers. arXiv preprint arXiv:2211.01324 (2022)."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14186"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.02062"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.02121"},{"key":"e_1_2_1_8_1","volume-title":"MagicDance: Realistic Human Dance Video Generation with Motions & Facial Expressions Transfer. arXiv preprint arXiv:2311.12052","author":"Chang Di","year":"2023","unstructured":"Di Chang, Yichun Shi, Quankai Gao, Jessica Fu, Hongyi Xu, Guoxian Song, Qing Yan, Xiao Yang, and Mohammad Soleymani. 2023. MagicDance: Realistic Human Dance Video Generation with Motions & Facial Expressions Transfer. arXiv preprint arXiv:2311.12052 (2023)."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3592116"},{"key":"e_1_2_1_10_1","volume-title":"Improved Visual Story Generation with Adaptive Context Modeling. arXiv preprint arXiv:2305.16811","author":"Feng Zhangyin","year":"2023","unstructured":"Zhangyin Feng, Yuchen Ren, Xinmiao Yu, Xiaocheng Feng, Duyu Tang, Shuming Shi, and Bing Qin. 2023. Improved Visual Story Generation with Adaptive Context Modeling. arXiv preprint arXiv:2305.16811 (2023)."},{"key":"e_1_2_1_11_1","volume-title":"Thirty-seventh Conference on Neural Information Processing Systems. https:\/\/openreview.net\/forum?id=DEiNSfh1k7","author":"Fu Stephanie","year":"2023","unstructured":"Stephanie Fu, Netanel Yakir Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. 2023. DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data. In Thirty-seventh Conference on Neural Information Processing Systems. https:\/\/openreview.net\/forum?id=DEiNSfh1k7"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","unstructured":"Rinon Gal Yuval Alaluf Yuval Atzmon Or Patashnik Amit H. Bermano Gal Chechik and Daniel Cohen-Or. 2022. An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion. 10.48550\/ARXIV.2208.01618","DOI":"10.48550\/ARXIV.2208.01618"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3592133"},{"key":"e_1_2_1_14_1","volume-title":"TokenFlow: Consistent Diffusion Features for Consistent Video Editing. arXiv preprint arxiv:2307.10373","author":"Geyer Michal","year":"2023","unstructured":"Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. 2023. TokenFlow: Consistent Diffusion Features for Consistent Video Editing. arXiv preprint arxiv:2307.10373 (2023)."},{"key":"e_1_2_1_15_1","volume-title":"TaleCrafter: Interactive Story Visualization with Multiple Characters. arXiv preprint arXiv:2305.18247","author":"Gong Yuan","year":"2023","unstructured":"Yuan Gong, Youxin Pang, Xiaodong Cun, Menghan Xia, Haoxin Chen, Longyue Wang, Yong Zhang, Xintao Wang, Ying Shan, and Yujiu Yang. 2023. TaleCrafter: Interactive Story Visualization with Multiple Characters. arXiv preprint arXiv:2305.18247 (2023)."},{"key":"e_1_2_1_16_1","volume-title":"Yujun Shi, Yunpeng Chen, Zihan Fan, Wuyou Xiao, Rui Zhao, Shuning Chang, Weijia Wu, et al.","author":"Gu Yuchao","year":"2023","unstructured":"Yuchao Gu, Xintao Wang, Jay Zhangjie Wu, Yujun Shi, Yunpeng Chen, Zihan Fan, Wuyou Xiao, Rui Zhao, Shuning Chang, Weijia Wu, et al. 2023. Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models. arXiv preprint arXiv:2305.18292 (2023)."},{"key":"e_1_2_1_17_1","unstructured":"Eric Hedlin Gopal Sharma Shweta Mahajan Hossam Isack Abhishek Kar Andrea Tagliasacchi and Kwang Moo Yi. 2023. Unsupervised Semantic Correspondence Using Stable Diffusion. (2023). arXiv:2305.15581 [cs.CV]"},{"key":"e_1_2_1_18_1","unstructured":"Amir Hertz Ron Mokady Jay Tenenbaum Kfir Aberman Yael Pritch and Daniel Cohen-Or. 2022. Prompt-to-prompt image editing with cross attention control. (2022)."},{"key":"e_1_2_1_19_1","doi-asserted-by":"crossref","unstructured":"Amir Hertz Andrey Voynov Shlomi Fruchter and Daniel Cohen-Or. 2023. Style Aligned Image Generation via Shared Attention. (2023).","DOI":"10.1109\/CVPR52733.2024.00457"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.595"},{"key":"e_1_2_1_21_1","volume-title":"LoRA: Low-Rank Adaptation of Large Language Models. ArXiv abs\/2106.09685","author":"Hu Edward J.","year":"2021","unstructured":"Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2021. LoRA: Low-Rank Adaptation of Large Language Models. ArXiv abs\/2106.09685 (2021)."},{"key":"e_1_2_1_22_1","volume-title":"Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation. arXiv preprint arXiv:2311.17117","author":"Hu Li","year":"2023","unstructured":"Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, and Liefeng Bo. 2023. Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation. arXiv preprint arXiv:2311.17117 (2023)."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.167"},{"key":"e_1_2_1_24_1","volume-title":"An Edit Friendly DDPM Noise Space: Inversion and Manipulations. arXiv preprint arXiv:2304.06140","author":"Huberman-Spiegelglas Inbar","year":"2023","unstructured":"Inbar Huberman-Spiegelglas, Vladimir Kulikov, and Tomer Michaeli. 2023. An Edit Friendly DDPM Noise Space: Inversion and Manipulations. arXiv preprint arXiv:2304.06140 (2023)."},{"key":"e_1_2_1_25_1","volume-title":"Zero-shot generation of coherent storybook from plain text story using diffusion models. arXiv preprint arXiv:2302.03900","author":"Jeong Hyeonho","year":"2023","unstructured":"Hyeonho Jeong, Gihyun Kwon, and Jong Chul Ye. 2023. Zero-shot generation of coherent storybook from plain text story using diffusion models. arXiv preprint arXiv:2302.03900 (2023)."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.01462"},{"key":"e_1_2_1_27_1","volume-title":"Multi-Concept Customization of Text-to-Image Diffusion. arXiv","author":"Kumari Nupur","year":"2022","unstructured":"Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. 2022. Multi-Concept Customization of Text-to-Image Diffusion. arXiv (2022)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073683"},{"key":"e_1_2_1_29_1","volume-title":"Intelligent Grimm-Open-ended Visual Storytelling via Latent Diffusion Models. arXiv preprint arXiv:2306.00973","author":"Liu Chang","year":"2023","unstructured":"Chang Liu, Haoning Wu, Yujie Zhong, Xiaoyun Zhang, and Weidi Xie. 2023. Intelligent Grimm-Open-ended Visual Storytelling via Latent Diffusion Models. arXiv preprint arXiv:2306.00973 (2023)."},{"key":"e_1_2_1_30_1","volume-title":"Aleksander Holynski, and Trevor Darrell.","author":"Luo Grace","year":"2023","unstructured":"Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holynski, and Trevor Darrell. 2023b. Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence. In Advances in Neural Information Processing Systems."},{"key":"e_1_2_1_31_1","volume-title":"Dimitris Dimos, and Ren\u00e9 Vidal.","author":"Luo Jinqi","year":"2023","unstructured":"Jinqi Luo, Kwan Ho Ryan Chan, Dimitris Dimos, and Ren\u00e9 Vidal. 2023a. Knowledge Pursuit Prompting for Zero-Shot Multimodal Synthesis. arXiv preprint arXiv:2311.17898 (2023)."},{"key":"e_1_2_1_32_1","volume-title":"Dragon-diffusion: Enabling drag-style manipulation on diffusion models. arXiv preprint arXiv:2307.02421","author":"Mou Chong","year":"2023","unstructured":"Chong Mou, Xintao Wang, Jiechong Song, Ying Shan, and Jian Zhang. 2023. Dragon-diffusion: Enabling drag-style manipulation on diffusion models. arXiv preprint arXiv:2307.02421 (2023)."},{"key":"e_1_2_1_33_1","unstructured":"OpenAI. 2022. ChatGPT. https:\/\/chat.openai.com\/. Accessed: 2023-10-15."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.1979.4310076"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588432.3591513"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.02107"},{"key":"e_1_2_1_37_1","volume-title":"Orthogonal Adaptation for Modular Customization of Diffusion Models. arXiv preprint arXiv:2312.02432","author":"Po Ryan","year":"2023","unstructured":"Ryan Po, Guandao Yang, Kfir Aberman, and Gordon Wetzstein. 2023. Orthogonal Adaptation for Modular Customization of Diffusion Models. arXiv preprint arXiv:2312.02432 (2023)."},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV). 15932--15942","author":"Xiaodong Cun Chenyang QI","year":"2023","unstructured":"Chenyang QI, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. 2023. FateZero: Fusing Attentions for Zero-shot Text-based Video Editing. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV). 15932--15942."},{"key":"e_1_2_1_39_1","doi-asserted-by":"crossref","unstructured":"Nataniel Ruiz Yuanzhen Li Varun Jampani Yael Pritch Michael Rubinstein and Kfir Aberman. 2022. DreamBooth: Fine Tuning Text-to-image Diffusion Models for Subject-Driven Generation. (2022).","DOI":"10.1109\/CVPR52729.2023.02155"},{"key":"e_1_2_1_40_1","unstructured":"Simo Ryu. 2023. Low-rank Adaptation for Fast Text-to-Image Diffusion Fine-tuning. https:\/\/github.com\/cloneofsimo\/lora."},{"key":"e_1_2_1_41_1","volume-title":"FreeU: Free Lunch in Diffusion U-Net. arXiv preprint arXiv:2309.11497","author":"Si Chenyang","year":"2023","unstructured":"Chenyang Si, Ziqi Huang, Yuming Jiang, and Ziwei Liu. 2023. FreeU: Free Lunch in Diffusion U-Net. arXiv preprint arXiv:2309.11497 (2023)."},{"key":"e_1_2_1_42_1","volume-title":"Thirty-seventh Conference on Neural Information Processing Systems. https:\/\/openreview.net\/forum?id=ypOiXjdfnU","author":"Tang Luming","year":"2023","unstructured":"Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. 2023. Emergent Correspondence from Image Diffusion. In Thirty-seventh Conference on Neural Information Processing Systems. https:\/\/openreview.net\/forum?id=ypOiXjdfnU"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588432.3591506"},{"key":"e_1_2_1_44_1","volume-title":"MotionEditor","author":"Tu Shuyuan","year":"1883","unstructured":"Shuyuan Tu, Qi Dai, Zhi-Qi Cheng, Han Hu, Xintong Han, Zuxuan Wu, and Yu-Gang Jiang. 2023. MotionEditor: Editing Video Motion via Content-Aware Diffusion. arXiv preprint arXiv:2311.18830 (2023)."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01048"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00191"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.01461"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00701"},{"key":"e_1_2_1_49_1","volume-title":"Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou.","author":"Xu Zhongcong","year":"2023","unstructured":"Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. 2023. MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model."},{"key":"e_1_2_1_50_1","volume-title":"Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721","author":"Ye Hu","year":"2023","unstructured":"Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. 2023. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721 (2023)."},{"key":"e_1_2_1_51_1","volume-title":"Varun Jampani, Deqing Sun, and Ming-Hsuan Yang.","author":"Zhang Junyi","year":"2023","unstructured":"Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Polania Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. 2023a. A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence. (2023)."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00355"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3658157","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3658157","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:05:54Z","timestamp":1750291554000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3658157"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,19]]},"references-count":52,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,7,19]]}},"alternative-id":["10.1145\/3658157"],"URL":"https:\/\/doi.org\/10.1145\/3658157","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,19]]},"assertion":[{"value":"2024-07-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}