{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T06:09:58Z","timestamp":1784268598529,"version":"3.55.0"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T00:00:00Z","timestamp":1731974400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2024,12,19]]},"abstract":"<jats:p>\n            Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a T2I model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight\n            <jats:italic>Spatial Adapters<\/jats:italic>\n            that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on\n            <jats:italic>\"frozen videos\"<\/jats:italic>\n            (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel\n            <jats:italic>Motion Adapter<\/jats:italic>\n            module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.\n          <\/jats:p>","DOI":"10.1145\/3687945","type":"journal-article","created":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T15:46:04Z","timestamp":1732031164000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":27,"title":["Still-Moving: Customized Video Generation without Customized Video Data"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1585-8782","authenticated-orcid":false,"given":"Hila","family":"Chefer","sequence":"first","affiliation":[{"name":"Google Research, Tel-Aviv, Israel"},{"name":"Tel Aviv University, Tel-Aviv, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-7807-0606","authenticated-orcid":false,"given":"Shiran","family":"Zada","sequence":"additional","affiliation":[{"name":"Google Research, Tel-Aviv, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3792-1770","authenticated-orcid":false,"given":"Roni","family":"Paiss","sequence":"additional","affiliation":[{"name":"Google Research, Tel Aviv, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-7737-9880","authenticated-orcid":false,"given":"Ariel","family":"Ephrat","sequence":"additional","affiliation":[{"name":"Google Research, Tel Aviv, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5910-2659","authenticated-orcid":false,"given":"Omer","family":"Tov","sequence":"additional","affiliation":[{"name":"Google Research, Tel Aviv, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3707-3807","authenticated-orcid":false,"given":"Michael","family":"Rubinstein","sequence":"additional","affiliation":[{"name":"Google Research, Boston, United States of America"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5578-8892","authenticated-orcid":false,"given":"Lior","family":"Wolf","sequence":"additional","affiliation":[{"name":"Tel Aviv University, Tel Aviv, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3703-0783","authenticated-orcid":false,"given":"Tali","family":"Dekel","sequence":"additional","affiliation":[{"name":"Google Research, Tel Aviv, Israel"},{"name":"Weizmann Institute of Science, Tel Aviv, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0525-8054","authenticated-orcid":false,"given":"Tomer","family":"Michaeli","sequence":"additional","affiliation":[{"name":"Google Research, Tel Aviv, Israel"},{"name":"Technion- Israel Institute of Technology, Haifa, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8757-7790","authenticated-orcid":false,"given":"Inbar","family":"Mosseri","sequence":"additional","affiliation":[{"name":"Google Research, Tel Aviv, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,11,19]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2022. Low-rank Adaptation for Fast Text-to-Image Diffusion Fine-tuning. https:\/\/github.com\/cloneofsimo\/lora."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3618322"},{"key":"e_1_2_1_3_1","volume-title":"Domain-agnostic tuning-encoder for fast personalization of text-to-image models. arXiv preprint arXiv:2307.06925","author":"Arar Moab","year":"2023","unstructured":"Moab Arar, Rinon Gal, Yuval Atzmon, Gal Chechik, Daniel Cohen-Or, Ariel Shamir, and Amit H Bermano. 2023. Domain-agnostic tuning-encoder for fast personalization of text-to-image models. arXiv preprint arXiv:2307.06925 (2023)."},{"key":"e_1_2_1_4_1","volume-title":"PALP: Prompt Aligned Personalization of Text-to-Image Models. arXiv:arXiv:2401.06105","author":"Arar Moab","year":"2024","unstructured":"Moab Arar, Andrey Voynov, Amir Hertz, Omri Avrahami, Shlomi Fruchter, Yael Pritch, Daniel Cohen-Or, and Ariel Shamir. 2024. PALP: Prompt Aligned Personalization of Text-to-Image Models. arXiv:arXiv:2401.06105"},{"key":"e_1_2_1_5_1","volume-title":"The Chosen One: Consistent Characters in Text-to-Image Diffusion Models. arXiv preprint arXiv:2311.10093","author":"Avrahami Omri","year":"2023","unstructured":"Omri Avrahami, Amir Hertz, Yael Vinker, Moab Arar, Shlomi Fruchter, Ohad Fried, Daniel Cohen-Or, and Dani Lischinski. 2023. The Chosen One: Consistent Characters in Text-to-Image Diffusion Models. arXiv preprint arXiv:2311.10093 (2023)."},{"key":"e_1_2_1_6_1","volume-title":"Lumiere: A Space-Time Diffusion Model for Video Generation. arXiv:arXiv:2401.12945","author":"Bar-Tal Omer","year":"2024","unstructured":"Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Guanghui Liu, Amit Raj, Yuanzhen Li, Michael Rubinstein, Tomer Michaeli, Oliver Wang, Deqing Sun, Tali Dekel, and Inbar Mosseri. 2024. Lumiere: A Space-Time Diffusion Model for Video Generation. arXiv:arXiv:2401.12945"},{"key":"e_1_2_1_7_1","volume-title":"Sanja Fidler, and Karsten Kreis.","author":"Blattmann Andreas","year":"2023","unstructured":"Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. 2023. Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00951"},{"key":"e_1_2_1_9_1","unstructured":"Hila Chefer Oran Lang Mor Geva Volodymyr Polosukhin Assaf Shocher Michal Irani Inbar Mosseri and Lior Wolf. 2023. The Hidden Language of Diffusion Models. arXiv:arXiv:2306.00966"},{"key":"e_1_2_1_10_1","volume-title":"Make-a-scene: Scene-based text-to-image generation with human priors. arXiv preprint arXiv:2203.13131","author":"Gafni Oran","year":"2022","unstructured":"Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman. 2022. Make-a-scene: Scene-based text-to-image generation with human priors. arXiv preprint arXiv:2203.13131 (2022)."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","unstructured":"Rinon Gal Yuval Alaluf Yuval Atzmon Or Patashnik Amit H. Bermano Gal Chechik and Daniel Cohen-Or. 2022. An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion. 10.48550\/ARXIV.2208.01618","DOI":"10.48550\/ARXIV.2208.01618"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3592133"},{"key":"e_1_2_1_13_1","unstructured":"Songwei Ge Seungjun Nah Guilin Liu Tyler Poon Andrew Tao Bryan Catanzaro David Jacobs Jia-Bin Huang Ming-Yu Liu and Yogesh Balaji. 2023. Preserve your own correlation: A noise prior for video diffusion models. 22930--22941."},{"key":"e_1_2_1_14_1","volume-title":"Akbar Shah, Xi Yin, Devi Parikh, and Ishan Misra.","author":"Girdhar Rohit","year":"2023","unstructured":"Rohit Girdhar, Mannat Singh, Andrew Brown, Quentin Duval, Samaneh Azadi, Sai Saketh Rambhatla, Akbar Shah, Xi Yin, Devi Parikh, and Ishan Misra. 2023. Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning. arXiv preprint arXiv:2311.10709 (2023)."},{"key":"e_1_2_1_15_1","volume-title":"Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725","author":"Guo Yuwei","year":"2023","unstructured":"Yuwei Guo, Ceyuan Yang, Anyi Rao, Yaohui Wang, Yu Qiao, Dahua Lin, and Bo Dai. 2023. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725 (2023)."},{"key":"e_1_2_1_16_1","volume-title":"SVDiff: Compact Parameter Space for Diffusion Fine-Tuning. arXiv preprint arXiv:2303.11305","author":"Han Ligong","year":"2023","unstructured":"Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, and Feng Yang. 2023. SVDiff: Compact Parameter Space for Diffusion Fine-Tuning. arXiv preprint arXiv:2303.11305 (2023)."},{"key":"e_1_2_1_17_1","doi-asserted-by":"crossref","unstructured":"Amir Hertz Andrey Voynov Shlomi Fruchter and Daniel Cohen-Or. 2023. Style Aligned Image Generation via Shared Attention. (2023).","DOI":"10.1109\/CVPR52733.2024.00457"},{"key":"e_1_2_1_18_1","unstructured":"Jonathan Ho William Chan Chitwan Saharia Jay Whang Ruiqi Gao Alexey Gritsenko Diederik P Kingma Ben Poole Mohammad Norouzi David J Fleet et al. 2022. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303 (2022)."},{"key":"e_1_2_1_19_1","volume-title":"LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=nZeVKeeFYf9","author":"Hu Edward J","year":"2022","unstructured":"Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=nZeVKeeFYf9"},{"key":"e_1_2_1_20_1","volume-title":"Chen Change Loy, and Ziwei Liu","author":"Jiang Yuming","year":"2023","unstructured":"Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, and Ziwei Liu. 2023. VideoBooth: Diffusion-based Video Generation with Image Prompts. (2023)."},{"key":"e_1_2_1_21_1","doi-asserted-by":"crossref","unstructured":"Nupur Kumari Bingliang Zhang Richard Zhang Eli Shechtman and Jun-Yan Zhu. 2023. Multi-Concept Customization of Text-to-Image Diffusion. (2023).","DOI":"10.1109\/CVPR52729.2023.00192"},{"key":"e_1_2_1_22_1","unstructured":"Jun Hao Liew Hanshu Yan Jianfeng Zhang Zhongcong Xu and Jiashi Feng. 2023. MagicEdit: High-Fidelity and Temporally Coherent Video Editing. In arXiv."},{"key":"e_1_2_1_23_1","volume-title":"Kwang-Ting Cheng, and Min-Hung Chen.","author":"Liu Shih-Yang","year":"2024","unstructured":"Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. DoRA: Weight-Decomposed Low-Rank Adaptation. arXiv:arXiv:2402.09353"},{"key":"e_1_2_1_24_1","volume-title":"Yossi Matias, Yael Pritch, Yaniv Leviathan, and Yedid Hoshen.","author":"Molad Eyal","year":"2023","unstructured":"Eyal Molad, Eliahu Horwitz, Dani Valevski, Alex Rav Acha, Yossi Matias, Yael Pritch, Yaniv Leviathan, and Yedid Hoshen. 2023. Dreamix: Video Diffusion Models are General Video Editors. arXiv preprint arXiv:2302.01329 (2023)."},{"key":"e_1_2_1_25_1","unstructured":"OpenAI. 2024. Video Generation Models as World Simulators. https:\/\/openai.com\/index\/video-generation-models-as-world-simulators"},{"key":"e_1_2_1_26_1","volume-title":"International Conference on Machine Learning. PMLR, 8748--8763","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning. PMLR, 8748--8763."},{"key":"e_1_2_1_27_1","doi-asserted-by":"crossref","unstructured":"Robin Rombach Andreas Blattmann Dominik Lorenz Patrick Esser and Bj\u00f6rn Ommer. 2021. High-Resolution Image Synthesis with Latent Diffusion Models. arXiv:2112.10752 [cs.CV]","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"e_1_2_1_28_1","doi-asserted-by":"crossref","unstructured":"Nataniel Ruiz Yuanzhen Li Varun Jampani Yael Pritch Michael Rubinstein and Kfir Aberman. 2022. DreamBooth: Fine Tuning Text-to-image Diffusion Models for Subject-Driven Generation. (2022).","DOI":"10.1109\/CVPR52729.2023.02155"},{"key":"e_1_2_1_29_1","volume-title":"Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models. arXiv preprint arXiv:2307.06949","author":"Ruiz Nataniel","year":"2023","unstructured":"Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein, and Kfir Aberman. 2023. Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models. arXiv preprint arXiv:2307.06949 (2023)."},{"key":"e_1_2_1_30_1","volume-title":"Instantbooth: Personalized text-to-image generation without test-time finetuning. arXiv preprint arXiv:2304.03411","author":"Shi Jing","year":"2023","unstructured":"Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. 2023. Instantbooth: Personalized text-to-image generation without test-time finetuning. arXiv preprint arXiv:2304.03411 (2023)."},{"key":"e_1_2_1_31_1","volume-title":"Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792","author":"Singer Uriel","year":"2022","unstructured":"Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. 2022. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 (2022)."},{"key":"e_1_2_1_32_1","doi-asserted-by":"crossref","unstructured":"Uriel Singer Amit Zohar Yuval Kirstain Shelly Sheynin Adam Polyak Devi Parikh and Yaniv Taigman. 2024. Video Editing via Factorized Diffusion Distillation. arXiv:2403.09334 [cs.CV]","DOI":"10.1007\/978-3-031-73116-7_26"},{"key":"e_1_2_1_33_1","volume-title":"Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al.","author":"Sohn Kihyuk","year":"2023","unstructured":"Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al. 2023. StyleDrop: Text-to-Image Generation in Any Style. arXiv preprint arXiv:2306.00983 (2023)."},{"key":"e_1_2_1_34_1","volume-title":"Extended Textual Conditioning in Text-to-Image Generation. arXiv preprint arXiv:2303.09522","author":"Voynov Andrey","year":"2023","unstructured":"Andrey Voynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. 2023. P+: Extended Textual Conditioning in Text-to-Image Generation. arXiv preprint arXiv:2303.09522 (2023)."},{"key":"e_1_2_1_35_1","volume-title":"Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519","author":"Wang Qixun","year":"2024","unstructured":"Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and Anthony Chen. 2024. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519 (2024)."},{"key":"e_1_2_1_36_1","volume-title":"ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation. arXiv preprint arXiv:2302.13848","author":"Wei Yuxiang","year":"2023","unstructured":"Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. 2023. ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation. arXiv preprint arXiv:2302.13848 (2023)."},{"key":"e_1_2_1_37_1","unstructured":"Hu Ye Jun Zhang Sibo Liu Xiao Han and Wei Yang. 2023. IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models. (2023)."},{"key":"e_1_2_1_38_1","doi-asserted-by":"crossref","unstructured":"Lvmin Zhang Anyi Rao and Maneesh Agrawala. 2023. Adding Conditional Control to Text-to-Image Diffusion Models.","DOI":"10.1109\/ICCV51070.2023.00355"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3687945","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3687945","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:09:57Z","timestamp":1750295397000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3687945"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,19]]},"references-count":38,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,12,19]]}},"alternative-id":["10.1145\/3687945"],"URL":"https:\/\/doi.org\/10.1145\/3687945","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,19]]},"assertion":[{"value":"2024-11-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}