{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T18:02:27Z","timestamp":1771956147013,"version":"3.50.1"},"reference-count":56,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,11,30]],"date-time":"2023-11-30T00:00:00Z","timestamp":1701302400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100003977","name":"Israeli Science Foundation","doi-asserted-by":"crossref","award":["2303\/20"],"award-info":[{"award-number":["2303\/20"]}],"id":[{"id":"10.13039\/501100003977","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100016068","name":"Carolito Stiftung","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100016068","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2024,2,29]]},"abstract":"<jats:p>\n            We present a method for semantically transferring the visual appearance of one natural image to another. Specifically, our goal is to generate an image in which objects in a source structure image are \u201cpainted\u201d with the visual appearance of their semantically related objects in a target appearance image. To integrate semantic information into our framework, our key idea is to leverage a pre-trained and fixed Vision Transformer (ViT) model. Specifically, we derive novel disentangled representations of structure and appearance extracted from deep ViT features. We then establish an objective function that splices the desired structure and appearance representations, interweaving them together in the space of ViT features. Based on our objective function, we propose two frameworks of semantic appearance transfer \u2013 \u201cSplice\u201d, which works by training a generator on a\n            <jats:italic>single and arbitrary<\/jats:italic>\n            pair of structure-appearance images, and \u201cSpliceNet\u201d, a\n            <jats:italic>feed-forward<\/jats:italic>\n            real-time appearance transfer model trained on a\n            <jats:italic>dataset<\/jats:italic>\n            of images from a\n            <jats:italic>specific domain<\/jats:italic>\n            . Our frameworks do not involve adversarial training, nor do they require any additional input information such as semantic segmentation or correspondences. We demonstrate high-resolution results on a variety of in-the-wild image pairs, under significant variations in the number of objects, pose, and appearance. Code and supplementary material are available in our project page: splice-vit.github.io.\n          <\/jats:p>","DOI":"10.1145\/3630096","type":"journal-article","created":{"date-parts":[[2023,11,2]],"date-time":"2023-11-02T04:40:28Z","timestamp":1698900028000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Disentangling Structure and Appearance in ViT Feature Space"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2734-8480","authenticated-orcid":false,"given":"Narek","family":"Tumanyan","sequence":"first","affiliation":[{"name":"Weizmann Institute of Science, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1622-3674","authenticated-orcid":false,"given":"Omer","family":"Bar-Tal","sequence":"additional","affiliation":[{"name":"Weizmann Institute of Science, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-4453-6920","authenticated-orcid":false,"given":"Shir","family":"Amir","sequence":"additional","affiliation":[{"name":"Weizmann Institute of Science, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6057-4263","authenticated-orcid":false,"given":"Shai","family":"Bagon","sequence":"additional","affiliation":[{"name":"Weizmann Institute of Science, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3703-0783","authenticated-orcid":false,"given":"Tali","family":"Dekel","sequence":"additional","affiliation":[{"name":"Weizmann Institute of Science, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,11,30]]},"reference":[{"key":"e_1_3_3_2_1","article-title":"Deep ViT features as dense visual descriptors","author":"Amir Shir","year":"2022","unstructured":"Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel. 2022. Deep ViT features as dense visual descriptors.ECCVW What is Motion For? (2022).","journal-title":"ECCVW What is Motion For?"},{"key":"e_1_3_3_3_1","article-title":"Splicing ropes illustrated","author":"Beech Frank","year":"2005","unstructured":"Frank Beech. 2005. Splicing ropes illustrated. CCCBR (2005).","journal-title":"CCCBR"},{"key":"e_1_3_3_4_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14186"},{"key":"e_1_3_3_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00951"},{"key":"e_1_3_3_6_1","article-title":"Semantic style transfer and turning two-bit doodles into fine artworks","author":"Champandard Alex J.","year":"2016","unstructured":"Alex J. Champandard. 2016. Semantic style transfer and turning two-bit doodles into fine artworks. arXiv (2016).","journal-title":"arXiv"},{"key":"e_1_3_3_7_1","article-title":"Fast patch-based style transfer of arbitrary style","author":"Chen Tian Qi","year":"2016","unstructured":"Tian Qi Chen and Mark Schmidt. 2016. Fast patch-based style transfer of arbitrary style. arXiv preprint arXiv:1612.04337 (2016).","journal-title":"arXiv preprint arXiv:1612.04337"},{"key":"e_1_3_3_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00821"},{"key":"e_1_3_3_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00187"},{"key":"e_1_3_3_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298813"},{"key":"e_1_3_3_11_1","article-title":"An image is worth 16x16 words: Transformers for image recognition at scale","author":"Dosovitskiy Alexey","year":"2021","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR (2021).","journal-title":"ICLR"},{"key":"e_1_3_3_12_1","unstructured":"Vincent Dumoulin Jonathon Shlens and Manjunath Kudlur. 2017. A learned representation For artistic style. In International Conference on Learning Representations ."},{"key":"e_1_3_3_13_1","doi-asserted-by":"crossref","unstructured":"Leon A. Gatys Alexander S. Ecker Matthias Bethge Aaron Hertzmann and Eli Shechtman. 2017. Controlling perceptual factors in neural style transfer.","DOI":"10.1109\/CVPR.2017.397"},{"key":"e_1_3_3_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.322"},{"key":"e_1_3_3_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_3_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.167"},{"key":"e_1_3_3_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.632"},{"key":"e_1_3_3_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2921336"},{"key":"e_1_3_3_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46475-6_43"},{"key":"e_1_3_3_20_1","volume-title":"Proc. CVPR","author":"Karras Tero","year":"2020","unstructured":"Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. 2020. Analyzing and improving the image quality of StyleGAN. In Proc. CVPR."},{"key":"e_1_3_3_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01770"},{"key":"e_1_3_3_22_1","unstructured":"Sunnie S. Y. Kim Nicholas Kolkin Jason Salavon and Gregory Shakhnarovich. 2020. Deformable style transfer."},{"key":"e_1_3_3_23_1","volume-title":"International Conference on Machine Learning","author":"Kim Taeksoo","year":"2017","unstructured":"Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, and Jiwon Kim. 2017. Learning to discover cross-domain relations with generative adversarial networks. In International Conference on Machine Learning. PMLR."},{"key":"e_1_3_3_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01029"},{"key":"e_1_3_3_25_1","unstructured":"Chuan Li and Michael Wand. 2016a. Combining Markov random fields and convolutional neural networks for image synthesis."},{"key":"e_1_3_3_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46487-9_43"},{"key":"e_1_3_3_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.36"},{"key":"e_1_3_3_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073683"},{"key":"e_1_3_3_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58548-8_2"},{"key":"e_1_3_3_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/3294771.3294838"},{"key":"e_1_3_3_31_1","doi-asserted-by":"crossref","unstructured":"Aravindh Mahendran and Andrea Vedaldi. 2014. Understanding deep image representations by inverting them. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201914) . 5188\u20135196.","DOI":"10.1109\/CVPR.2015.7299155"},{"key":"e_1_3_3_32_1","doi-asserted-by":"crossref","unstructured":"Roey Mechrez Itamar Talmi and Lihi Zelnik-Manor. 2018. The contextual loss for image transformation with non-aligned data.","DOI":"10.1007\/978-3-030-01264-9_47"},{"key":"e_1_3_3_33_1","volume-title":"CVPR","author":"Melas-Kyriazi Luke","year":"2022","unstructured":"Luke Melas-Kyriazi, Christian Rupprecht, Iro Laina, and Andrea Vedaldi. 2022. Deep spectral methods: A surprisingly strong baseline for unsupervised semantic segmentation and localization. In CVPR."},{"key":"e_1_3_3_34_1","doi-asserted-by":"crossref","unstructured":"Ron Mokady Omer Tov Michal Yarom Oran Lang Inbar Mosseri Tali Dekel Daniel Cohen-Or and Michal Irani. 2022. Self-Distilled StyleGAN: Towards generation from internet photos. In ACM SIGGRAPH 2022 Conference Proceedings (SIGGRAPH\u201922) . Association for Computing Machinery 9 pages.","DOI":"10.1145\/3528233.3530708"},{"key":"e_1_3_3_35_1","unstructured":"Muzammal Naseer Kanchana Ranasinghe Salman Khan Munawar Hayat Fahad Khan and Ming-Hsuan Yang. 2021. Intriguing properties of vision transformers. In Advances in Neural Information Processing Systems A. Beygelzimer Y. Dauphin P. Liang and J. Wortman Vaughan (Eds.)."},{"key":"e_1_3_3_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICVGIP.2008.47"},{"key":"e_1_3_3_37_1","doi-asserted-by":"publisher","DOI":"10.23915\/distill.00007"},{"key":"e_1_3_3_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58545-7_19"},{"key":"e_1_3_3_39_1","volume-title":"Advances in Neural Information Processing Systems","author":"Park Taesung","year":"2020","unstructured":"Taesung Park, Jun-Yan Zhu, Oliver Wang, Jingwan Lu, Eli Shechtman, Alexei A. Efros, and Richard Zhang. 2020b. Swapping autoencoder for deep image manipulation. In Advances in Neural Information Processing Systems."},{"key":"e_1_3_3_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_3_3_41_1","volume-title":"CVPR","author":"Shechtman Eli","year":"2007","unstructured":"Eli Shechtman and Michal Irani. 2007. Matching local self-similarities across images and videos. In CVPR."},{"key":"e_1_3_3_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601137"},{"key":"e_1_3_3_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508419"},{"key":"e_1_3_3_44_1","article-title":"Localizing objects with self-supervised transformers and no labels","author":"Sim\u00e9oni Oriane","year":"2021","unstructured":"Oriane Sim\u00e9oni, Gilles Puy, Huy V. Vo, Simon Roburin, Spyros Gidaris, Andrei Bursuc, Patrick P\u00e9rez, Renaud Marlet, and Jean Ponce. 2021. Localizing objects with self-supervised transformers and no labels. Proceedings of the British Machine Vision Conference (BMVC).","journal-title":"Proceedings of the British Machine Vision Conference (BMVC)"},{"key":"e_1_3_3_45_1","volume-title":"Workshop at International Conference on Learning Representations","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Deep inside convolutional networks: Visualising image classification models and saliency maps. InWorkshop at International Conference on Learning Representations."},{"key":"e_1_3_3_46_1","volume-title":"5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings","author":"Taigman Yaniv","year":"2017","unstructured":"Yaniv Taigman, Adam Polyak, and Lior Wolf. 2017. Unsupervised cross-domain image generation. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. https:\/\/openreview.net\/forum?id=Sk2Im59ex"},{"key":"e_1_3_3_47_1","first-page":"4","volume-title":"ICML","author":"Ulyanov Dmitry","year":"2016","unstructured":"Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, and Victor S. Lempitsky. 2016. Texture networks: Feed-forward synthesis of textures and stylized images. In ICML, Vol. 1. 4."},{"key":"e_1_3_3_48_1","unstructured":"Dmitry Ulyanov Andrea Vedaldi and Victor Lempitsky. 2018. Deep image prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ."},{"key":"e_1_3_3_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3208159.3208165"},{"key":"e_1_3_3_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01414"},{"key":"e_1_3_3_51_1","article-title":"Stable and controllable neural texture synthesis and style transfer using histogram losses","author":"Wilmot Pierre","year":"2017","unstructured":"Pierre Wilmot, Eric Risser, and Connelly Barnes. 2017. Stable and controllable neural texture synthesis and style transfer using histogram losses. ArXiv (2017).","journal-title":"ArXiv"},{"key":"e_1_3_3_52_1","unstructured":"Zhongyou Xu Tingting Wang Faming Fang Yun Sheng and Guixu Zhang. 2020. Stylization-based architecture for fast deep exemplar colorization."},{"key":"e_1_3_3_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.310"},{"key":"e_1_3_3_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00913"},{"key":"e_1_3_3_55_1","volume-title":"CVPR","author":"Zhang Richard","year":"2018","unstructured":"Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR."},{"key":"e_1_3_3_56_1","article-title":"Semantic understanding of scenes through the ADE20K dataset","author":"Zhou Bolei","year":"2018","unstructured":"Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. 2018. Semantic understanding of scenes through the ADE20K dataset. International Journal on Computer Vision (2018).","journal-title":"International Journal on Computer Vision"},{"key":"e_1_3_3_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.244"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3630096","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3630096","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:56Z","timestamp":1750287056000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3630096"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,30]]},"references-count":56,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,2,29]]}},"alternative-id":["10.1145\/3630096"],"URL":"https:\/\/doi.org\/10.1145\/3630096","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,30]]},"assertion":[{"value":"2023-02-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-12","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}