{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,28]],"date-time":"2026-05-28T04:52:09Z","timestamp":1779943929308,"version":"3.53.1"},"publisher-location":"New York, NY, USA","reference-count":57,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T00:00:00Z","timestamp":1730073600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,10,28]]},"DOI":"10.1145\/3664647.3681296","type":"proceedings-article","created":{"date-parts":[[2024,10,26]],"date-time":"2024-10-26T06:59:33Z","timestamp":1729925973000},"page":"3994-4003","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["BrainRAM: Cross-Modality Retrieval-Augmented Image Reconstruction from Human Brain Activity"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8452-9045","authenticated-orcid":false,"given":"Dian","family":"Xie","sequence":"first","affiliation":[{"name":"Hefei University of Technology, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-0451-9902","authenticated-orcid":false,"given":"Peiang","family":"Zhao","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0928-8336","authenticated-orcid":false,"given":"Jiarui","family":"Zhang","sequence":"additional","affiliation":[{"name":"Hefei University of Technology, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-4175-4687","authenticated-orcid":false,"given":"Kangqi","family":"Wei","sequence":"additional","affiliation":[{"name":"Hefei University of Technology, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8272-1066","authenticated-orcid":false,"given":"Xiaobao","family":"Ni","sequence":"additional","affiliation":[{"name":"Hefei University of Technology, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-3622-0605","authenticated-orcid":false,"given":"Jiong","family":"Xia","sequence":"additional","affiliation":[{"name":"Hefei University of Technology, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,10,28]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Emily J Allen Ghislain St-Yves Yihan Wu Jesse L Breedlove Jacob S Prince Logan T Dowdle Matthias Nau Brad Caron Franco Pestilli Ian Charest et al. 2022. A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nature neuroscience Vol. 25 1 (2022) 116--126.","DOI":"10.1038\/s41593-021-00962-x"},{"key":"e_1_3_2_1_2_1","volume-title":"Clip Retrieval: Easily compute clip embeddings and build a clip retrieval system with them. https:\/\/github.com\/rom1504\/clip-retrieval.","author":"Beaumont Romain","year":"2022","unstructured":"Romain Beaumont. 2022. Clip Retrieval: Easily compute clip embeddings and build a clip retrieval system with them. https:\/\/github.com\/rom1504\/clip-retrieval."},{"key":"e_1_3_2_1_3_1","volume-title":"Advances in Neural Information Processing Systems","volume":"32","author":"Beliy Roman","year":"2019","unstructured":"Roman Beliy, Guy Gaziv, Assaf Hoogi, Francesca Strappini, Tal Golan, and Michal Irani. 2019. From voxels to pixels and back: Self-supervision in natural-image reconstruction from fmri. Advances in Neural Information Processing Systems, Vol. 32 (2019)."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2021.07.006"},{"key":"e_1_3_2_1_5_1","first-page":"15309","article-title":"Retrieval-augmented diffusion models","volume":"35","author":"Blattmann Andreas","year":"2022","unstructured":"Andreas Blattmann, Robin Rombach, Kaan Oktay, Jonas M\u00fcller, and Bj\u00f6rn Ommer. 2022. Retrieval-augmented diffusion models. Advances in Neural Information Processing Systems, Vol. 35 (2022), 15309--15324.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_6_1","volume-title":"Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems","author":"Caron Mathilde","year":"2020","unstructured":"Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. 2020. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, Vol. 33 (2020), 9912--9924."},{"key":"e_1_3_2_1_7_1","first-page":"27517","article-title":"Instance-conditioned gan","volume":"34","author":"Casanova Arantxa","year":"2021","unstructured":"Arantxa Casanova, Marlene Careil, Jakob Verbeek, Michal Drozdzal, and Adriana Romero Soriano. 2021. Instance-conditioned gan. Advances in Neural Information Processing Systems, Vol. 34 (2021), 27517--27529.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02175"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2013.2260168"},{"key":"e_1_3_2_1_10_1","volume-title":"Decoding the representation of numerical values from brain activation patterns. Human brain mapping","author":"Damarla Saudamini Roy","year":"2013","unstructured":"Saudamini Roy Damarla and Marcel Adam Just. 2013. Decoding the representation of numerical values from brain activation patterns. Human brain mapping, Vol. 34, 10 (2013), 2624--2634."},{"key":"e_1_3_2_1_11_1","volume-title":"Diffusion models beat gans on image synthesis. Advances in neural information processing systems","author":"Dhariwal Prafulla","year":"2021","unstructured":"Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, Vol. 34 (2021), 8780--8794."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2023.3263181"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00958"},{"key":"e_1_3_2_1_14_1","volume-title":"Modular encoding and decoding models derived from Bayesian canonical correlation analysis. Neural computation","author":"Fujiwara Yusuke","year":"2013","unstructured":"Yusuke Fujiwara, Yoichi Miyawaki, and Yukiyasu Kamitani. 2013. Modular encoding and decoding models derived from Bayesian canonical correlation analysis. Neural computation, Vol. 25, 4 (2013), 979--1005."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2022.119121"},{"key":"e_1_3_2_1_16_1","volume-title":"Generative adversarial nets. Advances in neural information processing systems","author":"Goodfellow Ian","year":"2014","unstructured":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems, Vol. 27 (2014)."},{"key":"e_1_3_2_1_17_1","volume-title":"Decoding natural image stimuli from fMRI data with a surface-based convolutional network. arXiv preprint arXiv:2212.02409","author":"Gu Zijin","year":"2022","unstructured":"Zijin Gu, Keith Jamison, Amy Kuceyeski, and Mert Sabuncu. 2022. Decoding natural image stimuli from fMRI data with a surface-based convolutional network. arXiv preprint arXiv:2212.02409 (2022)."},{"key":"e_1_3_2_1_18_1","volume-title":"Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nature neuroscience","author":"Haynes John-Dylan","year":"2005","unstructured":"John-Dylan Haynes and Geraint Rees. 2005. Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nature neuroscience, Vol. 8, 5 (2005), 686--691."},{"key":"e_1_3_2_1_19_1","volume-title":"Denoising diffusion probabilistic models. Advances in neural information processing systems","author":"Ho Jonathan","year":"2020","unstructured":"Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, Vol. 33 (2020), 6840--6851."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2019.8852121"},{"key":"e_1_3_2_1_21_1","volume-title":"Nature","volume":"452","author":"Kay Kendrick N","year":"2008","unstructured":"Kendrick N Kay, Thomas Naselaris, Ryan J Prenger, and Jack L Gallant. 2008. Identifying natural images from human brain activity. Nature, Vol. 452, 7185 (2008), 352--355."},{"key":"e_1_3_2_1_22_1","volume-title":"Mixco: Mix-up contrastive learning for visual representation. arXiv preprint arXiv:2010.06300","author":"Kim Sungnyun","year":"2020","unstructured":"Sungnyun Kim, Gihun Lee, Sangmin Bae, and Se-Young Yun. 2020. Mixco: Mix-up contrastive learning for visual representation. arXiv preprint arXiv:2010.06300 (2020)."},{"key":"e_1_3_2_1_23_1","volume-title":"Brain-optimized inference improves reconstructions of fMRI brain activity. ArXiv","author":"Kneeland Reese","year":"2023","unstructured":"Reese Kneeland, Jordyn Ojeda, Ghislain St-Yves, and Thomas Naselaris. 2023. Brain-optimized inference improves reconstructions of fMRI brain activity. ArXiv (2023)."},{"key":"e_1_3_2_1_24_1","volume-title":"Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, Vol. 25 (2012)."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00060"},{"key":"e_1_3_2_1_26_1","first-page":"29624","article-title":"Mind reader: Reconstructing complex images from brain activities","volume":"35","author":"Lin Sikun","year":"2022","unstructured":"Sikun Lin, Thomas Sprague, and Ambuj K Singh. 2022. Mind reader: Reconstructing complex images from brain activities. Advances in Neural Information Processing Systems, Vol. 35 (2022), 29624--29636.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_27_1","volume-title":"Proceedings, Part V 13","author":"Lin Tsung-Yi","year":"2014","unstructured":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll\u00e1r, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6--12, 2014, Proceedings, Part V 13. Springer, 740--755."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3581783.3613832"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2008.11.004"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2010.07.073"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2009.09.006"},{"key":"e_1_3_2_1_32_1","volume-title":"Reconstructing visual experiences from brain activity evoked by natural movies. Current biology","author":"Nishimoto Shinji","year":"2011","unstructured":"Shinji Nishimoto, An T Vu, Thomas Naselaris, Yuval Benjamini, Bin Yu, and Jack L Gallant. 2011. Reconstructing visual experiences from brain activity evoked by natural movies. Current biology, Vol. 21, 19 (2011), 1641--1646."},{"key":"e_1_3_2_1_33_1","volume-title":"Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748","author":"van den Oord Aaron","year":"2018","unstructured":"Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN55064.2022.9892673"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-42891-8"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1101\/153759"},{"key":"e_1_3_2_1_37_1","volume-title":"International conference on machine learning. PMLR, 8748--8763","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748--8763."},{"key":"e_1_3_2_1_38_1","volume-title":"Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125","author":"Ramesh Aditya","year":"2022","unstructured":"Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, Vol. 1, 2 (2022), 3."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"e_1_3_2_1_40_1","volume-title":"U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention--MICCAI 2015: 18th international conference","author":"Ronneberger Olaf","year":"2015","unstructured":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention--MICCAI 2015: 18th international conference, Munich, Germany, October 5--9, 2015, proceedings, part III 18. Springer, 234--241."},{"key":"e_1_3_2_1_41_1","first-page":"25278","article-title":"Laion-5b: An open large-scale dataset for training next generation image-text models","volume":"35","author":"Schuhmann Christoph","year":"2022","unstructured":"Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. 2022. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, Vol. 35 (2022), 25278--25294.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_42_1","volume-title":"Advances in Neural Information Processing Systems","volume":"36","author":"Scotti Paul","year":"2023","unstructured":"Paul Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Shabalin, Alex Nguyen, Aidan Dempster, Nathalie Verlinde, Elad Yundler, David Weisberg, Kenneth Norman, et al. 2023. Reconstructing the mind's eye: fMRI-to-image with contrastive learning and diffusion priors. Advances in Neural Information Processing Systems, Vol. 36 (2023)."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2018.07.043"},{"key":"e_1_3_2_1_44_1","volume-title":"End-to-end deep image reconstruction from human brain activity. Frontiers in computational neuroscience","author":"Shen Guohua","year":"2019","unstructured":"Guohua Shen, Kshitij Dwivedi, Kei Majima, Tomoyasu Horikawa, and Yukiyasu Kamitani. 2019. End-to-end deep image reconstruction from human brain activity. Frontiers in computational neuroscience, Vol. 13 (2019), 432276."},{"key":"e_1_3_2_1_45_1","volume-title":"Knn-diffusion: Image generation via large-scale retrieval. arXiv preprint arXiv:2204.02849","author":"Sheynin Shelly","year":"2022","unstructured":"Shelly Sheynin, Oron Ashual, Adam Polyak, Uriel Singer, Oran Gafni, Eliya Nachmani, and Yaniv Taigman. 2022. Knn-diffusion: Image generation via large-scale retrieval. arXiv preprint arXiv:2204.02849 (2022)."},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01234"},{"key":"e_1_3_2_1_47_1","volume-title":"Advances in Neural Information Processing Systems","volume":"36","author":"Sun Jingyuan","year":"2023","unstructured":"Jingyuan Sun, Mingxiao Li, Zijiao Chen, Yunhao Zhang, Shaonan Wang, and Marie-Francine Moens. 2023. Contrast, attend and diffuse to decode high-resolution images from brain activities. Advances in Neural Information Processing Systems, Vol. 36 (2023)."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01389"},{"key":"e_1_3_2_1_50_1","volume-title":"International conference on machine learning. PMLR, 6105--6114","author":"Tan Mingxing","year":"2019","unstructured":"Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning. PMLR, 6105--6114."},{"key":"e_1_3_2_1_51_1","volume-title":"Visual working memory directly alters perception. Nature human behaviour","author":"Teng Chunyue","year":"2019","unstructured":"Chunyue Teng and Dwight J Kravitz. 2019. Visual working memory directly alters perception. Nature human behaviour, Vol. 3, 8 (2019), 827--836."},{"key":"e_1_3_2_1_52_1","volume-title":"Proceedings, Part VIII 16","author":"Tseng Hung-Yu","year":"2020","unstructured":"Hung-Yu Tseng, Hsin-Ying Lee, Lu Jiang, Ming-Hsuan Yang, and Weilong Yang. 2020. Retrievegan: Image synthesis via differentiable patch retrieval. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part VIII 16. Springer, 242--257."},{"key":"e_1_3_2_1_53_1","volume-title":"Image quality assessment: from error visibility to structural similarity","author":"Wang Zhou","year":"2004","unstructured":"Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, Vol. 13, 4 (2004), 600--612."},{"key":"e_1_3_2_1_54_1","volume-title":"Memorizing transformers. arXiv preprint arXiv:2203.08913","author":"Wu Yuhuai","year":"2022","unstructured":"Yuhuai Wu, Markus N Rabe, DeLesley Hutchins, and Christian Szegedy. 2022. Memorizing transformers. arXiv preprint arXiv:2203.08913 (2022)."},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV57701.2024.00804"},{"key":"e_1_3_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00713"},{"key":"e_1_3_2_1_57_1","volume-title":"Retrieval-Augmented Generation for AI-Generated Content: A Survey. arXiv preprint arXiv:2402.19473","author":"Zhao Penghao","year":"2024","unstructured":"Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, and Bin Cui. 2024. Retrieval-Augmented Generation for AI-Generated Content: A Survey. arXiv preprint arXiv:2402.19473 (2024)."}],"event":{"name":"MM '24: The 32nd ACM International Conference on Multimedia","location":"Melbourne VIC Australia","acronym":"MM '24","sponsor":["SIGMM ACM Special Interest Group on Multimedia"]},"container-title":["Proceedings of the 32nd ACM International Conference on Multimedia"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3664647.3681296","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3664647.3681296","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:17:43Z","timestamp":1750295863000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3664647.3681296"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,28]]},"references-count":57,"alternative-id":["10.1145\/3664647.3681296","10.1145\/3664647"],"URL":"https:\/\/doi.org\/10.1145\/3664647.3681296","relation":{},"subject":[],"published":{"date-parts":[[2024,10,28]]},"assertion":[{"value":"2024-10-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}