{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T14:12:01Z","timestamp":1779372721690,"version":"3.53.1"},"reference-count":66,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T00:00:00Z","timestamp":1779321600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62572365, 62271360"],"award-info":[{"award-number":["62572365, 62271360"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Key Research and Development Program of China","award":["2024YFF0907002"],"award-info":[{"award-number":["2024YFF0907002"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>Proxy hashing methods have attracted increasing attention in cross-modal retrieval, because they are able to learn the mapping of different modalities into a common low-dimensional hash space by leveraging global proxies. However, existing approaches typically suffer from two limitations: (1) They solely utilize prior labels to capture the global semantic information, and hence lack the ability to explore necessary fine-grained semantic information to bridge the modality gap effectively. (2) They seldom consider the guidance information of intrinsic semantic similarity on proxy-centered space, and thus fail to leverage the similarity relations among instances sufficiently. To mitigate these limitations, we propose RAPH, a novel Relation-Aware Proxy Hashing framework that learns the semantic relations between different modalities and different semantic levels to enhance the discriminative capability of hash codes. Specifically, we first propose a Local Semantic Interaction (LSI) module based on masked language modeling to achieve the interaction of multi-modal fine-grained semantic features. Second, a relation-aware hashing learning scheme is designed to simultaneously explore the intrinsic semantic relationships and the global semantic information based on proxies. This is achieved by minimizing the reconstruction error between the multi-modal affinity matrices derived from learned features and the cross-modal similarity matrix of the hash codes. The proposed framework is able to learn more discriminative hash codes and achieves superior performance to many baselines on three public datasets.<\/jats:p>","DOI":"10.1145\/3811910","type":"journal-article","created":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T15:40:47Z","timestamp":1777390847000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Relation-Aware Proxy Hashing for Cross-Modal Retrieval"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9246-4800","authenticated-orcid":false,"given":"Jinyu","family":"Xu","sequence":"first","affiliation":[{"name":"School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, China and Engineering Research Center of Intelligent Service Technology for Digital Publishing, Ministry of Education, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4530-588X","authenticated-orcid":false,"given":"Qing","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, China and Engineering Research Center of Intelligent Service Technology for Digital Publishing, Ministry of Education, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0602-9360","authenticated-orcid":false,"given":"Jiachen","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, China and Engineering Research Center of Intelligent Service Technology for Digital Publishing, Ministry of Education, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-4600-749X","authenticated-orcid":false,"given":"Yanchun","family":"Ma","sequence":"additional","affiliation":[{"name":"Wuhan Vocational College of Software and Engineering, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-8388-6736","authenticated-orcid":false,"given":"Yongjian","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, China and Engineering Research Center of Intelligent Service Technology for Digital Publishing, Ministry of Education, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2477-381X","authenticated-orcid":false,"given":"Zhifeng","family":"Bao","sequence":"additional","affiliation":[{"name":"School of Computing Technologies, RMIT University, Melbourne, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,5,21]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Jimmy Lei Ba Jamie Ryan Kiros and Geoffrey E. Hinton. 2016. Layer normalization. arXiv:1607.06450. Retrieved from https:\/\/arxiv.org\/abs\/1607.06450"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3372278.3390711"},{"key":"e_1_3_2_4_2","first-page":"202","volume-title":"European Conference on Computer Vision","author":"Cao Yue","year":"2018","unstructured":"Yue Cao, Bin Liu, Mingsheng Long, and Jianmin Wang. 2018. Cross-modal hamming hashing. In European Conference on Computer Vision, 202\u2013218."},{"key":"e_1_3_2_5_2","unstructured":"Ken Chatfield Karen Simonyan Andrea Vedaldi and Andrew Zisserman. 2014. Return of the devil in the details delving deep into convolutional nets. arXiv:1405.3531. Retrieved from https:\/\/arxiv.org\/abs\/1405.3531"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","first-page":"10635","DOI":"10.1109\/CVPR42600.2020.01065","volume-title":"2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Chen Shizhe","year":"2020","unstructured":"Shizhe Chen, Yida Zhao, Qin Jin, and Qi Wu. 2020. Fine-grained video-text retrieval with hierarchical graph reasoning. In 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10635\u201310644."},{"key":"e_1_3_2_7_2","first-page":"1","volume-title":"Proceedings of the ACM International Conference on Image and Video Retrieval","volume":"48","author":"Chua Tat-Seng","year":"2009","unstructured":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. 2009. NUS-WIDE: A real-world web image database from national university of Singapore. In Proceedings of the ACM International Conference on Image and Video Retrieval, Article 48, 1\u20139."},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2940693"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2821921"},{"key":"e_1_3_2_10_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171\u20134186."},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","unstructured":"Guiguang Ding Yuchen Guo and Jile Zhou. 2014. Collective Matrix Factorization Hashing for Multimodal Data. In 2014 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2083\u20132090.","DOI":"10.1109\/CVPR.2014.267"},{"key":"e_1_3_2_12_2","volume-title":"International Conference on Learning Representations","author":"Dosovitskiy Alexey","year":"2021","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3580501"},{"key":"e_1_3_2_14_2","first-page":"16146","volume-title":"2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Ge Yuying","year":"2022","unstructured":"Yuying Ge, Yixiao Ge, Xihui Liu, Dian Li, Ying Shan, Xiaohu Qie, and Ping Luo. 2022. Bridging video-text retrieval with multiple choice questions. In 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 16146\u201316155."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3632527"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/1460096.1460104"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643639"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2023.3285266"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00273"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.348"},{"key":"e_1_3_2_21_2","volume-title":"Advances in Neural Information Processing Systems","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems."},{"key":"e_1_3_2_22_2","first-page":"1360","volume-title":"Proceedings of the 22nd International Joint Conference on Artificial Intelligence","author":"Kumar Shaishav","year":"2011","unstructured":"Shaishav Kumar and Raghavendra Udupa. 2011. Learning hash functions for cross-view similarity search. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence, 1360\u20131365."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00446"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3698771"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2025.112549"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401086"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2019.2940446"},{"key":"e_1_3_2_29_2","first-page":"1","volume-title":"Proceedings of the 33rd International Conference on Neural Information Processing Systems","volume":"2","author":"Lu Jiasen","year":"2019","unstructured":"Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. ViLBERT: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, Article 2, 1\u201311."},{"issue":"4","key":"e_1_3_2_30_2","first-page":"1","article-title":"Complex scenario image retrieval via deep similarity-aware hashing","volume":"20","author":"Nie Xiushan","year":"2023","unstructured":"Xiushan Nie, Yang Shi, Ziyu Meng, Jin Huang, Weili Guan, and Yilong Yin. 2023. Complex scenario image retrieval via deep similarity-aware hashing. ACM Transactions on Multimedia Computing, Communications, and Applications 20, 4 (2023), 1\u201324.","journal-title":"ACM Transactions on Multimedia Computing, Communications, and Applications"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2023.3311646"},{"key":"e_1_3_2_32_2","first-page":"8748","volume-title":"Proceedings of the 38th International Conference on Machine Learning","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, 8748\u20138763."},{"key":"e_1_3_2_33_2","article-title":"Language models are unsupervised multitask learners","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog (2019). Retrieved from https:\/\/cdn.openai.com\/better-language-models\/language_models_are_unsupervised_multitask_learners.pdf","journal-title":"OpenAI blog"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1162"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2018.2789887"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3637442"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.5555\/3367471.3367706"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2465274"},{"key":"e_1_3_2_39_2","first-page":"3027","volume-title":"2019 IEEE\/CVF International Conference on Computer Vision (ICCV)","author":"Su Shupeng","year":"2019","unstructured":"Shupeng Su, Zhisheng Zhong, and Chao Zhang. 2019. Deep joint-semantics reconstructing hashing for large-scale unsupervised cross-modal retrieval. In 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), 3027\u20133035."},{"key":"e_1_3_2_40_2","volume-title":"International Conference on Learning Representations","author":"Su Weijie","year":"2020","unstructured":"Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. 2020. VL-BERT: Pre-training of generic visual-linguistic representations. In International Conference on Learning Representations."},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2016.2564638"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1177\/107769905303000401"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503161.3548187"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591660"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2023.3243608"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3187023"},{"key":"e_1_3_2_47_2","first-page":"3466","volume-title":"Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI \u201920)","author":"Tu Rong-Cheng","year":"2020","unstructured":"Rong-Cheng Tu, Xian-Ling Mao, and Wei Wei. 2020. MLS3RDUH: Deep unsupervised hashing via manifold based local semantic similarity structure reconstructing. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI \u201920), 3466\u20133472."},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295349"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650205"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.03.019"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2020.2974825"},{"key":"e_1_3_2_52_2","first-page":"1753","volume-title":"Proceedings of the 21st International Conference on Neural Information Processing Systems","author":"Weiss Yair","year":"2008","unstructured":"Yair Weiss, Antonio Torralba, and Rob Fergus. 2008. Spectral hashing. In Proceedings of the 21st International Conference on Neural Information Processing Systems, 1753\u20131760."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.5555\/3304889.3305057"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2023.101968"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.2963957"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2022.3222598"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.5555\/3367032.3367172"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.10719"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3631356"},{"key":"e_1_3_2_60_2","article-title":"Scaling autoregressive models for content-rich text-to-image generation","author":"Yu Jiahui","year":"2022","unstructured":"Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. 2022. Scaling autoregressive models for content-rich text-to-image generation. Transactions on Machine Learning Research (2022). Retrieved from https:\/\/openreview.net\/forum?id=AFDcYJKhND","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i5.16592"},{"key":"e_1_3_2_62_2","doi-asserted-by":"crossref","first-page":"3080","DOI":"10.1109\/CVPR42600.2020.00315","volume-title":"2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Yuan Li","year":"2020","unstructured":"Li Yuan, Tao Wang, Xiaopeng Zhang, Francis E. H. Tay, Zequn Jie, Wei Liu, and Jiashi Feng. 2020. Central similarity quantization for efficient image and video retrieval. In 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 3080\u20133089."},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2021.3053766"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01629"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/2600428.2609610"},{"key":"e_1_3_2_66_2","unstructured":"Lei Zhu Tianshi Wang Fengling Li Jingjing Li Zheng Zhang and Heng Tao Shen. 2023. CrossModal retrieval: A systematic review of methods and future directions. arXiv:2308.14263. Retrieved from https:\/\/arxiv.org\/abs\/2308.14263"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3218656"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3811910","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T13:33:28Z","timestamp":1779370408000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3811910"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,21]]},"references-count":66,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3811910"],"URL":"https:\/\/doi.org\/10.1145\/3811910","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,21]]},"assertion":[{"value":"2024-10-12","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-16","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}