{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T07:47:19Z","timestamp":1770277639782,"version":"3.49.0"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2024,1,11]],"date-time":"2024-01-11T00:00:00Z","timestamp":1704931200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62076044"],"award-info":[{"award-number":["62076044"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"The Chongqing Talent Plan Project","award":["cstc2022ycjh-bgzxm0160"],"award-info":[{"award-number":["cstc2022ycjh-bgzxm0160"]}]},{"name":"The Chongqing Graduate Research Innovation Project of China","award":["CYS21307"],"award-info":[{"award-number":["CYS21307"]}]},{"name":"The Doctoral Program of Chongqing University of Posts and Telecommunications","award":["BYJS202013"],"award-info":[{"award-number":["BYJS202013"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,5,31]]},"abstract":"<jats:p>\n            Medical multi-modal retrieval aims to provide doctors with similar medical images from different modalities, which can greatly promote the efficiency and accuracy of clinical diagnosis. However, most existing medical retrieval methods hardly support the retrieval of multi-modal medical images, i.e., the number of modalities is greater than 2, and just convert retrieval to classification or clustering. It futilely breaks the gap between the visual information and the semantic information in different medical image modalities. To solve the problem, a\n            <jats:bold>S<\/jats:bold>\n            upervised\n            <jats:bold>C<\/jats:bold>\n            ontrast\n            <jats:bold>L<\/jats:bold>\n            earning method based on a\n            <jats:bold>M<\/jats:bold>\n            ultiple\n            <jats:bold>P<\/jats:bold>\n            seudo-\n            <jats:bold>S<\/jats:bold>\n            iamese network (SCL-MPS) is proposed for multi-modal medical image retrieval. In order to make the samples with semantic similarity close neighbors on Riemann manifold, the multiple constraints based on semantic consistency and modal invariance are designed in different forward stages of SCL-MPS. We theoretically demonstrate the feasibility of the designed constraints. Finally, experiments on four benchmark datasets (ADNI1, ADNI2, ADNI3, and OASIS3) show that SCL-MPS achieves state-of-the-art performance compared to 15 retrieval methods. Especially, SCL-MPS achieves a 100% mAP score in medical cross-modal retrieval on ADNI1.\n          <\/jats:p>\n          <jats:p\/>","DOI":"10.1145\/3637441","type":"journal-article","created":{"date-parts":[[2023,12,13]],"date-time":"2023-12-13T11:40:14Z","timestamp":1702467614000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Multiple Pseudo-Siamese Network with Supervised Contrast Learning for Medical Multi-modal Retrieval"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5892-2372","authenticated-orcid":false,"given":"Xianhua","family":"Zeng","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0151-9133","authenticated-orcid":false,"given":"Xinyu","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, China and Chongqing Military Industry Group Co., LTD., China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2877-867X","authenticated-orcid":false,"given":"Yicai","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Mathematics and Computer Science, Gannan Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,1,11]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"1247","volume-title":"International Conference on Machine Learning","author":"Andrew Galen","year":"2013","unstructured":"Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. 2013. Deep canonical correlation analysis. In International Conference on Machine Learning. PMLR, 1247\u20131255."},{"key":"e_1_3_2_3_2","article-title":"Laplacian eigenmaps and spectral techniques for embedding and clustering","volume":"14","author":"Belkin Mikhail","year":"2001","unstructured":"Mikhail Belkin and Partha Niyogi. 2001. Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in Neural Information Processing Systems 14 (2001).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_4_2","first-page":"CIN\u2013S14053","article-title":"Medical image retrieval: A multimodal approach","volume":"13","year":"2014","unstructured":"Yu Cao, Shawn Steffey, Jianbiao He, Degui Xiao, Cui Tao, Ping Chen, and Henning M\u00fcller. 2014. Medical image retrieval: A multimodal approach. Cancer Informatics 13 (2014), CIN\u2013S14053.","journal-title":"Cancer Informatics"},{"key":"e_1_3_2_5_2","first-page":"1597","volume-title":"International Conference on Machine Learning","author":"Chen Ting","year":"2020","unstructured":"Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning. PMLR, 1597\u20131607."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01549"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2019.12.078"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3499027"},{"key":"e_1_3_2_9_2","first-page":"825","volume-title":"International Joint Conference on Artificial Intelligence","author":"Fan Lixin","year":"2020","unstructured":"Lixin Fan, KamWoh Ng, Ce Ju, Tianyu Zhang, and Chee Seng Chan. 2020. Deep polarized network for supervised learning of accurate binary hashing codes. In International Joint Conference on Artificial Intelligence. 825\u2013831."},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.101981"},{"key":"e_1_3_2_11_2","article-title":"MKVSE: Multimodal knowledge enhanced visual-semantic embedding for image-text retrieval","author":"Feng Duoduo","year":"2023","unstructured":"Duoduo Feng, Xiangteng He, and Yuxin Peng. 2023. MKVSE: Multimodal knowledge enhanced visual-semantic embedding for image-text retrieval. ACM Transactions on Multimedia Computing, Communications and Applications (2023).","journal-title":"ACM Transactions on Multimedia Computing, Communications and Applications"},{"key":"e_1_3_2_12_2","first-page":"86","volume-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention","year":"2015","unstructured":"Yue Gao, Ehsan Adeli-M, Minjeong Kim, Panteleimon Giannakopoulos, Sven Haller, and Dinggang Shen. 2015. Medical image retrieval using multi-graph learning for MCI diagnostic assistance. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 86\u201393."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.100"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2022.3152247"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3483381"},{"issue":"4","key":"e_1_3_2_16_2","doi-asserted-by":"crossref","first-page":"685","DOI":"10.1002\/jmri.21049","article-title":"The Alzheimer\u2019s disease neuroimaging initiative (ADNI): MRI methods","volume":"27","year":"2008","unstructured":"Clifford R. Jack Jr, Matt A. Bernstein, Nick C. Fox, Paul Thompson, Gene Alexander, Danielle Harvey, Bret Borowski, Paula J. Britson, Jennifer L. Whitwell, Chadwick Ward, Anders M. Dale, Joel P. Felmlee, Jeffrey L. Gunter, Derek L. G. Hill, Ron Killiany, Norbert Schuff, Sabrina Fox-Bosetti, Chen Lin, Colin Studholme, Charles DeCarli, Gunnar Krueger, Heidi A. Ward, Gregory J. Metzger, Katherine T. Scott, Richard Mallozzi, Daniel Blezek, Joshua Levy, Josef P. Debbins, Adam S. Fleisher, Marilyn Albert, Robert Green, George Bartzokis, Gary Glover, John Mugler, and Michael W. Weiner. 2008. The Alzheimer\u2019s disease neuroimaging initiative (ADNI): MRI methods. Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine 27, 4 (2008), 685\u2013691.","journal-title":"Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11814"},{"key":"e_1_3_2_18_2","first-page":"18661","article-title":"Supervised contrastive learning","volume":"33","year":"2020","unstructured":"Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. Advances in Neural Information Processing Systems 33 (2020), 18661\u201318673.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_19_2","first-page":"2983","article-title":"Adversarial self-supervised contrastive learning","volume":"33","author":"Kim Minseon","year":"2020","unstructured":"Minseon Kim, Jihoon Tack, and Sung Ju Hwang. 2020. Adversarial self-supervised contrastive learning. Advances in Neural Information Processing Systems 33 (2020), 2983\u20132994.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_20_2","article-title":"Adam: A method for stochastic optimization","author":"Kingma Diederik P.","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).","journal-title":"arXiv preprint arXiv:1412.6980"},{"key":"e_1_3_2_21_2","volume-title":"Advances in Neural Information Processing Systems","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, Vol. 25. Curran Associates, Inc."},{"key":"e_1_3_2_22_2","unstructured":"MedRxiv 2019 OASIS-3: Longitudinal neuroimaging clinical and cognitive dataset for normal aging and Alzheimer disease"},{"key":"e_1_3_2_23_2","article-title":"Deep supervised discrete hashing","volume":"30","author":"Li Qi","year":"2017","unstructured":"Qi Li, Zhenan Sun, Ran He, and Tieniu Tan. 2017. Deep supervised discrete hashing. Advances in Neural Information Processing Systems 30 (2017).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2022.3157517"},{"key":"e_1_3_2_25_2","article-title":"Feature learning based deep supervised hashing with pairwise labels","author":"Li Wu-Jun","year":"2015","unstructured":"Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. 2015. Feature learning based deep supervised hashing with pairwise labels. arXiv preprint arXiv:1511.03855 (2015).","journal-title":"arXiv preprint arXiv:1511.03855"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2017.07.005"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.227"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2020.2969792"},{"key":"e_1_3_2_29_2","volume-title":"International Conference on Machine Learning","author":"Mo Yujie","year":"2023","unstructured":"Yujie Mo, Yajie Lei, Jialie Shen, Xiaoshuang Shi, Heng Tao Shen, and Xiaofeng Zhu. 2023. Disentangled multiplex graph representation learning. In International Conference on Machine Learning."},{"key":"e_1_3_2_30_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TNNLS.2022.3230979","article-title":"GRLC: Graph representation learning with constraints","author":"Peng Liang","year":"2023","unstructured":"Liang Peng, Yujie Mo, Jie Xu, Jialie Shen, Xiaoshuang Shi, Xiaoxiao Li, Heng Tao Shen, and Xiaofeng Zhu. 2023. GRLC: Graph representation learning with constraints. IEEE Transactions on Neural Networks and Learning Systems (2023), 1\u201314.","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_3_2_31_2","first-page":"3846","volume-title":"International Joint Conference on Artificial Intelligence","author":"Peng Yuxin","year":"2016","unstructured":"Yuxin Peng, Xin Huang, and Jinwei Qi. 2016. Cross-media shared representation by hierarchical learning with multiple deep networks. In International Joint Conference on Artificial Intelligence. 3846\u20133853."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2017.05.025"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11517-021-02392-0"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123266.3123345"},{"key":"e_1_3_2_35_2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).","journal-title":"arXiv preprint arXiv:1409.1556"},{"issue":"11","key":"e_1_3_2_36_2","article-title":"Visualizing data using t-SNE.","volume":"9","author":"Maaten Laurens Van der","year":"2008","unstructured":"Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 11 (2008).","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3297001.3297007"},{"issue":"1","key":"e_1_3_2_38_2","doi-asserted-by":"crossref","first-page":"2316","DOI":"10.1038\/s41598-023-29320-6","article-title":"Deep consistency-preserving hash auto-encoders for neuroimage cross-modal retrieval","volume":"13","author":"Wang Xinyu","year":"2023","unstructured":"Xinyu Wang and Xianhua Zeng. 2023. Deep consistency-preserving hash auto-encoders for neuroimage cross-modal retrieval. Scientific Reports 13, 1 (2023), 2316.","journal-title":"Scientific Reports"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00928"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2878970"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2022.3171081"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.2975798"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485042"},{"issue":"2","key":"e_1_3_2_44_2","first-page":"503","article-title":"Deep Bayesian hashing with center prior for multi-modal neuroimage retrieval","volume":"40","year":"2020","unstructured":"Erkun Yang, Mingxia Liu, Dongren Yao, Bing Cao, Chunfeng Lian, Pew-Thian Yap, and Dinggang Shen. 2020. Deep Bayesian hashing with center prior for multi-modal neuroimage retrieval. IEEE Transactions on Medical Imaging 40, 2 (2020), 503\u2013513.","journal-title":"IEEE Transactions on Medical Imaging"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2004.1261097"},{"issue":"4","key":"e_1_3_2_46_2","doi-asserted-by":"crossref","first-page":"977","DOI":"10.1109\/TMI.2018.2790962","article-title":"Predicting CT image from MRI data through feature matching with learned nonlinear local descriptors","volume":"37","year":"2018","unstructured":"Wei Yang, Liming Zhong, Yang Chen, Liyan Lin, Zhentai Lu, Shupeng Liu, Yao Wu, Qianjin Feng, and Wufan Chen. 2018. Predicting CT image from MRI data through feature matching with learned nonlinear local descriptors. IEEE Transactions on Medical Imaging 37, 4 (2018), 977\u2013987.","journal-title":"IEEE Transactions on Medical Imaging"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00315"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3478642"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3575658"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01064"},{"issue":"5","key":"e_1_3_2_51_2","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1109\/TPAMI.2016.2645565","article-title":"Hetero-manifold regularisation for cross-modal hashing","volume":"40","author":"Zheng Feng","year":"2016","unstructured":"Feng Zheng, Yi Tang, and Ling Shao. 2016. Hetero-manifold regularisation for cross-modal hashing. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 5 (2016), 1059\u20131071.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_2_52_2","first-page":"4325","volume-title":"IEEE International Conference on Acoustics, Speech and Signal Processing","year":"2021","unstructured":"Yu Zhou, Yong Feng, Mingliang Zhou, Baohua Qiang, Leong Hou U, and Jiajie Zhu. 2021. Deep adversarial quantization network for cross-modal retrieval. In IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 4325\u20134329."},{"key":"e_1_3_2_53_2","first-page":"3567","volume-title":"International Joint Conference on Artificial Intelligence","year":"2017","unstructured":"Hao Zhu and Shenghua Gao. 2017. Locality constrained deep supervised hashing for image retrieval. In International Joint Conference on Artificial Intelligence. 3567\u20133573."},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.10235"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3637441","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3637441","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:25Z","timestamp":1750291405000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3637441"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,11]]},"references-count":53,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,5,31]]}},"alternative-id":["10.1145\/3637441"],"URL":"https:\/\/doi.org\/10.1145\/3637441","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,11]]},"assertion":[{"value":"2022-11-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-06","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}