{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T00:13:52Z","timestamp":1785543232306,"version":"3.56.0"},"reference-count":27,"publisher":"Association for Computing Machinery (ACM)","issue":"8","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2014,4]]},"abstract":"<jats:p>Multi-modal retrieval is emerging as a new search paradigm that enables seamless information retrieval from various types of media. For example, users can simply snap a movie poster to search relevant reviews and trailers. To solve the problem, a set of mapping functions are learned to project high-dimensional features extracted from data of different media types into a common low-dimensional space so that metric distance measures can be applied. In this paper, we propose an effective mapping mechanism based on deep learning (i.e., stacked auto-encoders) for multi-modal retrieval. Mapping functions are learned by optimizing a new objective function, which captures both intra-modal and inter-modal semantic relationships of data from heterogeneous sources effectively. Compared with previous works which require a substantial amount of prior knowledge such as similarity matrices of intra-modal data and ranking examples, our method requires little prior knowledge. Given a large training dataset, we split it into mini-batches and continually adjust the mapping functions for each batch of input. Hence, our method is memory efficient with respect to the data volume. Experiments on three real datasets illustrate that our proposed method achieves significant improvement in search accuracy over the state-of-the-art methods.<\/jats:p>","DOI":"10.14778\/2732296.2732301","type":"journal-article","created":{"date-parts":[[2015,5,12]],"date-time":"2015-05-12T15:37:52Z","timestamp":1431445072000},"page":"649-660","source":"Crossref","is-referenced-by-count":114,"title":["Effective multi-modal retrieval based on stacked auto-encoders"],"prefix":"10.14778","volume":"7","author":[{"given":"Wei","family":"Wang","sequence":"first","affiliation":[{"name":"National University of Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Beng Chin","family":"Ooi","sequence":"additional","affiliation":[{"name":"National University of Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoyan","family":"Yang","sequence":"additional","affiliation":[{"name":"Illinois at Singapore Pte, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dongxiang","family":"Zhang","sequence":"additional","affiliation":[{"name":"National University of Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yueting","family":"Zhuang","sequence":"additional","affiliation":[{"name":"Zhejiang University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2014,4]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5539928"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1646396.1646452"},{"key":"e_1_2_1_3_1","first-page":"1232","volume-title":"NIPS","author":"Dean J.","year":"2012","unstructured":"J. Dean , G. Corrado , R. Monga , K. Chen , M. Devin , Q. V. Le , M. Z. Mao , M. Ranzato , A. W. Senior , P. A. Tucker , K. Yang , and A. Y. Ng . Large scale distributed deep networks . In NIPS , pages 1232 -- 1240 , 2012 . J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, Q. V. Le, M. Z. Mao, M. Ranzato, A. W. Senior, P. A. Tucker, K. Yang, and A. Y. Ng. Large scale distributed deep networks. In NIPS, pages 1232--1240, 2012."},{"key":"e_1_2_1_4_1","volume-title":"Saturating auto-encoder. CoRR, abs\/1301.3577","author":"Goroshin R.","year":"2013","unstructured":"R. Goroshin and Y. LeCun . Saturating auto-encoder. CoRR, abs\/1301.3577 , 2013 . R. Goroshin and Y. LeCun. Saturating auto-encoder. CoRR, abs\/1301.3577, 2013."},{"key":"e_1_2_1_5_1","volume-title":"A Practical Guide to Training Restricted Boltzmann Machines. Technical report","author":"Hinton G.","year":"2010","unstructured":"G. Hinton . A Practical Guide to Training Restricted Boltzmann Machines. Technical report , 2010 . G. Hinton. A Practical Guide to Training Restricted Boltzmann Machines. Technical report, 2010."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/958942.958948"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1460096.1460104"},{"key":"e_1_2_1_8_1","volume-title":"Learning multiple layers of features from tiny images. Technical report","author":"Krizhevsky A.","year":"2009","unstructured":"A. Krizhevsky . Learning multiple layers of features from tiny images. Technical report , 2009 . A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/2283516.2283623"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/645754.668382"},{"key":"e_1_2_1_11_1","first-page":"1","volume-title":"ICML","author":"Liu W.","year":"2011","unstructured":"W. Liu , J. Wang , S. Kumar , and S.-F. Chang . Hashing with graphs . In ICML , pages 1 -- 8 , 2011 . W. Liu, J. Wang, S. Kumar, and S.-F. Chang. Hashing with graphs. In ICML, pages 1--8, 2011."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484039"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2012-6"},{"key":"e_1_2_1_14_1","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1017\/CBO9780511809071","volume-title":"Introduction to information retrieval","author":"Manning C. D.","year":"2008","unstructured":"C. D. Manning , P. Raghavan , and H. Sch\u00fctze . Introduction to information retrieval , pages 151 -- 175 . Cambridge University Press , 2008 . C. D. Manning, P. Raghavan, and H. Sch\u00fctze. Introduction to information retrieval, pages 151--175. Cambridge University Press, 2008."},{"key":"e_1_2_1_15_1","first-page":"689","volume-title":"ICML","author":"Ngiam J.","year":"2011","unstructured":"J. Ngiam , A. Khosla , M. Kim , J. Nam , H. Lee , and A. Y. Ng . Multimodal deep learning . In ICML , pages 689 -- 696 , 2011 . J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng. Multimodal deep learning. In ICML, pages 689--696, 2011."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1873951.1873987"},{"key":"e_1_2_1_17_1","first-page":"833","volume-title":"ICML","author":"Rifai S.","year":"2011","unstructured":"S. Rifai , P. Vincent , X. Muller , X. Glorot , and Y. Bengio . Contractive auto-encoders: Explicit invariance during feature extraction . In ICML , pages 833 -- 840 , 2011 . S. Rifai, P. Vincent, X. Muller, X. Glorot, and Y. Bengio. Contractive auto-encoders: Explicit invariance during feature extraction. In ICML, pages 833--840, 2011."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijar.2008.11.006"},{"key":"e_1_2_1_19_1","first-page":"151","volume-title":"EMNLP","author":"Socher R.","year":"2011","unstructured":"R. Socher , J. Pennington , E. H. Huang , A. Y. Ng , and C. D. Manning . Semi-supervised recursive autoencoders for predicting sentiment distributions . In EMNLP , pages 151 -- 161 , 2011 . R. Socher, J. Pennington, E. H. Huang, A. Y. Ng, and C. D. Manning. Semi-supervised recursive autoencoders for predicting sentiment distributions. In EMNLP, pages 151--161, 2011."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2465274"},{"key":"e_1_2_1_21_1","first-page":"2231","volume-title":"NIPS","author":"Srivastava N.","year":"2012","unstructured":"N. Srivastava and R. Salakhutdinov . Multimodal learning with deep boltzmann machines . In NIPS , pages 2231 -- 2239 , 2012 . N. Srivastava and R. Salakhutdinov. Multimodal learning with deep boltzmann machines. In NIPS, pages 2231--2239, 2012."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390294"},{"key":"e_1_2_1_23_1","first-page":"194","volume-title":"VLDB","author":"Weber R.","year":"1998","unstructured":"R. Weber , H.-J. Schek , and S. Blott . A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces . In VLDB , pages 194 -- 205 , 1998 . R. Weber, H.-J. Schek, and S. Blott. A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces. In VLDB, pages 194--205, 1998."},{"key":"e_1_2_1_24_1","first-page":"1753","volume-title":"NIPS","author":"Weiss Y.","year":"2008","unstructured":"Y. Weiss , A. Torralba , and R. Fergus . Spectral hashing . In NIPS , pages 1753 -- 1760 , 2008 . Y. Weiss, A. Torralba, and R. Fergus. Spectral hashing. In NIPS, pages 1753--1760, 2008."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339678"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2502081.2502107"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2007.911822"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/2732296.2732301","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T09:17:44Z","timestamp":1672219064000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/2732296.2732301"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,4]]},"references-count":27,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2014,4]]}},"alternative-id":["10.14778\/2732296.2732301"],"URL":"https:\/\/doi.org\/10.14778\/2732296.2732301","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2014,4]]}}}