{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T00:27:09Z","timestamp":1778632029800,"version":"3.51.4"},"reference-count":40,"publisher":"Association for Computing Machinery (ACM)","issue":"1s","license":[{"start":{"date-parts":[[2019,1,24]],"date-time":"2019-01-24T00:00:00Z","timestamp":1548288000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"crossref","award":["DUT18RC(3)025"],"award-info":[{"award-number":["DUT18RC(3)025"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100011002","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61672123"],"award-info":[{"award-number":["61672123"]}],"id":[{"id":"10.13039\/501100011002","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Key Science and Technology Planning Project of Guangdong Province, China","award":["2015B010110006"],"award-info":[{"award-number":["2015B010110006"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2019,1,31]]},"abstract":"<jats:p>Transfer learning, which focuses on finding a favorable representation for instances of different domains based on auxiliary data, can mitigate the divergence between domains through knowledge transfer. Recently, increasing efforts on transfer learning have employed<jats:bold>d<\/jats:bold>eep<jats:bold>n<\/jats:bold>eural<jats:bold>n<\/jats:bold>etworks (DNN) to learn more robust and higher level feature representations to better tackle cross-media disparities. However, only a few articles consider the correction and semantic matching between multi-layer heterogeneous domain networks. In this article, we propose a<jats:bold>d<\/jats:bold>eep semantic mapping model for<jats:bold>h<\/jats:bold>eterogeneous multimedia<jats:bold>t<\/jats:bold>ransfer<jats:bold>l<\/jats:bold>earning (DHTL) using co-occurrence data. More specifically, we integrate the DNN with<jats:bold>c<\/jats:bold>anonical<jats:bold>c<\/jats:bold>orrelation<jats:bold>a<\/jats:bold>nalysis (CCA) to derive a deep correlation subspace as the joint semantic representation for associating data across different domains. In the proposed DHTL, a multi-layer correlation matching network across domains is constructed, in which the CCA is combined to bridge each pair of domain-specific hidden layers. To train the network, a joint objective function is defined and the optimization processes are presented. When the deep semantic representation is achieved, the shared features of the source domain are transferred for task learning in the target domain. Extensive experiments for three multimedia recognition applications demonstrate that the proposed DHTL can effectively find deep semantic representations for heterogeneous domains, and it is superior to the several existing state-of-the-art methods for deep transfer learning.<\/jats:p>","DOI":"10.1145\/3241055","type":"journal-article","created":{"date-parts":[[2019,1,28]],"date-time":"2019-01-28T14:01:39Z","timestamp":1548684099000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Deep Semantic Mapping for Heterogeneous Multimedia Transfer Learning Using Co-Occurrence Data"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6301-1311","authenticated-orcid":false,"given":"Liang","family":"Zhao","sequence":"first","affiliation":[{"name":"Dalian University of Technology, Dalian, Liaoning, China"}]},{"given":"Zhikui","family":"Chen","sequence":"additional","affiliation":[{"name":"Dalian University of Technology, Dalian, Liaoning, China"}]},{"given":"Laurence T.","family":"Yang","sequence":"additional","affiliation":[{"name":"St. Francis Xavier University, Antigonish, NS, Canada"}]},{"given":"M. Jamal","family":"Deen","sequence":"additional","affiliation":[{"name":"McMaster University, Hamilton, ON, Canada"}]},{"given":"Z. Jane","family":"Wang","sequence":"additional","affiliation":[{"name":"University of British Columbia, Vancouver, BC, Canada"}]}],"member":"320","published-online":{"date-parts":[[2019,1,24]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"International Conference on Machine Learning. 1247--1255","author":"Andrew G.","unstructured":"G. Andrew , R. Arora , J. Bilmes , and K. Livescu . 2013. Deep canonical correlation analysis . In International Conference on Machine Learning. 1247--1255 . G. Andrew, R. Arora, J. Bilmes, and K. Livescu. 2013. Deep canonical correlation analysis. In International Conference on Machine Learning. 1247--1255."},{"key":"e_1_2_1_2_1","volume-title":"International Conference on Machine Learning. 1--8.","author":"Chen M.","unstructured":"M. Chen , Z. Xu , K. Weinberger , and F. Sha . 2012. Marginalized denoising autoencoders for domain adaptation . In International Conference on Machine Learning. 1--8. M. Chen, Z. Xu, K. Weinberger, and F. Sha. 2012. Marginalized denoising autoencoders for domain adaptation. In International Conference on Machine Learning. 1--8."},{"key":"e_1_2_1_3_1","volume-title":"IEEE International Conference on Acoustics, Speech and Signal Processing. 2414--2418","author":"Ding Z.","unstructured":"Z. Ding , N. M. Nasrabadi , and Y. Fu . 2016. Task-driven deep transfer learning for image classification . In IEEE International Conference on Acoustics, Speech and Signal Processing. 2414--2418 . Z. Ding, N. M. Nasrabadi, and Y. Fu. 2016. Task-driven deep transfer learning for image classification. In IEEE International Conference on Acoustics, Speech and Signal Processing. 2414--2418."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.92"},{"key":"e_1_2_1_5_1","volume-title":"International Conference on Machine Learning. 711--718","author":"Duan L.","unstructured":"L. Duan , D. Xu , and I. Tsang . 2012. Learning with augmented features for heterogeneous domain adaptation . In International Conference on Machine Learning. 711--718 . L. Duan, D. Xu, and I. Tsang. 2012. Learning with augmented features for heterogeneous domain adaptation. In International Conference on Machine Learning. 711--718."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2014.12.020"},{"key":"e_1_2_1_7_1","volume-title":"International Conference on Machine Learning. 513--520","author":"Glorot X.","unstructured":"X. Glorot , A. Bordes , and Y. Bengio . 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach . In International Conference on Machine Learning. 513--520 . X. Glorot, A. Bordes, and Y. Bengio. 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach. In International Conference on Machine Learning. 513--520."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1162\/0899766042321814"},{"key":"e_1_2_1_9_1","unstructured":"J. Hoffman D. Wang F. Yu and T. Darrell. 2016. FCNS in the wild: Pixel-level adversarial and constraint-based adaptation. In Computer Vision and Pattern Recognition. arXiv preprint arXiv:1612.02649. J. Hoffman D. Wang F. Yu and T. Darrell. 2016. FCNS in the wild: Pixel-level adversarial and constraint-based adaptation. In Computer Vision and Pattern Recognition. arXiv preprint arXiv:1612.02649."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2012.2206040"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995702"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.167"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.183"},{"key":"e_1_2_1_14_1","volume-title":"International Conference on Machine Learning. 689--696","author":"Ngiam J.","unstructured":"J. Ngiam , A. Khosla , M. Kim , J. Nam , H. Lee , and A. Y. Ng . 2012. Multimodal deep learning . In International Conference on Machine Learning. 689--696 . J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng. 2012. Multimodal deep learning. In International Conference on Machine Learning. 689--696."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.95"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2010.2091281"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2009.191"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2013.01.003"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.142"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273592"},{"key":"e_1_2_1_21_1","volume-title":"IEEE International Conference on Acoustics, Speech and Signal Processing. 5800--5804","author":"Sagha H.","unstructured":"H. Sagha , J. Deng , M. Gavryukova , J. Han , and B. Schuller . 2016. Cross lingual speech emotion recognition using canonical correlation analysis on principal component subspace . In IEEE International Conference on Acoustics, Speech and Signal Processing. 5800--5804 . H. Sagha, J. Deng, M. Gavryukova, J. Han, and B. Schuller. 2016. Cross lingual speech emotion recognition using canonical correlation analysis on principal component subspace. In IEEE International Conference on Acoustics, Speech and Signal Processing. 5800--5804."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2733373.2806216"},{"key":"e_1_2_1_23_1","unstructured":"R. Socher M. Ganjoo H. Sridhar O. Bastani C. D. Manning and A. Y. Ng. 2013. Zero-shot learning through cross-modal transfer. In Advances in Neural Information Processing Systems. 935--943. R. Socher M. Ganjoo H. Sridhar O. Bastani C. D. Manning and A. Y. Ng. 2013. Zero-shot learning through cross-modal transfer. In Advances in Neural Information Processing Systems. 935--943."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998574"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2015.2487860"},{"key":"e_1_2_1_26_1","doi-asserted-by":"crossref","unstructured":"E. Tzeng J. Hoffman K. Saenko and T. Darrell. 2017. Adversarial discriminative domain adaptation. In Computer Vision and Pattern Recognition. 7167--7176. E. Tzeng J. Hoffman K. Saenko and T. Darrell. 2017. Adversarial discriminative domain adaptation. In Computer Vision and Pattern Recognition. 7167--7176.","DOI":"10.1109\/CVPR.2017.316"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2015.2465157"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2015.2472457"},{"key":"e_1_2_1_29_1","volume-title":"Joint Conference of the Meeting of the ACL and the International Joint Conference on Natural Language Processing. 1--9.","author":"Yang Q.","unstructured":"Q. Yang , Y. Chen , G. R. Xue , W. Dai , and Y. Yu . 2009. Heterogeneous transfer learning for image clustering via the social web . In Joint Conference of the Meeting of the ACL and the International Joint Conference on Natural Language Processing. 1--9. Q. Yang, Y. Chen, G. R. Xue, W. Dai, and Y. Yu. 2009. Heterogeneous transfer learning for image clustering via the social web. In Joint Conference of the Meeting of the ACL and the International Joint Conference on Natural Language Processing. 1--9."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2700286"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2014.2310992"},{"key":"e_1_2_1_32_1","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1109\/TSC.2015.2497705","article-title":"Deep computation model for unsupervised feature learning on big data","volume":"9","author":"Zhang Q.","year":"2016","unstructured":"Q. Zhang , L. T. Yang , and Z. Chen . 2016 . Deep computation model for unsupervised feature learning on big data . IEEE Transactions on Services Computing 9 , 1 (2016), 161 -- 171 . Q. Zhang, L. T. Yang, and Z. Chen. 2016. Deep computation model for unsupervised feature learning on big data. IEEE Transactions on Services Computing 9, 1 (2016), 161--171.","journal-title":"IEEE Transactions on Services Computing"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2886779"},{"key":"e_1_2_1_34_1","unstructured":"X. Zhang F. X. Yu S. F. Chang and S. Wang. 2015. Deep transfer network: Unsupervised domain adaptation. In Computer Vision and Pattern Recognition. X. Zhang F. X. Yu S. F. Chang and S. Wang. 2015. Deep transfer network: Unsupervised domain adaptation. In Computer Vision and Pattern Recognition."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2017.2769086"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSYST.2016.2576026"},{"key":"e_1_2_1_37_1","volume-title":"28th AAAI Conference on Artificial Intelligence. 2213--2219","author":"Zhou J. T.","unstructured":"J. T. Zhou , S. J. Pan , I. W. Tsang , and Y. Yan . 2014. Hybrid heterogeneous transfer learning through deep learning . In 28th AAAI Conference on Artificial Intelligence. 2213--2219 . J. T. Zhou, S. J. Pan, I. W. Tsang, and Y. Yan. 2014. Hybrid heterogeneous transfer learning through deep learning. In 28th AAAI Conference on Artificial Intelligence. 2213--2219."},{"key":"e_1_2_1_38_1","unstructured":"J. T. Zhou I. W. Tsang S. J. Pan and M. Tan. 2014. Heterogeneous domain adaptation for multiple classes. In Artificial Intelligence and Statistics. 1095--1103. J. T. Zhou I. W. Tsang S. J. Pan and M. Tan. 2014. Heterogeneous domain adaptation for multiple classes. In Artificial Intelligence and Statistics. 1095--1103."},{"key":"e_1_2_1_39_1","volume-title":"AAAI Conference on Artificial Intelligence. 1304--1309","author":"Zhu Y.","unstructured":"Y. Zhu , Y. Chen , Z. Lu , S. J. Pan , G. R. Xue , Y. Yu , and Q. Yang . 2011. Heterogeneous transfer learning for image classification . In AAAI Conference on Artificial Intelligence. 1304--1309 . Y. Zhu, Y. Chen, Z. Lu, S. J. Pan, G. R. Xue, Y. Yu, and Q. Yang. 2011. Heterogeneous transfer learning for image classification. In AAAI Conference on Artificial Intelligence. 1304--1309."},{"key":"e_1_2_1_40_1","volume-title":"International Conference on Artificial Intelligence. 4119--4125","author":"Zhuang F.","unstructured":"F. Zhuang , X. Cheng , P. Luo , S. J. Pan , and Q. He . 2015. Supervised representation learning: Transfer learning with deep autoencoders . In International Conference on Artificial Intelligence. 4119--4125 . F. Zhuang, X. Cheng, P. Luo, S. J. Pan, and Q. He. 2015. Supervised representation learning: Transfer learning with deep autoencoders. In International Conference on Artificial Intelligence. 4119--4125."}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3241055","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3241055","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:43:46Z","timestamp":1750207426000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3241055"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,24]]},"references-count":40,"journal-issue":{"issue":"1s","published-print":{"date-parts":[[2019,1,31]]}},"alternative-id":["10.1145\/3241055"],"URL":"https:\/\/doi.org\/10.1145\/3241055","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,24]]},"assertion":[{"value":"2017-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-01-24","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}