{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T11:21:09Z","timestamp":1768821669412,"version":"3.49.0"},"reference-count":43,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,4,18]],"date-time":"2020-04-18T00:00:00Z","timestamp":1587168000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61601230"],"award-info":[{"award-number":["61601230"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Domain adaptation is critical to transfer the invaluable source domain knowledge to the target domain. In this paper, for a particular visual attention model, saying hard attention, we consider to adapt the learned hard attention to the unlabeled target domain. To tackle this kind of hard attention adaptation, a novel adversarial reward strategy is proposed to train the policy of the target domain agent. In this adversarial training framework, the target domain agent competes with the discriminator which takes the attention features generated from the both domain agents as input and tries its best to distinguish them, and thus the target domain policy is learned to align the local attention feature to its source domain counterpart. We evaluated our model on the benchmarks of the cross-domain tasks, such as the centered digits datasets and the enlarged non-centered digits datasets. The experimental results show that our model outperforms the ADDA and other existing methods.<\/jats:p>","DOI":"10.3390\/info11040224","type":"journal-article","created":{"date-parts":[[2020,4,21]],"date-time":"2020-04-21T03:23:06Z","timestamp":1587439386000},"page":"224","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Adversarial Hard Attention Adaptation"],"prefix":"10.3390","volume":"11","author":[{"given":"Hui","family":"Tao","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing 210044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1671-2283","authenticated-orcid":false,"given":"Jun","family":"He","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing 210044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Quanjie","family":"Cao","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing 210044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lei","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Electrical and Automation Engineering, Nanjing Normal University, Nanjing 210023, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,18]]},"reference":[{"key":"ref_1","unstructured":"Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2014, January 8\u201313). How transferable are features in deep neural networks?. Proceedings of the Advances in Neural Information Processing Systems 27, Montreal, QC, Canada."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Oquab, M., Bottou, L., Laptev, I., and Sivic, J. (2014, January 24\u201327). Learning and transferring mid-level image representations using convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.222"},{"key":"ref_3","unstructured":"Gretton, A., Smola, A., Huang, J., Schmittfull, M., Borgwardt, K., and Sch\u00f6lkopf, B. (2009). Covariate shift and local learning by distribution matching. Dataset Shift in Machine Learning, MIT Press."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N.D. (2009). Dataset Shift in Machine Learning, The MIT Press.","DOI":"10.7551\/mitpress\/9780262170055.001.0001"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1109\/TASLP.2016.2621669","article-title":"Bayesian Unsupervised Batch and Online Speaker Adaptation of Activation Function Parameters in Deep Models for Automatic Speech Recognition","volume":"25","author":"Huang","year":"2017","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"key":"ref_6","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems 27, Montreal, QC, Canada."},{"key":"ref_7","unstructured":"Ganin, Y., and Lempitsky, V. (2015, January 6\u201311). Unsupervised Domain Adaptation by Backpropagation. Proceedings of the 32nd International Conference on International Conference on Machine Learning, ICML\u201915, Lille, France."},{"key":"ref_8","first-page":"2096","article-title":"Domain-adversarial training of neural networks","volume":"17","author":"Ganin","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Tzeng, E., Hoffman, J., Darrell, T., and Saenko, K. (2015, January 7\u201313). Simultaneous deep transfer across domains and tasks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.463"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Saito, K., Watanabe, K., Ushiku, Y., and Harada, T. (2018, January 18\u201322). Maximum classifier discrepancy for unsupervised domain adaptation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00392"},{"key":"ref_11","unstructured":"Saito, K., Ushiku, Y., Harada, T., and Saenko, K. (May, January 30). Adversarial Dropout Regularization. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Tzeng, E., Hoffman, J., Saenko, K., and Darrell, T. (2017, January 21\u201326). Adversarial discriminative domain adaptation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.316"},{"key":"ref_13","unstructured":"Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. arXiv."},{"key":"ref_14","unstructured":"Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., and Bengio, Y. (2015, January 6\u201311). Show, attend and tell: Neural image caption generation with visual attention. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Vinyals, O., Toshev, A., Bengio, S., and Erhan, D. (2015, January 7\u201312). Show and tell: A neural image caption generator. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298935"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"You, Q., Jin, H., Wang, Z., Fang, C., and Luo, J. (2016, January 27\u201330). Image captioning with semantic attention. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.503"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chen, L., Zhang, H., Xiao, J., Nie, L., Shao, J., Liu, W., and Chua, T.S. (2017, January 21\u201326). Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.667"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S., and Zhang, L. (2018, January 18\u201322). Bottom-up and top-down attention for image captioning and visual question answering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00636"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Xu, H., and Saenko, K. (2016, January 11\u201314). Ask, attend and answer: Exploring question-guided spatial attention for visual question answering. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46478-7_28"},{"key":"ref_20","unstructured":"Lu, J., Yang, J., Batra, D., and Parikh, D. (2016, January 5\u201310). Hierarchical question-image co-attention for visual question answering. Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, Barcelona, Spain."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Groth, O., Bernstein, M., and Fei-Fei, L. (2016, January 27\u201330). Visual7w: Grounded question answering in images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.540"},{"key":"ref_22","unstructured":"Wang, X., Li, L., Ye, W., Long, M., and Wang, J. (February, January 27). Transferable attention for domain adaptation. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), Honolulu, HI, USA."},{"key":"ref_23","unstructured":"Mnih, V., Heess, N., and Graves, A. (2014, January 8\u201313). Recurrent models of visual attention. Proceedings of the Advances in Neural Information Processing Systems 27, Montreal, QC, Canada."},{"key":"ref_24","unstructured":"Ba, J., Mnih, V., and Kavukcuoglu, K. (2015, January 7\u20139). Multiple Object Recognition with Visual Attention. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"40707","DOI":"10.1109\/ACCESS.2019.2907992","article-title":"Long Document Classification From Local Word Glimpses via Recurrent Attention Learning","volume":"7","author":"He","year":"2019","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2287","DOI":"10.1109\/JSEN.2018.2885796","article-title":"Weakly Supervised Human Activity Recognition From Wearable Sensors by Recurrent Attention Learning","volume":"19","author":"He","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Huang, J., Gretton, A., Borgwardt, K., Sch\u00f6lkopf, B., and Smola, A.J. (2007, January 3\u20136). Correcting sample selection bias by unlabeled data. Proceedings of the Advances in Neural Information Processing Systems 20, Vancouver, BC, Canada.","DOI":"10.7551\/mitpress\/7503.003.0080"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yan, H., Ding, Y., Li, P., Wang, Q., Xu, Y., and Zuo, W. (2017, January 21\u201326). Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.107"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Sun, B., and Saenko, K. (2016, January 11\u201314). Deep coral: Correlation alignment for deep domain adaptation. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-49409-8_35"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sun, B., Feng, J., and Saenko, K. (2016, January 12\u201317). Return of frustratingly easy domain adaptation. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10306"},{"key":"ref_31","unstructured":"Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., and Darrell, T. (2014). Deep Domain Confusion: Maximizing for Domain Invariance. arXiv."},{"key":"ref_32","unstructured":"Long, M., Cao, Y., Wang, J., and Jordan, M.I. (2015, January 6\u201311). Learning Transferable Features with Deep Adaptation Networks. Proceedings of the 32nd International Conference on International Conference on Machine Learning, ICML\u201915, Lille, France."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"e49","DOI":"10.1093\/bioinformatics\/btl242","article-title":"Integrating structured biological data by kernel maximum mean discrepancy","volume":"22","author":"Borgwardt","year":"2006","journal-title":"Bioinformatics"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22\u201329). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_35","unstructured":"Hoffman, J., Tzeng, E., Park, T., Zhu, J.Y., Isola, P., Saenko, K., Efros, A., and Darrell, T. (2018, January 10\u201315). CyCADA: Cycle-Consistent Adversarial Domain Adaptation. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Graves, A., Mohamed, A.R., and Hinton, G. (2013, January 26\u201331). Speech recognition with deep recurrent neural networks. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638947"},{"key":"ref_37","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_38","unstructured":"Netzer, Y., Tao, W., Coates, A., Bissacco, A., Bo, W., and Ng, A.Y. (2011, January 16\u201317). Reading Digits in Natural Images with Unsupervised Feature Learning. Proceedings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning, Sierra Nevada, Spain."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_41","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA."},{"key":"ref_42","unstructured":"Liu, M.Y., and Tuzel, O. (2016, January 5\u201310). Coupled generative adversarial networks. Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, Barcelona, Spain."},{"key":"ref_43","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Mach. Learn. Res."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/4\/224\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:31:29Z","timestamp":1760362289000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/4\/224"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,18]]},"references-count":43,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["info11040224"],"URL":"https:\/\/doi.org\/10.3390\/info11040224","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,18]]}}}