{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:20:56Z","timestamp":1760239256239,"version":"build-2065373602"},"reference-count":35,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2020,10,22]],"date-time":"2020-10-22T00:00:00Z","timestamp":1603324800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"China-Korea Young Scientist Exchange Program (2020)","award":["-"],"award-info":[{"award-number":["-"]}]},{"name":"Science Foundation of Shenyang University of Chemical Technology","award":["LQ2020020"],"award-info":[{"award-number":["LQ2020020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>To classify the image material on the internet, the deep learning methodology, especially deep neural network, is the most optimal and costliest method of all computer vision methods. Convolutional neural networks (CNNs) learn a comprehensive feature representation by exploiting local information with a fixed receptive field, demonstrating distinguished capacities on image classification. Recent works concentrate on efficient feature exploration, which neglect the global information for holistic consideration. There is large effort to reduce the computational costs of deep neural networks. Here, we provide a hierarchical global attention mechanism that improve the network representation with restricted increase of computation complexity. Different from nonlocal-based methods, the hierarchical global attention mechanism requires no matrix multiplication and can be flexibly applied in various modern network designs. Experimental results demonstrate that proposed hierarchical global attention mechanism can conspicuously improve the image classification precision\u2014a reduction of 7.94% and 16.63% percent in Top 1 and Top 5 errors separately\u2014with little increase of computation complexity (6.23%) in comparison to competing approaches.<\/jats:p>","DOI":"10.3390\/fi12110178","type":"journal-article","created":{"date-parts":[[2020,10,22]],"date-time":"2020-10-22T10:27:58Z","timestamp":1603362478000},"page":"178","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Learning a Hierarchical Global Attention for Image Classification"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4053-7166","authenticated-orcid":false,"given":"Kerang","family":"Cao","sequence":"first","affiliation":[{"name":"Shenyang University of Chemical Technology, Shenyang 110000, China"}]},{"given":"Jingyu","family":"Gao","sequence":"additional","affiliation":[{"name":"Shenyang University of Chemical Technology, Shenyang 110000, China"}]},{"given":"Kwang-nam","family":"Choi","sequence":"additional","affiliation":[{"name":"NTIS Center, Korea Institute of Science and Technology Information, Seoul 02792, Korea"}]},{"given":"Lini","family":"Duan","sequence":"additional","affiliation":[{"name":"Shenyang University of Chemical Technology, Shenyang 110000, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,10,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Florea, G., and Mihailescu, R.C. (2020). Deep Learning for Group Activity Recognition in Smart Office Environments. Future Internet, 12.","DOI":"10.3390\/fi12080133"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Song, X., Yang, H., and Zhou, C. (2019). Pedestrian Attribute Recognition with Graph Convolutional Network in Surveillance Scenarios. Future Internet, 11.","DOI":"10.3390\/fi11110245"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Liu, W., Qian, J., Yao, Z., and Pan, J. (2019). Convolutional Two-Stream Network Using Multi-Facial Feature Fusion for Driver Fatigue Detection. Future Internet, 11.","DOI":"10.3390\/fi11050115"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Song, A., Wu, Z., Ding, X., Hu, Q., and Di, X. (2018). Neurologist Standard Classification of Facial Nerve Paralysis with Deep Neural Networks. Future Internet, 10.","DOI":"10.3390\/fi10110111"},{"key":"ref_6","unstructured":"Roychowdhury, S., Hage, P., and Vasquez, J. (2020). Azure-Based Smart Monitoring System for Anemia-Like Pallor. Appl. Sci., 10."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201322). Squeeze-and-Excitation Networks. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., and Tang, X. (2017, January 21\u201326). Residual Attention Network for Image Classification. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.683"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wang, X., Girshick, R.B., Gupta, A., and He, K. (2018, January 18\u201322). Non-Local Neural Networks. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00813"},{"key":"ref_10","unstructured":"Krizhevsky, A., and Hinton, G. (2009). Learning Multiple Layers of Features from Tiny Images, Computer Science Department, University of Toronto. Technically Report."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Li, F. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_12","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012, Lake Tahoe, NV, USA."},{"key":"ref_13","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S.E., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7\u201312). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_16","unstructured":"Gao, S., Cheng, M., Zhao, K., Zhang, X., Yang, M., and Torr, P.H.S. (2019). Res2Net: A New Multi-scale Backbone Architecture. IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Xie, S., Girshick, R.B., Doll\u00e1r, P., Tu, Z., and He, K. (2017, January 21\u201326). Aggregated Residual Transformations for Deep Neural Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.634"},{"key":"ref_18","unstructured":"Zhang, H., Wu, C., Zhang, Z., Zhu, Y., Zhang, Z., Lin, H., Sun, Y., He, T., Mueller, J., and Manmatha, R. (2020). ResNeSt: Split-Attention Networks. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zagoruyko, S., and Komodakis, N. (2016, January 19\u201322). Wide Residual Networks. Proceedings of the British Machine Vision Conference 2016, BMVC 2016, York, UK.","DOI":"10.5244\/C.30.87"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Han, D., Kim, J., and Kim, J. (2017, January 21\u201326). Deep Pyramidal Residual Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.668"},{"key":"ref_22","unstructured":"Larsson, G., Maire, M., and Shakhnarovich, G. (2017, January 24\u201326). FractalNet: Ultra-Deep Neural Networks without Residuals. Proceedings of the 5th International Conference on Learning Representations, ICLR 2017, Toulon, France."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhang, T., Qi, G., Xiao, B., and Wang, J. (2017). Interleaved Group Convolutions for Deep Neural Networks. arXiv.","DOI":"10.1109\/ICCV.2017.469"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Xie, G., Wang, J., Zhang, T., Lai, J., Hong, R., and Qi, G. (2018). IGCV2: Interleaved Structured Sparse Convolutional Neural Networks. arXiv.","DOI":"10.1109\/CVPR.2018.00922"},{"key":"ref_25","unstructured":"Sun, K., Li, M., Liu, D., and Wang, J. (2018, January 3\u20136). IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks. Proceedings of the British Machine Vision Conference 2018, BMVC 2018, Newcastle, UK."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep Learning with Depthwise Separable Convolutions. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhang, X., Li, Z., Loy, C.C., and Lin, D. (2017, January 21\u201326). PolyNet: A Pursuit of Structural Diversity in Very Deep Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.415"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Xie, L., and Yuille, A.L. (2017, January 22\u201329). Genetic CNN. Proceedings of the IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy.","DOI":"10.1109\/ICCV.2017.154"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018, January 18\u201322). Learning Transferable Architectures for Scalable Image Recognition. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00907"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Liu, C., Zoph, B., Neumann, M., Shlens, J., Hua, W., Li, F., Yuille, A.L., Huang, J., and Murphy, K. (2018, January 8\u201314). Progressive Neural Architecture Search. Proceedings of the Computer Vision\u2014ECCV 2018\u201415th European Conference, Munich, Germany.","DOI":"10.1007\/978-3-030-01246-5_2"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., and Le, Q.V. (2019, January 16\u201320). MnasNet: Platform-Aware Neural Architecture Search for Mobile. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00293"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhong, Z., Yan, J., Wu, W., Shao, J., and Liu, C. (2018, January 18\u201322). Practical Block-Wise Neural Network Architecture Generation. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00257"},{"key":"ref_33","unstructured":"Baker, B., Gupta, O., Naik, N., and Raskar, R. (2017, January 24\u201326). Designing Neural Network Architectures using Reinforcement Learning. Proceedings of the 5th International Conference on Learning Representations, ICLR 2017, Toulon, France."},{"key":"ref_34","unstructured":"Liu, H., Simonyan, K., and Yang, Y. (2019, January 6\u20139). DARTS: Differentiable Architecture Search. Proceedings of the 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Li, X., Wang, W., Hu, X., and Yang, J. (2019, January 16\u201320). Selective Kernel Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00060"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/12\/11\/178\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:25:56Z","timestamp":1760178356000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/12\/11\/178"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,22]]},"references-count":35,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["fi12110178"],"URL":"https:\/\/doi.org\/10.3390\/fi12110178","relation":{},"ISSN":["1999-5903"],"issn-type":[{"type":"electronic","value":"1999-5903"}],"subject":[],"published":{"date-parts":[[2020,10,22]]}}}