{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,26]],"date-time":"2026-09-26T13:59:23Z","timestamp":1790431163420,"version":"4.1.0"},"reference-count":35,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,9,4]],"date-time":"2022-09-04T00:00:00Z","timestamp":1662249600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Taiwan Ministry of Science and Technology","award":["MOST 111-2410-H-A49-019"],"award-info":[{"award-number":["MOST 111-2410-H-A49-019"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>With the successful development in computer vision, building a deep convolutional neural network (CNNs) has been mainstream, considering the character of shared parameters in a convolutional layer. Stacking convolutional layers into a deep structure improves performance, but over-stacking also ramps up the needed resources for GPUs. Seeing another surge of Transformers in computer vision, the issue has aroused severely. A resource-hungry model is hardly implemented for limited hardware or single-customers-based GPU. Therefore, this work focuses on these concerns and proposes an efficient but robust backbone, which equips with channel and spatial direction attentions, so the attentions help to expand receptive fields in shallow convolutional layers and pass the information to every layer. An attention-boosted network based on already efficient CNNs, Universal Pixel Attention Networks (UPANets), is proposed. Through a series of experiments, UPANets fulfil the purposes of learning global information with less needed resources and outshine many existing SOTAs in CIFAR-{10, 100}.<\/jats:p>","DOI":"10.3390\/e24091243","type":"journal-article","created":{"date-parts":[[2022,9,5]],"date-time":"2022-09-05T23:35:57Z","timestamp":1662420957000},"page":"1243","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["UPANets: Learning from the Universal Pixel Attention Neworks"],"prefix":"10.3390","volume":"24","author":[{"given":"Ching-Hsun","family":"Tseng","sequence":"first","affiliation":[{"name":"Department of Computer Science, The University of Manchester, Manchester M13 9PL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shin-Jye","family":"Lee","sequence":"additional","affiliation":[{"name":"Institute of Management of Technology, National Chiao Tung University, Hsinchu 300, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6701-2730","authenticated-orcid":false,"given":"Jianan","family":"Feng","sequence":"additional","affiliation":[{"name":"School of Software, Yunnan University, Kunming 650504, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shengzhong","family":"Mao","sequence":"additional","affiliation":[{"name":"Department of Computer Science, The University of Manchester, Manchester M13 9PL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6399-9710","authenticated-orcid":false,"given":"Yu-Ping","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, The University of Manchester, Manchester M13 9PL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jia-Yu","family":"Shang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, The University of Manchester, Manchester M13 9PL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiao-Jun","family":"Zeng","sequence":"additional","affiliation":[{"name":"Department of Computer Science, The University of Manchester, Manchester M13 9PL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,4]]},"reference":[{"key":"ref_1","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_2","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the inception architecture for computer vision. Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2011","DOI":"10.1109\/TPAMI.2019.2913372","article-title":"Squeeze-and-Excitation Networks","volume":"42","author":"Hu","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_7","unstructured":"Yung, J., Gelly, S., and Houlsby, N. (2020). Big Transfer (BiT): General Visual Representation Learning. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ridnik, T., Lawen, H., Noy, A., Baruch, E.B., Sharir, G., and Friedman, I. (2021, January 5\u20139). Tresnet: High performance gpu-dedicated architecture. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV48630.2021.00144"},{"key":"ref_9","unstructured":"Zhao, S., Zhou, L., Wang, W., Cai, D., Lam, T.L., and Xu, Y. (2020). SplitNet: Divide and Co-training. arXiv."},{"key":"ref_10","unstructured":"Kabir, H.M.D., Abdar, M., Khosravi, A., Jalali, S.M.J., Atiya, A.F., Nahavandi, S., and Srinivasan, D. (2020). Spinalnet: Deep neural network with gradual input. arXiv."},{"key":"ref_11","unstructured":"Lee, H., Kim, H.-E., and Nam, H. (November, January 27). Srm: A style-based recalibration module for convolutional neural networks. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Shah, A., Kadam, E., Shah, H., Shinde, S., and Shingade, S. (2016, January 21\u201324). Deep residual networks with exponential linear unit. Proceedings of the Third International Symposium on Computer Vision and the Internet, Jaipur, Rajasthan, India.","DOI":"10.1145\/2983402.2983406"},{"key":"ref_13","unstructured":"Misra, D. (2019). Mish: A self regularized non-monotonic activation function. arXiv."},{"key":"ref_14","unstructured":"Deng, W., Feng, Q., Gao, L., Liang, F., and Lin, G. (2020, January 13\u201318). Non-convex Learning via Replica Exchange Stochastic Gradient MCMC. Proceedings of the International Conference on Machine Learning, PMLR, Virtual."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1007\/s41095-019-0149-9","article-title":"Salient object detection: A survey","volume":"5","author":"Borji","year":"2019","journal-title":"Comput. Vis. Media"},{"key":"ref_16","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_17","unstructured":"Han, K., Xiao, A., Wu, E., Guo, J., Xu, C., and Wang, Y. (2021). Transformer in Transformer. arXiv."},{"key":"ref_18","unstructured":"Hudson, D.A., and Zitnick, C.L. (2021). Generative Adversarial Transformers. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. (2020). Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. arXiv.","DOI":"10.1609\/aaai.v35i12.17325"},{"key":"ref_20","unstructured":"Li, H., Xu, Z., Taylor, G., Studer, C., and Goldstein, T. (2017). Visualizing the loss landscape of neural nets. arXiv."},{"key":"ref_21","unstructured":"Tan, M., and Le, Q. (2019, January 10\u201315). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_22","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. arXiv."},{"key":"ref_23","unstructured":"Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and J\u00e9gou, H. (2020). Training data-efficient image transformers & distillation through attention. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8\u201314). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201322). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Xie, S., Girshick, R., Doll\u00e1r, P., Tu, Z., and He, K. (2017, January 21\u201326). Aggregated residual transformations for deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.634"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., and Lu, H. (2019, January 15\u201320). Dual attention network for scene segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00326"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Cao, Y., Xu, J., Lin, S., Wei, F., and Hu, H. (2019, January 27\u201328). Gcnet: Non-local networks meet squeeze-excitation networks and beyond. Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops, Seoul, Korea.","DOI":"10.1109\/ICCVW.2019.00246"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1386","DOI":"10.1109\/50.90937","article-title":"Shuffle net: An application of generalized perfect shuffles to multihop lightwave networks","volume":"9","author":"Hluchyj","year":"1991","journal-title":"J. Lightwave Technol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ma, N., Zhang, X., Zheng, H.-T., and Sun, J. (2018, January 8\u201314). Shufflenet v2: Practical guidelines for efficient cnn architecture design. Proceedings of the European conference on computer vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_8"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3780","DOI":"10.1109\/TIP.2022.3175601","article-title":"Joint Representation Learning and Keypoint Detection for Cross-view Geo-localization","volume":"31","author":"Lin","year":"2022","journal-title":"IEEE Trans. Image Process."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"867","DOI":"10.1109\/TCSVT.2021.3061265","article-title":"Each part matters: Local patterns facilitate cross-view geo-localization","volume":"32","author":"Wang","year":"2021","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_34","unstructured":"Abai, Z., and Rajmalwar, N. (2019). DenseNet Models for Tiny ImageNet Classification. arXiv."},{"key":"ref_35","unstructured":"Kim, J.-H., Choo, W., and Song, H.O. (2020, January 13\u201318). Puzzle mix: Exploiting saliency and local statistics for optimal mixup. Proceedings of the International Conference on Machine Learning, PMLR, Virtual."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/9\/1243\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:23:07Z","timestamp":1760142187000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/9\/1243"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,4]]},"references-count":35,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["e24091243"],"URL":"https:\/\/doi.org\/10.3390\/e24091243","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,4]]}}}