{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T02:10:16Z","timestamp":1778811016838,"version":"3.51.4"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,1,5]],"date-time":"2023-01-05T00:00:00Z","timestamp":1672876800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2023,1,31]]},"abstract":"<jats:p>\n            The task of image multi-label classification is to accurately recognize multiple objects in an input image. Most of the recent works need to leverage the label co-occurrence matrix counted from training data to construct the graph structure, which are inflexible and may degrade model generalizability. In addition, these methods fail to capture the semantic correlation between the channel feature maps to further improve model performance. To address these issues, we propose DA-GAT (a\n            <jats:italic>D<\/jats:italic>\n            ouble\n            <jats:italic>A<\/jats:italic>\n            ttention framework based on the\n            <jats:italic>G<\/jats:italic>\n            raph\n            <jats:italic>A<\/jats:italic>\n            ttention ne\n            <jats:italic>T<\/jats:italic>\n            work) to effectively learn the correlation between labels from training data. First, we devise a new channel attention mechanism to enhance the semantic correlation between channel feature maps, so as to implicitly capture the correlation between labels. Second, we propose a new label attention mechanism to avoid the adverse impact of a manually constructed label co-occurrence matrix. It only needs to leverage the label embedding as the input of network, then automatically constructs the label relation matrix to explicitly establish the correlation between labels. Finally, we effectively fuse the output of these two attention mechanisms to further improve model performance. Extensive experiments are conducted on three public multi-label classification benchmarks. Our DA-GAT model achieves mean average precision of 87.1%, 96.6%, and 64.3% on MS-COCO 2014, PASCAL VOC 2007, and NUS-WIDE, respectively, and obviously outperforms other existing state-of-the-art methods. In addition, visual analysis experiments demonstrate that each attention mechanism can capture the correlation between labels well and significantly promote the model performance.\n          <\/jats:p>","DOI":"10.1145\/3519030","type":"journal-article","created":{"date-parts":[[2022,3,12]],"date-time":"2022-03-12T11:50:20Z","timestamp":1647085820000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":36,"title":["Double Attention Based on Graph Attention Network for Image Multi-Label Classification"],"prefix":"10.1145","volume":"19","author":[{"given":"Wei","family":"Zhou","sequence":"first","affiliation":[{"name":"Sun Yat-sen University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhiwu","family":"Xia","sequence":"additional","affiliation":[{"name":"Sun Yat-sen University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peng","family":"Dou","sequence":"additional","affiliation":[{"name":"Sun Yat-sen University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tao","family":"Su","sequence":"additional","affiliation":[{"name":"Sun Yat-sen University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haifeng","family":"Hu","sequence":"additional","affiliation":[{"name":"Sun Yat-sen University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,1,5]]},"reference":[{"key":"e_1_3_1_2_2","article-title":"Asymmetric loss for multi-label classification","author":"Ben-Baruch Emanuel","year":"2020","unstructured":"Emanuel Ben-Baruch, Tal Ridnik, Nadav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, and Lihi Zelnik-Manor. 2020. Asymmetric loss for multi-label classification. arXiv preprint arXiv:2009.14119 (2020).","journal-title":"arXiv preprint arXiv:2009.14119"},{"key":"e_1_3_1_3_2","doi-asserted-by":"crossref","unstructured":"Hakan Cevikalp Burak Benligiray \u00d6mer Nezih Gerek and Hasan Saribas. 2020. Semi-supervised robust deep neural networks for multi-label classification. Pattern Recognition 100 (2020) 107164.","DOI":"10.1016\/j.patcog.2019.107164"},{"key":"e_1_3_1_4_2","article-title":"Knowledge-guided multi-label few-shot learning for general image recognition","author":"Chen Tianshui","year":"2022","unstructured":"Tianshui Chen, Liang Lin, Xiaolu Hui, Riquan Chen, and Hefeng Wu. 2022. Knowledge-guided multi-label few-shot learning for general image recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (2022), 1371\u20131384.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_1_5_2","first-page":"522","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Chen Tianshui","year":"2019","unstructured":"Tianshui Chen, Muxin Xu, Xiaolu Hui, Hefeng Wu, and Liang Lin. 2019. Learning semantic-specific graph representation for multi-label image recognition. In Proceedings of the IEEE International Conference on Computer Vision. 522\u2013531."},{"key":"e_1_3_1_6_2","doi-asserted-by":"crossref","first-page":"622","DOI":"10.1109\/ICME.2019.00113","volume-title":"Proceedings of the 2019 IEEE International Conference on Multimedia and Expo (ICME\u201919)","author":"Chen Zhao-Min","year":"2019","unstructured":"Zhao-Min Chen, Xiu-Shen Wei, Xin Jin, and Yanwen Guo. 2019. Multi-label image recognition with joint class-aware map disentangling and label correlation embedding. In Proceedings of the 2019 IEEE International Conference on Multimedia and Expo (ICME\u201919). IEEE, Los Alamitos, CA, 622\u2013627."},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00532"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/1646396.1646452"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_1_10_2","first-page":"6599","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Ding Yao","year":"2019","unstructured":"Yao Ding, Yanzhao Zhou, Yi Zhu, Qixiang Ye, and Jianbin Jiao. 2019. Selective sparse sampling for fine-grained image recognition. In Proceedings of the IEEE\/CVF International Conference on Computer Vision. 6599\u20136608."},{"key":"e_1_3_1_11_2","first-page":"642","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Durand Thibaut","year":"2017","unstructured":"Thibaut Durand, Taylor Mordan, Nicolas Thome, and Matthieu Cord. 2017. WILDCAT: Weakly supervised learning of deep ConvNets for image classification, pointwise localization and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 642\u2013651."},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2788435"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-009-0275-4"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00326"},{"key":"e_1_3_1_15_2","article-title":"Multi-label image recognition with multi-class attentional regions","author":"Gao Bin-Bin","year":"2020","unstructured":"Bin-Bin Gao and Hong-Yu Zhou. 2020. Multi-label image recognition with multi-class attentional regions. arXiv preprint arXiv:2007.01755 (2020).","journal-title":"arXiv preprint arXiv:2007.01755"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00082"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3436494"},{"key":"e_1_3_1_18_2","article-title":"Learning discriminative representations for multi-label image recognition","author":"Hassanin Mohammed","year":"2021","unstructured":"Mohammed Hassanin, Ibrahim Radwan, Salman Khan, and Murat Tahtali. 2021. Learning discriminative representations for multi-label image recognition. arXiv preprint arXiv:2107.11159 (2021).","journal-title":"arXiv preprint arXiv:2107.11159"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00745"},{"key":"e_1_3_1_21_2","first-page":"2452","volume-title":"Proceedings of the 2016 23rd International Conference on Pattern Recognition (ICPR\u201916)","author":"Jin Jiren","year":"2016","unstructured":"Jiren Jin and Hideki Nakayama. 2016. Annotation order matters: Recurrent image annotator for arbitrary length image tagging. In Proceedings of the 2016 23rd International Conference on Pattern Recognition (ICPR\u201916). IEEE, Los Alamitos, CA, 2452\u20132457."},{"key":"e_1_3_1_22_2","article-title":"Semi-supervised classification with graph convolutional networks","author":"Kipf Thomas N.","year":"2016","unstructured":"Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).","journal-title":"arXiv preprint arXiv:1609.02907"},{"key":"e_1_3_1_23_2","first-page":"736","volume-title":"Proceedings of the International Conference on Artificial Neural Networks","author":"Li Junbing","year":"2020","unstructured":"Junbing Li, Changqing Zhang, Xueman Wang, and Ling Du. 2020. Multi-scale cross-modal spatial attention fusion for multi-label image recognition. In Proceedings of the International Conference on Artificial Neural Networks. 736\u2013747."},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3359753"},{"key":"e_1_3_1_25_2","article-title":"Learning category correlations for multi-label image recognition with graph networks","author":"Li Qing","year":"2019","unstructured":"Qing Li, Xiaojiang Peng, Yu Qiao, and Qiang Peng. 2019. Learning category correlations for multi-label image recognition with graph networks. arXiv preprint arXiv:1909.13005 (2019).","journal-title":"arXiv preprint arXiv:1909.13005"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2021.3099369"},{"issue":"1","key":"e_1_3_1_27_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3426974","article-title":"A semi-supervised learning approach based on adaptive weighted fusion for automatic image annotation","volume":"17","author":"Li Zhixin","year":"2021","unstructured":"Zhixin Li, Lan Lin, Canlong Zhang, Huifang Ma, Weizhong Zhao, and Zhiping Shi. 2021. A semi-supervised learning approach based on adaptive weighted fusion for automatic image annotation. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 1 (2021), 1\u201323.","journal-title":"ACM Transactions on Multimedia Computing, Communications, and Applications"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3362988"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2651061"},{"key":"e_1_3_1_31_2","article-title":"Query2Label: A simple transformer way to multi-label classification","author":"Liu Shilong","year":"2021","unstructured":"Shilong Liu, Lei Zhang, Xiao Yang, Hang Su, and Jun Zhu. 2021. Query2Label: A simple transformer way to multi-label classification. arXiv preprint arXiv:2107.10834 (2021).","journal-title":"arXiv preprint arXiv:2107.10834"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2019.2894964"},{"key":"e_1_3_1_33_2","first-page":"2579","article-title":"Visualizing data using t-SNE","author":"Maaten Laurens van der","year":"2008","unstructured":"Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9 (Nov. 2008), 2579\u20132605.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_1_34_2","first-page":"Article 41, 6 p","volume-title":"Proceedings of the ACM Multimedia Asia Conference (MMAsia\u201919)","author":"Meng Quanling","year":"2019","unstructured":"Quanling Meng and Weigang Zhang. 2019. Multi-label image classification with attention mechanism and graph convolutional networks. In Proceedings of the ACM Multimedia Asia Conference (MMAsia\u201919). Article 41, 6 pages."},{"key":"e_1_3_1_35_2","first-page":"9092","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"35","author":"Nguyen Hoang D.","year":"2021","unstructured":"Hoang D. Nguyen, Xuan-Son Vu, and Duc-Trong Le. 2021. Modular graph transformer networks for multi-label image classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 9092\u20139100."},{"key":"e_1_3_1_36_2","unstructured":"Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in PyTorch. In Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS\u201917) . 1\u20134."},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01240-3_25"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3451215"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.74"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3386249"},{"key":"e_1_3_1_41_2","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998\u20136008."},{"key":"e_1_3_1_42_2","article-title":"Graph attention networks","author":"Veli\u010dkovi\u0107 Petar","year":"2017","unstructured":"Petar Veli\u010dkovi\u0107, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).","journal-title":"arXiv preprint arXiv:1710.10903"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3414047"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.251"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01155"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00813"},{"key":"e_1_3_1_47_2","unstructured":"Yikai Wang Wenbing Huang Fuchun Sun Tingyang Xu Yu Rong and Junzhou Huang. 2020. Deep multimodal fusion by channel exchanging. In Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS\u201920) . 1\u201311."},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2020.2995290"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3411880"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.58"},{"key":"e_1_3_1_51_2","article-title":"Learning dual semantic relations with graph attention for image-text matching","author":"Wen Keyu","year":"2020","unstructured":"Keyu Wen, Xiaodong Gu, and Qingrong Cheng. 2020. Learning dual semantic relations with graph attention for image-text matching. IEEE Transactions on Circuits and Systems for Video Technology PP, 99 (2020), 1.","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"e_1_3_1_52_2","article-title":"Multilabel image classification via feature\/label co-projection","author":"Wen Shiping","year":"2021","unstructured":"Shiping Wen, Weiwei Liu, Yin Yang, Pan Zhou, Zhenyuan Guo, Zheng Yan, Yiran Chen, and Tingwen Huang. 2021. Multilabel image classification via feature\/label co-projection. IEEE Transactions on Systems, Man, and Cybernetics: Systems 51, 11 (2021), 7250\u20137259.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics: Systems"},{"key":"e_1_3_1_53_2","first-page":"284","volume-title":"Proceedings of the 28th ACM International Conference on Multimedia","author":"Wu Xiangping","year":"2020","unstructured":"Xiangping Wu, Qingcai Chen, Wei Li, Yulun Xiao, and Baotian Hu. 2020. AdaHGNN: Adaptive hypergraph neural networks for multi-label image classification. In Proceedings of the 28th ACM International Conference on Multimedia. 284\u2013293."},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.634"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2929512"},{"key":"e_1_3_1_56_2","first-page":"649","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Ye Jin","year":"2020","unstructured":"Jin Ye, Junjun He, Xiaojiang Peng, Wenhao Wu, and Yu Qiao. 2020. Attention-driven dynamic graph convolutional network for multi-label image recognition. In Proceedings of the European Conference on Computer Vision. 649\u2013665."},{"key":"e_1_3_1_57_2","first-page":"12709","volume-title":"Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI\u201920)","author":"You Renchun","year":"2020","unstructured":"Renchun You, Zhiyao Guo, Lei Cui, Xiang Long, Yingze Bao, and Shilei Wen. 2020. Cross-modality attention with semantic graph embedding for multi-label classification. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI\u201920). 12709\u201312716."},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2019.03.006"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2018.2812605"},{"key":"e_1_3_1_60_2","first-page":"9050","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zhang Kaihua","year":"2020","unstructured":"Kaihua Zhang, Tengpeng Li, Shiwen Shen, Bo Liu, Jin Chen, and Qingshan Liu. 2020. Adaptive graph convolutional network with attention graph clustering for co-saliency detection. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 9050\u20139059."},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3044446"},{"key":"e_1_3_1_62_2","article-title":"Deep semantic dictionary learning for multi-label image classification","author":"Zhou Fengtao","year":"2020","unstructured":"Fengtao Zhou, Sheng Huang, and Yun Xing. 2020. Deep semantic dictionary learning for multi-label image classification. arXiv preprint arXiv:2012.12509 (2020).","journal-title":"arXiv preprint arXiv:2012.12509"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.219"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3519030","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3519030","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:12:20Z","timestamp":1750191140000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3519030"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,5]]},"references-count":62,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,1,31]]}},"alternative-id":["10.1145\/3519030"],"URL":"https:\/\/doi.org\/10.1145\/3519030","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,5]]},"assertion":[{"value":"2021-09-04","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-02-14","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-01-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}