{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T12:13:10Z","timestamp":1770207190622,"version":"3.49.0"},"reference-count":48,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2020,1,2]],"date-time":"2020-01-02T00:00:00Z","timestamp":1577923200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"published-print":{"date-parts":[[2020,2,6]]},"abstract":"<jats:p>With the wide application of attention mechanism in multitudinous field of natural language processing (NLP), to date various deep neural networks based on this mechanism have been introduced and developed. However, a major problem with this kind of application is that a long time will be consumed due to the current networks still need to rely on their own ability to form attention values from scratch during the training. In this paper, we propose an auxiliary method called the Guided Attention Mechanism (GAM), which utilizes the prior knowledge to guide the network to form attention values in NLP field, thereby shortening the network training time and making the attention values more accurate. This work designed two sets of prior knowledge generation processes based on the regularization method and the deep learning method respectively. And the prior knowledge is used to guide the attention values of the original network in terms of values and angles. The experimental results show that compared with the original network, the classification accuracy of the network using GAM is improved by about 2%, and the training time is reduced by 5\u223c9%.<\/jats:p>","DOI":"10.3233\/jifs-191257","type":"journal-article","created":{"date-parts":[[2020,2,7]],"date-time":"2020-02-07T14:53:21Z","timestamp":1581087201000},"page":"2323-2335","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":2,"title":["Guided attention mechanism: Training network more efficiently"],"prefix":"10.1177","volume":"38","author":[{"given":"Chiyu","family":"Wang","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, Changsha, Hunan, China"}]},{"given":"Hong","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, Changsha, Hunan, China"}]},{"given":"Xinrong","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, Changsha, Hunan, China"}]},{"given":"Feifei","family":"Hou","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, Changsha, Hunan, China"}]},{"given":"Xun","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, Changsha, Hunan, China"}]}],"member":"179","published-online":{"date-parts":[[2020,1,2]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","author":"Krizhevsky A.","year":"2012","unstructured":"KrizhevskyA., SutskeverI. and HintonG.E., Imagenet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems (2012), 1097\u20131105.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_3_2","first-page":"580","article-title":"Rich feature hierarchies for accurate object detection and semantic segmentation","author":"Girshick R.","year":"2014","unstructured":"GirshickR., DonahueJ., DarrellT. and MalikJ., Rich feature hierarchies for accurate object detection and semantic segmentation, Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (2014), 580\u2013587.","journal-title":"Proceedings of the IEEE conference on Computer Vision and Pattern Recognition"},{"key":"e_1_3_2_4_2","article-title":"Learning phrase representations using RNN encoder-decoder for statistical machine translation","author":"Cho K.","year":"2014","unstructured":"ChoK., Van Merri\u00ebnboerB., GulcehreC., BahdanauD., BougaresF., SchwenkH. and BengioY., Learning phrase representations using RNN encoder-decoder for statistical machine translation, arXiv preprint arXiv:1406.1078 (2014).","journal-title":"arXiv preprint arXiv:1406.1078"},{"issue":"3","key":"e_1_3_2_5_2","first-page":"1","article-title":"ANFIS system for classification of brain signals","volume":"37","author":"Rubio J.d.J.","year":"2019","unstructured":"RubioJ.d.J., Ricardo CruzD., EliasI., OchoaG., BalcazarandR. and AguilarA., ANFIS system for classification of brain signals, Journal of Intelligent & Fuzzy Systems 37(3) (2019), 1\u20139.","journal-title":"Journal of Intelligent & Fuzzy Systems"},{"issue":"2","key":"e_1_3_2_6_2","first-page":"1","article-title":"Neural network updating via argument Kalman filter for modeling of Takagi-Sugeno fuzzy models","volume":"35","author":"de Jes\u00fas Rubio J.","year":"2018","unstructured":"de Jes\u00fas RubioJ., LughoferE., Meda-Campa\u00f1aJ.A., P\u00e1ramoL.A., Novoa J.F. and PachecoJ., Neural network updating via argument Kalman filter for modeling of Takagi-Sugeno fuzzy models, Journal of Intelligent & Fuzzy Systems 35(2) (2018), 1\u201312.","journal-title":"Journal of Intelligent & Fuzzy Systems"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_2_8_2","article-title":"On the properties of neural machine translation: Encoder-decoder approaches","author":"Cho K.","year":"2014","unstructured":"ChoK., Van Merri\u00ebnboerB., BahdanauD. and BengioY., On the properties of neural machine translation: Encoder-decoder approaches, arXiv preprint arXiv:1409.1259 (2014).","journal-title":"arXiv preprint arXiv:1409.1259"},{"key":"e_1_3_2_9_2","first-page":"3111","article-title":"Distributed representations of words and phrases and their compositionality","author":"Mikolov T.","year":"2013","unstructured":"MikolovT., SutskeverI., ChenK., CorradoG.S. and DeanJ., Distributed representations of words and phrases and their compositionality, Advances in Neural Information Processing Systems (2013), 3111\u20133119.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_2_11_2","article-title":"Neural machine translation by jointly learning to align and translate","author":"Bahdanau D.","year":"2014","unstructured":"BahdanauD., ChoK. and BengioY., Neural machine translation by jointly learning to align and translate, arXiv preprint arXiv:1409.0473 (2014).","journal-title":"arXiv preprint arXiv:1409.0473"},{"key":"e_1_3_2_12_2","first-page":"2440","article-title":"End-to-end memory networks","author":"Sukhbaatar S.","year":"2015","unstructured":"SukhbaatarS., WestonJ. and FergusR., End-to-end memory networks, Advances in Neural Information Processing Systems (2015), 2440\u20132448.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.15837\/ijccc.2018.1.3142"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1058"},{"key":"e_1_3_2_15_2","first-page":"3","article-title":"Cbam: Convolutional block attention module","author":"Woo S.","year":"2018","unstructured":"WooS., ParkJ., LeeJ.-Y. and So KweonI., Cbam: Convolutional block attention module, Proceedings of the European Conference on Computer Vision (ECCV) (2018), 3\u201319.","journal-title":"Proceedings of the European Conference on Computer Vision (ECCV)"},{"key":"e_1_3_2_16_2","first-page":"5998","article-title":"Attention is all you need","author":"Vaswani A.","year":"2017","unstructured":"VaswaniA., ShazeerN., ParmarN., UszkoreitJ., JonesL., GomezA.N., Kaiser\u0141. and PolosukhinI., Attention is all you need, Advances in Neural Information Processing Systems (2017), 5998\u20136008.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_17_2","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin J.","year":"2018","unstructured":"DevlinJ., ChangM.-W., LeeK. and ToutanovaK., Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018).","journal-title":"arXiv preprint arXiv:1810.04805"},{"key":"e_1_3_2_18_2","unstructured":"RadfordA. NarasimhanK. SalimansT. SutskeverI. Improving language understanding by generative pre-training URL https:\/\/s3-us-west-2.amazonaws.com\/openai-assets\/researchcovers\/languageunsupervised\/languageunderstandingpaper.pdf (2018)."},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1016\/j.patrec.2018.10.020","article-title":"Fusion-Attention Network for person search with free-form natural language","volume":"116","author":"Ji Z.","year":"2018","unstructured":"JiZ., LiS. and PangY., Fusion-Attention Network for person search with free-form natural language, Pattern Recognition Letters 116 (2018), 205\u2013211.","journal-title":"Pattern Recognition Letters"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1016\/j.patcog.2018.12.011","article-title":"Attention guided deep audio-face fusion for efficient speaker naming","volume":"88","author":"Liu X.","year":"2019","unstructured":"LiuX., GengJ., LingH. and CheungY.-m., Attention guided deep audio-face fusion for efficient speaker naming, Pattern Recognition 88 (2019), 557\u2013568.","journal-title":"Pattern Recognition"},{"key":"e_1_3_2_21_2","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/j.patrec.2019.01.010","article-title":"Image-attribute reciprocally guided attention network for pedestrian attribute recognition","volume":"120","author":"Ji Z.","year":"2019","unstructured":"JiZ., HeE., WangH. and YangA., Image-attribute reciprocally guided attention network for pedestrian attribute recognition, Pattern Recognition Letters 120 (2019), 89\u201395.","journal-title":"Pattern Recognition Letters"},{"key":"e_1_3_2_22_2","first-page":"432","article-title":"An introductory survey on attention mechanisms in nlp problems","author":"Hu D.","year":"2019","unstructured":"HuD., An introductory survey on attention mechanisms in nlp problems, Proceedings of SAI Intelligent Systems Conference (2019), 432\u2013448.","journal-title":"Proceedings of SAI Intelligent Systems Conference"},{"key":"e_1_3_2_23_2","first-page":"1693","article-title":"Teaching machines to read and comprehend","author":"Hermann K. M.","year":"2015","unstructured":"HermannK. M., KociskyT., GrefenstetteE., EspeholtL., KayW., SuleymanM. and BlunsomP., Teaching machines to read and comprehend, Advances in Neural Information Processing Systems (2015), 1693\u20131701.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_24_2","first-page":"1480","article-title":"Hierarchical attention networks for document classification","author":"Yang Z.","year":"2016","unstructured":"YangZ., YangD., DyerC., HeX., SmolaA. and HovyE., Hierarchical attention networks for document classification, Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2016), 1480\u20131489.","journal-title":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies"},{"key":"e_1_3_2_25_2","article-title":"Factorization tricks for LSTM networks","author":"Kuchaiev O.","year":"2017","unstructured":"KuchaievO. and GinsburgB., Factorization tricks for LSTM networks, arXiv preprint arXiv:1703.10722 (2017).","journal-title":"arXiv preprint arXiv:1703.10722"},{"key":"e_1_3_2_26_2","article-title":"Outrageously large neural networks: The sparsely-gated mixture-of-experts layer","author":"Shazeer N.","year":"2017","unstructured":"ShazeerN., MirhoseiniA., MaziarzK., DavisA., LeQ., HintonG. and DeanJ., Outrageously large neural networks: The sparsely-gated mixture-of-experts layer, arXiv preprint arXiv:1701.06538 (2017).","journal-title":"arXiv preprint arXiv:1701.06538"},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.csl.2017.01.007","article-title":"Context-dependent word representation for neural machine translation","volume":"45","author":"Choi H.","year":"2017","unstructured":"ChoiH., ChoK. and BengioY., Context-dependent word representation for neural machine translation, Computer Speech & Language 45 (2017), 149\u2013160.","journal-title":"Computer Speech & Language"},{"key":"e_1_3_2_28_2","article-title":"Deep contextualized word representations","author":"Peters M. E.","year":"2018","unstructured":"PetersM. E., NeumannM., IyyerM., GardnerM., ClarkC., LeeK. and ZettlemoyerL., Deep contextualized word representations, arXiv preprint arXiv:1802.05365 (2018).","journal-title":"arXiv preprint arXiv:1802.05365"},{"key":"e_1_3_2_29_2","first-page":"417","article-title":"Sentiwordnet: A publicly available lexical resource for opinion mining","volume":"6","author":"Esuli A.","year":"2006","unstructured":"EsuliA. and SebastianiF., Sentiwordnet: A publicly available lexical resource for opinion mining, LREC 6 (2006), 417\u2013422.","journal-title":"LREC"},{"key":"e_1_3_2_30_2","first-page":"2200","article-title":"Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining","volume":"10","author":"Baccianella S.","year":"2010","unstructured":"BaccianellaS., EsuliA. and SebastianiF., Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining, Lrec 10 (2010), 2200\u20132204.","journal-title":"Lrec"},{"key":"e_1_3_2_31_2","first-page":"649","article-title":"Character-level convolutional networks for text classification","author":"Zhang X.","year":"2015","unstructured":"ZhangX., ZhaoJ. and LeCunY., Character-level convolutional networks for text classification, Advances in Neural Information Processing Systems (2015), 649\u2013657.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_32_2","article-title":"Sequential short-text classification with recurrent and convolutional neural networks","author":"Lee J. Y.","year":"2016","unstructured":"LeeJ. Y. and DernoncourtF., Sequential short-text classification with recurrent and convolutional neural networks, arXiv preprint arXiv:1603.03827 (2016).","journal-title":"arXiv preprint arXiv:1603.03827"},{"key":"e_1_3_2_33_2","first-page":"115","article-title":"Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales","author":"Pang B.","year":"2005","unstructured":"PangB. and LeeL., Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales, Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics (2005), 115\u2013124.","journal-title":"Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D13-1170"},{"key":"e_1_3_2_35_2","first-page":"271","article-title":"A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts","author":"Pang B.","year":"2004","unstructured":"PangB. and LeeL., A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics (2004), 271.","journal-title":"Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics"},{"key":"e_1_3_2_36_2","first-page":"1","article-title":"Learning question classifiers","volume":"1","author":"Li X.","year":"2002","unstructured":"LiX. and RothD., Learning question classifiers, Proceedings of the 19th International Conference on Computational Linguistics 1 (2002), 1\u20137.","journal-title":"Proceedings of the 19th International Conference on Computational Linguistics"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-005-7880-9"},{"key":"e_1_3_2_38_2","article-title":"Sentiment lexicon enhanced attention-based LSTM for sentiment classification","author":"Lei Z.","year":"2018","unstructured":"LeiZ., YangY. and YangM., Sentiment lexicon enhanced attention-based LSTM for sentiment classification, Thirty-Second AAAI Conference on Artificial Intelligence (2018).","journal-title":"Thirty-Second AAAI Conference on Artificial Intelligence"},{"key":"e_1_3_2_39_2","article-title":"Disan: Directional self-attention network for rnn\/cnn-free language understanding","author":"Shen T.","year":"2018","unstructured":"ShenT., ZhouT., LongG., JiangJ., PanS. and ZhangC., Disan: Directional self-attention network for rnn\/cnn-free language understanding, Thirty-Second AAAI Conference on Artificial Intelligence (2018).","journal-title":"Thirty-Second AAAI Conference on Artificial Intelligence"},{"key":"e_1_3_2_40_2","first-page":"183","article-title":"A convolutional attention model for text classification","author":"Du J.","year":"2017","unstructured":"DuJ., GuiL., XuR. and HeY., A convolutional attention model for text classification, National CCF Conference on Natural Language Processing and Chinese Computing (2017), 183\u2013195.","journal-title":"National CCF Conference on Natural Language Processing and Chinese Computing"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2018.06.019"},{"key":"e_1_3_2_42_2","article-title":"Text classification improved by integrating bidirectional LSTM with two-dimensional max pooling","author":"Zhou P.","year":"2016","unstructured":"ZhouP., QiZ., ZhengS., XuJ., BaoH. and XuB., Text classification improved by integrating bidirectional LSTM with two-dimensional max pooling, arXiv preprint arXiv:1611.06639 (2016).","journal-title":"arXiv preprint arXiv:1611.06639"},{"key":"e_1_3_2_43_2","article-title":"Compositional distributional semantics with long short term memory","author":"Le P.","year":"2015","unstructured":"LeP. and ZuidemaW., Compositional distributional semantics with long short term memory, arXiv preprint arXiv:1503.02510 (2015).","journal-title":"arXiv preprint arXiv:1503.02510"},{"key":"e_1_3_2_44_2","first-page":"1201","article-title":"Semantic compositionality through recursive matrix-vector spaces","author":"Socher R.","year":"2012","unstructured":"SocherR., HuvalB., ManningC.D. and NgA.Y., Semantic compositionality through recursive matrix-vector spaces, Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (2012), 1201\u20131211.","journal-title":"Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning"},{"key":"e_1_3_2_45_2","article-title":"Convolutional neural networks for sentence classification","author":"Kim Y.","year":"2014","unstructured":"KimY., Convolutional neural networks for sentence classification, arXiv preprint arXiv:1408.5882 (2014).","journal-title":"arXiv preprint arXiv:1408.5882"},{"key":"e_1_3_2_46_2","article-title":"A convolutional neural network for modelling sentences","author":"Kalchbrenner N.","year":"2014","unstructured":"KalchbrennerN., GrefenstetteE. and BlunsomP., A convolutional neural network for modelling sentences, arXiv preprint arXiv:1404.2188 (2014).","journal-title":"arXiv preprint arXiv:1404.2188"},{"key":"e_1_3_2_47_2","article-title":"Multichannel variable-size convolution for sentence classification","author":"Yin W.","year":"2016","unstructured":"YinW. and Sch\u00fctzeH., Multichannel variable-size convolution for sentence classification, arXiv preprint arXiv:1603.04513 (2016).","journal-title":"arXiv preprint arXiv:1603.04513"},{"key":"e_1_3_2_48_2","article-title":"MGNC-CNN: A simple approach to exploiting multiple word embeddings for sentence classification","author":"Zhang Y.","year":"2016","unstructured":"ZhangY., RollerS. and WallaceB., MGNC-CNN: A simple approach to exploiting multiple word embeddings for sentence classification, arXiv preprint arXiv:1603.00968 (2016).","journal-title":"arXiv preprint arXiv:1603.00968"},{"key":"e_1_3_2_49_2","first-page":"1681","article-title":"Deep unordered composition rivals syntactic methods for text classification","volume":"1","author":"Iyyer M.","year":"2015","unstructured":"IyyerM., ManjunathaV., Boyd-GraberJ. and Daum\u00e9H.III, Deep unordered composition rivals syntactic methods for text classification, Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing 1 (2015), 1681\u20131691.","journal-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing"}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JIFS-191257","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.3233\/JIFS-191257","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JIFS-191257","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T12:26:21Z","timestamp":1770121581000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.3233\/JIFS-191257"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,2]]},"references-count":48,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,2,6]]}},"alternative-id":["10.3233\/JIFS-191257"],"URL":"https:\/\/doi.org\/10.3233\/jifs-191257","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1,2]]}}}