{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T16:08:38Z","timestamp":1769702918187,"version":"3.49.0"},"reference-count":7,"publisher":"SAGE Publications","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IFS"],"published-print":{"date-parts":[[2023,7,2]]},"abstract":"<jats:p>In recent mathematical reasoning tasks, self-attention has achieved better results in public datasets. However, self-attention performs poorly on more complex mathematical problems due to the lack of capacity to capture local features and the ill-conditioned training after deepening the number of layers. To tackle the problem and enhance its ability of extracting local features while learning the global contexts, we propose an implicit mathematical reasoning model that improves Transformer by combining self-attention and convolution to achieve joint modeling of global and local context. Also, by introducing Reweight connection and adversarial loss function, we prevent the model gradient from disappearing or exploding in a deep neural network while ensuring the convergence speed and avoiding overfitting. Experimental results show that the proposed model improves the accuracy by 4.47% on average for complex mathematical problems compared to the best existing results. In addition, we verify the validity of our model using ablation analysis and further demonstrate the interpretability of the model by attention mapping and task role analysis.<\/jats:p>","DOI":"10.3233\/jifs-224598","type":"journal-article","created":{"date-parts":[[2023,4,28]],"date-time":"2023-04-28T12:13:53Z","timestamp":1682684033000},"page":"975-988","source":"Crossref","is-referenced-by-count":0,"title":["The implicit mathematical reasoning model combining self-attention and convolution"],"prefix":"10.1177","volume":"45","author":[{"given":"Zhuangkai","family":"Yao","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, P.R. China"}]},{"given":"Bi","family":"Zeng","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, P.R. China"}]},{"given":"Huiting","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, Zhongkai University of Agriculture and Engineering, Guangzhou, P.R. China"}]},{"given":"Pengfei","family":"Wei","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, P.R. China"}]}],"member":"179","reference":[{"key":"10.3233\/JIFS-224598_ref6","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI blog"},{"issue":"5","key":"10.3233\/JIFS-224598_ref10","first-page":"1465","article-title":"Review of natural scene textdetection and recognition based on deep learning","volume":"31","author":"Wang","year":"2020","journal-title":"Journal ofSoftware"},{"issue":"12","key":"10.3233\/JIFS-224598_ref11","first-page":"1120","article-title":"CNN with part-of-speech and attentionmechanism for targeted sentiment classification","volume":"31","author":"Du","year":"2018","journal-title":"Pattern RecognArtif Intell"},{"key":"10.3233\/JIFS-224598_ref16","first-page":"5036","article-title":"Conformer: Convolution-augmented Transformer for SpeechRecognition,","volume":"2020","author":"Gulati","year":"2020","journal-title":"Proc. Interspeech"},{"key":"10.3233\/JIFS-224598_ref17","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/j.future.2020.08.005","article-title":"ABCDM: An attention-based bidirectional CNN-RNN deep model forsentiment analysis","volume":"115","author":"Basiri","year":"2021","journal-title":"Future Generation Computer Systems"},{"issue":"6","key":"10.3233\/JIFS-224598_ref18","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","article-title":"Deep neural networks for acoustic modeling inspeech recognition: The shared views of four research groups","volume":"29","author":"Hinton","year":"2012","journal-title":"IEEE Signal Processing Magazine"},{"issue":"3","key":"10.3233\/JIFS-224598_ref23","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1145\/3446776","article-title":"Understanding deep learning (still) requires rethinkinggeneralization","volume":"64","author":"Zhang","year":"2021","journal-title":"Communications of the ACM"}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/JIFS-224598","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T05:25:44Z","timestamp":1769664344000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/JIFS-224598"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,2]]},"references-count":7,"journal-issue":{"issue":"1"},"URL":"https:\/\/doi.org\/10.3233\/jifs-224598","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,2]]}}}