{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T08:49:41Z","timestamp":1782290981991,"version":"3.54.5"},"reference-count":54,"publisher":"PeerJ","license":[{"start":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T00:00:00Z","timestamp":1782259200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Science and Technology Research Program of Chongqing Municipal Education Commission","award":["KJQN202401915"],"award-info":[{"award-number":["KJQN202401915"]}]},{"name":"Open Research Project of Chongqing Engineering Research Center for Intelligent Applications of Financial Big Data","award":["2025KFKT-001"],"award-info":[{"award-number":["2025KFKT-001"]}]},{"name":"2022 High-Level Talent Research Startup Fund of Chongqing Engineering College","award":["2022gcky03"],"award-info":[{"award-number":["2022gcky03"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:p>This study introduces the multi-head self-attention rotary-positional encoding Informer (MHSA-RPEI), a novel enterprise credit rating model that leverages self-correlation weights and positional information of financial indicators to enhance rating accuracy. Corporate credit rating presents challenges such as temporal dynamics, complex non-linear relationships, and cross-regional variations, which traditional and deep learning models struggle to address. The MHSA-RPEI model overcomes these limitations through four innovations: (1) objective MHSA-RPEI model, which is specifically designed for credit rating, achieving high precision using only financial data and serving as an objective and validated tool; (2) revolutionary rotary positional encoding (RoPE), which enhances temporal modeling, improving dynamic risk assessment; (3) strategic transfer learning (TL), which facilitates cross-regional adaptations, mitigating data scarcity and domain variations; and (4) optimized informer architecture, which improves long-term risk forecasting based on extensive financial time series. Experiments on the U.S., Chinese, and Japanese financial data showed that MHSA-RPEI achieved 92.16% accuracy (U.S.), 90.32% (China), and 92.50% (Japan), with area under the receiver operating characteristic curve scores ranging from 88.47% to 91.95%, surpassing traditional and state-of-the-art models. Furthermore, TL experiments highlighted the exceptional adaptability of the MHSA-RPEI for cross-regional applications, yielding performance gains of 5\u20136%. Ablation studies confirmed that the RoPE and self-correlation weight modules were crucial to the effectiveness of the model.<\/jats:p>","DOI":"10.7717\/peerj-cs.3920","type":"journal-article","created":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T08:27:11Z","timestamp":1782289631000},"page":"e3920","source":"Crossref","is-referenced-by-count":0,"title":["Advancing enterprise credit rating through multi-head self-attention and transfer learning"],"prefix":"10.7717","volume":"12","author":[{"given":"Rui","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Integrated Innovative Technology, King Mongkut\u2019s Institute of Technology Ladkrabang, Bangkok, Thailand"},{"name":"School of Big Data and Artificial Intelligence, Chongqing Institute of Engineering, Chongqing, China"},{"name":"Chongqing Engineering Research Center for Intelligent Applications of Financial Big Data, Chongqing Institute of Engineering, Chongqing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qing","family":"Li","sequence":"additional","affiliation":[{"name":"School of Big Data and Artificial Intelligence, Chongqing Institute of Engineering, Chongqing, China"},{"name":"Chongqing Engineering Research Center for Intelligent Applications of Financial Big Data, Chongqing Institute of Engineering, Chongqing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiangchuan","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Big Data and Artificial Intelligence, Chongqing Institute of Engineering, Chongqing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jie","family":"Li","sequence":"additional","affiliation":[{"name":"School of Big Data and Artificial Intelligence, Chongqing Institute of Engineering, Chongqing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rachsak","family":"Sakdanuphab","sequence":"additional","affiliation":[{"name":"School of Integrated Innovative Technology, King Mongkut\u2019s Institute of Technology Ladkrabang, Bangkok, Thailand"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hualin","family":"Li","sequence":"additional","affiliation":[{"name":"School of Big Data and Artificial Intelligence, Chongqing Institute of Engineering, Chongqing, China"},{"name":"Chongqing Engineering Research Center for Intelligent Applications of Financial Big Data, Chongqing Institute of Engineering, Chongqing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"4443","published-online":{"date-parts":[[2026,6,24]]},"reference":[{"key":"10.7717\/peerj-cs.3920\/ref-1","doi-asserted-by":"publisher","first-page":"64873","DOI":"10.1109\/ACCESS.2020.2984412","article-title":"Explainability of a machine learning granting scoring model in P2P lending","volume":"8","author":"Ariza-Garc\u00f3n","year":"2020","journal-title":"IEEE Access"},{"issue":"3","key":"10.7717\/peerj-cs.3920\/ref-2","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1287\/mnsc.49.3.312.12739","article-title":"Using neural network rule extraction and decision tables for credit-risk evaluation","volume":"49","author":"Baesens","year":"2003","journal-title":"Management Science"},{"key":"10.7717\/peerj-cs.3920\/ref-3","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1409.0473","article-title":"Neural machine translation by jointly learning to align and translate","author":"Bahdanau","year":"2014"},{"issue":"3","key":"10.7717\/peerj-cs.3920\/ref-4","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1007\/s42521-024-00114-3","article-title":"Improving credit risk assessment in P2P lending with explainable machine learning survival analysis","volume":"6","author":"Bone-Winkel","year":"2024","journal-title":"Digital Finance"},{"key":"10.7717\/peerj-cs.3920\/ref-5","doi-asserted-by":"publisher","first-page":"15301","DOI":"10.5555\/3540261.3541433","article-title":"Attention approximates sparse distributed memory","volume":"34","author":"Bricken","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"10.7717\/peerj-cs.3920\/ref-6","doi-asserted-by":"publisher","first-page":"203876\u2013203889","DOI":"10.1109\/access.2020.3036469","article-title":"A novel end-to-end corporate credit rating model based on self-attention mechanism","volume":"8","author":"Chen","year":"2020","journal-title":"IEEE Access"},{"key":"10.7717\/peerj-cs.3920\/ref-7","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1409.1259","article-title":"On the properties of neural machine translation: encoder-decoder approaches","author":"Cho","year":"2014"},{"issue":"13","key":"10.7717\/peerj-cs.3920\/ref-8","doi-asserted-by":"publisher","first-page":"1061","DOI":"10.1080\/09603100600749220","article-title":"Are credit ratings valuable information?","volume":"17","author":"Czarnitzki","year":"2007","journal-title":"Applied Financial Economics"},{"key":"10.7717\/peerj-cs.3920\/ref-9","first-page":"443","article-title":"Bond rating: a nonconservative application of neural networks","volume":"2","author":"Dutta","year":"1988"},{"issue":"4","key":"10.7717\/peerj-cs.3920\/ref-10","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1111\/j.1540-6288.1985.tb00306.x","article-title":"Classification models and bond ratings","volume":"20","author":"Ederington","year":"1985","journal-title":"Financial Review"},{"issue":"3","key":"10.7717\/peerj-cs.3920\/ref-11","doi-asserted-by":"publisher","first-page":"102000","DOI":"10.1016\/j.jksuci.2024.102000","article-title":"Transfer learning model for cash-instrument prediction adopting a transformer derivative","volume":"36","author":"Feng","year":"2024","journal-title":"Journal of King Saud University: Computer and Information Sciences"},{"key":"10.7717\/peerj-cs.3920\/ref-12","first-page":"1578","article-title":"Every corporation owns its image: corporate credit ratings via convolutional neural networks","author":"Feng","year":"2020"},{"key":"10.7717\/peerj-cs.3920\/ref-13","doi-asserted-by":"publisher","first-page":"78","DOI":"10.2139\/ssrn.4282842","article-title":"Stock price prediction model based on dual attention and Tcn","author":"Fu","year":"2022","journal-title":"SSRN Electronic Journal"},{"issue":"1","key":"10.7717\/peerj-cs.3920\/ref-14","doi-asserted-by":"publisher","first-page":"101251","DOI":"10.1016\/j.najef.2020.101251","article-title":"A comparative study of forecasting corporate credit ratings using neural networks, support vector machines, and decision trees","volume":"54","author":"Golbayani","year":"2020","journal-title":"The North American Journal of Economics and Finance"},{"key":"10.7717\/peerj-cs.3920\/ref-15","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2003.02334","article-title":"Application of deep neural networks to assess corporate credit rating","author":"Golbayani","year":"2020"},{"key":"10.7717\/peerj-cs.3920\/ref-16","first-page":"227","article-title":"Multi-source transfer learning with ensemble for financial time series forecasting","author":"He","year":"2020"},{"key":"10.7717\/peerj-cs.3920\/ref-17","first-page":"895","article-title":"Bank credit risk analysis based on network data mining and pre-training-fine-tuning ANN","author":"Hu","year":"2022"},{"key":"10.7717\/peerj-cs.3920\/ref-18","article-title":"A primer on machine learning methods for credit rating modeling","author":"Jiang","year":"2022"},{"issue":"1","key":"10.7717\/peerj-cs.3920\/ref-19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4018\/joeuc.361650","article-title":"Innovation in financial enterprise risk prediction model: a hybrid deep learning technique based on CNN-transformer-WT","volume":"36","author":"Jin","year":"2024","journal-title":"Journal of Organizational and End User Computing"},{"key":"10.7717\/peerj-cs.3920\/ref-20","doi-asserted-by":"crossref","DOI":"10.1109\/ICETSIS61505.2024.10459393","article-title":"The use of machine learning to forecast financial performance: a literature review","author":"Khudhur","year":"2024"},{"key":"10.7717\/peerj-cs.3920\/ref-21","doi-asserted-by":"crossref","DOI":"10.1109\/SITA.2015.7358416","article-title":"A hybrid deep belief network approach for financial distress prediction","author":"Lanbouri","year":"2015"},{"key":"10.7717\/peerj-cs.3920\/ref-22","doi-asserted-by":"publisher","first-page":"114754\u2013114766","DOI":"10.1109\/access.2021.3104472","article-title":"Transfer learning strategies for credit card fraud detection","volume":"9","author":"Lebichot","year":"2021","journal-title":"IEEE Access"},{"issue":"1","key":"10.7717\/peerj-cs.3920\/ref-23","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1016\/j.eswa.2006.04.018","article-title":"Application of support vector machines to corporate credit rating prediction","volume":"33","author":"Lee","year":"2007","journal-title":"Expert Systems with Applications"},{"key":"10.7717\/peerj-cs.3920\/ref-24","first-page":"241","article-title":"LSTM-based deep learning for time series forecasting: the case of corporate credit score prediction","volume":"29","author":"Lee","year":"2020","journal-title":"The Journal of Information Systems"},{"key":"10.7717\/peerj-cs.3920\/ref-25","first-page":"330","article-title":"The hybrid credit scoring strategies based on KNN classifier","author":"Li","year":"2009"},{"issue":"2","key":"10.7717\/peerj-cs.3920\/ref-26","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1016\/j.engappai.2016.12.002","article-title":"A deep learning approach for credit scoring using credit default swaps","volume":"65","author":"Luo","year":"2017","journal-title":"Engineering Applications of Artificial Intelligence"},{"key":"10.7717\/peerj-cs.3920\/ref-27","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1508.04025","article-title":"Effective approaches to attention-based neural machine translation","author":"Luong","year":"2015"},{"issue":"12","key":"10.7717\/peerj-cs.3920\/ref-28","doi-asserted-by":"publisher","first-page":"2033","DOI":"10.1080\/14697688.2019.1634277","article-title":"Forecasting jump arrivals in stock prices: new attention-based network architecture using limit order book data","volume":"19","author":"M\u00e4kinen","year":"2019","journal-title":"Quantitative Finance"},{"issue":"22","key":"10.7717\/peerj-cs.3920\/ref-29","doi-asserted-by":"publisher","first-page":"4745","DOI":"10.3390\/app9224745","article-title":"A novel approach to short-term stock price movement prediction using transfer learning","volume":"9","author":"Nguyen","year":"2019","journal-title":"Applied Sciences"},{"key":"10.7717\/peerj-cs.3920\/ref-30","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2405.08707","article-title":"Beyond scaling laws: understanding transformer performance with associative memory","author":"Niu","year":"2024"},{"key":"10.7717\/peerj-cs.3920\/ref-31","doi-asserted-by":"publisher","first-page":"96","DOI":"10.3390\/make6030085","article-title":"Opening a new era with machine learning in financial services? Forecasting corporate credit ratings based on annual financial statements","volume":"11","author":"Pamuk","year":"2023","journal-title":"International Journal of Financial Studies"},{"key":"10.7717\/peerj-cs.3920\/ref-32","first-page":"883","article-title":"Predicting stock market trends with a transfer learning model based on CNN-LSTM architecture","author":"Patil","year":"2023"},{"key":"10.7717\/peerj-cs.3920\/ref-33","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2008.02217","article-title":"Hopfield networks is all you need","author":"Ramsauer","year":"2020"},{"key":"10.7717\/peerj-cs.3920\/ref-34","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2402.13725","article-title":"Sparse and structured hopfield networks","author":"Santos","year":"2024"},{"key":"10.7717\/peerj-cs.3920\/ref-35","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2402.10202","article-title":"Bridging associative memory and probabilistic modeling","author":"Schaeffer","year":"2024"},{"key":"10.7717\/peerj-cs.3920\/ref-36","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2107.09323","article-title":"Transfer learning for credit card fraud detection: a journey from research to production","author":"Siblini","year":"2021"},{"issue":"1","key":"10.7717\/peerj-cs.3920\/ref-37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4018\/joeuc.337607","article-title":"Research on financial risk intelligent monitoring and early warning model based on LSTM, transformer, and deep learning","volume":"36","author":"Song","year":"2024","journal-title":"Journal of Organizational and End User Computing"},{"key":"10.7717\/peerj-cs.3920\/ref-38","first-page":"157","article-title":"Neural networks for bond rating improved by multiple hidden layers","author":"Surkan","year":"1990"},{"key":"10.7717\/peerj-cs.3920\/ref-39","doi-asserted-by":"publisher","first-page":"868232","DOI":"10.3389\/frai.2022.868232","article-title":"Credit risk modeling using transfer learning and domain adaptation","volume":"5","author":"Suryanto","year":"2022","journal-title":"Frontiers in Artificial Intelligence"},{"key":"10.7717\/peerj-cs.3920\/ref-40","doi-asserted-by":"publisher","first-page":"1407","DOI":"10.1109\/TNNLS.2018.2869225","article-title":"Temporal attention-augmented bilinear network for financial time-series data analysis","volume":"30","author":"Tran","year":"2018","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"10.7717\/peerj-cs.3920\/ref-41","first-page":"6000","article-title":"Attention is all you need","author":"Vaswani","year":"2017"},{"key":"10.7717\/peerj-cs.3920\/ref-42","first-page":"180","article-title":"Credit rating forecasting using machine learning techniques","author":"Wallis","year":"2019"},{"key":"10.7717\/peerj-cs.3920\/ref-43","first-page":"159","article-title":"Research on personal credit risk assessment model based on instance-based transfer learning","author":"Wang","year":"2021"},{"key":"10.7717\/peerj-cs.3920\/ref-44","doi-asserted-by":"publisher","first-page":"1613","DOI":"10.1111\/poms.13634","article-title":"Credit rating prediction through supply chains: a machine learning approach","volume":"31","author":"Wu","year":"2022","journal-title":"Production and Operations Management"},{"issue":"8","key":"10.7717\/peerj-cs.3920\/ref-45","doi-asserted-by":"publisher","first-page":"5175","DOI":"10.3390\/app13085175","article-title":"A financial time-series prediction model based on multiplex attention and linear transformer structure","volume":"13","author":"Xu","year":"2023","journal-title":"Applied Sciences"},{"key":"10.7717\/peerj-cs.3920\/ref-46","first-page":"57","article-title":"A multiclass machine learning approach to credit rating prediction","author":"Ye","year":"2008"},{"key":"10.7717\/peerj-cs.3920\/ref-47","first-page":"159","article-title":"Deep belief networks for predicting corporate defaults","author":"Yeh","year":"2015"},{"key":"10.7717\/peerj-cs.3920\/ref-48","first-page":"1268","article-title":"Interpreting attention of stock price prediction","author":"Yoshida","year":"2022"},{"key":"10.7717\/peerj-cs.3920\/ref-49","doi-asserted-by":"publisher","first-page":"6826573","DOI":"10.1155\/2022\/6826573","article-title":"The evaluation on the credit risk of enterprises with the CNN-LSTM-ATT model","volume":"2022","author":"Zhang","year":"2022","journal-title":"Computational Intelligence and Neuroscience"},{"key":"10.7717\/peerj-cs.3920\/ref-50","first-page":"338","article-title":"An improved neural manufacturing corporate credit rating model based on LSTM","volume":"12","author":"Zhang","year":"2023","journal-title":"International Journal of Intelligent Systems and Applications in Engineering"},{"key":"10.7717\/peerj-cs.3920\/ref-51","first-page":"052037","article-title":"At-LSTM: an attention-based LSTM model for financial time series prediction","author":"Zhang","year":"2019"},{"key":"10.7717\/peerj-cs.3920\/ref-52","first-page":"11106","article-title":"Informer: beyond efficient transformer for long sequence time-series forecasting","author":"Zhou","year":"2021"},{"issue":"12","key":"10.7717\/peerj-cs.3920\/ref-53","doi-asserted-by":"publisher","first-page":"11106","DOI":"10.1609\/aaai.v35i12.17325","article-title":"Beyond efficient transformer for long sequence time-series forecasting","volume":"35","author":"Zhou","year":"2023","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"10.7717\/peerj-cs.3920\/ref-54","first-page":"928","article-title":"Modeling users\u2019 behavior sequences with hierarchical explainable network for cross-domain fraud detection","author":"Zhu","year":"2020"}],"container-title":["PeerJ Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/peerj.com\/articles\/cs-3920.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3920.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3920.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3920.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T08:27:16Z","timestamp":1782289636000},"score":1,"resource":{"primary":{"URL":"https:\/\/peerj.com\/articles\/cs-3920"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,24]]},"references-count":54,"alternative-id":["10.7717\/peerj-cs.3920"],"URL":"https:\/\/doi.org\/10.7717\/peerj-cs.3920","archive":["CLOCKSS","LOCKSS","Portico"],"relation":{},"ISSN":["2376-5992"],"issn-type":[{"value":"2376-5992","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,24]]},"article-number":"e3920"}}