{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T11:21:46Z","timestamp":1771845706376,"version":"3.50.1"},"reference-count":36,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2025,7,11]],"date-time":"2025-07-11T00:00:00Z","timestamp":1752192000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Anhui Provincial Department of Education\u2019s Higher Education Institution Scientific Research Project (Humanities and Social Sciences)","award":["SK2020A0373"],"award-info":[{"award-number":["SK2020A0373"]}]},{"name":"Anhui Provincial Department of Education\u2019s Higher Education Institution Scientific Research Project (Humanities and Social Sciences)","award":["2023dzxkc030"],"award-info":[{"award-number":["2023dzxkc030"]}]},{"name":"Anhui Provincial Department of Education\u2019s Higher Education Institution Scientific Research Project (Humanities and Social Sciences)","award":["2024AH040383"],"award-info":[{"award-number":["2024AH040383"]}]},{"name":"Anhui Provincial Department of Education\u2019s Higher Education Institution Scientific Research Project (Humanities and Social Sciences)","award":["202410368070"],"award-info":[{"award-number":["202410368070"]}]},{"name":"Quality Engineering Project","award":["SK2020A0373"],"award-info":[{"award-number":["SK2020A0373"]}]},{"name":"Quality Engineering Project","award":["2023dzxkc030"],"award-info":[{"award-number":["2023dzxkc030"]}]},{"name":"Quality Engineering Project","award":["2024AH040383"],"award-info":[{"award-number":["2024AH040383"]}]},{"name":"Quality Engineering Project","award":["202410368070"],"award-info":[{"award-number":["202410368070"]}]},{"name":"Higher Education Institution Scientific Research Project (Humanities and Social Sciences)","award":["SK2020A0373"],"award-info":[{"award-number":["SK2020A0373"]}]},{"name":"Higher Education Institution Scientific Research Project (Humanities and Social Sciences)","award":["2023dzxkc030"],"award-info":[{"award-number":["2023dzxkc030"]}]},{"name":"Higher Education Institution Scientific Research Project (Humanities and Social Sciences)","award":["2024AH040383"],"award-info":[{"award-number":["2024AH040383"]}]},{"name":"Higher Education Institution Scientific Research Project (Humanities and Social Sciences)","award":["202410368070"],"award-info":[{"award-number":["202410368070"]}]},{"name":"Student Innovation Training Program","award":["SK2020A0373"],"award-info":[{"award-number":["SK2020A0373"]}]},{"name":"Student Innovation Training Program","award":["2023dzxkc030"],"award-info":[{"award-number":["2023dzxkc030"]}]},{"name":"Student Innovation Training Program","award":["2024AH040383"],"award-info":[{"award-number":["2024AH040383"]}]},{"name":"Student Innovation Training Program","award":["202410368070"],"award-info":[{"award-number":["202410368070"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Accurate and interpretable prediction plays a vital role in natural language processing (NLP) tasks, particularly for enhancing user trust and model transparency. However, existing models often struggle with poor adaptability and limited interpretability when applied to dynamic language prediction tasks such as Wordle. To address these challenges, this study proposes an interpretable reinforcement learning framework based on an Enhanced Deep Deterministic Policy Gradient (Enhanced-DDPG) algorithm. By leveraging a custom simulation environment and integrating key linguistic features word frequency, letter frequency, and repeated letter patterns (rep) the model dynamically predicts the number of attempts needed to solve Wordle puzzles. Experimental results demonstrate that Enhanced-DDPG outperforms traditional methods such as Random Forest Regression (RFR), XGBoost, LightGBM, METRA, and SQIRL in terms of both prediction accuracy (MSE = 0.0134, R2 = 0.8439) and robustness under noisy conditions. Furthermore, SHapley Additive exPlanations (SHAP) are employed to interpret the model\u2019s decision process, revealing that repeated letter patterns significantly influence low-attempt predictions, while word and letter frequencies are more relevant for higher attempt scenarios. This research highlights the potential of combining interpretable artificial intelligence (I-AI) and reinforcement learning to develop robust, transparent, and high-performance NLP prediction systems for real-world applications.<\/jats:p>","DOI":"10.3390\/a18070427","type":"journal-article","created":{"date-parts":[[2025,7,11]],"date-time":"2025-07-11T10:26:53Z","timestamp":1752229613000},"page":"427","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Interpretable Reinforcement Learning for Sequential Strategy Prediction in Language-Based Games"],"prefix":"10.3390","volume":"18","author":[{"given":"Jun","family":"Zhao","sequence":"first","affiliation":[{"name":"Department of Public Foundation, Wannan Medical College, Wuhu 241000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-1775-8200","authenticated-orcid":false,"given":"Jintian","family":"Ji","sequence":"additional","affiliation":[{"name":"Department of Medical Information, Wannan Medical College, Wuhu 241000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1673-1946","authenticated-orcid":false,"given":"Robail","family":"Yasrab","sequence":"additional","affiliation":[{"name":"MRC Biostatistics Unit, University of Cambridge, Cambridge CB2 1TN, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuxin","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Medical Information, Wannan Medical College, Wuhu 241000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liang","family":"Yu","sequence":"additional","affiliation":[{"name":"Department of Public Foundation, Wannan Medical College, Wuhu 241000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lingzhen","family":"Zhao","sequence":"additional","affiliation":[{"name":"Department of Public Foundation, Wannan Medical College, Wuhu 241000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,7,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1007\/s11749-016-0481-7","article-title":"A random forest guided tour","volume":"25","author":"Biau","year":"2016","journal-title":"Test"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1937","DOI":"10.1007\/s10462-020-09896-5","article-title":"A comparative analysis of gradient boosting algorithms","volume":"54","author":"Bentejac","year":"2021","journal-title":"Artif. Intell. Rev."},{"key":"ref_5","unstructured":"Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.Y. (2017). Lightgbm: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems 30, Neural Information Processing Systems Foundation, Inc. (NeurIPS)."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, S., Dong, X., Ma, D., Dang, B., Zang, H., and Gong, Y. (2024). Utilizing the lightgbm algorithm for operator user credit assessment research. arXiv.","DOI":"10.54254\/2755-2721\/75\/20240503"},{"key":"ref_7","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_8","unstructured":"Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10\u201315). Addressing function approximation error in actor-critic methods. Proceedings of the International Conference on Machine Learning, PMLR, Stockholm, Sweden."},{"key":"ref_9","unstructured":"Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 22\u201324). Deterministic policy gradient algorithms. Proceedings of the International Conference on Machine Learning, PMLR, Beijing, China."},{"key":"ref_10","unstructured":"Balduzzi, D., and Ghifary, M. (2015). Compatible value gradients for reinforcement learning of continuous deep policies. arXiv."},{"key":"ref_11","unstructured":"Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., and Abbeel, P. (2019). Soft Actor-Critic Algorithms and Applications. arXiv."},{"key":"ref_12","unstructured":"Park, S., Rybkin, O., and Levine, S. (2024). METRA: Scalable Unsupervised RL with Metric-Aware Abstraction. arXiv."},{"key":"ref_13","unstructured":"Laidlaw, C., Zhu, B., Russell, S., and Dragan, A. (2024). The Effective Horizon Explains Deep RL Performance in Stochastic Environments. arXiv."},{"key":"ref_14","unstructured":"Reddi, A., T\u00f6lle, M., Peters, J., Chalvatzaki, G., and D\u2019Eramo, C. (2023). Robust Adversarial Reinforcement Learning via Bounded Rationality Curricula. arXiv."},{"key":"ref_15","unstructured":"Futuhi, E., Karimi, S., Gao, C., and M\u00fcller, M. (2025). ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse Reward Continuous Control. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3623377","article-title":"Explainability in deep reinforcement learning: A review into current methods and applications","volume":"56","author":"Hickling","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Anderson, A., Dodge, J., Sadarangani, A., Juozapaitis, Z., Newman, E., Irvine, J., Chattopadhyay, S., Fern, A., and Burnett, M. (2019). Explaining Reinforcement Learning to Mere Mortals: An Empirical Study. arXiv.","DOI":"10.24963\/ijcai.2019\/184"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2930","DOI":"10.1016\/j.cja.2020.05.001","article-title":"Coactive design of explainable agent-based task planning and deep reinforcement learning for human-UAVs teamwork","volume":"33","author":"Wang","year":"2020","journal-title":"Chin. J. Aeronaut."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Douglas, N., Yim, D., Kartal, B., Hernandez-Leal, P., Maurer, F., and Taylor, M.E. (2019, January 10\u201313). Towers of saliency: A reinforcement learning visualization using immersive environments. Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces, Daejeon, Republic of Korea.","DOI":"10.1145\/3343055.3360747"},{"key":"ref_20","unstructured":"Huber, T., Schiller, D., and Andre, E. (2019). Enhancing explainability of deep reinforcement learning through selective layer-wise relevance propagation. KI 2019: Advances in Artificial Intelligence: 42nd German Conference on AI, Kassel, Germany, September 23\u201326, 2019, Springer International Publishing. Proceedings 42."},{"key":"ref_21","unstructured":"Muschalik, M., Baniecki, H., Fumagalli, F., Kolpaczki, P., Hammer, B., and Hullermeier, E. (2024). shapiq: Shapley interactions for machine learning. Advances in Neural Information Processing Systems 37, Neural Information Processing Systems Foundation, Inc. (NeurIPS)."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"103502","DOI":"10.1016\/j.artint.2021.103502","article-title":"Explaining individual predictions when features are dependent: More accurate approximations to Shapley values","volume":"298","author":"Aas","year":"2021","journal-title":"Artif. Intell."},{"key":"ref_23","first-page":"1","article-title":"Dalex: Responsible machine learning with interactive explainability and fairness in python","volume":"22","author":"Baniecki","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Oksuz, A.C., Halimi, A., and Ayday, E. (2024, January 15\u201320). AUTOLYCUS: Exploiting Explainable Artificial Intelligence (XAI) for Model Extraction Attacks against Interpretable Models. Proceedings of the Privacy Enhancing Technologies, Bristol, UK.","DOI":"10.56553\/popets-2024-0137"},{"key":"ref_25","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016). Model-Agnostic Interpretability of Machine Learning. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1038\/s42256-019-0138-9","article-title":"From local explanations to global understanding with explainable AI for trees","volume":"2","author":"Lundberg","year":"2020","journal-title":"Nat. Mach. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"919","DOI":"10.1007\/s11831-023-10002-5","article-title":"Comparative analysis of diabetic retinopathy classification approaches using machine learning and deep learning techniques","volume":"31","author":"Bala","year":"2024","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_28","unstructured":"Letoffe, O., Huang, X., and Marques-Silva, J. (2024). On correcting SHAP scores. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1453","DOI":"10.1007\/s12541-023-00816-5","article-title":"Digital Twin Data-Driven Multi-Disciplinary and Multi-Objective Optimization Framework for Automatic Design of Negative Stiffness Honeycomb","volume":"24","author":"Choi","year":"2023","journal-title":"Int. J. Precis. Eng. Manuf."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Feretzakis, G., Sakagianni, A., Anastasiou, A., Kapogianni, I., Bazakidou, E., Koufopoulos, P., Koumpouros, Y., Koufopoulou, C., Kaldis, V., and Verykios, V.S. (2024). Integrating Shapley values into machine learning techniques for enhanced predictions of hospital admissions. Appl. Sci., 14.","DOI":"10.3390\/app14135925"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Salih, A., Raisi-Estabragh, Z., Galazzo, I.B., Radeva, P., Petersen, E., Menegaz, G., and Lekadir, K. (2023). Commentary on explainable artificial intelligence methods: SHAP and LIME. arXiv.","DOI":"10.1002\/aisy.202400304"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.inffus.2021.07.016","article-title":"Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond","volume":"77","author":"Yang","year":"2022","journal-title":"Inf. Fusion"},{"key":"ref_33","first-page":"1803","article-title":"How to explain individual classification decisions","volume":"11","author":"Baehrens","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_34","unstructured":"Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., and Kim, B. (2018). Sanity checks for saliency maps. Advances in Neural Information Processing Systems 31, Neural Information Processing Systems Foundation, Inc. (NeurIPS)."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). Why should I trust you? Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_36","unstructured":"Teso, S., Alkan, O., Daly, E., and Stammer, W. (March, January 22). Explanations in Interactive Machine Learning. Proceedings of the AAAI Conference on Artificial Intelligence, Online."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/7\/427\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:08:16Z","timestamp":1760033296000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/7\/427"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,11]]},"references-count":36,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2025,7]]}},"alternative-id":["a18070427"],"URL":"https:\/\/doi.org\/10.3390\/a18070427","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,11]]}}}