{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:05:28Z","timestamp":1760144728392,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2024,5,13]],"date-time":"2024-05-13T00:00:00Z","timestamp":1715558400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"MOE Humanities and the Social Sciences Foundation of China","award":["20YJCZH102","EP\/N020170\/1"],"award-info":[{"award-number":["20YJCZH102","EP\/N020170\/1"]}]},{"name":"Singapore\u2013UK Cyber Security of EPSRC","award":["20YJCZH102","EP\/N020170\/1"],"award-info":[{"award-number":["20YJCZH102","EP\/N020170\/1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>In text classifier models, the complexity of recurrent neural networks (RNNs) is very high because of the vast state space and uncertainty of transitions, which makes the RNN classifier\u2019s explainability insufficient. It is almost impossible to explain the large-scale RNN directly. A feasible method is to generalize the rules undermining it, that is, model abstraction. To deal with the low efficiency and excessive information loss in existing model abstraction for RNNs, this work proposes a PSO (Particle Swarm Optimization)-based model abstraction and explanation generation method for RNNs. Firstly, the k-means clustering is applied to preliminarily partition the RNN decision process state. Secondly, a frequency prefix tree is constructed based on the traces, and a PSO algorithm is designed to implement state merging to address the problem of vast state space. Then, a PFA (probabilistic finite automata) is constructed to explain the RNN structure with preserving the origin RNN information as much as possible. Finally, the quantitative keywords are labeled as an explanation for classification results, which are automatically generated with the abstract model PFA. We demonstrate the feasibility and effectiveness of the proposed method in some cases.<\/jats:p>","DOI":"10.3390\/a17050210","type":"journal-article","created":{"date-parts":[[2024,5,13]],"date-time":"2024-05-13T11:18:17Z","timestamp":1715599097000},"page":"210","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Particle Swarm Optimization-Based Model Abstraction and Explanation Generation for a Recurrent Neural Network"],"prefix":"10.3390","volume":"17","author":[{"given":"Yang","family":"Liu","sequence":"first","affiliation":[{"name":"Institute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 200120, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-6452-1547","authenticated-orcid":false,"given":"Huadong","family":"Wang","sequence":"additional","affiliation":[{"name":"Institute of Logistics Science and Engineering, Shanghai Maritime University, Shanghai 200120, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan","family":"Ma","sequence":"additional","affiliation":[{"name":"School of Accounting, Nanjing University of Finance and Economics, Nanjing 210023, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,5,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"105697","DOI":"10.1016\/j.engappai.2022.105697","article-title":"Review of artificial intelligence applications in engineering design perspective","volume":"118","author":"Sezer","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_2","unstructured":"Zhang, S., Wu, L., Yu, S.G., Shi, E.Z., Qiang, N., Gao, H., Zhao, J.Y., and Zhao, S.J. (2022). An Explainable and Generalizable Recurrent Neural Network Approach for Differentiating Human Brain States on EEG Dataset. Ieee Trans. Neural Netw. Learn. Syst., Article ASAP."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.neucom.2023.01.071","article-title":"TextGuise: Adaptive adversarial example attacks on text classification model","volume":"529","author":"Chang","year":"2023","journal-title":"Neurocomputing"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"106458","DOI":"10.1016\/j.knosys.2020.106458","article-title":"A deep neural network based multi-task learning approach to hate speech detection","volume":"210","author":"Kapil","year":"2020","journal-title":"Knowl.-Based Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"107506","DOI":"10.1016\/j.asoc.2021.107506","article-title":"Semantics aware adversarial malware examples generation for black-box attacks","volume":"109","author":"Peng","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Du, M., Liu, N., Yang, F., Ji, S., and Hu, X. (2019, January 13\u201317). On Attribution of Recurrent Neural Network Predictions via Additive Decomposition. Proceedings of the The World Wide Web Conference, San Francisco, CA, USA.","DOI":"10.1145\/3308558.3313545"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Li, X.Z., Lin, F.F., Wang, H., Zhang, X., Ma, H., Wen, C.Y., and Blaabjerg, F. (2024). Temporal Modeling for Power Converters with Physics-in-Architecture Recurrent Neural Network. Ieee Trans. Ind. Electron., Article ASAP.","DOI":"10.1109\/TIE.2024.3352119"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4025","DOI":"10.1007\/s11280-023-01211-w","article-title":"Death comes but why: A multi-task memory-fused prediction for accurate and explainable illness severity in ICUs","volume":"26","author":"Chen","year":"2023","journal-title":"World Wide Web-Internet Web Inf. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Bennetot","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"106687","DOI":"10.1016\/j.knosys.2020.106687","article-title":"Accurate and Explainable Recommendation via Hierarchical Attention Network Oriented towards Crowd Intelligence","volume":"213","author":"Yang","year":"2021","journal-title":"Knowl.-Based Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hong, D., Segre, A.M., and Wang, T. (2022, January 14\u201318). AdaAX: Explaining Recurrent Neural Networks by Learning Automata with Adaptive States. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington DC, USA.","DOI":"10.1145\/3534678.3539356"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yang, M., Moon, J., Yang, S., Oh, H., Lee, S., Kim, Y., and Jeong, J. (2022). Design and Implementation of an Explainable Bidirectional LSTM Model Based on Transition System Approach for Cooperative AI-Workers. Appl. Sci., 12.","DOI":"10.3390\/app12136390"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1007\/s10009-022-00684-w","article-title":"Analysis of recurrent neural networks via property-directed verification of surrogate models","volume":"25","author":"Khmelnitsky","year":"2023","journal-title":"Int. J. Softw. Tools Technol. Transf."},{"key":"ref_14","unstructured":"Guillaumier, K., and Abela, J. (2021, January 23). Learning DFAs by Evolving Short Sequences of Merges. Proceedings of the ICGI 2021\u201415th International Conference on Grammatical Inference, Virtual\/New York City, NY, USA."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/0890-5401(87)90052-6","article-title":"Learning regular sets from queries and counterexamples","volume":"75","author":"Angluin","year":"1987","journal-title":"Inf. Comput."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1145\/2967606","article-title":"Model learning","volume":"60","author":"Vaandrager","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_17","first-page":"2267","article-title":"Learning With Interpretable Structure From Gated RNN","volume":"31","author":"Hou","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4117","DOI":"10.1109\/TIFS.2021.3103064","article-title":"Text Backdoor Detection Using an Interpretable RNN Abstract Model","volume":"16","author":"Fan","year":"2021","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"100907","DOI":"10.1016\/j.jlamp.2023.100907","article-title":"Weighted automata extraction and explanation of recurrent neural networks for natural language tasks","volume":"136","author":"Wei","year":"2024","journal-title":"J. Log. Algebr. Methods Program."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yellin, D.M., and Weiss, G. (April, January 27). Synthesizing Context-free Grammars from Recurrent Neural Networks. Proceedings of the Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2021, Luxembourg City, Luxembourg.","DOI":"10.1007\/978-3-030-72016-2_19"},{"key":"ref_21","unstructured":"Weiss, G., Goldberg, Y., and Yahav, E. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 10\u201315 July 2018, Proceedings of Machine Learning Research."},{"key":"ref_22","unstructured":"Barbot, B., Bollig, B., Finkel, A., Haddad, S., Khmelnitsky, I., Leucker, M., Neider, D., Roy, R., and Ye, L. (2021, January 23). Extracting Context-Free Grammars from Recurrent Neural Networks using Tree-Automata Learning and A* Search. Proceedings of the ICGI 2021\u201415th International Conference on Grammatical Inference, Virtual\/New York City, NY, USA."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1007\/s10009-018-0492-7","article-title":"Learning probabilistic models for model checking: An evolutionary approach and an empirical study","volume":"20","author":"Wang","year":"2018","journal-title":"Int. J. Softw. Tools Technol. Transf."},{"key":"ref_24","unstructured":"Weiss, G., Goldberg, Y., and Yahav, E. (2019, January 8\u201314). Learning deterministic weighted automata with queries and counterexamples. Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Dong, G., Wang, J., Sun, J., Zhang, Y., Wang, X., Dai, T., Dong, J.S., and Wang, X. (2021, January 21\u201325). Towards interpreting recurrent neural networks through probabilistic abstraction. Proceedings of the 35th IEEE\/ACM International Conference on Automated Software Engineering, Virtual Event.","DOI":"10.1145\/3324884.3416592"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Mao, H., Chen, Y., Jaeger, M., Nielsen, T.D., Larsen, K.G., and Nielsen, B. (2011, January 5\u20138). Learning Probabilistic Automata for Model Checking. Proceedings of the 2011 Eighth International Conference on Quantitative Evaluation of SysTems, Aachen, Germany.","DOI":"10.1109\/QEST.2011.21"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"108213","DOI":"10.1016\/j.ymssp.2021.108213","article-title":"Thresholdless Classification of chaotic dynamics and combustion instability via probabilistic finite state automata","volume":"164","author":"Bhattacharya","year":"2022","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"107117","DOI":"10.1016\/j.infsof.2022.107117","article-title":"PAFL: Probabilistic Automaton-Based Fault Localization for Recurrent Neural Networks","volume":"155","author":"Ishimoto","year":"2023","journal-title":"Inf. Softw. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2568","DOI":"10.1162\/neco_a_01111","article-title":"An Empirical Evaluation of Rule Extraction from Recurrent Neural Networks","volume":"30","author":"Wang","year":"2018","journal-title":"Neural Comput."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"819","DOI":"10.1613\/jair.1.12963","article-title":"Task-Aware Verifiable RNN-Based Policies for Partially Observable Markov Decision Processes","volume":"72","author":"Carr","year":"2021","journal-title":"J. Artif. Intell. Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"7739","DOI":"10.1109\/TPAMI.2022.3225334","article-title":"State-Regularized Recurrent Neural Networks to Extract Automata and Explain Predictions","volume":"45","author":"Wang","year":"2023","journal-title":"Ieee Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Du, X., Xie, X., Li, Y., Ma, L., Liu, Y., and Zhao, J. (2019, January 26\u201330). DeepStellar: Model-based quantitative analysis of stateful deep learning systems. Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Tallinn, Estonia.","DOI":"10.1145\/3338906.3338954"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Maes, P. (1987, January 4\u20138). Concepts and experiments in computational reflection. Proceedings of the Conference Proceedings on Object-Oriented Programming Systems, Languages and Applications, Orlando, FL, USA.","DOI":"10.1145\/38765.38821"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1145\/3527448","article-title":"Explainable Deep Reinforcement Learning: State of the Art and Challenges","volume":"55","author":"Vouros","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Kwiatkowska, M., Norman, G., and Parker, D. (2011, January 14\u201320). PRISM 4.0: Verification of Probabilistic Real-Time Systems. Proceedings of the International Conference on Computer Aided Verification, CAV 2011, Snowbird, UT, USA.","DOI":"10.1007\/978-3-642-22110-1_47"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1016\/j.neucom.2021.04.105","article-title":"Mitigating backdoor attacks in LSTM-based text classification systems by Backdoor Keyword Identification","volume":"452","author":"Chen","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Wang, B., Yao, Y., Shan, S., Li, H., Viswanath, B., Zheng, H., and Zhao, B.Y. (2019, January 19\u201323). Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks. Proceedings of the 2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA.","DOI":"10.1109\/SP.2019.00031"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Li, J., Ji, S., Du, T., Li, B., and Wang, T. (2018). TextBugger: Generating Adversarial Text Against Real-World Applications. arXiv.","DOI":"10.14722\/ndss.2019.23138"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/5\/210\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:41:40Z","timestamp":1760107300000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/5\/210"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,13]]},"references-count":38,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2024,5]]}},"alternative-id":["a17050210"],"URL":"https:\/\/doi.org\/10.3390\/a17050210","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2024,5,13]]}}}