{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:17:40Z","timestamp":1760059060228,"version":"build-2065373602"},"reference-count":15,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T00:00:00Z","timestamp":1747699200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Funda\u00e7\u00e3o para a Ci\u00eancia e Tecnologia (Portuguese Foundation for Science and Technology)","award":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"],"award-info":[{"award-number":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"]}]},{"name":"FCT and Google Cloud partnership","award":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"],"award-info":[{"award-number":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"]}]},{"name":"OptXAI: Constrained Optimization in NNs for Explainable, Ethical and Greener AI","award":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"],"award-info":[{"award-number":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"]}]},{"name":"FCT funding","award":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"],"award-info":[{"award-number":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"]}]},{"name":"KIBIDZ project","award":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"],"award-info":[{"award-number":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"]}]},{"name":"FCT\/MCTES (PIDDAC)","award":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"],"award-info":[{"award-number":["UIDB\/00013\/2020","UIDP\/00013\/2020","CPCA-IAC\/AV\/589164\/2023","CPCA-IAC\/AF\/589140\/2023","2024.00191.CPCA.A1","2021.05201.BD","2022.06672.PTDC"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["AI"],"abstract":"<jats:p>Neural Ordinary Differential Equations (Neural ODEs) have emerged as a promising approach for learning the continuous-time behaviour of dynamical systems from data. However, Neural ODEs are black-box models, posing challenges in interpreting and understanding their decision-making processes. This raises concerns about their application in critical domains such as healthcare and autonomous systems. To address this challenge and provide insight into the decision-making process of Neural ODEs, we introduce the eXplainable Neural ODE (XNODE) framework, a suite of eXplainable Artificial Intelligence (XAI) techniques specifically designed for Neural ODEs. Drawing inspiration from classical visualisation methods for differential equations, including time series, state space, and vector field plots, XNODE aims to offer intuitive insights into model behaviour. Although relatively simple, these techniques are intended to furnish researchers with a deeper understanding of the underlying mathematical tools, thereby serving as a practical guide for interpreting results obtained with Neural ODEs. The effectiveness of XNODE is verified through case studies involving a Resistor\u2013Capacitor (RC) circuit, the Lotka\u2013Volterra predator-prey dynamics, and a chemical reaction. The proposed XNODE suite offers a more nuanced perspective for cases where low Mean Squared Error values are obtained, which initially suggests successful learning of the data dynamics. This reveals that a low training error does not necessarily equate to comprehensive understanding or accurate modelling of the underlying data dynamics.<\/jats:p>","DOI":"10.3390\/ai6050105","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T06:10:46Z","timestamp":1747721446000},"page":"105","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["XNODE: A XAI Suite to Understand Neural Ordinary Differential Equations"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-4502-937X","authenticated-orcid":false,"given":"Cec\u00edlia","family":"Coelho","sequence":"first","affiliation":[{"name":"Institute for Artificial Intelligence, Helmut Schmidt University, 22043 Hamburg, Germany"},{"name":"Centre of Mathematics (CMAT), University of Minho, 4710-057 Braga, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6235-286X","authenticated-orcid":false,"given":"Maria Fernanda Pires","family":"da Costa","sequence":"additional","affiliation":[{"name":"Centre of Mathematics (CMAT), University of Minho, 4710-057 Braga, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5477-3226","authenticated-orcid":false,"given":"Lu\u00eds L.","family":"Ferr\u00e1s","sequence":"additional","affiliation":[{"name":"Centre of Mathematics (CMAT), University of Minho, 4710-057 Braga, Portugal"},{"name":"Centro de Estudos de Fen\u00f3menos de Transporte, Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal"},{"name":"ALiCE Associate Laboratory in Chemical Engineering, Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2025,5,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"106905","DOI":"10.1016\/j.compeleceng.2020.106905","article-title":"Predicting Market Movement Direction for Bitcoin: A Comparison of Time Series Modeling Methods","volume":"89","author":"Ibrahim","year":"2021","journal-title":"Comput. Electr. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"107225","DOI":"10.1016\/j.compeleceng.2021.107225","article-title":"Application of Neural Networks to Forecast Changes in the Technical Condition of Critical Production Facilities","volume":"93","author":"Yemelyanov","year":"2021","journal-title":"Comput. Electr. Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1016\/0893-6080(89)90020-8","article-title":"Multilayer Feedforward Networks Are Universal Approximators","volume":"2","author":"Hornik","year":"1989","journal-title":"Neural Netw."},{"key":"ref_4","unstructured":"Chen, R.T., Rubanova, Y., Bettencourt, J., and Duvenaud, D.K. (2018, January 3\u20138). Neural ordinary differential equations. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montr\u00e9al, QC, Canada."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Xing, Y., Ye, H., Zhang, X., Cao, W., Zheng, S., Bian, J., and Guo, Y. (2022, January 6\u20138). A Continuous Glucose Monitoring Measurements Forecasting Approach via Sporadic Blood Glucose Monitoring. Proceedings of the 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Las Vegas, NV, USA.","DOI":"10.1109\/BIBM55620.2022.9995522"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Su, X., Ji, W., An, J., Ren, Z., Deng, S., and Law, C.K. (2022). Kinetics Parameter Optimization via Neural Ordinary Differential Equations. arXiv.","DOI":"10.1016\/j.combustflame.2023.112732"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Laurie, M., and Lu, J. (2023). Explainable Deep Learning for Tumor Dynamic Modeling and Overall Survival Prediction Using Neural-ODE. arXiv.","DOI":"10.1038\/s41540-023-00317-1"},{"key":"ref_8","first-page":"53","article-title":"Understanding Neural ODE prediction decision using SHAP","volume":"Volume 233","author":"Lutchyn","year":"2024","journal-title":"Proceedings of the 5th Northern Lights Deep Learning Conference (NLDL)"},{"key":"ref_9","first-page":"5837","article-title":"Explainable Tensorized Neural Ordinary Differential Equations for Arbitrary-Step Time Series Prediction","volume":"35","author":"Gao","year":"2023","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3503","DOI":"10.1007\/s10462-021-10088-y","article-title":"Explainable Artificial Intelligence: A Comprehensive Review","volume":"55","author":"Minh","year":"2022","journal-title":"Artif. Intell. Rev."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"025025","DOI":"10.1088\/2632-2153\/acd5a9","article-title":"Interpretable Machine Learning Model to Predict Survival Days of Malignant Brain Tumor Patients","volume":"4","author":"Rajput","year":"2023","journal-title":"Mach. Learn. Sci. Technol."},{"key":"ref_12","unstructured":"Rojat, T., Puget, R., Filliat, D., Del Ser, J., Gelin, R., and D\u00edaz-Rodr\u00edguez, N. (2021). Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1007\/s40096-021-00442-0","article-title":"On Discrete Fractional-Order Lotka-Volterra Model Based on the Caputo Difference Discrete Operator","volume":"17","author":"Elsonbaty","year":"2023","journal-title":"Math. Sci."},{"key":"ref_14","unstructured":"Coelho, C., Costa, M.F.P., and Ferr\u00e1s, L.L. (2023, November 20). Synthetic Chemical Reaction. Available online: https:\/\/www.kaggle.com\/datasets\/cici118\/synthetic-chemical-reaction."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Coelho, C., Costa, M.F.P., and Ferr\u00e1s, L.L. (2025). A Two-Stage Training Method for Modeling Constrained Systems With Neural Networks. J. Forecast.","DOI":"10.1002\/for.3270"}],"container-title":["AI"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2673-2688\/6\/5\/105\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:35:35Z","timestamp":1760031335000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2673-2688\/6\/5\/105"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,20]]},"references-count":15,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5]]}},"alternative-id":["ai6050105"],"URL":"https:\/\/doi.org\/10.3390\/ai6050105","relation":{},"ISSN":["2673-2688"],"issn-type":[{"type":"electronic","value":"2673-2688"}],"subject":[],"published":{"date-parts":[[2025,5,20]]}}}