{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:31:40Z","timestamp":1760059900279,"version":"build-2065373602"},"reference-count":30,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2025,7,20]],"date-time":"2025-07-20T00:00:00Z","timestamp":1752969600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Funda\u00e7\u00e3o para a Ci\u00eancia e Tecnologia (Portuguese Foundation for Science and Technology)","award":["UIDB\/00013\/2020","2024.00191.CPCA.A1","2022.06672.PTDC\u2014iMAD"],"award-info":[{"award-number":["UIDB\/00013\/2020","2024.00191.CPCA.A1","2022.06672.PTDC\u2014iMAD"]}]},{"name":"FCT I.P.","award":["UIDB\/00013\/2020","2024.00191.CPCA.A1","2022.06672.PTDC\u2014iMAD"],"award-info":[{"award-number":["UIDB\/00013\/2020","2024.00191.CPCA.A1","2022.06672.PTDC\u2014iMAD"]}]},{"name":"dtec.bw\u2014Digitalization and Technology Research Center of the Bundeswehr","award":["UIDB\/00013\/2020","2024.00191.CPCA.A1","2022.06672.PTDC\u2014iMAD"],"award-info":[{"award-number":["UIDB\/00013\/2020","2024.00191.CPCA.A1","2022.06672.PTDC\u2014iMAD"]}]},{"name":"European Union\u2014NextGenerationEU","award":["UIDB\/00013\/2020","2024.00191.CPCA.A1","2022.06672.PTDC\u2014iMAD"],"award-info":[{"award-number":["UIDB\/00013\/2020","2024.00191.CPCA.A1","2022.06672.PTDC\u2014iMAD"]}]},{"name":"FCT\/MCTES (PIDDAC)","award":["UIDB\/00013\/2020","2024.00191.CPCA.A1","2022.06672.PTDC\u2014iMAD"],"award-info":[{"award-number":["UIDB\/00013\/2020","2024.00191.CPCA.A1","2022.06672.PTDC\u2014iMAD"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Fractal Fract"],"abstract":"<jats:p>This work presents and compares different methodologies for the joint optimisation of the fractional derivative order and the parameters of the right-hand-side neural network in Neural Fractional Differential Equation models. The proposed strategies aim to tackle the training difficulties typically encountered when learning the fractional order \u03b1 together with the network weights. One approach is based on regulating the gradient magnitude of the loss function with respect to \u03b1, encouraging more stable and effective updates. Another strategy introduces an online pre-training scheme, where the network parameters are initially optimised over progressively longer time intervals, while \u03b1 is updated more conservatively using the full time trajectory. The study focuses only on a foundational setting with one-dimensional problems, and numerical experiments demonstrate that the proposed techniques improve both training stability and accuracy. Nonetheless, the issue of non-uniqueness in the optimal derivative order remains, particularly in less well-posed scenarios, suggesting the need for further research in data-driven modelling of fractional-order systems.<\/jats:p>","DOI":"10.3390\/fractalfract9070471","type":"journal-article","created":{"date-parts":[[2025,7,21]],"date-time":"2025-07-21T08:47:14Z","timestamp":1753087634000},"page":"471","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Methodologies for Improved Optimisation of the Derivative Order and Neural Network Parameters in Neural FDE Models"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-4502-937X","authenticated-orcid":false,"given":"Cec\u00edlia","family":"Coelho","sequence":"first","affiliation":[{"name":"Institute for Artificial Intelligence, Helmut Schmidt University, 22043 Hamburg, Germany"},{"name":"Centre of Mathematics (CMAT), University of Minho, 4710-57 Braga, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6235-286X","authenticated-orcid":false,"given":"M. Fernanda P.","family":"Costa","sequence":"additional","affiliation":[{"name":"Centre of Mathematics (CMAT), University of Minho, 4710-57 Braga, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8747-3596","authenticated-orcid":false,"given":"Oliver","family":"Niggemann","sequence":"additional","affiliation":[{"name":"Institute for Artificial Intelligence, Helmut Schmidt University, 22043 Hamburg, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5477-3226","authenticated-orcid":false,"given":"Lu\u00eds L.","family":"Ferr\u00e1s","sequence":"additional","affiliation":[{"name":"Centre of Mathematics (CMAT), University of Minho, 4710-57 Braga, Portugal"},{"name":"Centro de Estudos de Fen\u00f3menos de Transporte CEFT, Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal"},{"name":"ALiCE Associate Laboratory in Chemical Engineering, Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2025,7,20]]},"reference":[{"key":"ref_1","unstructured":"Coelho, C., Costa, M.F.P., and Ferr\u00e1s, L.L. (2024, January 11). Tracing Footprints: Neural Networks Meet Non-integer Order Differential Equations For Modelling Systems with Memory. Proceedings of the The Second Tiny Papers Track at ICLR 2024, Tiny Papers @ ICLR 2024, Vienna, Austria."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"116060","DOI":"10.1016\/j.apm.2025.116060","article-title":"Neural Fractional Differential Equations","volume":"144","author":"Coelho","year":"2025","journal-title":"Appl. Math. Model."},{"key":"ref_3","unstructured":"Kang, Q., Zhao, K., Ding, Q., Ji, F., Li, X., Liang, W., Song, Y., and Tay, W.P. (2024). Unleashing the potential of fractional calculus in graph neural networks with FROND. arXiv."},{"key":"ref_4","unstructured":"Cui, W., Kang, Q., Li, X., Zhao, K., Tay, W.P., Deng, W., and Li, Y. (March, January 25). Neural variable-order Fractional Differential Equation networks. Proceedings of the AAAI Conference on Artificial Intelligence, Philadelphia, PA, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"116127","DOI":"10.1016\/j.apm.2025.116127","article-title":"Physics-informed Neural Fractional Differential Equations","volume":"145","author":"Vellappandi","year":"2025","journal-title":"Appl. Math. Model."},{"key":"ref_6","unstructured":"Kang, Q., Li, X., Zhao, K., Cui, W., Zhao, Y., Deng, W., and Tay, W.P. (March, January 25). Efficient training of neural fractional-order differential equation via adjoint backpropagation. Proceedings of the AAAI Conference on Artificial Intelligence, Philadelphia, PA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"128143","DOI":"10.1016\/j.neucom.2024.128143","article-title":"FDE-Net: A memory-efficiency densely connected network inspired from fractional-order differential equations for single image super-resolution","volume":"600","author":"Zhang","year":"2024","journal-title":"Neurocomputing"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1111\/j.1365-246X.1967.tb02303.x","article-title":"Linear Models of Dissipation whose Q is almost Frequency Independent\u2014II","volume":"13","author":"Caputo","year":"1967","journal-title":"Geophys. J. Int."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Diethelm, K. (2010). The Analysis of Fractional Differential Equations: An Application-Oriented Exposition Using Differential Operators of Caputo Type, Springer.","DOI":"10.1007\/978-3-642-14574-2"},{"key":"ref_10","unstructured":"Podlubny, I. (1998). Fractional Differential Equations: An Introduction to fRactional Derivatives, Fractional Differential Equations, to Methods of Their Solution and Some of Their Applications, Elsevier."},{"key":"ref_11","unstructured":"Kilbas, A.A., Srivastava, H.M., and Trujillo, J.J. (2014). Theory and Applications of Fractional Differential Equations, North-Holland Mathematics Studies; Elsevier Science & Technology."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Jin, B. (2021). Fractional Differential Equations: An Approach via Fractional Derivatives, Springer International Publishing.","DOI":"10.1007\/978-3-030-76043-4"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Wang, J., and Zhang, L. (2016). Basic Theory of Fractional Differential Equations, WORLD SCIENTIFIC.","DOI":"10.1142\/10238"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Abbas, S., Benchohra, M., and N\u2019Gu\u00e9r\u00e9kata, G.M. (2012). Topics in Fractional Differential Equations, Springer.","DOI":"10.1007\/978-1-4614-4036-9"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Milici, C., Dr\u0103g\u0103nescu, G., and Tenreiro Machado, J. (2019). Introduction to Fractional Differential Equations, Springer International Publishing.","DOI":"10.1007\/978-3-030-00895-6"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"4846","DOI":"10.1002\/mma.3818","article-title":"Modeling some real phenomena by Fractional Differential Equations","volume":"39","author":"Almeida","year":"2015","journal-title":"Math. Methods Appl. Sci."},{"key":"ref_17","unstructured":"Chen, R.T., Rubanova, Y., Bettencourt, J., and Duvenaud, D.K. (2018). Neural ordinary differential equations. Adv. Neural Inf. Process. Syst., 31."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Coelho, C., Costa, M.F.P., and Ferr\u00e1s, L.L. (2024). Neural Fractional Differential Equations: Optimising the Order of the Fractional Derivative. Fractal Fract., 8.","DOI":"10.20944\/preprints202407.2572.v1"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1006\/jmaa.2000.7194","article-title":"Analysis of Fractional Differential Equations","volume":"265","author":"Diethelm","year":"2002","journal-title":"J. Math. Anal. Appl."},{"key":"ref_20","unstructured":"Zhang, J., He, T., Sra, S., and Jadbabaie, A. (2019). Why gradient clipping accelerates training: A theoretical justification for adaptivity. arXiv."},{"key":"ref_21","first-page":"15511","article-title":"Improved analysis of clipping algorithms for non-convex optimization","volume":"33","author":"Zhang","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1007\/s00521-013-1534-4","article-title":"A review of online learning in supervised neural networks","volume":"25","author":"Jain","year":"2014","journal-title":"Neural Comput. Appl."},{"key":"ref_23","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems 32, Curran Associates, Inc."},{"key":"ref_24","unstructured":"Zimmering, B., Coelho, C., and Niggemann, O. (2024, January 20). Optimising Neural Fractional Differential Equations for Performance and Efficiency. Proceedings of the 1st ECAI Workshop on \u201cMachine Learning Meets Differential Equations: From Theory to Applications\u201d, Santiago de Compostela, Spain."},{"key":"ref_25","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1151","DOI":"10.1080\/00207160.2017.1381691","article-title":"A note on finite difference methods for nonlinear fractional differential equations with non-uniform meshes","volume":"95","author":"Liu","year":"2018","journal-title":"Int. J. Comput. Math."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/s11071-012-0656-z","article-title":"Chaos in the fractional-order complex Lorenz system and its synchronization","volume":"71","author":"Luo","year":"2012","journal-title":"Nonlinear Dyn."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1393","DOI":"10.1016\/j.physa.2007.10.052","article-title":"The synchronization of fractional-order R\u00f6ssler hyperchaotic systems","volume":"387","author":"Yu","year":"2008","journal-title":"Phys. A Stat. Mech. Its Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1007\/s11071-010-9873-5","article-title":"Time-fractional KdV equation: Formulation and solution using variational methods","volume":"65","author":"Abulwafa","year":"2011","journal-title":"Nonlinear Dyn."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"056108","DOI":"10.1103\/PhysRevE.66.056108","article-title":"Fractional schr\u00f6dinger equation","volume":"66","author":"Laskin","year":"2002","journal-title":"Phys. Rev. E"}],"container-title":["Fractal and Fractional"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-3110\/9\/7\/471\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:12:52Z","timestamp":1760033572000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-3110\/9\/7\/471"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,20]]},"references-count":30,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2025,7]]}},"alternative-id":["fractalfract9070471"],"URL":"https:\/\/doi.org\/10.3390\/fractalfract9070471","relation":{},"ISSN":["2504-3110"],"issn-type":[{"type":"electronic","value":"2504-3110"}],"subject":[],"published":{"date-parts":[[2025,7,20]]}}}