{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T23:59:34Z","timestamp":1740182374989,"version":"3.37.3"},"reference-count":64,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2024,10,2]],"date-time":"2024-10-02T00:00:00Z","timestamp":1727827200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,10,2]],"date-time":"2024-10-02T00:00:00Z","timestamp":1727827200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Link\u00f6ping University, Sweden"},{"name":"Link\u00f6ping University Cancer Research Network"},{"name":"Excellence Center at Link\u00f6ping-Lund in Information Technology"},{"name":"Link\u00f6ping University Center for Industrial Information Technology"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Oper. Res. Forum"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The inverse problem of supervised reconstruction of depth-variable (time-dependent) parameters in ordinary differential equations is considered, with the typical application of finding weights of a neural ordinary differential equation (NODE) for a residual network with time continuous layers. The differential equation is treated as an abstract and isolated entity, termed a standalone NODE (sNODE), to facilitate for a wide range of applications. The proposed parameter reconstruction is performed by minimizing a cost functional covering a variety of loss functions and penalty terms. Regularization via penalty terms is incorporated to enhance ethical and trustworthy AI formulations. A nonlinear conjugate gradient mini-batch optimization scheme (NCG) is derived for the training having the benefit of including a sensitivity problem. The model (differential equation)-based approach is thus combined with a data-driven learning procedure. Mathematical properties are stated for the differential equation and the cost functional. The adjoint problem needed is derived together with the sensitivity problem. The sensitivity problem itself can estimate changes in the output under perturbation of the trained parameters. To preserve smoothness during the iterations, the Sobolev gradient is calculated and incorporated. Numerical results are included to validate the procedure for a NODE and synthetic datasets and compared with standard gradient approaches. For stability, using the sensitivity problem, a strategy for adversarial attacks is constructed, and it is shown that the given method with Sobolev gradients is more robust than standard approaches for parameter identification.<\/jats:p>","DOI":"10.1007\/s43069-024-00377-x","type":"journal-article","created":{"date-parts":[[2024,10,2]],"date-time":"2024-10-02T11:02:17Z","timestamp":1727866937000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Hybrid Sobolev Gradient Method for Learning NODEs"],"prefix":"10.1007","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9810-3539","authenticated-orcid":false,"given":"George","family":"Baravdish","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9217-9997","authenticated-orcid":false,"given":"Gabriel","family":"Eilertsen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8445-0129","authenticated-orcid":false,"given":"Rym","family":"Jaroudi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9066-7922","authenticated-orcid":false,"given":"B. Tomas","family":"Johansson","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2083-9180","authenticated-orcid":false,"given":"Luk\u00e1\u0161","family":"Mal\u00fd","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7765-1747","authenticated-orcid":false,"given":"Jonas","family":"Unger","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,10,2]]},"reference":[{"issue":"4","key":"377_CR1","doi-asserted-by":"publisher","first-page":"797","DOI":"10.1080\/00207160.2017.1296955","volume":"95","author":"K Cao","year":"2018","unstructured":"Cao K, Lesnic D (2018) Reconstruction of the perfusion coefficient from temperature measurements using the conjugate gradient method. Int J Comput Math 95(4):797\u2013814. https:\/\/doi.org\/10.1080\/00207160.2017.1296955","journal-title":"Int J Comput Math"},{"key":"377_CR2","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1016\/j.cam.2018.01.010","volume":"337","author":"K Cao","year":"2018","unstructured":"Cao K, Lesnic D (2018) Reconstruction of the space-dependent perfusion coefficient from final time or time-average temperature measurements. J Comput Appl Math 337:150\u2013165. https:\/\/doi.org\/10.1016\/j.cam.2018.01.010","journal-title":"J Comput Appl Math"},{"key":"377_CR3","doi-asserted-by":"publisher","unstructured":"Alosaimi M, Lesnic D, Johansson BT (2021) Solution of the Cauchy problem for the wave equation using iterative regularization. Inverse Probl Sci Eng 29:2757\u20132771. https:\/\/doi.org\/10.1080\/17415977.2021.1949590","DOI":"10.1080\/17415977.2021.1949590"},{"key":"377_CR4","doi-asserted-by":"crossref","unstructured":"Baravdish G, Johansson BT, Ssebunjo W, Svensson O (2021) Identifying the response of radiation therapy for brain tumors. IMA J. Appl. Math. 88(2023), 378\u2013404","DOI":"10.1093\/imamat\/hxad013"},{"issue":"3","key":"377_CR5","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1007\/s10915-022-01939-z","volume":"92","author":"S Cuomo","year":"2022","unstructured":"Cuomo S, Di Cola VS, Giampaolo F, Rozza G, Raissi M, Piccialli F (2022) Scientific machine learning through physics-informed neural networks: where we are and what\u2019s next. J Sci Comput 92(3):88. https:\/\/doi.org\/10.1007\/s10915-022-01939-z","journal-title":"J Sci Comput"},{"key":"377_CR6","doi-asserted-by":"publisher","unstructured":"Fompeyrine DA, Vorm ES, Ricka N, Rose F, Pellegrin G (2021) Enhancing human-machine teaming for medical prognosis through neural ordinary differential equations (NODEs). Human Intell Syst Integr 3:263\u2013275. https:\/\/doi.org\/10.1007\/s42454-021-00037-z","DOI":"10.1007\/s42454-021-00037-z"},{"issue":"1","key":"377_CR7","doi-asserted-by":"publisher","first-page":"216","DOI":"10.1038\/msb.2008.53","volume":"4","author":"S Nelander","year":"2008","unstructured":"Nelander S, Wang W, Nilsson B, She Q-B, Pratilas C, Rosen N, Gennemark P, Sander C (2008) Models from experiments: combinatorial drug perturbations of cancer cells. Mol Syst Biol 4(1):216. https:\/\/doi.org\/10.1038\/msb.2008.53","journal-title":"Mol Syst Biol"},{"issue":"7","key":"377_CR8","doi-asserted-by":"publisher","first-page":"1007909","DOI":"10.1371\/journal.pcbi.1007909","volume":"16","author":"E Nyman","year":"2020","unstructured":"Nyman E, Stein RR, Jing X, Wang W, Marks B, Zervantonakis IK, Korkut A, Gauthier NP, Sander C (2020) Perturbation biology links temporal protein changes to drug responses in a melanoma cell line. PLoS Comput Biol 16(7):1007909. https:\/\/doi.org\/10.1371\/journal.pcbi.1007909","journal-title":"PLoS Comput Biol"},{"key":"377_CR9","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1016\/j.cherd.2022.01.041","volume":"179","author":"MSF Bangi","year":"2022","unstructured":"Bangi MSF, Kao K, Kwon JS-I (2022) Physics-informed neural networks for hybrid modeling of lab-scale batch fermentation for $$\\beta $$-carotene production using saccharomyces cerevisiae. Chem Eng Res Des 179:415\u2013423. https:\/\/doi.org\/10.1016\/j.cherd.2022.01.041","journal-title":"Chem Eng Res Des"},{"issue":"9","key":"377_CR10","doi-asserted-by":"publisher","first-page":"620","DOI":"10.1016\/j.ifacol.2021.06.124","volume":"54","author":"M Benning","year":"2021","unstructured":"Benning M, Celledoni E, Ehrhardt MJ, Owren B, Sch\u00f6nlieb C-B (2021) Deep learning as optimal control problems. IFAC-PapersOnLine 54(9):620\u2013623. https:\/\/doi.org\/10.1016\/j.ifacol.2021.06.124","journal-title":"IFAC-PapersOnLine"},{"key":"377_CR11","doi-asserted-by":"publisher","unstructured":"Giesecke E, Kr\u00f6ner A (2021) Classification with Runge-Kutta networks and feature space augmentation. J Comput Dyn 8(4):495\u2013520. https:\/\/doi.org\/10.3934\/jcd.2021018","DOI":"10.3934\/jcd.2021018"},{"key":"377_CR12","doi-asserted-by":"publisher","first-page":"116196","DOI":"10.1016\/j.jsv.2021.116196","volume":"508","author":"Z Lai","year":"2021","unstructured":"Lai Z, Mylonas C, Nagarajaiah S, Chatzi E (2021) Structural identification with physics-informed neural ordinary differential equations. J Sound Vib 508:116196. https:\/\/doi.org\/10.1016\/j.jsv.2021.116196","journal-title":"J Sound Vib"},{"key":"377_CR13","doi-asserted-by":"publisher","unstructured":"Lai Z, Liu W, Jian X, Bacsa K, Sun L, Chatzi E (2022) Neural modal ODEs: integrating physics-based modeling with neural ODEs for modeling high dimensional monitored structures. Data-Centric Engineering 3:e34. https:\/\/doi.org\/10.1017\/dce.2022.35","DOI":"10.1017\/dce.2022.35"},{"key":"377_CR14","doi-asserted-by":"publisher","unstructured":"Parvini\u00a0Ahmadi S, Hansson A (2023) Distributed optimal control of nonlinear systems using a second-order augmented Lagrangian method. Eur J Control 70. https:\/\/doi.org\/10.1016\/j.ejcon.2022.100768","DOI":"10.1016\/j.ejcon.2022.100768"},{"key":"377_CR15","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1016\/j.jcp.2018.10.045","volume":"378","author":"M Raissi","year":"2019","unstructured":"Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686\u2013707. https:\/\/doi.org\/10.1016\/j.jcp.2018.10.045","journal-title":"J Comput Phys"},{"key":"377_CR16","unstructured":"Chen RTQ, Rubanova Y, Bettencourt J, Duvenaud D (2018) Neural ordinary differential equations. In: Bengio S, Wallach HM, Larochelle H, Grauman K, Cesa-Bianchi N (eds) Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp 6572\u20136583. Curran Associates Inc, Red Hook, USA"},{"key":"377_CR17","volume-title":"Modeling and identification of dynamic systems","author":"L Ljung","year":"2021","unstructured":"Ljung L, Glad T, Hansson A (2021) Modeling and identification of dynamic systems. Studentlitteratur, Sweden"},{"key":"377_CR18","doi-asserted-by":"publisher","unstructured":"Gholami A, Keutzer K, Biros G (2019) Anode: unconditionally accurate memory-efficient gradients for neural odes. arXiv:1902.10298. https:\/\/doi.org\/10.48550\/arXiv.1902.10298","DOI":"10.48550\/arXiv.1902.10298"},{"key":"377_CR19","unstructured":"Mannion P, Heintz F, Karimpanal TG, Vamplew P (2021) Multi-objective decision making for trustworthy AI. In: Proceedings of the multi-objective decision making (MODeM) Workshop"},{"key":"377_CR20","doi-asserted-by":"crossref","unstructured":"Lombardi M, Baldo F, Borghesi A, Milano M (2021) An analysis of regularized approaches for constrained machine learning. In: Trustworthy AI-integrating learning, optimization and reasoning: First International Workshop, TAILOR 2020, Virtual Event, September 4\u20135, 2020, Revised Selected Papers 1. Springer, pp 112\u2013119","DOI":"10.1007\/978-3-030-73959-1_11"},{"issue":"3","key":"377_CR21","doi-asserted-by":"publisher","first-page":"745","DOI":"10.1088\/0266-5611\/15\/3\/308","volume":"15","author":"HE Kunze","year":"1999","unstructured":"Kunze HE, Vrscay ER (1999) Solving inverse problems for ordinary differential equations using the Picard contraction mapping. Inverse Probl 15(3):745\u2013770. https:\/\/doi.org\/10.1088\/0266-5611\/15\/3\/308","journal-title":"Inverse Probl"},{"key":"377_CR22","doi-asserted-by":"publisher","unstructured":"Llibre J, Ram\u00edrez R (2016) Inverse problems in ordinary differential equations and applications vol. 313. Springer, Cham. https:\/\/doi.org\/10.1007\/978-3-319-26339-7","DOI":"10.1007\/978-3-319-26339-7"},{"issue":"2","key":"377_CR23","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/BF00941054","volume":"62","author":"AA Brown","year":"1989","unstructured":"Brown AA, Bartholomew-Biggs MC (1989) Some effective methods for unconstrained optimization based on the solution of systems of ordinary differential equations. J Optim Theory Appl 62(2):211\u2013224. https:\/\/doi.org\/10.1007\/BF00941054","journal-title":"J Optim Theory Appl"},{"key":"377_CR24","doi-asserted-by":"publisher","unstructured":"Arridge S, de Hoop M, Maass P, \u00d6ktem O, Sch\u00f6nlieb C, Unser M (2019) Deep learning and inverse problems. Snapshots of Modern Mathematics from Oberwolfach, 1\u201313. https:\/\/doi.org\/10.4171\/OWR\/2021\/13","DOI":"10.4171\/OWR\/2021\/13"},{"issue":"4","key":"377_CR25","doi-asserted-by":"publisher","first-page":"860","DOI":"10.1137\/18M1165748","volume":"61","author":"CF Higham","year":"2019","unstructured":"Higham CF, Higham DJ (2019) Deep learning: an introduction for applied mathematicians. SIAM Rev 61(4):860\u2013891. https:\/\/doi.org\/10.1137\/18M1165748","journal-title":"SIAM Rev"},{"key":"377_CR26","unstructured":"Lu Y, Zhong A, Li Q, Dong B (2018) Beyond finite layer neural networks: bridging deep architectures and numerical differential equations. In: Dy J, Krause A (eds) Proceedings of the 35th international conference on machine learning. proceedings of machine learning research, vol 80. PMLR, pp 3276\u20133285. https:\/\/proceedings.mlr.press\/v80\/lu18d.html"},{"key":"377_CR27","unstructured":"Dupont E, Doucet A, Teh YW (2019) Augmented neural ODEs. In: Wallach H, Larochelle H, Beygelzimer A, d\u2019Alch\u00e9 Buc F, Fox E, Garnet R (eds) Advances in neural information processing systems, vol 32, pp 3140\u20133150. Curran Associates Inc, USA. http:\/\/papers.neurips.cc\/paper\/by-source-2019-1771"},{"key":"377_CR28","doi-asserted-by":"publisher","unstructured":"Yan H, Du J, Tan VY, Feng J (2019) On robustness of neural ordinary differential equations. In: International conference on learning representations. https:\/\/doi.org\/10.48550\/arXiv.1910.05513","DOI":"10.48550\/arXiv.1910.05513"},{"key":"377_CR29","doi-asserted-by":"publisher","unstructured":"Liu X, Xiao T, Si S, Cao Q, Kumar S, Hsieh C-J (2019) Neural SDE: stabilizing neural ODE networks with stochastic noise. https:\/\/doi.org\/10.48550\/arXiv.1906.02355","DOI":"10.48550\/arXiv.1906.02355"},{"key":"377_CR30","unstructured":"Matsubara T, Miyatake Y, Yaguchi T (2021) Symplectic adjoint method for exact gradient of neural ODE with minimal memory. Adv Neural Inf Process Syst 34"},{"key":"377_CR31","first-page":"3952","volume":"33","author":"S Massaroli","year":"2020","unstructured":"Massaroli S, Poli M, Park J, Yamashita A, Asama H (2020) Dissecting neural ODEs. Adv Neural Inf Process Syst 33:3952\u20133963","journal-title":"Adv Neural Inf Process Syst"},{"key":"377_CR32","unstructured":"Queiruga A, Erichson NB, Hodgkinson L, Mahoney MW (2021) Stateful ODE-Nets using basis function expansions. Adv Neural Inf Process Syst 34"},{"key":"377_CR33","doi-asserted-by":"publisher","unstructured":"Simonyan K, Vedaldi A, Zisserman A (2013) Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv:1312.6034. https:\/\/doi.org\/10.48550\/arXiv.1312.6034","DOI":"10.48550\/arXiv.1312.6034"},{"key":"377_CR34","doi-asserted-by":"publisher","unstructured":"Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer, pp 818\u2013833. https:\/\/doi.org\/10.1007\/978-3-319-10590-1_53","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"377_CR35","doi-asserted-by":"publisher","unstructured":"Yosinski J, Clune J, Nguyen A, Fuchs T, Lipson H (2015) Understanding neural networks through deep visualization. arXiv:1506.06579. https:\/\/doi.org\/10.48550\/arXiv.1506.06579","DOI":"10.48550\/arXiv.1506.06579"},{"key":"377_CR36","doi-asserted-by":"publisher","unstructured":"Novak R, Bahri Y, Abolafia DA, Pennington J, Sohl-Dickstein J (2018) Sensitivity and generalization in neural networks: an empirical study. arXiv:1802.08760. https:\/\/doi.org\/10.48550\/arXiv.1802.08760","DOI":"10.48550\/arXiv.1802.08760"},{"key":"377_CR37","doi-asserted-by":"publisher","unstructured":"Pizarroso J, Portela J, Mu\u00f1oz A (2020) NeuralSens: sensitivity analysis of neural networks. arXiv:2002.11423. https:\/\/doi.org\/10.48550\/arXiv.2002.11423","DOI":"10.48550\/arXiv.2002.11423"},{"key":"377_CR38","doi-asserted-by":"publisher","unstructured":"Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R (2013) Intriguing properties of neural networks. arXiv:1312.6199. https:\/\/doi.org\/10.48550\/arXiv.1312.6199","DOI":"10.48550\/arXiv.1312.6199"},{"key":"377_CR39","doi-asserted-by":"publisher","unstructured":"Goodfellow IJ, Shlens J, Szegedy C (2014) Explaining and harnessing adversarial examples. arXiv:1412.6572. https:\/\/doi.org\/10.48550\/arXiv.1412.6572","DOI":"10.48550\/arXiv.1412.6572"},{"key":"377_CR40","doi-asserted-by":"publisher","unstructured":"Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2018) Towards deep learning models resistant to adversarial attacks. In: International conference on learning representations. https:\/\/doi.org\/10.48550\/arXiv.1706.06083","DOI":"10.48550\/arXiv.1706.06083"},{"key":"377_CR41","doi-asserted-by":"publisher","unstructured":"Carrara F, Caldelli R, Falchi F, Amato G (2019) On the robustness to adversarial examples of neural ODE image classifiers. In: 2019 IEEE International workshop on information forensics and security (WIFS). IEEE, pp 1\u20136. https:\/\/doi.org\/10.1109\/WIFS47025.2019.9035109","DOI":"10.1109\/WIFS47025.2019.9035109"},{"key":"377_CR42","unstructured":"Kang Q, Song Y, Ding Q, Tay WP (2021) Stable neural ODE with Lyapunov-stable equilibrium points for defending against adversarial attacks. Adv Neural Inf Process Syst 34"},{"issue":"2","key":"377_CR43","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1137\/16M1080173","volume":"60","author":"L Bottou","year":"2018","unstructured":"Bottou L, Curtis FE, Nocedal J (2018) Optimization methods for large-scale machine learning. SIAM Rev 60(2):223\u2013311. https:\/\/doi.org\/10.1137\/16M1080173","journal-title":"SIAM Rev"},{"key":"377_CR44","unstructured":"Czarnecki WM, Osindero S, Jaderberg M, Swirszcz G, Pascanu R (2017) Sobolev training for neural networks. Adv Neural Inf Process Syst 30"},{"key":"377_CR45","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-017-1517-1","volume-title":"Existence theory for nonlinear ordinary differential equations","author":"D O\u2019Regan","year":"1997","unstructured":"O\u2019Regan D (1997) Existence theory for nonlinear ordinary differential equations. Springer, Dordrecht. https:\/\/doi.org\/10.1007\/978-94-017-1517-1"},{"key":"377_CR46","doi-asserted-by":"publisher","unstructured":"Hartman P (2002) Ordinary differential equations, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA. https:\/\/doi.org\/10.1137\/1.9780898719222","DOI":"10.1137\/1.9780898719222"},{"key":"377_CR47","doi-asserted-by":"publisher","first-page":"191","DOI":"10.4064\/ap-31-2-191-195","volume":"31","author":"C Ursescu","year":"1975","unstructured":"Ursescu C (1975) A differentiable dependence on the right-hand side of solutions of ordinary differential equations. Ann Pol Math 31:191\u2013195","journal-title":"Ann Pol Math"},{"issue":"2","key":"377_CR48","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1007\/s10957-014-0539-1","volume":"163","author":"KA Khan","year":"2014","unstructured":"Khan KA, Barton PI (2014) Generalized derivatives for solutions of parametric ordinary differential equations with non-differentiable right-hand sides. J Optim Theory Appl 163(2):355\u2013386. https:\/\/doi.org\/10.1007\/s10957-014-0539-1","journal-title":"J Optim Theory Appl"},{"key":"377_CR49","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-42950-8","volume-title":"Nonlinear conjugate gradient methods for unconstrained optimization","author":"N Andrei","year":"2020","unstructured":"Andrei N (2020) Nonlinear conjugate gradient methods for unconstrained optimization. Springer, Cham. https:\/\/doi.org\/10.1007\/978-3-030-42950-8"},{"key":"377_CR50","doi-asserted-by":"publisher","unstructured":"Alifanov OM (1994) Inverse heat transfer problems. Springer, Berlin, Heidelberg. https:\/\/doi.org\/10.1007\/978-3-642-76436-3","DOI":"10.1007\/978-3-642-76436-3"},{"issue":"3","key":"377_CR51","doi-asserted-by":"publisher","first-page":"677","DOI":"10.1093\/imanum\/drn066","volume":"30","author":"B Jin","year":"2010","unstructured":"Jin B, Zou J (2010) Numerical estimation of the Robin coefficient in a stationary diffusion equation. IMA J Numer Anal 30(3):677\u2013701. https:\/\/doi.org\/10.1093\/imanum\/drn066","journal-title":"IMA J Numer Anal"},{"key":"377_CR52","doi-asserted-by":"publisher","unstructured":"Neuberger JW (2010) Sobolev gradients and differential equations, 2nd edn. Springer, Berlin, Heidelberg. https:\/\/doi.org\/10.1007\/978-3-642-04041-2","DOI":"10.1007\/978-3-642-04041-2"},{"key":"377_CR53","doi-asserted-by":"publisher","unstructured":"Esteve C, Geshkovski B, Pighin D, Zuazua E (2020) Large-time asymptotics in deep learning. arXiv:2008.02491. https:\/\/doi.org\/10.48550\/arXiv.2008.02491","DOI":"10.48550\/arXiv.2008.02491"},{"key":"377_CR54","doi-asserted-by":"publisher","unstructured":"Schuster T, Kaltenbacher B, Hofmann B, Kazimierski KS (2012) Regularization methods in Banach spaces. Radon Series on computational and applied mathematics, vol 10, p 283. Walter de Gruyter GmbH & Co. KG, Berlin. https:\/\/doi.org\/10.1515\/9783110255720","DOI":"10.1515\/9783110255720"},{"issue":"3","key":"377_CR55","doi-asserted-by":"publisher","first-page":"331","DOI":"10.3390\/math8030331","volume":"8","author":"B Hofmann","year":"2020","unstructured":"Hofmann B, Hofmann C (2020) The impact of the discrepancy principle on the Tikhonov-regularized solutions with oversmoothing penalties. Mathematics 8(3):331. https:\/\/doi.org\/10.3390\/math8030331","journal-title":"Mathematics"},{"key":"377_CR56","doi-asserted-by":"publisher","unstructured":"Tabuada P, Gharesifard B (2020) Universal approximation power of deep neural networks via nonlinear control theory. arXiv:2007.06007. https:\/\/doi.org\/10.48550\/arXiv.2007.06007","DOI":"10.48550\/arXiv.2007.06007"},{"key":"377_CR57","doi-asserted-by":"publisher","unstructured":"Teshima T, Tojo K, Ikeda M, Ishikawa I, Oono K (2020) Universal approximation property of neural ordinary differential equations. arXiv:2012.02414. https:\/\/doi.org\/10.48550\/arXiv.2012.02414","DOI":"10.48550\/arXiv.2012.02414"},{"key":"377_CR58","doi-asserted-by":"publisher","unstructured":"Li Q, Lin T, Shen Z (2019) Deep learning via dynamical systems: an approximation perspective. arXiv:1912.10382. https:\/\/doi.org\/10.4171\/JEMS\/1221","DOI":"10.4171\/JEMS\/1221"},{"issue":"03","key":"377_CR59","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1142\/S0219530520400023","volume":"19","author":"B Avelin","year":"2021","unstructured":"Avelin B, Nystr\u00f6m K (2021) Neural ODEs as the deep limit of ResNets with constant weights. Anal Appl 19(03):397\u2013437. https:\/\/doi.org\/10.1142\/S0219530520400023","journal-title":"Anal Appl"},{"key":"377_CR60","unstructured":"LeCun Y, Cortes C, Burges CJC (1998) The MNIST database of handwritten digits. http:\/\/yann.lecun.com\/exdb\/mnist\/"},{"key":"377_CR61","unstructured":"Hinton G, Srivastava N, Swersky K (2012) Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. https:\/\/www.cs.toronto.edu\/~tijmen\/csc321\/slides\/lecture_slides_lec6.pdf"},{"key":"377_CR62","doi-asserted-by":"publisher","unstructured":"Carlini N, Wagner D (2017) Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP). IEEE, pp 39\u201357. https:\/\/doi.org\/10.1109\/SP.2017.49","DOI":"10.1109\/SP.2017.49"},{"key":"377_CR63","unstructured":"Alberti G, De\u00a0Vito E, Lassas M, Ratti L, Santacesaria M (2021) Learning the optimal Tikhonov regularizer for inverse problems. Adv Neural Inf Process Syst 34"},{"key":"377_CR64","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1016\/j.cam.2018.12.044","volume":"354","author":"IM Ross","year":"2019","unstructured":"Ross IM (2019) An optimal control theory for nonlinear optimization. J Comput Appl Math 354:39\u201351. https:\/\/doi.org\/10.1016\/j.cam.2018.12.044","journal-title":"J Comput Appl Math"}],"container-title":["Operations Research Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s43069-024-00377-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s43069-024-00377-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s43069-024-00377-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,20]],"date-time":"2024-12-20T16:12:52Z","timestamp":1734711172000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s43069-024-00377-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,2]]},"references-count":64,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["377"],"URL":"https:\/\/doi.org\/10.1007\/s43069-024-00377-x","relation":{},"ISSN":["2662-2556"],"issn-type":[{"type":"electronic","value":"2662-2556"}],"subject":[],"published":{"date-parts":[[2024,10,2]]},"assertion":[{"value":"15 March 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 September 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 October 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}],"article-number":"91"}}