{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T06:43:54Z","timestamp":1791528234804,"version":"4.3.4"},"reference-count":79,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"5","funder":[{"DOI":"10.13039\/501100001711","name":"Swiss National Science Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Platform for Advanced Scientific Computing"},{"DOI":"10.13039\/501100001711","name":"Swiss National Science Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Vannevar Bush Faculty Fellowship"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Sci. Comput."],"published-print":{"date-parts":[[2024,10,31]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>We propose to enhance the training of physics-informed neural networks. To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain decomposition framework, where the parameters of the network are decomposed in a layerwise manner. Through a series of numerical experiments, we demonstrate that both additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS optimizer while providing more accurate solutions of the underlying PDEs. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism.<\/jats:p>","DOI":"10.1137\/23m1583375","type":"journal-article","created":{"date-parts":[[2024,4,11]],"date-time":"2024-04-11T11:19:42Z","timestamp":1712834382000},"page":"S46-S67","source":"Crossref","is-referenced-by-count":31,"title":["Enhancing Training of Physics-Informed Neural Networks Using Domain Decomposition\u2013Based Preconditioning Strategies"],"prefix":"10.1137","volume":"46","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8388-5518","authenticated-orcid":true,"given":"Alena","family":"Kopani\u010d\u00e1kov\u00e1","sequence":"first","affiliation":[{"name":"Euler Institute, Universit\u00e0 della Svizzera italiana, Lugano, Switzerland, and Division of Applied Mathematics, Brown University, Providence, RI 02906 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0110-5384","authenticated-orcid":true,"given":"Hardik","family":"Kothari","sequence":"additional","affiliation":[{"name":"Euler Institute, Universit\u00e0 della Svizzera italiana, Lugano, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9713-7120","authenticated-orcid":true,"given":"George E.","family":"Karniadakis","sequence":"additional","affiliation":[{"name":"Division of Applied Mathematics, Brown University, Providence, RI 02906 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rolf","family":"Krause","sequence":"additional","affiliation":[{"name":"Euler Institute, Universit\u00e0 della Svizzera italiana, Lugano, Switzerland, and UniDistance, Brig-Glis, Switzerland."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2024,4,11]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1145\/3320060"},{"key":"ref2","unstructured":"A. Botev, H. Ritter, and D. Barber, Practical Gauss-Newton optimisation for deep learning, in International Conference on Machine Learning, PMLR, 2017, pp. 557\u2013565, https:\/\/arxiv.org\/abs\/1706.03662."},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1137\/130936725"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1007\/BF01582063"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1137\/S106482750037620X"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1016\/j.apor.2020.102339"},{"key":"ref7","first-page":"512","volume-title":"Mathematical and Scientific Machine Learning","author":"Cyr E. C.","year":"2020"},{"key":"ref8","first-page":"1223","volume-title":"Advances in Neural Information Processing Systems","author":"Dean J.","year":"2012"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611971200"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23899-4"},{"key":"ref11","doi-asserted-by":"crossref","unstructured":"V. Dolean, A. Heinlein, S. Mishra, and B. Moseley, Finite basis physics-informed neural networks as a Schwarz domain decomposition method, in International Conference on Domain Decomposition Methods, Springer, 2022, pp. 165\u2013172.","DOI":"10.1007\/978-3-031-50769-4_19"},{"key":"ref12","doi-asserted-by":"crossref","unstructured":"V. Dolean, A. Heinlein, S. Mishra, and B. Moseley, Multilevel Domain Decomposition-Based Architectures for Physics-Informed Neural Networks, preprint, arXiv:2306.05486, 2023.","DOI":"10.1016\/j.cma.2024.117116"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.09.006"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1090\/noti2259"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1007\/s40304-018-0127-z"},{"key":"ref16","unstructured":"Z. Fang, S. Wang, and P. Perdikaris, Ensemble Learning for Physics Informed Neural Networks:\u00a0A Gradient Boosting Approach, preprint, arXiv:2302.13143, 2023."},{"key":"ref17","doi-asserted-by":"crossref","unstructured":"L. Gaedke-Merzh\u00e4user, A. Kopani\u010d\u00e1kov\u00e1, and R. Krause, Multilevel minimization for deep residual networks, in Proceedings of French-German-Swiss Optimization Conference (FGS\u20192019), 2021, https:\/\/doi.org\/10.1051\/proc\/202171131.","DOI":"10.1051\/proc\/202171131"},{"key":"ref18","unstructured":"H. P. Gavin, The Levenberg-Marquardt Algorithm for Nonlinear Least Squares Curve-Fitting Problems, Duke University, 2019, pp. 1\u201319."},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1016\/0893-9659(93)90032-I"},{"key":"ref20","unstructured":"X. Glorot and Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, 2010, pp. 249\u2013256."},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.1088\/0266-5611\/19\/2\/201"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1137\/23M1553455"},{"key":"ref23","doi-asserted-by":"crossref","unstructured":"S. Gratton, V. Mercier, E. Riccietti, and P. L. Toint, A Block-Coordinate Approach of Multi-Level Optimization with an Application to Physics-Informed Neural Networks, preprint, arXiv:2305.14477, 2023.","DOI":"10.1007\/s10589-024-00597-1"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1553\/etna_vol56s157"},{"key":"ref25","first-page":"153","volume-title":"Domain Decomposition Methods in Science and Engineering XXVI","author":"Gu L.","year":"2023"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1137\/19M1247620"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1016\/j.apm.2014.11.025"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2023.107183"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.1137\/21M1447039"},{"key":"ref30","doi-asserted-by":"crossref","unstructured":"A. D. Jagtap and G. E. Karniadakis, Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations, in AAAI Spring Symposium: MLPS, 2021, pp. 2002\u20132041.","DOI":"10.4208\/cicp.OA-2020-0164"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1098\/rspa.2020.0334"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2019.109136"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2020.113028"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.1038\/s42254-021-00314-5"},{"key":"ref35","unstructured":"D. P. Kingma and J. Ba, Adam:\u00a0A Method for Stochastic Optimization, preprint, arXiv:1412.6980, 2014."},{"key":"ref36","first-page":"337","volume-title":"Domain Decomposition Methods in Science and Engineering XXVI","author":"Klawonn A.","year":"2023"},{"key":"ref37","unstructured":"A. Klawonn, M. Lanser, and J. Weber, A Domain Decomposition-Based CNN-DNN Architecture for Model Parallel Training Applied to Image Recognition Problems, preprint, arXiv:2302.06564, https:\/\/arxiv.org\/abs\/2302.06564, 2023."},{"key":"ref38","author":"Kopani\u010d\u00e1kov\u00e1 A.","year":"2022","journal-title":"Git repository"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2022.115733"},{"key":"ref40","doi-asserted-by":"publisher","DOI":"10.1137\/18M1194870"},{"key":"ref41","first-page":"S254","author":"Kopani\u010d\u00e1kov\u00e1 A.","year":"2022","journal-title":"SIAM J. Sci. Comput."},{"key":"ref42","doi-asserted-by":"crossref","unstructured":"H. Kothari, Nonlinear Schwarz preconditioning for Quasi-Newton methods, in International Conference on Domain Decomposition Methods, Springer, 2022, pp. 311\u2013318.","DOI":"10.1007\/978-3-031-50769-4_37"},{"key":"ref43","doi-asserted-by":"crossref","unstructured":"H. Kothari, A. Kopani\u010d\u00e1kov\u00e1, and R. Krause, Nonlinear Schwarz preconditioning for nonlinear optimization problems with bound constraints, in International Conference on Domain Decomposition Methods, Springer, 2022, pp. 319\u2013326.","DOI":"10.1007\/978-3-031-50769-4_38"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.1109\/72.712178"},{"key":"ref45","first-page":"180","volume-title":"CEUR Workshop Proceedings","author":"Lee K.","year":"2021"},{"key":"ref46","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2957200"},{"key":"ref47","first-page":"269","volume-title":"Mathematical and Scientific Machine Learning","author":"Li W.","year":"2020"},{"key":"ref48","doi-asserted-by":"publisher","DOI":"10.1007\/BF01589116"},{"key":"ref49","doi-asserted-by":"publisher","DOI":"10.1137\/140970379"},{"key":"ref50","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-021-00302-5"},{"key":"ref51","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2022.114778"},{"key":"ref52","doi-asserted-by":"publisher","DOI":"10.1137\/22M1507942"},{"key":"ref53","first-page":"566","volume":"6","author":"Lydia A.","year":"2019","journal-title":"Int. J. Inf. Comput. Sci."},{"key":"ref54","doi-asserted-by":"crossref","unstructured":"S. Mahboubi, S. Indrapriyadarsini, H. Ninomiya, and H. Asai, Momentum acceleration of quasi-Newton training for neural networks, in PRICAI 2019: Trends in Artificial Intelligence: 16th Pacific Rim International Conference on Artificial Intelligence, Cuvu, Yanuca Island, Fiji, August 26\u201330, 2019, Proceedings, Part II 16, Springer-Verlag, Berlin, 2019, pp. 268\u2013281, https:\/\/doi.org\/10.1007\/978-3-030-29911-8_21.","DOI":"10.1007\/978-3-030-29911-8_21"},{"key":"ref55","unstructured":"L. McClenny and U. Braga-Neto, Self-Adaptive Physics-Informed Neural Networks Using a Soft Attention Mechanism, preprint, arXiv:2009.04544, https:\/\/arxiv.org\/abs\/2009.04544, 2020."},{"key":"ref56","unstructured":"V. Mercier, S. Gratton, and P. Boudier, A Coarse Space Acceleration of Deep-DDM, preprint, arXiv:2112.03732, https:\/\/arxiv.org\/abs\/2112.03732, 2021."},{"key":"ref57","doi-asserted-by":"publisher","DOI":"10.1093\/imanum\/drab032"},{"key":"ref58","unstructured":"V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, Asynchronous methods for deep reinforcement learning, in International Conference on Machine Learning, 2016, pp. 1928\u20131937, https:\/\/arxiv.org\/abs\/1602.01783."},{"key":"ref59","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0067700"},{"key":"ref60","doi-asserted-by":"publisher","DOI":"10.1007\/s10444-023-10065-9"},{"key":"ref61","doi-asserted-by":"publisher","DOI":"10.1137\/20M1359511"},{"key":"ref62","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-012-9492-9"},{"key":"ref63","volume":"32","author":"Paszke A.","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref64","doi-asserted-by":"crossref","unstructured":"J. Rafati and R. F. Marcia, Improving L-BFGS initialization for trust-region methods in deep learning, in 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), IEEE, New York, 2018, pp. 501\u2013508, https:\/\/doi.org\/10.1109\/icmla.2018.00081.","DOI":"10.1109\/ICMLA.2018.00081"},{"key":"ref65","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2018.10.045"},{"key":"ref66","doi-asserted-by":"publisher","DOI":"10.1162\/08997660260028683"},{"key":"ref67","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2020.110085"},{"key":"ref68","doi-asserted-by":"publisher","DOI":"10.4208\/cicp.OA-2022-0114"},{"key":"ref69","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2021.110683"},{"key":"ref70","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2018.08.029"},{"key":"ref71","doi-asserted-by":"crossref","unstructured":"P. Stiller, F. Bethke, M. B\u00f6hme, R. Pausch, S. Torge, A. Debus, J. Vorberger, M. Bussmann, and N. Hoffmann, Large-scale neural solvers for partial differential equations, in Driving Scientific and Engineering Discoveries through the Convergence of HPC, Big Data and AI: 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Oak Ridge, TN, August 2628, 2020, Revised Selected Papers 17, Springer-Verlag, Berlin, 2020, pp. 20\u201334, https:\/\/doi.org\/10.1007\/978-3-030-63393-6_2.","DOI":"10.1007\/978-3-030-63393-6_2"},{"key":"ref72","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2021.114333"},{"key":"ref73","doi-asserted-by":"publisher","DOI":"10.2172\/2004363"},{"key":"ref74","first-page":"6000","volume":"30","author":"Vaswani A.","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref75","doi-asserted-by":"publisher","DOI":"10.1137\/20M1318043"},{"key":"ref76","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2021.110768"},{"key":"ref77","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-016-0043-6"},{"key":"ref78","doi-asserted-by":"publisher","DOI":"10.1201\/9781315218427-12"},{"key":"ref79","first-page":"807","volume-title":"Proceedings of the International Congress of Mathematics","author":"Yuan Y.-X.","year":"2014"}],"container-title":["SIAM Journal on Scientific Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/23M1583375","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T15:44:28Z","timestamp":1787327068000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/23M1583375"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,11]]},"references-count":79,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,10,31]]}},"alternative-id":["10.1137\/23M1583375"],"URL":"https:\/\/doi.org\/10.1137\/23m1583375","relation":{},"ISSN":["1064-8275","1095-7197"],"issn-type":[{"value":"1064-8275","type":"print"},{"value":"1095-7197","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,11]]}}}