{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T19:20:27Z","timestamp":1790882427086,"version":"4.1.0"},"reference-count":41,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"4","funder":[{"name":"Singapore Ministry of Education Academic Research Fund Tier 1","award":["T1 251RES2207"],"award-info":[{"award-number":["T1 251RES2207"]}]},{"DOI":"10.13039\/501100001381","name":"National Research Foundation Singapore","doi-asserted-by":"crossref","award":["AISG2-TC-2023-010-SGIL"],"award-info":[{"award-number":["AISG2-TC-2023-010-SGIL"]}],"id":[{"id":"10.13039\/501100001381","id-type":"DOI","asserted-by":"crossref"}]},{"name":"DOE SEA-CROGS","award":["DE-SC0023191"],"award-info":[{"award-number":["DE-SC0023191"]}]},{"name":"ONR Vannevar Bush Faculty Fellowship","award":["N00014-22-1-2795"],"award-info":[{"award-number":["N00014-22-1-2795"]}]},{"name":"MURI-AFOSR","award":["FA9550-20-1-0358"],"award-info":[{"award-number":["FA9550-20-1-0358"]}]},{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"publisher","award":["FA2386-24-1-4011"],"award-info":[{"award-number":["FA2386-24-1-4011"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Sci. Comput."],"published-print":{"date-parts":[[2025,8,31]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>Physics-informed neural networks (PINNs) have triggered a paradigm shift in scientific computing, leveraging mesh-free properties and robust approximation capabilities. While proving effective for low-dimensional partial differential equations (PDEs), the computational cost of PINNs remains a hurdle in high-dimensional scenarios. This is particularly pronounced when computing high-order and high-dimensional derivatives in the physics-informed loss. Randomized smoothing PINN (RS-PINN) introduces Gaussian noise for stochastic smoothing of the original neural net model, enabling the use of Monte Carlo methods for derivative approximation, which eliminates the need for costly automatic differentiation. Despite its computational efficiency, especially in the approximation of high-dimensional derivatives, RS-PINN introduces biases in both loss and gradients, negatively impacting convergence, especially when coupled with stochastic gradient descent algorithms. We present a comprehensive analysis of biases in RS-PINN, attributing them to the nonlinearity of the mean squared error loss as well as the intrinsic nonlinearity of the PDE itself. We propose tailored bias correction techniques, delineating their application based on the order of PDE nonlinearity. The derivation of an unbiased RS-PINN allows for a detailed examination of its advantages and disadvantages compared to the biased version. Specifically, the biased version has a lower variance and runs faster than the unbiased version, but it is less accurate due to the bias. To optimize the bias-variance trade-off, we combine the two approaches in a hybrid method that balances the rapid convergence of the biased version with the high accuracy of the unbiased version. In addition to methodological contributions, we present an enhanced implementation of RS-PINN. Extensive experiments on diverse high-dimensional PDEs, including Fokker\u2013Planck, Hamilton\u2013Jacobi\u2013Bellman, viscous Burgers\u2019, Allen\u2013Cahn, and Sine\u2013Gordon equations, illustrate the bias-variance trade-off and highlight the effectiveness of the hybrid RS-PINN. Empirical guidelines are provided for selecting biased, unbiased, or hybrid versions, depending on the dimensionality and nonlinearity of the specific PDE problem. The code is available at https:\/\/github.com\/zheyuanhu01\/RS_PINN_Bias_Variance .<\/jats:p>\n                  <jats:p>Reproducibility of computational results. This paper has been awarded the \u201cSIAM Reproducibility Badge: Code and data available\u201d as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https:\/\/github.com\/zheyuanhu01\/RS_PINN_Bias_Variance and in the supplementary materials ( M162135_Supplement.pdf [719KB], RS_PINN_Bias_Variance-main.zip [23.7KB]). [Formula: see text]<\/jats:p>","DOI":"10.1137\/23m1621356","type":"journal-article","created":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T08:04:18Z","timestamp":1752480258000},"page":"C846-C872","source":"Crossref","is-referenced-by-count":10,"title":["Bias-Variance Trade-Off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs"],"prefix":"10.1137","volume":"47","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0116-9874","authenticated-orcid":true,"given":"Zheyuan","family":"Hu","sequence":"first","affiliation":[{"name":"Department of Computer Science, National University of Singapore, Singapore, 119077. The first three authors contributed equally."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhouhao","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, National University of Singapore, Singapore, 119077. The first three authors contributed equally."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yezhen","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, National University of Singapore, Singapore, 119077. The first three authors contributed equally."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9713-7120","authenticated-orcid":true,"given":"George E.","family":"Karniadakis","sequence":"additional","affiliation":[{"name":"Division of Applied Mathematics, Brown University, Providence, RI 02912 USA, and Advanced Computing, Mathematics and Data Division, Pacific Northwest National Laboratory, Richland, WA 99352 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kenji","family":"Kawaguchi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, National University of Singapore, Singapore, 119077. The first three authors contributed equally."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2025,7,14]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1137\/19M1297919"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1007\/s00332-018-9525-3"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1007\/s42985-024-00272-4"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1515\/jnma-2019-0074"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.4208\/cicp.OA-2020-0130"},{"key":"ref6","unstructured":"J. Bettencourt, M. J. Johnson, and D. Duvenaud, Taylor-mode automatic differentiation for higher-order derivatives in JAX, in Proceedings of Program Transformations for ML Workshop at NeurIPS, 2019, https:\/\/openreview.net\/forum?id=SkxEF3FNPH."},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-019-00908-3"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2022.114909"},{"key":"ref9","doi-asserted-by":"crossref","first-page":"23761","DOI":"10.52202\/075280-1032","volume":"36","author":"Cho J.","year":"2023","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"ref10","unstructured":"J. Cohen, E. Rosenfeld, and Z. Kolter, Certified adversarial robustness via randomized smoothing, in Proceedings of the 36th International Conference on Machine Learning, Proc. Mach. Learn. Res. 97, K. Chaudhuri and R. Salakhutdinov, 2019, pp. 1310\u20131320, https:\/\/proceedings.mlr.press\/v97\/cohen19c.html."},{"key":"ref11","doi-asserted-by":"crossref","unstructured":"H. Feng, T. Pang, C. Du, W. Chen, S. Yan, and M. Lin, BAFFLE: A baseline of backpropagation-free federated learning, in European Conference on Computer Vision, Springer, 2024, pp. 89\u2013109.","DOI":"10.1007\/978-3-031-73226-3_6"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1718942115"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1007\/s40304-017-0117-6"},{"key":"ref14","unstructured":"D. He, S. Li, W. Shi, X. Gao, J. Zhang, J. Bian, L. Wang, and T.Y. Liu, Learning physics-informed neural networks without stacked back-propagation, in Proceedings of the International Conference on Artificial Intelligence and Statistics, 2023, pp. 3034\u20133047."},{"key":"ref15","doi-asserted-by":"crossref","unstructured":"P. Henry-Labordere, Deep Primal-Dual Algorithm for BSDEs: Applications of Machine Learning to CVA and IM, https:\/\/dx.doi.org\/10.2139\/ssrn.3071506, 2017.","DOI":"10.2139\/ssrn.3071506"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.1137\/21M1447039"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2024.106369"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1137\/24M1638768"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1090\/mcom\/3514"},{"key":"ref20","first-page":"20190630","volume":"476","author":"Hutzenthaler M.","year":"2020","journal-title":"Proc. A"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.1007\/s42985-021-00089-5"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2019.109136"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2020.2971597"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1038\/s42254-021-00314-5"},{"key":"ref25","unstructured":"K. Kawaguchi, Deep learning without poor local minima, in Proceedings of Advances in Neural Information Processing Systems (NeurIPS), 2016, pp. 586\u2013594."},{"key":"ref26","unstructured":"K. Kawaguchi, Z. Deng, X. Ji, and J. Huang, How does information bottleneck help deep learning? in Proceedings of the International Conference on Machine Learning (ICML), 2023."},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1017\/9781009025096.003"},{"key":"ref28","unstructured":"D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, in Proceedings of the International Conference on Learning Representations, 2015."},{"key":"ref29","doi-asserted-by":"crossref","unstructured":"M. Lecuyer, V. Atlidakis, R. Geambasu, D. Hsu, and S. Jana, Certified robustness to adversarial examples with differential privacy, in Proceedings of the IEEE Symposium on Security and Privacy, IEEE, 2019, pp. 656\u2013672.","DOI":"10.1109\/SP.2019.00044"},{"key":"ref30","unstructured":"Y. Liao, Z. Lin, J. Liu, Q. Sun, Y. Wang, T. Wu, H. Xie, and M. He, Solving Schr\u00f6dinger Equation Using Tensor Neural Network, preprint, https:\/\/arxiv.org\/abs\/2209.12572, 2022."},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1137\/21M1397908"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.1142\/S0129183123500821"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1093\/imanum\/drab093"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.1007\/s42985-021-00102-x"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1137\/18M1229845"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.1142\/9789811280306_0018"},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2018.10.045"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.4208\/cicp.OA-2020-0193"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2018.08.029"},{"key":"ref40","doi-asserted-by":"crossref","unstructured":"C. Wang, S. Li, D. He, and L. Wang, Is L\n                      2\n                      physics informed loss always suitable for training physics informed neural network? in Proceedings of Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, eds. 2022, https:\/\/openreview.net\/forum?id=cy1TKLRAEML.","DOI":"10.52202\/068431-0602"},{"key":"ref41","doi-asserted-by":"publisher","DOI":"10.4208\/jcm.2307-m2022-0233"}],"container-title":["SIAM Journal on Scientific Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/23M1621356","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T16:19:28Z","timestamp":1787329168000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/23M1621356"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,14]]},"references-count":41,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,8,31]]}},"alternative-id":["10.1137\/23M1621356"],"URL":"https:\/\/doi.org\/10.1137\/23m1621356","relation":{},"ISSN":["1064-8275","1095-7197"],"issn-type":[{"value":"1064-8275","type":"print"},{"value":"1095-7197","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,14]]}}}