{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T05:05:13Z","timestamp":1773810313036,"version":"3.50.1"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2022,4,25]],"date-time":"2022-04-25T00:00:00Z","timestamp":1650844800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,4,25]],"date-time":"2022-04-25T00:00:00Z","timestamp":1650844800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The theory-guided neural network (TgNN) is a kind of method which improves the effectiveness and efficiency of neural network architectures by incorporating scientific knowledge or physical information. Despite its great success, the theory-guided (deep) neural network possesses certain limits when maintaining a tradeoff between training data and domain knowledge during the training process. In this paper, the Lagrangian dual-based TgNN (TgNN-LD) is proposed to improve the effectiveness of the training process. We convert the original loss function into a constrained form with several items, in which partial differential equations (PDEs), engineering controls (ECs), and expert knowledge (EK) are regarded as constraints, with one Lagrangian variable per constraint. These Lagrangian variables are incorporated to achieve an equitable trade-off between observation data and corresponding constraints, to improve prediction accuracy and training efficiency. To investigate the performance of the proposed method, the original TgNN model with a set of optimized weight values adjusted by ad-hoc procedures is compared on a subsurface flow problem, with their L2 error, R square (R2), and computational time being analyzed. Experimental results demonstrate the superiority of the Lagrangian dual-based TgNN.<\/jats:p>","DOI":"10.1007\/s40747-022-00738-1","type":"journal-article","created":{"date-parts":[[2022,4,25]],"date-time":"2022-04-25T05:02:57Z","timestamp":1650862977000},"page":"4849-4862","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["A Lagrangian dual-based theory-guided deep neural network"],"prefix":"10.1007","volume":"8","author":[{"given":"Miao","family":"Rong","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6930-5994","authenticated-orcid":false,"given":"Dongxiao","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Nanzhe","family":"Wang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,4,25]]},"reference":[{"issue":"7587","key":"738_CR1","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","volume":"529","author":"D Silver","year":"2016","unstructured":"Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529(7587):484\u2013489","journal-title":"Nature"},{"key":"738_CR2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2019.2953131","author":"Y Chen","year":"2019","unstructured":"Chen Y, Sun X, Jin Y (2019) Communication-efficient federated deep learning with layerwise asynchronous model update and temporally weighted aggregation. IEEE Trans Neural Netw Learn Syst. https:\/\/doi.org\/10.1109\/TNNLS.2019.2953131","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"738_CR3","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-45778-5_1","author":"Y Chen","year":"2019","unstructured":"Chen Y, Zhang J, Yeo CK (2019) Network anomaly detection using federated deep autoencoding gaussian mixture model. Int Conf Mach Learn Netw. https:\/\/doi.org\/10.1007\/978-3-030-45778-5_1","journal-title":"Int Conf Mach Learn Netw"},{"key":"738_CR4","doi-asserted-by":"crossref","unstructured":"Tan M, Chen B, Pang R, Vasudevan V, Sandier M, Howard A, Le QV (2019) MnasNet: platform-aware neural architecture search for mobile. In: IEEE conference on computer vision and pattern recognition, pp 2815\u20132823","DOI":"10.1109\/CVPR.2019.00293"},{"key":"738_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.enbuild.2020.109831","author":"T Hong","year":"2020","unstructured":"Hong T, Wang Z, Luo X, Zhang W (2020) State-of-the-art on research and applications of machine learning in the building life cycle. Energy Build. https:\/\/doi.org\/10.1016\/j.enbuild.2020.109831","journal-title":"Energy Build"},{"issue":"10","key":"738_CR6","doi-asserted-by":"publisher","first-page":"2318","DOI":"10.1109\/TKDE.2017.2720168","volume":"29","author":"A Karpatne","year":"2017","unstructured":"Karpatne A, Atluri G, Faghmous JH, Steinbach M, Banerjee A, Ganguly A, Shekhar S, Samatova N, Kumar V (2017) Theory-guided data science: a new paradigm for scientific discovery from data. IEEE Trans Knowl Data Eng 29(10):2318\u20132331","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"738_CR7","unstructured":"Karpatne A, Watkins W, Read J, Kumar V (2017) Physics-guided neural networks (pgnn): an application in lake temperature modeling. arXiv:1710.11431"},{"key":"738_CR8","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1016\/j.jcp.2018.10.045","volume":"378","author":"M Raissi","year":"2019","unstructured":"Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686\u2013707","journal-title":"J Comput Phys"},{"key":"738_CR9","doi-asserted-by":"publisher","DOI":"10.1016\/j.jhydrol.2020.124700","author":"N Wang","year":"2020","unstructured":"Wang N, Zhang D, Chang H, Li H (2020) Deep learning of subsurface flow via theory-guided neural network. J Hydrol. https:\/\/doi.org\/10.1016\/j.jhydrol.2020.124700","journal-title":"J Hydrol"},{"key":"738_CR10","doi-asserted-by":"crossref","unstructured":"Zoph B, Vasudevan V, Shlens J, Le QV (2018) Learning transferable architectures for scalable image recognition. In: IEEE conference on computer vision and pattern recognition, pp 8697\u20138710","DOI":"10.1109\/CVPR.2018.00907"},{"key":"738_CR11","doi-asserted-by":"crossref","unstructured":"Chen Y, Sun X, Hu Y (2019) Federated learning assisted interactive eda with dual probabilistic models for personalized search. In: International conference on swarm intelligence, pp 374\u2013383","DOI":"10.1007\/978-3-030-26369-0_35"},{"key":"738_CR12","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1016\/j.ins.2020.01.048","volume":"519","author":"X Wang","year":"2020","unstructured":"Wang X, Jin Y, Schmitt S, Olhofer M (2020) An adaptive Bayesian approach to surrogate-assisted evolutionary multi-objective optimization. Inf Sci 519:317\u2013331","journal-title":"Inf Sci"},{"key":"738_CR13","doi-asserted-by":"publisher","first-page":"202","DOI":"10.1016\/j.asoc.2019.03.037","volume":"80","author":"HA Fayed","year":"2019","unstructured":"Fayed HA, Atiya AF (2019) Speed up grid-search for parameter selection of support vector machines. Appl Soft Comput 80:202\u2013210","journal-title":"Appl Soft Comput"},{"key":"738_CR14","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1016\/j.vibspec.2017.10.006","volume":"94","author":"H Chen","year":"2018","unstructured":"Chen H, Liu Z, Cai K, Xu L, Chen A (2018) Grid search parametric optimization for FT-NIR quantitative analysis of solid soluble content in strawberry samples. Vib Spectrosc 94:7-15","journal-title":"Vib Spectrosc"},{"issue":"2","key":"738_CR15","doi-asserted-by":"publisher","first-page":"721","DOI":"10.1007\/s42835-020-00343-7","volume":"15","author":"J-H Han","year":"2020","unstructured":"Han J-H, Choi D-J, Park S-U, Hong S-K (2020) Hyperparameter optimization using a genetic algorithm considering verification time in a convolutional neural network. J Electric Eng Technol 15(2):721\u2013726","journal-title":"J Electric Eng Technol"},{"issue":"SI","key":"738_CR16","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1016\/j.neucom.2018.05.136","volume":"354","author":"FJ Martinez-de Pison","year":"2019","unstructured":"Martinez-de Pison FJ, Gonzalez-Sendino R, Aldama A, Ferreiro-Cabello J, Fraile-Garcia E (2019) Hybrid methodology based on Bayesian optimization and GA-PARSIMONY to search for parsimony models by combining hyperparameter optimization and feature selection. Neurocomputing 354(SI):20\u201326","journal-title":"Neurocomputing"},{"key":"738_CR17","doi-asserted-by":"publisher","DOI":"10.1016\/j.geomorph.2020.107201","author":"D Sun","year":"2020","unstructured":"Sun D, Wen H, Wang D, Xu J (2020) A random forest model of landslide susceptibility mapping based on hyperparameter optimization using Bayes algorithm. Geomorphology. https:\/\/doi.org\/10.1016\/j.geomorph.2020.107201","journal-title":"Geomorphology"},{"key":"738_CR18","doi-asserted-by":"crossref","unstructured":"Yao Y, Cao J, Ma Z (2018) A cost-effective deadline-constrained scheduling strategy for a hyperparameter optimization workflow for machine learning algorithms. In: International conference on service-oriented computing, pp 870\u2013878","DOI":"10.1007\/978-3-030-03596-9_62"},{"key":"738_CR19","first-page":"281","volume":"13","author":"J Bergstra","year":"2012","unstructured":"Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13:281\u2013305","journal-title":"J Mach Learn Res"},{"key":"738_CR20","doi-asserted-by":"crossref","unstructured":"Fioretto F, Hentenryck PV, Mak TW, Tran C, Baldo F, Lombardi M (2020) Lagrangian duality for constrained deep learning. arXiv:2001.09394","DOI":"10.1007\/978-3-030-67670-4_8"},{"key":"738_CR21","doi-asserted-by":"crossref","unstructured":"Lombardi M, Baldo F, Borghesi A, Milano M (2020) An analysis of regularized approaches for constrained machine learning. arXiv:2005.10674","DOI":"10.1007\/978-3-030-73959-1_11"},{"key":"738_CR22","unstructured":"Borghesi A, Baldo F, Milano M (2020) Improving deep learning models via constraint-based domain knowledge: a brief survey. arXiv:2005.10691"},{"key":"738_CR23","doi-asserted-by":"publisher","DOI":"10.1016\/j.jaerosci.2019.105468","author":"J Gan","year":"2020","unstructured":"Gan J, Liu P, Chakrabarty RK (2020) Deep learning enabled Lagrangian particle trajectory simulation. J Aerosol Sci. https:\/\/doi.org\/10.1016\/j.jaerosci.2019.105468","journal-title":"J Aerosol Sci"},{"key":"738_CR24","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1016\/j.ebiom.2019.01.028","volume":"40","author":"BN Walker","year":"2019","unstructured":"Walker BN, Rehg JM, Kalra A, Winters RM, Drews P, Dascalu J, David EO, Dascalu A (2019) Dermoscopy diagnosis of cancerous lesions utilizing dual deep learning algorithms via visual and audio (sonification) outputs: laboratory and prospective observational studies. Ebiomedicine 40:176\u2013183","journal-title":"Ebiomedicine"},{"issue":"6","key":"738_CR25","doi-asserted-by":"publisher","first-page":"2419","DOI":"10.1007\/s10694-019-00872-2","volume":"55","author":"AS Pundir","year":"2019","unstructured":"Pundir AS, Raman B (2019) Dual deep learning model for image based smoke detection. Fire Technol 55(6):2419\u20132442","journal-title":"Fire Technol"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-022-00738-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-022-00738-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-022-00738-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,27]],"date-time":"2022-10-27T12:03:48Z","timestamp":1666872228000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-022-00738-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,25]]},"references-count":25,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["738"],"URL":"https:\/\/doi.org\/10.1007\/s40747-022-00738-1","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,25]]},"assertion":[{"value":"6 October 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 April 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 April 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}