{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T14:34:02Z","timestamp":1787322842828,"version":"build-2736575974"},"reference-count":75,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"1","funder":[{"name":"NORCE Kompetanseoppbygging program"},{"name":"CRF","award":["C1013-21GF"],"award-info":[{"award-number":["C1013-21GF"]}]},{"DOI":"10.13039\/100007567","name":"City University of Hong Kong","doi-asserted-by":"publisher","award":["CityU1101120"],"award-info":[{"award-number":["CityU1101120"]}],"id":[{"id":"10.13039\/100007567","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007567","name":"City University of Hong Kong","doi-asserted-by":"publisher","award":["CityU11309922"],"award-info":[{"award-number":["CityU11309922"]}],"id":[{"id":"10.13039\/100007567","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001747","name":"Hong Kong Baptist University","doi-asserted-by":"publisher","award":["179356"],"award-info":[{"award-number":["179356"]}],"id":[{"id":"10.13039\/501100001747","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["N-HKBU214-19"],"award-info":[{"award-number":["N-HKBU214-19"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["12201530"],"award-info":[{"award-number":["12201530"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"HKRGC ECS","award":["22302123"],"award-info":[{"award-number":["22302123"]}]},{"name":"HKRGC CRF","award":["C1013-21GF"],"award-info":[{"award-number":["C1013-21GF"]}]},{"name":"HKRGC-NSFC","award":["N-CityU214-19"],"award-info":[{"award-number":["N-CityU214-19"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Imaging Sci."],"published-print":{"date-parts":[[2024,3,31]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>For problems in image processing and many other fields, a large class of effective neural networks has encoder-decoder-based architectures. Although these networks have shown impressive performance, mathematical explanations of their architectures are still underdeveloped. In this paper, we study the encoder-decoder-based network architecture from the algorithmic perspective and provide a mathematical explanation. We use the two-phase Potts model for image segmentation as an example for our explanations. We associate the segmentation problem with a control problem in the continuous setting. Then, the continuous control model is time discretized by an operator-splitting scheme, the PottsMGNet, and space discretized by the multigrid method. We show that the resulting discrete PottsMGNet is equivalent to an encoder-decoder-based network. With minor modifications, it is shown that a number of the popular encoder-decoder-based neural networks are just instances of the proposed PottsMGNet. By incorporating the soft-threshold-dynamics into the PottsMGNet as a regularizer, the PottsMGNet has shown to be robust with the network parameters such as network width and depth and has achieved remarkable performance on datasets with very large noise. In nearly all our experiments, the new network always performs better than or as well as on accuracy and dice score compared to existing networks for image segmentation.<\/jats:p>","DOI":"10.1137\/23m1586355","type":"journal-article","created":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T08:49:19Z","timestamp":1709801359000},"page":"540-594","source":"Crossref","is-referenced-by-count":21,"title":["PottsMGNet: A Mathematical Explanation of Encoder-Decoder Based Neural Networks"],"prefix":"10.1137","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3359-9104","authenticated-orcid":true,"given":"Xue-Cheng","family":"Tai","sequence":"first","affiliation":[{"name":"NORCE Norwegian Research Centre, Nyg\u00e5rdstangen, NO-5838 Bergen, Norway."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7504-9859","authenticated-orcid":true,"given":"Hao","family":"Liu","sequence":"additional","affiliation":[{"name":"Corresponding author. Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0910-4685","authenticated-orcid":true,"given":"Raymond","family":"Chan","sequence":"additional","affiliation":[{"name":"Department of Mathematics, City University of Hong Kong, Hong Kong; and Hong Kong Centre for Cerebro-Cardiovascular Health Engineering."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2024,3,7]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1017\/S0956792598003453"},{"key":"ref2","doi-asserted-by":"crossref","unstructured":"S. Anwar  and \nN. Barnes  , Real image denoising with feature attention, in Proceedings of the IEEE\/CVF International Conference on Computer Vision, 2019, pp. 3155\u20133164.","DOI":"10.1109\/ICCV.2019.00325"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2644615"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-010-0406-y"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.3934\/jcd.2019009"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-41589-5_21"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-006-7934-5"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2004.60"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1109\/34.969114"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1017\/S0956792521000139"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1017\/S0962492900002427"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1109\/83.902291"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2699184"},{"key":"ref15","doi-asserted-by":"crossref","unstructured":"L.C. Chen , \nY. Zhu , \nG. Papandreou , \nF. Schroff , and \nH. Adam  , Encoder-decoder with atrous separable convolution for semantic image segmentation, in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 801\u2013818.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref16","unstructured":"C.W. Cheng , \nC. Runkel , \nL. Liu , \nR. H. Chan , \nC.B. Sch\u00f6nlieb , and \nA. I. Aviles-Rivero  , Continuous U-Net: Faster, Greater and Noiseless, http:\/\/arxiv.org\/abs\/2302.00626, 2023."},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2014.2345401"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1137\/18M1226361"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-022-01910-y"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1007\/s40304-017-0103-z"},{"key":"ref21","doi-asserted-by":"crossref","unstructured":"W. E , \nC. Ma , and \nL. Wu  , Machine learning from a continuous viewpoint, I, Sci. China Math., 63 (2020), pp. 2233\u20132266, https:\/\/doi.org\/10.1007\/s11425-020-1773-8.","DOI":"10.1007\/s11425-020-1773-8"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1007\/BF01581204"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1002\/cpa.21527"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1016\/S0168-2024(08)70028-6"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(89)90003-8"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.1984.4767596"},{"key":"ref27","first-page":"3","volume":"9","author":"Glowinski R.","year":"2003","journal-title":"Handb. Numer. Anal."},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611970838"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.1137\/140992072"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-018-0839-y"},{"key":"ref31","doi-asserted-by":"crossref","unstructured":"R. Glowinski , \nS. Luo , and \nX.C. Tai  , Fast operator-splitting algorithms for variational imaging models: Some recent developments, in Processing, Analyzing and Learning of Images, Shapes, and Forms: Part 2, Handb. Numer. Anal. 20, Elsevier, New York, 2019, pp. 191\u2013232.","DOI":"10.1016\/bs.hna.2019.08.002"},{"key":"ref32","volume-title":"Splitting Methods in Communication, Imaging, Science, and Engineering","author":"Glowinski R.","year":"2017"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-41589-5_2"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6420\/aa9a90"},{"key":"ref35","doi-asserted-by":"crossref","unstructured":"E. Haber , \nL. Ruthotto , \nE. Holtham , and \nS. H. Jun  , Learning across scales\u2014Multiscale methods for convolution neural networks, in Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018, pp. 3142\u20133148, https:\/\/doi.org\/10.1609\/aaai.v32i1.11680.","DOI":"10.1609\/aaai.v32i1.11680"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.1007\/s11425-019-9547-2"},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.1137\/20M1314525"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(89)90020-8"},{"key":"ref39","unstructured":"P. Iakubovskii  , Segmentation Models PyTorch, https:\/\/github.com\/qubvel\/segmentation_models.pytorch, 2019."},{"key":"ref40","doi-asserted-by":"publisher","DOI":"10.1201\/9780203904190.ch7"},{"key":"ref41","unstructured":"Kaggle  , 2018 Data Science Bowl Dataset, https:\/\/www.kaggle.com\/competitions\/data-science-bowl-2018\/overview, 2018."},{"key":"ref42","doi-asserted-by":"crossref","unstructured":"Y. Lan , \nZ. Li , \nJ. Sun , and \nY. Xiang  , DOSnet as a Non-Black-Box PDE Solver: When Deep Learning Meets Operator Splitting, preprint, arXiv:2212.05571, 2022.","DOI":"10.1016\/j.jcp.2023.112343"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-019-01080-4"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.1137\/21M143772X"},{"key":"ref45","doi-asserted-by":"publisher","DOI":"10.1137\/20M1354532"},{"key":"ref46","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2023.111941"},{"key":"ref47","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2011.02.022"},{"key":"ref48","doi-asserted-by":"publisher","DOI":"10.1007\/s10851-022-01087-x"},{"key":"ref49","doi-asserted-by":"crossref","unstructured":"J. Long , \nE. Shelhamer , and \nT. Darrell  , Fully convolutional networks for semantic segmentation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3431\u20133440.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref50","doi-asserted-by":"publisher","DOI":"10.1051\/m2an\/1992260606731"},{"key":"ref51","first-page":"197","volume":"1","author":"Marchuk G. I.","year":"1990","journal-title":"Handb. Numer. Anal."},{"key":"ref52","volume-title":"Diffusion Generated Motion by Mean Curvature","author":"Merriman B.","year":"1992"},{"key":"ref53","first-page":"125","volume-title":"Ann. Fac. Sci. Toulouse Math.","volume":"16","author":"Miranda M.","year":"2007"},{"key":"ref54","doi-asserted-by":"publisher","DOI":"10.1002\/fld.5122"},{"key":"ref55","doi-asserted-by":"crossref","unstructured":"T. Pock , \nA. Chambolle , \nD. Cremers , and \nH. Bischof  , A convex relaxation approach for computing minimal partitions, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2009, pp. 810\u2013817.","DOI":"10.1109\/CVPR.2009.5206604"},{"key":"ref56","doi-asserted-by":"crossref","unstructured":"R. B. Potts  , Some Generalized Order-Disorder Transformations, Math. Proc. Cambridge Philos. Soc. 48, Cambridge University Press, Cambridge, 1952, pp. 106\u2013109.","DOI":"10.1017\/S0305004100027419"},{"key":"ref57","volume-title":"Numerical Recipes 3rd Edition: The Art of Scientific Computing","author":"Press W. H.","year":"2007"},{"key":"ref58","doi-asserted-by":"publisher","DOI":"10.1007\/s00211-020-01158-4"},{"key":"ref59","doi-asserted-by":"crossref","unstructured":"O. Ronneberger , \nP. Fischer , and \nT. Brox  , U-Net: Convolutional networks for biomedical image segmentation, in Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, New York, 2015, pp. 234\u2013241.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref60","doi-asserted-by":"publisher","DOI":"10.1137\/21M1411433"},{"key":"ref61","doi-asserted-by":"publisher","DOI":"10.1007\/s10851-019-00903-1"},{"key":"ref62","doi-asserted-by":"publisher","DOI":"10.1137\/20M1343956"},{"key":"ref63","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00409"},{"key":"ref64","first-page":"1","volume-title":"in Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging","author":"Tai X.","year":"2021"},{"key":"ref65","doi-asserted-by":"publisher","DOI":"10.1007\/s002110200404"},{"key":"ref66","unstructured":"X.C. Tai  and \nJ. Xu  , Subspace correction methods for convex optimization problems, in Proceedings of the 11th International Conference on Domain Decomposition Methods, London, 1998, pp. 130\u2013139."},{"key":"ref67","doi-asserted-by":"publisher","DOI":"10.1090\/S0025-5718-01-01311-4"},{"key":"ref68","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-021-01457-4"},{"key":"ref69","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2009.2021087"},{"key":"ref70","doi-asserted-by":"publisher","DOI":"10.4310\/AMSA.2018.v3.n1.a8"},{"key":"ref71","doi-asserted-by":"publisher","DOI":"10.1137\/1034116"},{"key":"ref72","doi-asserted-by":"crossref","unstructured":"J. Yuan , \nE. Bae , and \nX.C. Tai  , A study on continuous max-flow and min-cut approaches, in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, 2010, pp. 2217\u20132224.","DOI":"10.1109\/CVPR.2010.5539903"},{"key":"ref73","doi-asserted-by":"crossref","unstructured":"J. Yuan , \nE. Bae , \nX.C. Tai , and \nY. Boykov  , A continuous max-flow approach to Potts model, in Proceedings of the European Conference on Computer Vision, Springer, New York, 2010, pp. 379\u2013392.","DOI":"10.1007\/978-3-642-15567-3_28"},{"key":"ref74","doi-asserted-by":"crossref","unstructured":"K. Zhang , \nW. Zuo , \nY. Chen , \nD. Meng , and \nL. Zhang  , Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017, pp. 5743\u20135752.","DOI":"10.1109\/TIP.2017.2662206"},{"key":"ref75","doi-asserted-by":"crossref","unstructured":"D. X. Zhou  , Universality of deep convolutional neural networks, Appl. Comput. Harmon. Anal., 48 (2020), pp. 787\u2013794, https:\/\/doi.org\/10.1016\/j.acha.2019.06.004.","DOI":"10.1016\/j.acha.2019.06.004"},{"key":"ref76","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2019.2959609"}],"container-title":["SIAM Journal on Imaging Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/23M1586355","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T13:28:55Z","timestamp":1787318935000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/23M1586355"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,7]]},"references-count":75,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3,31]]}},"alternative-id":["10.1137\/23M1586355"],"URL":"https:\/\/doi.org\/10.1137\/23m1586355","relation":{},"ISSN":["1936-4954"],"issn-type":[{"value":"1936-4954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,7]]}}}