{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T16:33:23Z","timestamp":1775666003034,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2023,8,10]],"date-time":"2023-08-10T00:00:00Z","timestamp":1691625600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council (ERC)","doi-asserted-by":"publisher","award":["101002198"],"award-info":[{"award-number":["101002198"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000781","name":"European Research Council (ERC)","doi-asserted-by":"publisher","award":["462569370"],"award-info":[{"award-number":["462569370"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)","doi-asserted-by":"publisher","award":["101002198"],"award-info":[{"award-number":["101002198"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)","doi-asserted-by":"publisher","award":["462569370"],"award-info":[{"award-number":["462569370"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Photoacoustic imaging potentially allows for the real-time visualization of functional human tissue parameters such as oxygenation but is subject to a challenging underlying quantification problem. While in silico studies have revealed the great potential of deep learning (DL) methodology in solving this problem, the inherent lack of an efficient gold standard method for model training and validation remains a grand challenge. This work investigates whether DL can be leveraged to accurately and efficiently simulate photon propagation in biological tissue, enabling photoacoustic image synthesis. Our approach is based on estimating the initial pressure distribution of the photoacoustic waves from the underlying optical properties using a back-propagatable neural network trained on synthetic data. In proof-of-concept studies, we validated the performance of two complementary neural network architectures, namely a conventional U-Net-like model and a Fourier Neural Operator (FNO) network. Our in silico validation on multispectral human forearm images shows that DL methods can speed up image generation by a factor of 100 when compared to Monte Carlo simulations with 5\u00d7108 photons. While the FNO is slightly more accurate than the U-Net, when compared to Monte Carlo simulations performed with a reduced number of photons (5\u00d7106), both neural network architectures achieve equivalent accuracy. In contrast to Monte Carlo simulations, the proposed DL models can be used as inherently differentiable surrogate models in the photoacoustic image synthesis pipeline, allowing for back-propagation of the synthesis error and gradient-based optimization over the entire pipeline. Due to their efficiency, they have the potential to enable large-scale training data generation that can expedite the clinical application of photoacoustic imaging.<\/jats:p>","DOI":"10.3390\/s23167085","type":"journal-article","created":{"date-parts":[[2023,8,10]],"date-time":"2023-08-10T10:52:40Z","timestamp":1691664760000},"page":"7085","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Efficient Photoacoustic Image Synthesis with Deep Learning"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0322-0370","authenticated-orcid":false,"given":"Tom","family":"Rix","sequence":"first","affiliation":[{"name":"Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany"},{"name":"Faculty of Mathematics and Computer Sciences, Heidelberg University, 69120 Heidelberg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9179-9414","authenticated-orcid":false,"given":"Kris K.","family":"Dreher","sequence":"additional","affiliation":[{"name":"Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany"},{"name":"Faculty of Physics and Astronomy, Heidelberg University, 69120 Heidelberg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7600-3839","authenticated-orcid":false,"given":"Jan-Hinrich","family":"N\u00f6lke","sequence":"additional","affiliation":[{"name":"Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany"},{"name":"Faculty of Mathematics and Computer Sciences, Heidelberg University, 69120 Heidelberg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7911-5622","authenticated-orcid":false,"given":"Melanie","family":"Schellenberg","sequence":"additional","affiliation":[{"name":"Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany"},{"name":"Faculty of Mathematics and Computer Sciences, Heidelberg University, 69120 Heidelberg, Germany"},{"name":"HIDSS4Health\u2014Helmholtz Information and Data Science School for Health, 69120 Heidelberg, Germany"},{"name":"National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, 69120 Heidelberg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Minu D.","family":"Tizabi","sequence":"additional","affiliation":[{"name":"Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany"},{"name":"National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, 69120 Heidelberg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5919-9646","authenticated-orcid":false,"given":"Alexander","family":"Seitel","sequence":"additional","affiliation":[{"name":"Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany"},{"name":"National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, 69120 Heidelberg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4910-9368","authenticated-orcid":false,"given":"Lena","family":"Maier-Hein","sequence":"additional","affiliation":[{"name":"Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany"},{"name":"Faculty of Mathematics and Computer Sciences, Heidelberg University, 69120 Heidelberg, Germany"},{"name":"HIDSS4Health\u2014Helmholtz Information and Data Science School for Health, 69120 Heidelberg, Germany"},{"name":"National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, 69120 Heidelberg, Germany"},{"name":"Medical Faculty, Heidelberg University, 69120 Heidelberg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"41970","DOI":"10.1038\/srep41970","article-title":"Visualization of tumor-related blood vessels in human breast by photoacoustic imaging system with a hemispherical detector array","volume":"7","author":"Toi","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1148\/radiol.2017172228","article-title":"A pivotal study of optoacoustic imaging to diagnose benign and malignant breast masses: A new evaluation tool for radiologists","volume":"287","author":"Neuschler","year":"2018","journal-title":"Radiology"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"6912","DOI":"10.1158\/1078-0432.CCR-16-3200","article-title":"Multispectral optoacoustic tomography (MSOT) of human breast cancer","volume":"23","author":"Diot","year":"2017","journal-title":"Clin. Cancer Res."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1007\/s00330-017-5002-x","article-title":"Multispectral optoacoustic tomography of the human breast: Characterisation of healthy tissue and malignant lesions using a hybrid ultrasound-optoacoustic approach","volume":"28","author":"Becker","year":"2018","journal-title":"Eur. Radiol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Jnawali, K., Chinni, B.K., Dogra, V., Sinha, S., and Rao, N. (2019, January 15). Deep 3D convolutional neural network for automatic cancer tissue detection using multispectral photoacoustic imaging. Proceedings of the SPIE Medical Imaging 2019: Ultrasonic Imaging and Tomography, San Diego, CA, USA.","DOI":"10.1117\/12.2518686"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"eaav2169","DOI":"10.1126\/scitranslmed.aav2169","article-title":"Simultaneous transrectal ultrasound and photoacoustic human prostate imaging","volume":"11","author":"Kothapalli","year":"2019","journal-title":"Sci. Transl. Med."},{"key":"ref_7","first-page":"5457","article-title":"Photoacoustic image classification and segmentation of breast cancer: A feasibility study","volume":"7","author":"Zhang","year":"2018","journal-title":"IEEE Access Pract. Innov. Open Solut."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Periyasamy, V., \u00d6zsoy, \u00c7., Reiss, M., De\u00e1n-Ben, X.L., and Razansky, D. (2019, January 24\u201325). Tumor ablation and volumetric optoacoustic monitoring with a short-pulsed laser source. Proceedings of the Opto-Acoustic Methods and Applications in Biophotonics IV, Munich, Germany.","DOI":"10.1117\/12.2531536"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1038\/nphoton.2009.157","article-title":"Multiscale photoacoustic microscopy and computed tomography","volume":"3","author":"Wang","year":"2009","journal-title":"Nat. Photonics"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"0068","DOI":"10.1038\/s41551-017-0068","article-title":"Precision assessment of label-free psoriasis biomarkers with ultra-broadband optoacoustic mesoscopy","volume":"1","author":"Aguirre","year":"2017","journal-title":"Nat. Biomed. Eng."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1292","DOI":"10.1056\/NEJMc1612455","article-title":"Multispectral optoacoustic tomography for assessment of crohn\u2019s disease activity","volume":"376","author":"Knieling","year":"2017","journal-title":"N. Engl. J. Med."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"121205","DOI":"10.1117\/1.JBO.28.12.121205","article-title":"Photoacoustic imaging on its way toward clinical utility: A tutorial review focusing on practical application in medicine","volume":"28","author":"Riksen","year":"2023","journal-title":"J. Biomed. Opt."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"061202","DOI":"10.1117\/1.JBO.17.6.061202","article-title":"Quantitative Spectroscopic Photoacoustic Imaging: A Review","volume":"17","author":"Cox","year":"2012","journal-title":"J. Biomed. Opt."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1109\/TMI.2016.2607199","article-title":"Towards quantitative evaluation of tissue absorption coefficients using light fluence correction in optoacoustic tomography","volume":"36","author":"Brochu","year":"2017","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2752","DOI":"10.1364\/OL.43.002752","article-title":"End-to-end deep neural network for optical inversion in quantitative photoacoustic imaging","volume":"43","author":"Cai","year":"2018","journal-title":"Opt. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yang, C., Lan, H., Zhong, H., and Gao, F. (2019, January 8\u201311). Quantitative photoacoustic blood oxygenation imaging using deep residual and recurrent neural network. Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy.","DOI":"10.1109\/ISBI.2019.8759438"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Dreher, K.K., Ayala, L., Schellenberg, M., H\u00fcbner, M., N\u00f6lke, J.H., Adler, T.J., Seidlitz, S., Sellner, J., Studier-Fischer, A., and Gr\u00f6hl, J. (2023). Unsupervised domain transfer with conditional invertible neural networks. arXiv.","DOI":"10.1007\/978-3-031-43907-0_73"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"6069","DOI":"10.1029\/2018GL081646","article-title":"Using deep neural networks as Cost-Effective surrogate models for Super-Parameterized E3SM radiative transfer","volume":"46","author":"Pal","year":"2019","journal-title":"Geophys. Res. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"91","DOI":"10.3847\/PSJ\/abe3fd","article-title":"Accurate machine-learning atmospheric retrieval via a neural-network surrogate model for radiative transfer","volume":"3","author":"Himes","year":"2022","journal-title":"Planet. Sci. J."},{"key":"ref_20","unstructured":"Bench, C., and Cox, B.T. (2023). Enhancing synthetic training data for quantitative photoacoustic tomography with generative deep learning. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1364\/BOE.1.000165","article-title":"Mesh-based Monte Carlo Method Using Fast Ray-tracing in Pl\u00fccker Coordinates","volume":"1","author":"Fang","year":"2010","journal-title":"Biomed. Opt. Express"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_23","unstructured":"Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. (2021). Fourier Neural Operator for Parametric Partial Differential Equations. Technical Report. arXiv."},{"key":"ref_24","first-page":"083010","article-title":"SIMPA: An open-source toolkit for simulation and image processing for photonics and acoustics","volume":"27","author":"Dreher","year":"2022","journal-title":"J. Biomed. Opt."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Schellenberg, M., Gr\u00f6hl, J., Dreher, K., Holzwarth, N., Tizabi, M.D., Seitel, A., and Maier-Hein, L. (2021, January 6\u201311). Generation of Training Data for Quantitative Photoacoustic Imaging. Proceedings of the Photons Plus Ultrasound: Imaging and Sensing 2021, Online.","DOI":"10.1117\/12.2578180"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"R37","DOI":"10.1088\/0031-9155\/58\/11\/R37","article-title":"Optical properties of biological tissues: A review","volume":"58","author":"Jacques","year":"2013","journal-title":"Phys. Med. Biol."},{"key":"ref_27","unstructured":"Gr\u00f6hl, J., Dreher, K., and N\u00f6lke, J. (2022). GitHub-SIMPA-MSOTAcuityEcho. GitHub., 6, Available online: https:\/\/github.com\/IMSY-DKFZ\/simpa\/blob\/d3cb0c6eaf5981b23ee829f1eff4b4d537749a16\/simpa\/core\/device_digital_twins\/pa_devices\/ithera_msot_acuity.py."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Fujita, H., Ali, M., Selamat, A., Sasaki, J., and Kurematsu, M. (2016). Trends in Applied Knowledge-Based Systems and Data Science, Springer International Publishing.","DOI":"10.1007\/978-3-319-42007-3"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Khan, S.U., Zomaya, A.Y., and Abbas, A. (2017). Handbook of Large-Scale Distributed Computing in Smart Healthcare, Springer International Publishing.","DOI":"10.1007\/978-3-319-58280-1"},{"key":"ref_30","unstructured":"Hendrycks, D., and Gimpel, K. (2023). Gaussian Error Linear Units (GELUs). arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Guan, S., Hsu, K.T., and Chitnis, P.V. (2023). Fourier neural operator network for fast photoacoustic wave simulations. Algorithms, 16.","DOI":"10.3390\/a16020124"},{"key":"ref_32","unstructured":"Kingma, D.P., and Ba, J. (2017). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image Quality Assessment: From Error Visibility to Structural Similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"056008","DOI":"10.1117\/1.JBO.23.5.056008","article-title":"Context encoding enables machine learning-based quantitative photoacoustics","volume":"23","author":"Kirchner","year":"2018","journal-title":"J. Biomed. Opt."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Stoyanov, D., Taylor, Z., Aylward, S., Tavares, J.M.R., Xiao, Y., Simpson, A., Martel, A., Maier-Hein, L., Li, S., and Rivaz, H. (2018). Simulation, Image Processing, and Ultrasound Systems for Assisted Diagnosis and Navigation, Springer International Publishing. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-030-01045-4"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3360","DOI":"10.1364\/BOE.395683","article-title":"Deep learning improves contrast in low-fluence photoacoustic imaging","volume":"11","author":"Hariri","year":"2020","journal-title":"Biomed. Opt. Express"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1109\/TMI.2019.2922026","article-title":"A Partially-Learned Algorithm for Joint Photo-acoustic Reconstruction and Segmentation","volume":"39","author":"Boink","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"100241","DOI":"10.1016\/j.pacs.2021.100241","article-title":"Deep learning for biomedical photoacoustic imaging: A review","volume":"22","author":"Schellenberg","year":"2021","journal-title":"Photoacoustics"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"112903","DOI":"10.1117\/1.JBO.25.11.112903","article-title":"Deep learning in photoacoustic tomography: Current approaches and future directions","volume":"25","author":"Hauptmann","year":"2020","journal-title":"J. Biomed. Opt."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"100215","DOI":"10.1016\/j.pacs.2020.100215","article-title":"Review of deep learning for photoacoustic imaging","volume":"21","author":"Yang","year":"2021","journal-title":"Photoacoustics"},{"key":"ref_41","unstructured":"Hochuli, R. (2016). Monte Carlo Methods in Quantitative Photoacoustic Tomography. [Ph.D. Thesis, University College London]."},{"key":"ref_42","unstructured":"Ardakani, M.R., Yu, L., Kaeli, D.R., and Fang, Q. (February, January 28). A framework for enhancing Monte Carlo photon transport simulations using deep learning. Proceedings of the Multimodal Biomedical Imaging XVIII, San Francisco, CA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/16\/7085\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:30:44Z","timestamp":1760128244000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/16\/7085"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,10]]},"references-count":42,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["s23167085"],"URL":"https:\/\/doi.org\/10.3390\/s23167085","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,10]]}}}