{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T23:59:16Z","timestamp":1778543956594,"version":"3.51.4"},"reference-count":29,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2019,9,19]],"date-time":"2019-09-19T00:00:00Z","timestamp":1568851200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000147","name":"Canadian Cancer Society","doi-asserted-by":"publisher","award":["300028"],"award-info":[{"award-number":["300028"]}],"id":[{"id":"10.13039\/501100000147","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["300028"],"award-info":[{"award-number":["300028"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>We present a deep learning method used in conjunction with dual-modal microwave-ultrasound imaging to produce tomographic reconstructions of the complex-valued permittivity of numerical breast phantoms. We also assess tumor segmentation performance using the reconstructed permittivity as a feature. The contrast source inversion (CSI) technique is used to create the complex-permittivity images of the breast with ultrasound-derived tissue regions utilized as prior information. However, imaging artifacts make the detection of tumors difficult. To overcome this issue we train a convolutional neural network (CNN) that takes in, as input, the dual-modal CSI reconstruction and attempts to produce the true image of the complex tissue permittivity. The neural network consists of successive convolutional and downsampling layers, followed by successive deconvolutional and upsampling layers based on the U-Net architecture. To train the neural network, the input-output pairs consist of CSI\u2019s dual-modal reconstructions, along with the true numerical phantom images from which the microwave scattered field was synthetically generated. The reconstructed permittivity images produced by the CNN show that the network is not only able to remove the artifacts that are typical of CSI reconstructions, but can also improve the detectability of tumors. The performance of the CNN is assessed using a four-fold cross-validation on our dataset that shows improvement over CSI both in terms of reconstruction error and tumor segmentation performance.<\/jats:p>","DOI":"10.3390\/s19184050","type":"journal-article","created":{"date-parts":[[2019,9,19]],"date-time":"2019-09-19T11:02:01Z","timestamp":1568890921000},"page":"4050","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":51,"title":["Enhancement of Multimodal Microwave-Ultrasound Breast Imaging Using a Deep-Learning Technique"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6241-8467","authenticated-orcid":false,"given":"Vahab","family":"Khoshdel","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Manitoba, Winnipeg, MB R3T 5V6 1, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ahmed","family":"Ashraf","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Manitoba, Winnipeg, MB R3T 5V6 1, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joe","family":"LoVetri","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Manitoba, Winnipeg, MB R3T 5V6 1, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,9,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Pastorino, M. (2010). Microwave Imaging, John Wiley Sons.","DOI":"10.1002\/9780470602492"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2712","DOI":"10.1109\/TBME.2010.2059023","article-title":"Large-Domain, Low-Contrast Acoustic Inverse Scattering for Ultrasound Breast Imaging","volume":"57","author":"Haynes","year":"2010","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Sak, M., Duric, N., Littrup, P., Sherman, M., and Gierach, G. (2017, January 11\u201316). Ultrasound tomography imaging with waveform sound speed: Parenchymal changes in women undergoing tamoxifen therapy. Proceedings of the Medical Imaging 2017: Ultrasonic Imaging and Tomography, Orlando, FL, USA.","DOI":"10.1117\/12.2254472"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"6093","DOI":"10.1088\/0031-9155\/52\/20\/002","article-title":"A large-scale study of the ultrawideband microwave dielectric properties of normal, benign and malignant breast tissues obtained from cancer surgeries","volume":"52","author":"Lazebnik","year":"2007","journal-title":"Phys. Med. Biol."},{"key":"ref_5","unstructured":"Lakhtakia, A., and Furse, C.M. (2018). Crossed Viewpoints on Microwave-Based Imaging for Medical Diagnosis: From Genesis to Earliest Clinical Outcomes. The World of Applied Electromagnetics: In Appreciation of Magdy Fahmy Iskander, Springer International Publishing."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2580","DOI":"10.1109\/TBME.2018.2809541","article-title":"Microwave Breast Imaging: Clinical Advances and Remaining Challenges","volume":"65","author":"Moloney","year":"2018","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"S121","DOI":"10.1088\/0967-3334\/30\/6\/S08","article-title":"The correlation of in vivo and ex vivo tissue dielectric properties to validate electromagnetic breast imaging: initial clinical experience","volume":"30","author":"Halter","year":"2009","journal-title":"Physiol. Meas."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2528\/PIER18071704","article-title":"A Method for Effective Permittivity and Conductivity Mapping of Biological Scenarios via Segmented Contrast Source Inversion","volume":"164","author":"Bevacqua","year":"2019","journal-title":"Prog. Electromagn. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1109\/JMMCT.2018.2865111","article-title":"Proof-of-Concept of the Incorporation of Ultrasound-Derived Structural Information Into Microwave Radar Imaging","volume":"3","author":"Omer","year":"2018","journal-title":"IEEE J. Multiscale Multiphys. Comput. Tech."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1109\/JMMCT.2019.2905344","article-title":"Incorporation of Ultrasonic Prior Information for Improving Quantitative Microwave Imaging of Breast","volume":"4","author":"Abdollahi","year":"2019","journal-title":"IEEE J. Multiscale Multiphys. Comput. Tech."},{"key":"ref_11","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Available online: http:\/\/www.deeplearningbook.org."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1849","DOI":"10.1109\/TGRS.2018.2869221","article-title":"Deep-Learning Schemes for Full-Wave Nonlinear Inverse Scattering Problems","volume":"57","author":"Wei","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1109\/MSP.2017.2739299","article-title":"Convolutional Neural Networks for Inverse Problems in Imaging: A Review","volume":"34","author":"McCann","year":"2017","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Gerazov, B., and Conceicao, R.C. (2017, January 6\u20138). Deep learning for tumour classification in homogeneous breast tissue in medical microwave imaging. Proceedings of the IEEE EUROCON 2017-17th International Conference on Smart Technologies, Ohrid, Macedonia.","DOI":"10.1109\/EUROCON.2017.8011175"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the 18th Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015), Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Jin, K.H., McCann, M.T., Froustey, E., and Unser, M. (2016). Deep Convolutional Neural Network for Inverse Problems in Imaging. arXiv.","DOI":"10.1109\/TIP.2017.2713099"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4509","DOI":"10.1109\/TIP.2017.2713099","article-title":"Deep Convolutional Neural Network for Inverse Problems in Imaging","volume":"26","author":"Jin","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_18","first-page":"5119","article-title":"Novel Microwave Tomography System Using a Phased-Array Antenna","volume":"66","author":"Rahama","year":"2018","journal-title":"IEEE Trans. Microw. Theory Tech."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1061","DOI":"10.1109\/20.996272","article-title":"Neural-network-based inverse-scattering technique for online microwave medical imaging","volume":"38","author":"Rekanos","year":"2002","journal-title":"IEEE Trans. Magn."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1819","DOI":"10.1109\/TAP.2018.2885437","article-title":"DeepNIS: Deep Neural Network for Nonlinear Electromagnetic Inverse Scattering","volume":"67","author":"Li","year":"2019","journal-title":"IEEE Trans. Antennas Propag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1607","DOI":"10.1088\/0266-5611\/13\/6\/013","article-title":"A contrast source inversion method","volume":"13","author":"Kleinman","year":"1997","journal-title":"Inverse Probl."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"115010","DOI":"10.1088\/0266-5611\/26\/11\/115010","article-title":"Finite-element contrast source inversion method for microwave imaging","volume":"26","author":"Zakaria","year":"2010","journal-title":"Inverse Probl."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6461","DOI":"10.1002\/mp.12585","article-title":"Integrating prior information into microwave tomography Part 1: Impact of detail on image quality","volume":"44","author":"Kurrant","year":"2017","journal-title":"Med. Phys."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"6482","DOI":"10.1002\/mp.12584","article-title":"Integrating prior information into microwave tomography part 2: Impact of errors in prior information on microwave tomography image quality","volume":"44","author":"Baran","year":"2017","journal-title":"Med. Phys."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1027","DOI":"10.1007\/s11517-017-1740-9","article-title":"Automated 3D method for the construction of flexible and reconfigurable numerical breast models from MRI scans","volume":"56","author":"Omer","year":"2018","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_26","unstructured":"Trabelsi, C., Bilaniuk, O., Zhang, Y., Serdyuk, D., Subramanian, S., Santos, J.F., Mehri, S., Rostamzadeh, N., Bengio, Y., and Pal, C. (2018). Deep Complex Networks. arXiv."},{"key":"ref_27","unstructured":"Glorot, X., and Bengio, Y. (2010, January 13\u201315). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, Sardinia, Italy."},{"key":"ref_28","unstructured":"Bishop, C.M. (2006). Pattern Recognition and Machine Learning, Springer."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3946","DOI":"10.1109\/TMTT.2019.2906619","article-title":"An experimental phantom study for air-based quasi-resonant microwave breast imaging","volume":"67","author":"Asefi","year":"2019","journal-title":"IEEE Trans. Microw. Theory Tech."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/18\/4050\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:21:59Z","timestamp":1760188919000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/18\/4050"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,19]]},"references-count":29,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2019,9]]}},"alternative-id":["s19184050"],"URL":"https:\/\/doi.org\/10.3390\/s19184050","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,9,19]]}}}