{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,6]],"date-time":"2026-09-06T04:03:05Z","timestamp":1788667385163,"version":"build-2803163510"},"reference-count":32,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2021,10,4]],"date-time":"2021-10-04T00:00:00Z","timestamp":1633305600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["NA"],"award-info":[{"award-number":["NA"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Canada Research Chair","award":["NA"],"award-info":[{"award-number":["NA"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Mass spectrometry is an effective imaging tool for evaluating biological tissue to detect cancer. With the assistance of deep learning, this technology can be used as a perioperative tissue assessment tool that will facilitate informed surgical decisions. To achieve such a system requires the development of a database of mass spectrometry signals and their corresponding pathology labels. Assigning correct labels, in turn, necessitates precise spatial registration of histopathology and mass spectrometry data. This is a challenging task due to the domain differences and noisy nature of images. In this study, we create a registration framework for mass spectrometry and pathology images as a contribution to the development of perioperative tissue assessment. In doing so, we explore two opportunities in deep learning for medical image registration, namely, unsupervised, multi-modal deformable image registration and evaluation of the registration. We test this system on prostate needle biopsy cores that were imaged with desorption electrospray ionization mass spectrometry (DESI) and show that we can successfully register DESI and histology images to achieve accurate alignment and, consequently, labelling for future training. This automation is expected to improve the efficiency and development of a deep learning architecture that will benefit the use of mass spectrometry imaging for cancer diagnosis.<\/jats:p>","DOI":"10.3390\/jimaging7100203","type":"journal-article","created":{"date-parts":[[2021,10,7]],"date-time":"2021-10-07T16:23:58Z","timestamp":1633623838000},"page":"203","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Combined Mass Spectrometry and Histopathology Imaging for Perioperative Tissue Assessment in Cancer Surgery"],"prefix":"10.3390","volume":"7","author":[{"given":"Laura","family":"Connolly","sequence":"first","affiliation":[{"name":"School of Computing, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amoon","family":"Jamzad","sequence":"additional","affiliation":[{"name":"School of Computing, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Kaufmann","sequence":"additional","affiliation":[{"name":"Department of Surgery, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Catriona E.","family":"Farquharson","sequence":"additional","affiliation":[{"name":"School of Computing, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kevin","family":"Ren","sequence":"additional","affiliation":[{"name":"Department of Pathology and Molecular Medicine, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John F.","family":"Rudan","sequence":"additional","affiliation":[{"name":"Department of Surgery, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6354-262X","authenticated-orcid":false,"given":"Gabor","family":"Fichtinger","sequence":"additional","affiliation":[{"name":"School of Computing, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Parvin","family":"Mousavi","sequence":"additional","affiliation":[{"name":"School of Computing, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3146","DOI":"10.1002\/jssc.202000273","article-title":"Recent advances of ambient ionization mass spectrometry imaging in clinical research","volume":"43","author":"Li","year":"2020","journal-title":"J. Sep. Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S1361-8415(01)80026-8","article-title":"A survey of medical image registration","volume":"2","author":"Maintz","year":"1998","journal-title":"Med. Image Anal."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1016\/j.media.2016.06.030","article-title":"A survey of medical image registration\u2014Under review","volume":"33","author":"Viergever","year":"2016","journal-title":"Med. Image Anal."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1080\/10255842.2012.670855","article-title":"Medical image registration: A review","volume":"17","author":"Oliveira","year":"2014","journal-title":"Comput. Methods Biomech. Biomed. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1109\/TMI.2009.2035616","article-title":"Elastix: A toolbox for intensity-based medical image registration","volume":"29","author":"Klein","year":"2010","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_6","unstructured":"(2021, June 18). A Reproducible Evaluation of ANTs Similarity Metric Performance in Brain Image Registration\u2014ScienceDirect. Available online: https:\/\/www-sciencedirect-com.proxy.queensu.ca\/science\/article\/pii\/S1053811910012061."},{"key":"ref_7","unstructured":"Modat, M., Mcclelland, J.R., Mcclelland, J., and Ourselin, S. (2010). Lung Registration Using the NiftyReg Package, University College London. Available online: https:\/\/www.researchgate.net\/publication\/268354372."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1088\/1361-6560\/ab843e","article-title":"Deep learning in medical image registration: A review","volume":"65","author":"Fu","year":"2020","journal-title":"Phys. Med. Biol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"101919","DOI":"10.1016\/j.media.2020.101919","article-title":"ProsRegNet: A deep learning framework for registration of MRI and histopathology images of the prostate","volume":"68","author":"Shao","year":"2021","journal-title":"Med. Image Anal."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1007\/978-3-030-00928-1_87","article-title":"Adversarial deformation regularization for training image registration neural networks","volume":"Volume 11070","author":"Hu","year":"2018","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_11","first-page":"012003","article-title":"Deep learning in medical image registration","volume":"3","author":"Chen","year":"2020","journal-title":"Prog. Biomed. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"101939","DOI":"10.1016\/j.media.2020.101939","article-title":"Image registration: Maximum likelihood, minimum entropy and deep learning","volume":"69","author":"Sedghi","year":"2021","journal-title":"Med. Image Anal."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3329","DOI":"10.1002\/mp.13620","article-title":"Robust contour propagation using deep learning and image registration for online adaptive proton therapy of prostate cancer","volume":"46","author":"Elmahdy","year":"2019","journal-title":"Med. Phys."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1788","DOI":"10.1109\/TMI.2019.2897538","article-title":"VoxelMorph: A Learning Framework for Deformable Medical Image Registration","volume":"38","author":"Balakrishnan","year":"2019","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Zhou, Z., Liao, G., and Yuan, K. (2020). New Loss Functions for Medical Image Registration Based on VoxelMorph, SPIE (The International Society for Optical Engineering).","DOI":"10.1117\/12.2550030"},{"key":"ref_16","unstructured":"Liang, X., Morgan, H., Nguyen, D., and Jiang, S. (2021, June 29). Deep Learning Based CT-to-CBCT Deformable Image Registration for Autosegmentation in Head and Neck Adaptive Radiation Therapy. January 2021. Available online: http:\/\/arxiv.org\/abs\/2102.00590."},{"key":"ref_17","first-page":"9252","article-title":"An Unsupervised Learning Model for Deformable Medical Image Registration","volume":"11313","author":"Balakrishnan","year":"2018","journal-title":"Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"101822","DOI":"10.1016\/j.media.2020.101822","article-title":"Weakly-supervised learning of multi-modal features for regularised iterative descent in 3D image registration","volume":"67","author":"Blendowski","year":"2021","journal-title":"Med. Image Anal."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2879","DOI":"10.1109\/TIP.2007.909412","article-title":"Evaluation of optimization methods for nonrigid medical image registration using mutual information and B-splines","volume":"16","author":"Klein","year":"2007","journal-title":"IEEE Trans. Image Process."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"4490","DOI":"10.1002\/mp.13726","article-title":"Automatic large quantity landmark pairs detection in 4DCT lung images","volume":"46","author":"Fu","year":"2019","journal-title":"Med. Phys."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.media.2019.05.005","article-title":"Quantitative error prediction of medical image registration using regression forests","volume":"56","author":"Sokooti","year":"2019","journal-title":"Med. Image Anal."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"101698","DOI":"10.1016\/j.media.2020.101698","article-title":"Multi-atlas image registration of clinical data with automated quality assessment using ventricle segmentation","volume":"63","author":"Dubost","year":"2020","journal-title":"Med. Image Anal."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1002\/mp.13890","article-title":"A fast and scalable method for quality assurance of deformable image registration on lung CT scans using convolutional neural networks","volume":"47","author":"Galib","year":"2020","journal-title":"Med. Phys."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1016\/j.ijrobp.2016.05.012","article-title":"Multi-institutional Validation Study of Commercially Available Deformable Image Registration Software for Thoracic Images","volume":"96","author":"Kadoya","year":"2016","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"ref_25","unstructured":"(2020, December 14). Prostate cancer statistics\u2014Canadian Cancer Society. Available online: https:\/\/www.cancer.ca\/en\/cancer-information\/cancer-type\/prostate\/statistics\/?region=on."},{"key":"ref_26","unstructured":"Koch, G. (2015). Siamese Neural Networks for One-Shot Image Recognition, University of Toronto."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1561","DOI":"10.1038\/s41374-019-0265-2","article-title":"Reliable identification of prostate cancer using mass spectrometry metabolomic imaging in needle core biopsies","volume":"99","author":"Morse","year":"2019","journal-title":"Lab. Investig."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1373","DOI":"10.1007\/s13361-011-0142-8","article-title":"Software Lock Mass by Two-Dimensional Minimization of Peptide Mass Errors","volume":"22","author":"Cox","year":"2011","journal-title":"J. Am. Soc. Mass Spectrom."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1016\/j.jasms.2003.11.008","article-title":"On the normalization of a mass spectrum for comparison of two spectra","volume":"15","author":"Alfassi","year":"2004","journal-title":"J. Am. Soc. Mass Spectrom."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.media.2016.06.011","article-title":"Open-source platforms for navigated image-guided interventions","volume":"33","author":"Ungi","year":"2016","journal-title":"Med. Image Anal."},{"key":"ref_31","unstructured":"Chandra, S. (2021, September 06). Deep-Learning-Based 2D and 3D Affine Registration | by Sarath Chandra | Medium. Medium, 30 August 2020. Available online: https:\/\/medium.com\/@sarathchandra.knv31\/deep-learning-based-2d-and-3d-affine-registration-da73df8d2f24."},{"key":"ref_32","first-page":"2017","article-title":"Spatial Transformer Networks","volume":"28","author":"Jaderberg","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/7\/10\/203\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:09:20Z","timestamp":1760166560000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/7\/10\/203"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,4]]},"references-count":32,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["jimaging7100203"],"URL":"https:\/\/doi.org\/10.3390\/jimaging7100203","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,4]]}}}