{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T07:06:43Z","timestamp":1773472003848,"version":"3.50.1"},"reference-count":24,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2020,5,15]],"date-time":"2020-05-15T00:00:00Z","timestamp":1589500800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,5,15]],"date-time":"2020-05-15T00:00:00Z","timestamp":1589500800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["NS\/A000050\/1"],"award-info":[{"award-number":["NS\/A000050\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["NS\/A000027\/1"],"award-info":[{"award-number":["NS\/A000027\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/N027078\/1"],"award-info":[{"award-number":["EP\/N027078\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004440","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["203145Z\/16\/Z"],"award-info":[{"award-number":["203145Z\/16\/Z"]}],"id":[{"id":"10.13039\/100004440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004440","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["WT101957"],"award-info":[{"award-number":["WT101957"]}],"id":[{"id":"10.13039\/100004440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004440","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["203148\/Z\/16\/Z"],"award-info":[{"award-number":["203148\/Z\/16\/Z"]}],"id":[{"id":"10.13039\/100004440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000287","name":"Royal Academy of Engineering","doi-asserted-by":"crossref","award":["RCSRF1819\\7\\34"],"award-info":[{"award-number":["RCSRF1819\\7\\34"]}],"id":[{"id":"10.13039\/501100000287","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100010478","name":"Mauna Kea Technologies","doi-asserted-by":"crossref","award":["PhD Studentship"],"award-info":[{"award-number":["PhD Studentship"]}],"id":[{"id":"10.13039\/100010478","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"published-print":{"date-parts":[[2020,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n<jats:title>Purpose<\/jats:title>\n<jats:p>Probe-based confocal laser endomicroscopy (pCLE) enables performing an optical biopsy via a probe. pCLE probes consist of multiple optical fibres arranged in a bundle, which taken together generate signals in an irregularly sampled pattern. Current pCLE reconstruction is based on interpolating irregular signals onto an over-sampled Cartesian grid, using a naive linear interpolation. It was shown that convolutional neural networks (CNNs) could improve pCLE image quality. Yet classical CNNs may be suboptimal in regard to irregular data.\n<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Methods<\/jats:title>\n<jats:p>We compare pCLE reconstruction and super-resolution (SR) methods taking irregularly sampled or reconstructed pCLE images as input. We also propose to embed a Nadaraya\u2013Watson (NW) kernel regression into the CNN framework as a novel trainable CNN layer. We design deep learning architectures allowing for reconstructing high-quality pCLE images directly from the irregularly sampled input data. We created synthetic sparse pCLE images to evaluate our methodology.\n<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Results<\/jats:title>\n<jats:p>The results were validated through an image quality assessment based on a combination of the following metrics: peak signal-to-noise ratio and the structural similarity index. Our analysis indicates that both dense and sparse CNNs outperform the reconstruction method currently used in the clinic.\n<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Conclusion<\/jats:title>\n<jats:p>The main contributions of our study are a comparison of sparse and dense approach in pCLE image reconstruction. We also implement trainable generalised NW kernel regression as a novel sparse approach. We also generated synthetic data for training pCLE SR.<\/jats:p>\n<\/jats:sec>","DOI":"10.1007\/s11548-020-02170-7","type":"journal-article","created":{"date-parts":[[2020,5,15]],"date-time":"2020-05-15T17:03:52Z","timestamp":1589562232000},"page":"1167-1175","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Learning from irregularly sampled data for endomicroscopy super-resolution: a comparative study of sparse and dense approaches"],"prefix":"10.1007","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7393-0538","authenticated-orcid":false,"given":"Agnieszka Barbara","family":"Szczotka","sequence":"first","affiliation":[]},{"given":"Dzhoshkun Ismail","family":"Shakir","sequence":"additional","affiliation":[]},{"given":"Daniele","family":"Rav\u00ec","sequence":"additional","affiliation":[]},{"given":"Matthew J.","family":"Clarkson","sequence":"additional","affiliation":[]},{"given":"Stephen P.","family":"Pereira","sequence":"additional","affiliation":[]},{"given":"Tom","family":"Vercauteren","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,5,15]]},"reference":[{"issue":"4","key":"2170_CR1","doi-asserted-by":"publisher","first-page":"460","DOI":"10.1016\/j.media.2011.02.003","volume":"15","author":"B Andr\u00e9","year":"2011","unstructured":"Andr\u00e9 B, Vercauteren T, Buchner AM, Wallace MB, Ayache N (2011) A smart atlas for endomicroscopy using automated video retrieval. Med Image Anal 15(4):460\u2013476","journal-title":"Med Image Anal"},{"issue":"1","key":"2170_CR2","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1109\/JPROC.2019.2956422","volume":"108","author":"JS Duncan","year":"2019","unstructured":"Duncan JS, Insana MF, Ayache N (2019) Biomedical imaging and analysis in the age of big data and deep learning. Proc IEEE 108(1):3\u201310. https:\/\/doi.org\/10.1109\/JPROC.2019.2956422","journal-title":"Proc IEEE"},{"key":"2170_CR3","unstructured":"Eldesokey A, Felsberg M, Khan FS (2018) Propagating confidences through CNNs for sparse data regression. arXiv preprint arXiv:1805.11913"},{"issue":"1","key":"2170_CR4","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1038\/s41591-018-0316-z","volume":"25","author":"A Esteva","year":"2019","unstructured":"Esteva A, Robicquet A, Ramsundar B, Kuleshov V, DePristo M, Chou K et al (2019) A guide to deep learning in healthcare. Nat Med 25(1):24\u201329. https:\/\/doi.org\/10.1038\/s41591-018-0316-z","journal-title":"Nat Med"},{"key":"2170_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2016\/4638683","volume":"2016","author":"A Fugazza","year":"2016","unstructured":"Fugazza A, Gaiani F, Carra MC, Brunetti F, L\u00e9vy M, Sobhani I, Azoulay D, Catena F, de\u2019Angelis GL, de\u2019Angelis N (2016) Confocal laser endomicroscopy in gastrointestinal and pancreatobiliary diseases: a systematic review and meta-analysis. BioMed Res Int 2016:1\u201331","journal-title":"BioMed Res Int"},{"key":"2170_CR6","doi-asserted-by":"crossref","unstructured":"Hua J, Gong X (2018) A normalized convolutional neural network for guided sparse depth upsampling. In: International joint conferences on artificial intelligence, pp 2283\u20132290","DOI":"10.24963\/ijcai.2018\/316"},{"key":"2170_CR7","doi-asserted-by":"crossref","unstructured":"Izadi S, Moriarty KP, Hamarneh G (2018) Can deep learning relax endomicroscopy hardware miniaturization requirements? In: International conference on medical image computing and computer-assisted intervention, pp 57\u201364. Springer","DOI":"10.1007\/978-3-030-00928-1_7"},{"key":"2170_CR8","unstructured":"Jaderberg M, Dalibard V, Osindero S, Czarnecki WM, Donahue J, Razavi A, Vinyals O, Green T, Dunning I, Simonyan K, Fernando C, Kavukcuoglu K (2017) Population based training of neural networks. arXiv preprint arXiv:1711.09846"},{"issue":"1","key":"2170_CR9","doi-asserted-by":"publisher","first-page":"29","DOI":"10.4103\/2153-3539.186902","volume":"7","author":"A Janowczyk","year":"2016","unstructured":"Janowczyk A, Madabhushi A (2016) Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases. J Pathol Inform 7(1):29","journal-title":"J Pathol Inform"},{"key":"2170_CR10","doi-asserted-by":"publisher","first-page":"27988","DOI":"10.1038\/srep27988","volume":"6","author":"JN Kather","year":"2016","unstructured":"Kather JN, Weis CA, Bianconi F, Melchers SM, Schad LR, Gaiser T, Marx A, Z\u00f6llner FG (2016) Multi-class texture analysis in colorectal cancer histology. Sci Rep 6:27988","journal-title":"Sci Rep"},{"key":"2170_CR11","doi-asserted-by":"crossref","unstructured":"K\u00f6hler R, Schuler C, Sch\u00f6lkopf B, Harmeling S (2014) Mask-specific inpainting with deep neural networks. In: German conference on pattern recognition, pp 523\u2013534. Springer","DOI":"10.1007\/978-3-319-11752-2_43"},{"key":"2170_CR12","first-page":"761","volume":"2004","author":"G Le Goualher","year":"2004","unstructured":"Le Goualher G, Perchant A, Genet M, Cav\u00e9 C, Viellerobe B, Berier F, Abrat B, Ayache N (2004) Towards optical biopsies with an integrated fibered confocal fluorescence microscope. Med Image Comput Comput Assist Interv MICCAI 2004:761\u2013768","journal-title":"Med Image Comput Comput Assist Interv MICCAI"},{"key":"2170_CR13","doi-asserted-by":"crossref","unstructured":"Li B, Zhang T, Xia T (2016) Vehicle detection from 3D lidar using fully convolutional network. ArXiv arXiv:1608.07916","DOI":"10.15607\/RSS.2016.XII.042"},{"key":"2170_CR14","doi-asserted-by":"crossref","unstructured":"Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: The IEEE conference on computer vision and pattern recognition (CVPR) workshops","DOI":"10.1109\/CVPRW.2017.151"},{"key":"2170_CR15","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","volume":"42","author":"G Litjens","year":"2017","unstructured":"Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60\u201388. https:\/\/doi.org\/10.1016\/j.media.2017.07.005","journal-title":"Med Image Anal"},{"issue":"1","key":"2170_CR16","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1137\/1109020","volume":"9","author":"EA Nadaraya","year":"1964","unstructured":"Nadaraya EA (1964) On estimating regression. Theory Probab Appl 9(1):141\u2013142","journal-title":"Theory Probab Appl"},{"key":"2170_CR17","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1016\/j.media.2019.01.011","volume":"53","author":"D Rav\u00ec","year":"2019","unstructured":"Rav\u00ec D, Szczotka AB, Pereira SP, Vercauteren T (2019) Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy. Med Image Anal 53:123\u2013131","journal-title":"Med Image Anal"},{"key":"2170_CR18","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1007\/s11548-018-1764-0","volume":"13","author":"D Rav\u00ec","year":"2018","unstructured":"Rav\u00ec D, Szczotka AB, Shakir DI, Pereira SP, Vercauteren T (2018) Effective deep learning training for single-image super-resolution in endomicroscopy exploiting video-registration-based reconstruction. Int J Comput Assist Radiol Surg 13:917\u2013924","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"4","key":"2170_CR19","doi-asserted-by":"publisher","first-page":"e0153623","DOI":"10.1371\/journal.pone.0153623","volume":"11","author":"A Sundstrom","year":"2016","unstructured":"Sundstrom A (2016) Replication Data for: histological image processing features induce a quantitative characterization of chronic tumor hypoxia. PloS one 11(4):e0153623","journal-title":"PloS one"},{"key":"2170_CR20","doi-asserted-by":"crossref","unstructured":"Uhrig J, Schneider N, Schneider L, Franke U, Brox T, Geiger A (2017) Sparsity invariant cnns. In: IEEE International conference on 3D vision","DOI":"10.1109\/3DV.2017.00012"},{"key":"2170_CR21","unstructured":"Vercauteren T (2008) Image registration and mosaicing for dynamic in vivo fibered confocal microscopy. Ph.D. dissertation, Equipe-Projet Asclepios, INRIA Sophia Antipolis"},{"issue":"5","key":"2170_CR22","doi-asserted-by":"publisher","first-page":"673","DOI":"10.1016\/j.media.2006.06.006","volume":"10","author":"T Vercauteren","year":"2006","unstructured":"Vercauteren T, Perchant A, Malandain G, Pennec X, Ayache N (2006) Robust mosaicing with correction of motion distortions and tissue deformations for in vivo fibered microscopy. Med Image Anal 10(5):673\u2013692","journal-title":"Med Image Anal"},{"issue":"4","key":"2170_CR23","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1109\/TIP.2003.819861","volume":"13","author":"Z Wang","year":"2004","unstructured":"Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600\u2013612","journal-title":"IEEE Trans Image Process"},{"issue":"1","key":"2170_CR24","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1109\/TCI.2016.2644865","volume":"3","author":"H Zhao","year":"2017","unstructured":"Zhao H, Gallo O, Frosio I, Kautz J (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imag 3(1):47\u201357","journal-title":"IEEE Trans Comput Imag"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-020-02170-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-020-02170-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-020-02170-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,14]],"date-time":"2021-05-14T23:34:10Z","timestamp":1621035250000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-020-02170-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,15]]},"references-count":24,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2020,7]]}},"alternative-id":["2170"],"URL":"https:\/\/doi.org\/10.1007\/s11548-020-02170-7","relation":{},"ISSN":["1861-6410","1861-6429"],"issn-type":[{"value":"1861-6410","type":"print"},{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,5,15]]},"assertion":[{"value":"19 November 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 April 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 May 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"The Ph.D. studentship of Agnieszka Barbara Szczotka is funded by Mauna Kea Technologies, Paris, France. Tom Vercauteren owns stock in Mauna Kea Technologies, Paris, France. The other authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and\/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"For this type of study formal consent is not required. This article does not contain patient data.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}}]}}