{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T05:27:57Z","timestamp":1775280477663,"version":"3.50.1"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"21","license":[{"start":{"date-parts":[[2022,6,25]],"date-time":"2022-06-25T00:00:00Z","timestamp":1656115200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,6,25]],"date-time":"2022-06-25T00:00:00Z","timestamp":1656115200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/P020275\/1"],"award-info":[{"award-number":["EP\/P020275\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2022,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In this paper, we introduce our unique dataset of fluorescence lifetime imaging endo\/microscopy (FLIM), containing over 100,000 different FLIM images collected from 18 pairs of cancer\/non-cancer human lung tissues of 18 patients by our custom fibre-based FLIM system. The aim of providing this dataset is that more researchers from relevant fields can push forward this particular area of research. Afterwards, we describe the best practice of image post-processing suitable per the dataset. In addition, we propose a novel hierarchically aggregated multi-scale architecture to improve the binary classification performance of classic CNNs. The proposed model integrates the advantages of multi-scale feature extraction at different levels, where layer-wise global information is aggregated with branch-wise local information. We integrate the proposal, namely ResNetZ, into ResNet, and appraise it on the FLIM dataset. Since ResNetZ can be configured with a shortcut connection and the aggregations by <jats:italic>Addition<\/jats:italic> or <jats:italic>Concatenation<\/jats:italic>, we first evaluate the impact of different configurations on the performance. We thoroughly examine various ResNetZ variants to demonstrate the superiority. We also compare our model with a feature-level multi-scale model to illustrate the advantages and disadvantages of multi-scale architectures at different levels.<\/jats:p>","DOI":"10.1007\/s00521-022-07481-1","type":"journal-article","created":{"date-parts":[[2022,6,25]],"date-time":"2022-06-25T19:02:27Z","timestamp":1656183747000},"page":"18881-18894","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["A layer-level multi-scale architecture for lung cancer classification with fluorescence lifetime imaging endomicroscopy"],"prefix":"10.1007","volume":"34","author":[{"given":"Qiang","family":"Wang","sequence":"first","affiliation":[]},{"given":"James R.","family":"Hopgood","sequence":"additional","affiliation":[]},{"given":"Susan","family":"Fernandes","sequence":"additional","affiliation":[]},{"given":"Neil","family":"Finlayson","sequence":"additional","affiliation":[]},{"given":"Gareth O. S.","family":"Williams","sequence":"additional","affiliation":[]},{"given":"Ahsan R.","family":"Akram","sequence":"additional","affiliation":[]},{"given":"Kevin","family":"Dhaliwal","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9957-954X","authenticated-orcid":false,"given":"Marta","family":"Vallejo","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,6,25]]},"reference":[{"key":"7481_CR1","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1016\/j.medpho.2014.12.001","volume":"27","author":"K Suhling","year":"2015","unstructured":"Suhling K, Hirvonen LM, Levitt JA, Chung PH, Tregidgo C, Le Marois A, Rusakov DA, Zheng K, Ameer-Beg S, Poland S, Coelho S, Henderson R, Krstajic N (2015) Fluorescence lifetime imaging: Basic concepts and some recent developments. Med Photonics 27:3\u201340. https:\/\/doi.org\/10.1016\/j.medpho.2014.12.001","journal-title":"Med Photonics"},{"key":"7481_CR2","doi-asserted-by":"publisher","unstructured":"Jo JA, Cheng S, Cuenca-Martinez R, Duran-Sierra E, Malik B, Ahmed B, Maitland K, Cheng Y-SL, Wright J, Reese T (2018) Ous fluorescence lifetime imaging (FLIM) endoscopy for early detection of oral cancer and dysplasia. In: 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 3009\u20133012. https:\/\/doi.org\/10.1109\/EMBC.2018.8513027","DOI":"10.1109\/EMBC.2018.8513027"},{"issue":"2","key":"7481_CR3","doi-asserted-by":"publisher","first-page":"627","DOI":"10.1364\/BOE.1.000627","volume":"1","author":"J McGinty","year":"2010","unstructured":"McGinty J, Galletly NP, Dunsby C, Munro I, Elson DS, Requejo-Isidro J, Cohen P, Ahmad R, Forsyth A, Thillainayagam AV et al (2010) Wide-field fluorescence lifetime imaging of cancer. Biomed Opt Express 1(2):627\u2013640","journal-title":"Biomed Opt Express"},{"issue":"3","key":"7481_CR4","doi-asserted-by":"publisher","first-page":"921","DOI":"10.1364\/BOE.5.000921","volume":"5","author":"S Cheng","year":"2014","unstructured":"Cheng S, Cuenca RM, Liu B, Malik BH, Jabbour JM, Maitland KC, Wright J, Cheng Y-SL, Jo JA (2014) Handheld multispectral fluorescence lifetime imaging system for in vivo applications. Biomed Opt Express 5(3):921\u2013931","journal-title":"Biomed Opt Express"},{"issue":"10","key":"7481_CR5","doi-asserted-by":"publisher","first-page":"4550","DOI":"10.1109\/TNNLS.2017.2766168","volume":"29","author":"F Xing","year":"2018","unstructured":"Xing F, Xie Y, Su H, Liu F, Yang L (2018) Deep learning in microscopy image analysis: a survey. IEEE Trans Neural Netw Learn Sys 29(10):4550\u20134568. https:\/\/doi.org\/10.1109\/TNNLS.2017.2766168","journal-title":"IEEE Trans Neural Netw Learn Sys"},{"issue":"20","key":"7481_CR6","doi-asserted-by":"publisher","first-page":"10640","DOI":"10.1021\/acs.analchem.9b01866","volume":"91","author":"B Chen","year":"2019","unstructured":"Chen B, Lu Y, Pan W, Xiong J, Yang Z, Yan W, Liu L, Qu J (2019) Support vector machine classification of nonmelanoma skin lesions based on fluorescence lifetime imaging microscopy. Anal Chem 91(20):10640\u201310647. https:\/\/doi.org\/10.1021\/acs.analchem.9b01866","journal-title":"Anal Chem"},{"key":"7481_CR7","doi-asserted-by":"crossref","unstructured":"Wang Q, Hopgood JR, Finlayson N, Williams GO, Fernandes S, Williams E, Akram A, Dhaliwal K, Vallejo M (2020) Deep learning in ex-vivo lung cancer discrimination using fluorescence lifetime endomicroscopic images. In: 2020 42nd annual international conference of the IEEE engineering in medicine & biology society (EMBC), pp 1891\u20131894. IEEE","DOI":"10.1109\/EMBC44109.2020.9175598"},{"key":"7481_CR8","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition, pp 770\u2013778. https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"7481_CR9","doi-asserted-by":"publisher","unstructured":"Huang G, Liu Z, v.\u00a0d. Maaten L, Weinberger KQ (2017)Densely connected convolutional networks. In: IEEE conference on computer vision and pattern recognition, pp 2261\u20132269. https:\/\/doi.org\/10.1109\/CVPR.2017.243","DOI":"10.1109\/CVPR.2017.243"},{"key":"7481_CR10","doi-asserted-by":"crossref","unstructured":"Huang G, Sun Y, Liu Z, Sedra D, Weinberger KQ (2016) Deep networks with stochastic depth. In: European conference on computer vision, pp 646\u2013661. Springer","DOI":"10.1007\/978-3-319-46493-0_39"},{"key":"7481_CR11","doi-asserted-by":"publisher","unstructured":"Gao S, Cheng M, Zhao K, Zhang X, Yang M, Torr PHS (2019) Res2net: a new multi-scale backbone architecture. IEEE transactions on pattern analysis and machine intelligence, 1. https:\/\/doi.org\/10.1109\/TPAMI.2019.2938758","DOI":"10.1109\/TPAMI.2019.2938758"},{"issue":"6","key":"7481_CR12","doi-asserted-by":"publisher","first-page":"1705","DOI":"10.1109\/JSSC.2019.2894355","volume":"54","author":"AT Erdogan","year":"2019","unstructured":"Erdogan AT, Walker R, Finlayson N, Krstaji\u0107 N, Williams G, Girkin J, Henderson R (2019) A CMOS SPAD line sensor with per-pixel histogramming TDC for time-resolved multispectral imaging. IEEE J Solid-State Circuits 54(6):1705\u20131719","journal-title":"IEEE J Solid-State Circuits"},{"issue":"1","key":"7481_CR13","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37\u201346","journal-title":"Educ Psychol Meas"},{"issue":"7","key":"7481_CR14","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1002\/jbio.201200202","volume":"7","author":"J Gu","year":"2014","unstructured":"Gu J, Fu CY, Ng BK, Gulam Razul S, Lim SK (2014) Quantitative diagnosis of cervical neoplasia using fluorescence lifetime imaging on haematoxylin and eosin stained tissue sections. J Biophotonics 7(7):483\u2013491","journal-title":"J Biophotonics"},{"key":"7481_CR15","doi-asserted-by":"crossref","unstructured":"Cuenca R, Cheng S, Malik BH, Maitland KC, Ahmed B, Cheng Y-SL, Wright JM, Rees T, Jo JA (2018) Learning methods for fluorescence lifetime imaging (FLIM) based automated detection of early stage oral cancer and dysplasia (conference presentation). In: Optical imaging, therapeutics, and advanced technology in head and neck surgery and otolaryngology 2018, vol 10469, p 104690. International Society for Optics and Photonics","DOI":"10.1117\/12.2288840"},{"key":"7481_CR16","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2020.3010480","author":"M Marsden","year":"2020","unstructured":"Marsden M, Weyers BW, Bec J, Sun T, Gandour-Edwards RF, Birkeland AC, Abouyared M, Bewley AF, Farwell DG, Marcu L (2020) Intraoperative margin assessment in oral and oropharyngeal cancer using label-free fluorescence lifetime imaging and machine learning. Trans Biomed Eng. https:\/\/doi.org\/10.1109\/TBME.2020.3010480","journal-title":"Trans Biomed Eng"},{"key":"7481_CR17","doi-asserted-by":"publisher","unstructured":"Wang Q, Vallejo M, Hopgood J (2020) Fluorescence lifetime endomicroscopic image-based ex-vivo human lung cancer differentiation using machine learning. TechRxiv Preprint. https:\/\/doi.org\/10.36227\/techrxiv.11535708.v1","DOI":"10.36227\/techrxiv.11535708.v1"},{"key":"7481_CR18","doi-asserted-by":"crossref","unstructured":"Wang Q, Hopgood JR, Vallejo M (2021) Fluorescence lifetime imaging endomicroscopy based ex-vivo lung cancer prediction using multi-scale concatenated-dilation convolutional neural networks. In: Medical imaging 2021: computer-aided diagnosis, vol 11597, p 115972. International Society for Optics and Photonics","DOI":"10.1117\/12.2580467"},{"key":"7481_CR19","doi-asserted-by":"crossref","unstructured":"Wang Q, Hopgood JR, Vallejo M (2021) Multi-scale aggregated-dilation network for ex-vivo lung cancer detection with fluorescence lifetime imaging endomicroscopy. In: 2021 43rd annual international conference of the IEEE engineering in medicine & biology society (EMBC), pp 2918\u20132922. IEEE","DOI":"10.1109\/EMBC46164.2021.9630836"},{"issue":"5","key":"7481_CR20","doi-asserted-by":"publisher","first-page":"1252","DOI":"10.1109\/TMI.2016.2548501","volume":"35","author":"P Moeskops","year":"2016","unstructured":"Moeskops P, Viergever MA, Mendrik AM, de Vries LS, Benders MJNL, I\u0161gum I (2016) Automatic segmentation of mr brain images with a convolutional neural network. IEEE Trans Med Imaging 35(5):1252\u20131261. https:\/\/doi.org\/10.1109\/TMI.2016.2548501","journal-title":"IEEE Trans Med Imaging"},{"key":"7481_CR21","doi-asserted-by":"crossref","unstructured":"Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: IEEE conference on computer vision and pattern recognition, pp 2818\u20132826","DOI":"10.1109\/CVPR.2016.308"},{"issue":"5","key":"7481_CR22","doi-asserted-by":"publisher","first-page":"1160","DOI":"10.1109\/TMI.2016.2536809","volume":"35","author":"AAA Setio","year":"2016","unstructured":"Setio AAA, Ciompi F, Litjens G, Gerke P, Jacobs C, van Riel SJ, Wille MMW, Naqibullah M, S\u00e1nchez CI, van Ginneken B (2016) Pulmonary nodule detection in ct images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging 35(5):1160\u20131169. https:\/\/doi.org\/10.1109\/TMI.2016.2536809","journal-title":"IEEE Trans Med Imaging"},{"key":"7481_CR23","unstructured":"Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: 32nd international conference on machine learning, pp 448\u2013456"},{"key":"7481_CR24","doi-asserted-by":"publisher","unstructured":"Mou L, Chen L, Cheng J, Gu Z, Zhao Y, Liu J (2019) Dense dilated network with probability regularized walk for vessel detection. IEEE Transactions on medical imaging, 1. https:\/\/doi.org\/10.1109\/TMI.2019.2950051","DOI":"10.1109\/TMI.2019.2950051"},{"issue":"4","key":"7481_CR25","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","volume":"40","author":"L Chen","year":"2018","unstructured":"Chen L, Papandreou G, Kokkinos IKM et al (2018) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834\u2013848. https:\/\/doi.org\/10.1109\/TPAMI.2017.2699184","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"11","key":"7481_CR26","doi-asserted-by":"publisher","first-page":"2453","DOI":"10.1109\/TMI.2018.2835303","volume":"37","author":"L Chen","year":"2018","unstructured":"Chen L, Bentley P, Mori K, Misawa K, Fujiwara M, Rueckert D (2018) Drinet for medical image segmentation. IEEE Trans Med Imag 37(11):2453\u20132462. https:\/\/doi.org\/10.1109\/TMI.2018.2835303","journal-title":"IEEE Trans Med Imag"},{"key":"7481_CR27","unstructured":"Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp 1097\u20131105"},{"key":"7481_CR28","doi-asserted-by":"crossref","unstructured":"Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: IEEE conference on computer vision and pattern recognition, pp 1251\u20131258","DOI":"10.1109\/CVPR.2017.195"},{"key":"7481_CR29","unstructured":"Tan M, Le QV (2019) MixConv: mixed depthwise convolutional kernels. In: 30th british machine vision conference"},{"key":"7481_CR30","doi-asserted-by":"crossref","unstructured":"Zhang X, Zhou X, Lin M, Sun J (2018) Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: IEEE conference on computer vision and pattern recognition","DOI":"10.1109\/CVPR.2018.00716"},{"key":"7481_CR31","unstructured":"Zhang H, Wu C, Zhang Z, Zhu Y, Zhang Z, Lin H, Sun Y, He T, Mueller J, Manmatha R et al (2020) Resnest: split-attention networks. arXiv preprint arXiv:2004.08955"},{"key":"7481_CR32","unstructured":"Liu M, Yin H (2019) Feature pyramid encoding network for real-time semantic segmentation. In: british machine vision conference"},{"issue":"1","key":"7481_CR33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41467-021-26837-0","volume":"12","author":"GO Williams","year":"2021","unstructured":"Williams GO, Williams E, Finlayson N, Erdogan AT, Wang Q, Fernandes S, Akram AR, Dhaliwal K, Henderson RK, Girkin JM, Bradley M (2021) Full spectrum fluorescence lifetime imaging with 0.5 nm spectral and 50 ps temporal resolution. Nat Commun 12(1):1\u20139","journal-title":"Nat Commun"},{"issue":"1","key":"7481_CR34","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1021\/ac00176a007","volume":"61","author":"RM Ballew","year":"1989","unstructured":"Ballew RM, Demas J (1989) An error analysis of the rapid lifetime determination method for the evaluation of single exponential decays. Anal Chem 61(1):30\u201333","journal-title":"Anal Chem"},{"issue":"2","key":"7481_CR35","doi-asserted-by":"publisher","first-page":"368","DOI":"10.1364\/JOSAA.20.000368","volume":"20","author":"J Philip","year":"2003","unstructured":"Philip J, Carlsson K (2003) Theoretical investigation of the signal-to-noise ratio in fluorescence lifetime imaging. J Opt Soc Am 20(2):368\u2013379","journal-title":"J Opt Soc Am"},{"issue":"2","key":"7481_CR36","doi-asserted-by":"publisher","DOI":"10.1117\/1.jbo.17.2.021105","volume":"17","author":"TN Ford","year":"2012","unstructured":"Ford TN, Lim D, Mertz J (2012) Fast optically sectioned fluorescence HiLo endomicroscopy. J Biomed Opt 17(2):021105. https:\/\/doi.org\/10.1117\/1.jbo.17.2.021105","journal-title":"J Biomed Opt"},{"key":"7481_CR37","volume-title":"Image processing, analysis, and machine vision","author":"M Sonka","year":"2014","unstructured":"Sonka M, Hlavac V, Boyle R (2014) Image processing, analysis, and machine vision. Cengage Learning, Stamford"},{"key":"7481_CR38","unstructured":"Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: 14th international conference on artificial intelligence and statistics, vol 15, pp 315\u2013323"},{"key":"7481_CR39","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: international conference on computer vision, pp 1026\u20131034","DOI":"10.1109\/ICCV.2015.123"},{"key":"7481_CR40","doi-asserted-by":"crossref","unstructured":"Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: Visual explanations from deep networks via gradient-based localization. In: IEEE international conference on computer vision, pp 618\u2013626","DOI":"10.1109\/ICCV.2017.74"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-022-07481-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-022-07481-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-022-07481-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,20]],"date-time":"2022-10-20T21:31:50Z","timestamp":1666301510000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-022-07481-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,25]]},"references-count":40,"journal-issue":{"issue":"21","published-print":{"date-parts":[[2022,11]]}},"alternative-id":["7481"],"URL":"https:\/\/doi.org\/10.1007\/s00521-022-07481-1","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,25]]},"assertion":[{"value":"7 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 May 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 June 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}