{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T22:49:44Z","timestamp":1776206984460,"version":"3.50.1"},"reference-count":20,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2021,4,28]],"date-time":"2021-04-28T00:00:00Z","timestamp":1619568000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,28]],"date-time":"2021-04-28T00:00:00Z","timestamp":1619568000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["813782"],"award-info":[{"award-number":["813782"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"published-print":{"date-parts":[[2021,6]]},"abstract":"Abstract<\/jats:title>\n Purpose<\/jats:title>\n Ureteroscopy is an efficient endoscopic minimally invasive technique for the diagnosis and treatment of upper tract urothelial carcinoma. During ureteroscopy, the automatic segmentation of the hollow lumen is of primary importance, since it indicates the path that the endoscope should follow. In order to obtain an accurate segmentation of the hollow lumen, this paper presents an automatic method based on convolutional neural networks (CNNs).<\/jats:p>\n <\/jats:sec>\n Methods<\/jats:title>\n The proposed method is based on an ensemble of 4 parallel CNNs to simultaneously process single and multi-frame information. Of these, two architectures are taken as core-models, namely U-Net based in residual blocks ($$m_1$$<\/jats:tex-math>\n \n m<\/mml:mi>\n 1<\/mml:mn>\n <\/mml:msub>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>) and Mask-RCNN ($$m_2$$<\/jats:tex-math>\n \n m<\/mml:mi>\n 2<\/mml:mn>\n <\/mml:msub>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>), which are fed with single still-frames I<\/jats:italic>(t<\/jats:italic>). The other two models ($$M_1$$<\/jats:tex-math>\n \n M<\/mml:mi>\n 1<\/mml:mn>\n <\/mml:msub>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>, $$M_2$$<\/jats:tex-math>\n \n M<\/mml:mi>\n 2<\/mml:mn>\n <\/mml:msub>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>) are modifications of the former ones consisting on the addition of a stage which makes use of 3D convolutions to process temporal information. $$M_1$$<\/jats:tex-math>\n \n M<\/mml:mi>\n 1<\/mml:mn>\n <\/mml:msub>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>, $$M_2$$<\/jats:tex-math>\n \n M<\/mml:mi>\n 2<\/mml:mn>\n <\/mml:msub>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula> are fed with triplets of frames ($$I(t-1)$$<\/jats:tex-math>\n \n I<\/mml:mi>\n (<\/mml:mo>\n t<\/mml:mi>\n -<\/mml:mo>\n 1<\/mml:mn>\n )<\/mml:mo>\n <\/mml:mrow>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>, I<\/jats:italic>(t<\/jats:italic>), $$I(t+1)$$<\/jats:tex-math>\n \n I<\/mml:mi>\n (<\/mml:mo>\n t<\/mml:mi>\n +<\/mml:mo>\n 1<\/mml:mn>\n )<\/mml:mo>\n <\/mml:mrow>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>) to produce the segmentation for I<\/jats:italic>(t<\/jats:italic>).<\/jats:p>\n <\/jats:sec>\n Results<\/jats:title>\n The proposed method was evaluated using a custom dataset of 11 videos (2673 frames) which were collected and manually annotated from 6 patients. We obtain a Dice similarity coefficient of 0.80, outperforming previous state-of-the-art methods.<\/jats:p>\n <\/jats:sec>\n Conclusion<\/jats:title>\n The obtained results show that spatial-temporal information can be effectively exploited by the ensemble model to improve hollow lumen segmentation in ureteroscopic images. The method is effective also in the presence of poor visibility, occasional bleeding, or specular reflections.<\/jats:p>\n <\/jats:sec>","DOI":"10.1007\/s11548-021-02376-3","type":"journal-article","created":{"date-parts":[[2021,4,28]],"date-time":"2021-04-28T09:03:39Z","timestamp":1619600619000},"page":"915-922","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Using spatial-temporal ensembles of convolutional neural networks for lumen segmentation in ureteroscopy"],"prefix":"10.1007","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5508-0747","authenticated-orcid":false,"given":"Jorge F.","family":"Lazo","sequence":"first","affiliation":[]},{"given":"Aldo","family":"Marzullo","sequence":"additional","affiliation":[]},{"given":"Sara","family":"Moccia","sequence":"additional","affiliation":[]},{"given":"Michele","family":"Catellani","sequence":"additional","affiliation":[]},{"given":"Benoit","family":"Rosa","sequence":"additional","affiliation":[]},{"given":"Michel","family":"de Mathelin","sequence":"additional","affiliation":[]},{"given":"Elena","family":"De Momi","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,4,28]]},"reference":[{"issue":"1","key":"2376_CR1","doi-asserted-by":"publisher","first-page":"7","DOI":"10.3322\/caac.21601","volume":"70","author":"RL Siegel","year":"2020","unstructured":"Siegel RL, Miller KD, Jemal A (2020) Cancer statistics, 2020. CA A Cancer J Clin 70(1):7\u201330. https:\/\/doi.org\/10.3322\/caac.21601","journal-title":"CA A Cancer J Clin"},{"issue":"1","key":"2376_CR2","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1007\/s00345-012-0877-2","volume":"31","author":"M Cosentino","year":"2013","unstructured":"Cosentino M, Palou J, Gaya JM, Breda A, Rodriguez-Faba O, Villavicencio-Mavrich H (2013) Upper urinary tract urothelial cell carcinoma: location as a predictive factor for concomitant bladder carcinoma. World J Urol 31(1):141\u2013145. https:\/\/doi.org\/10.1007\/s00345-012-0877-2","journal-title":"World J Urol"},{"key":"2376_CR3","doi-asserted-by":"publisher","unstructured":"Rojas CP, Castle SM, Llanos CA, Cortes JAS, Bird V, Rodriguez S,Reis IM, Zhao W, Gomez-Fernandez C, Leveillee RJL, and Jorda M (2013) Low biopsy volume in ureteroscopy does not affect tumor biopsy grading in upper tract urothelial carcinoma. In: Urologic oncology: seminars and original investigations, vol. 31, Elsevier, pp 1696\u20131700. https:\/\/doi.org\/10.1016\/j.urolonc.2012.05.010","DOI":"10.1016\/j.urolonc.2012.05.010"},{"key":"2376_CR4","unstructured":"Wason SE, Leslie SW (2020) Ureteroscopy StatPearls. https:\/\/pubmed.ncbi.nlm.nih.gov\/32809391\/. Accessed 29 Nov 2020"},{"issue":"5","key":"2376_CR5","doi-asserted-by":"publisher","first-page":"991","DOI":"10.1016\/j.eururo.2006.02.033","volume":"50","author":"JJ de la Rosette","year":"2006","unstructured":"de la Rosette JJ, Skrekas T, Segura JW (2006) Handling and prevention of complications in stone basketing. Eur Urol 50(5):991\u2013999. https:\/\/doi.org\/10.1016\/j.eururo.2006.02.033","journal-title":"Eur Urol"},{"issue":"1","key":"2376_CR6","doi-asserted-by":"publisher","first-page":"1323","DOI":"10.1007\/s11042-016-4219-z","volume":"77","author":"B M\u00fcnzer","year":"2018","unstructured":"M\u00fcnzer B, Schoeffmann K, B\u00f6sz\u00f6rmenyi L (2018) Content-based processing and analysis of endoscopic images and videos: a survey. Multim Tools Appl 77(1):1323\u20131362. https:\/\/doi.org\/10.1007\/s11042-016-4219-z","journal-title":"Multim Tools Appl"},{"key":"2376_CR7","unstructured":"Borghesan G, Trauzettel F, Ansar MHD, Barata BF, Wu D, Li Z, Lazo JF, Finocchiaro M, Xuan TH, Lai C-F, Ramesh S, Sahu SK, Sestini L, Guiqiu L, Pore A (2020) ATLAS: autonomous intraluminal surgery\u2014system specifications for targeted intraluminal interventions. https:\/\/atlas-itn.eu\/d102_main\/. Accessed 10 Dec 2020"},{"key":"2376_CR8","doi-asserted-by":"publisher","unstructured":"V\u00e1zquez D, Bernal J, S\u00e1nchez FJ, Fern\u00e1ndez-Esparrach G, L\u00f3pez AM, Romero A, Drozdzal M, Courville A (2017) A benchmark for endoluminal scene segmentation of colonoscopy images. J Healthc Eng. https:\/\/doi.org\/10.1155\/2017\/4037190","DOI":"10.1155\/2017\/4037190"},{"key":"2376_CR9","unstructured":"Lazo JF, Marzullo A, Moccia S, Cattellani M, Rosa B, Calimeri F, de Mathelin M, De Momi E(2020) A lumen segmentation method in ureteroscopy images based on a deep residual u-net architecture. In: International conference on pattern recognition (ICPR)"},{"issue":"3","key":"2376_CR10","doi-asserted-by":"publisher","first-page":"2714","DOI":"10.1109\/LRA.2019.2917163","volume":"4","author":"E Colleoni","year":"2019","unstructured":"Colleoni E, Moccia S, Du X, De Momi E, Stoyanov D (2019) Deep learning based robotic tool detection and articulation estimation with spatio-temporal layers. IEEE Robot Autom Lett 4(3):2714\u20132721. https:\/\/doi.org\/10.1109\/LRA.2019.2917163","journal-title":"IEEE Robot Autom Lett"},{"key":"2376_CR11","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2019.2961448","author":"S Moccia","year":"2019","unstructured":"Moccia S, Migliorelli L, Carnielli V, Frontoni E (2019) Preterm infants\u2019 pose estimation with spatio-temporal features. IEEE Trans Biomed Eng. https:\/\/doi.org\/10.1109\/TBME.2019.2961448","journal-title":"IEEE Trans Biomed Eng"},{"key":"2376_CR12","doi-asserted-by":"publisher","DOI":"10.1109\/TCDS.2020.3035572","author":"X Wang","year":"2020","unstructured":"Wang X, Yang S, Lan J, Fang Y, He J, Wang M, Zhang J, Han X (2020) Automatic segmentation of pneumothorax in chest radiographs based on a two-stage deep learning method. IEEE Trans Cognit Dev Syst. https:\/\/doi.org\/10.1109\/TCDS.2020.3035572","journal-title":"IEEE Trans Cognit Dev Syst"},{"key":"2376_CR13","doi-asserted-by":"publisher","first-page":"5909","DOI":"10.1609\/aaai.v33i01.33015909","volume":"33","author":"H Zheng","year":"2019","unstructured":"Zheng H, Zhang Y, Yang L, Liang P, Zhao Z, Wang C, Chen DZ (2019) A new ensemble learning framework for 3d biomedical image segmentation. Proc AAAI Conf Artif Intell 33:5909\u20135916. https:\/\/doi.org\/10.1609\/aaai.v33i01.33015909","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"2376_CR14","doi-asserted-by":"crossref","unstructured":"Vuola AO, Akram SU, Kannala J (2019) Mask-RCNN and U-net ensembled for nuclei segmentation. In: 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019). IEEE, pp 208\u2013212","DOI":"10.1109\/ISBI.2019.8759574"},{"key":"2376_CR15","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770\u2013778. https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"2376_CR16","doi-asserted-by":"publisher","unstructured":"He K, Gkioxari G, Doll\u00e1r P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp. 2980\u20132988. https:\/\/doi.org\/10.1109\/ICCV.2017.322","DOI":"10.1109\/ICCV.2017.322"},{"key":"2376_CR17","doi-asserted-by":"publisher","first-page":"17039","DOI":"10.1109\/ACCESS.2020.2967474","volume":"8","author":"Y He","year":"2020","unstructured":"He Y, Zhang P, Qi X, Zhao B, Li S, Hu Y (2020) Endoscopic path planning in robot-assisted endoscopic nasal surgery. IEEE Access 8:17039\u201317048. https:\/\/doi.org\/10.1109\/ACCESS.2020.2967474","journal-title":"IEEE Access"},{"issue":"14","key":"2376_CR18","doi-asserted-by":"publisher","first-page":"8098","DOI":"10.1109\/JSEN.2020.2979513","volume":"20","author":"FN Alsunaydih","year":"2020","unstructured":"Alsunaydih FN, Arefin MS, Redoute J-M, Yuce MR (2020) A navigation and pressure monitoring system toward autonomous wireless capsule endoscopy. IEEE Sens J 20(14):8098\u20138107. https:\/\/doi.org\/10.1109\/JSEN.2020.2979513","journal-title":"IEEE Sens J"},{"key":"2376_CR19","doi-asserted-by":"crossref","unstructured":"Blasch E, Liu S, Liu Z, Zheng Y (2018) Deep learning measures of effectiveness. In: NAECON 2018-IEEE national aerospace and electronics conference. IEEE, pp. 254\u2013261","DOI":"10.1109\/NAECON.2018.8556808"},{"issue":"9","key":"2376_CR20","first-page":"5471","volume":"60","author":"Y Xie","year":"2019","unstructured":"Xie Y, Nguyen Q, Bellemo V, Yip MY, Lee XQ, Hamzah H, Lim G, Hsu W, Lee ML, Wang JJ et al (2019) Cost-effectiveness analysis of an artificial intelligence-assisted deep learning system implemented in the national tele-medicine diabetic retinopathy screening in singapore. Invest Ophthalmol Vis Sci 60(9):5471","journal-title":"Invest Ophthalmol Vis Sci"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-021-02376-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-021-02376-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-021-02376-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,31]],"date-time":"2021-05-31T10:53:02Z","timestamp":1622458382000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-021-02376-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,28]]},"references-count":20,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,6]]}},"alternative-id":["2376"],"URL":"https:\/\/doi.org\/10.1007\/s11548-021-02376-3","relation":{},"ISSN":["1861-6410","1861-6429"],"issn-type":[{"value":"1861-6410","type":"print"},{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,28]]},"assertion":[{"value":"11 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 April 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 April 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The collection of data was in accordance with the ethical standards of the Istituto Europeo di Oncologia and with the 1964 Helsinki Declaration, revised in 2000. All the subjects involved in this research were informed and agreed to data treatment before the intervention.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Written informed consent was obtained from all patients included in the study.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}}]}}