{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T02:52:35Z","timestamp":1772247155677,"version":"3.50.1"},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2020,4,22]],"date-time":"2020-04-22T00:00:00Z","timestamp":1587513600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,4,22]],"date-time":"2020-04-22T00:00:00Z","timestamp":1587513600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000272","name":"National Institute for Health Research","doi-asserted-by":"crossref","award":["II-LB-0716-20002"],"award-info":[{"award-number":["II-LB-0716-20002"]}],"id":[{"id":"10.13039\/501100000272","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["714562"],"award-info":[{"award-number":["714562"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]},{"name":"The Michael Uren Foundation"},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/P012841\/1"],"award-info":[{"award-number":["EP\/P012841\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"name":"RAEng Chair","award":["CiET1819\\2\\36"],"award-info":[{"award-number":["CiET1819\\2\\36"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"published-print":{"date-parts":[[2020,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Purpose<\/jats:title><jats:p>Sustained delivery of regenerative retinal therapies by robotic systems requires intra-operative tracking of the retinal fundus. We propose a supervised deep convolutional neural network to densely predict semantic segmentation and optical flow of the retina as mutually supportive tasks, implicitly inpainting retinal flow information missing due to occlusion by surgical tools.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>As manual annotation of optical flow is infeasible, we propose a flexible algorithm for generation of large synthetic training datasets on the basis of given intra-operative retinal images. We evaluate optical flow estimation by tracking a grid and sparsely annotated ground truth points on a benchmark of challenging real intra-operative clips obtained from an extensive internally acquired dataset encompassing representative vitreoretinal surgical cases.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The U-Net-based network trained on the synthetic dataset is shown to generalise well to the benchmark of real surgical videos. When used to track retinal points of interest, our flow estimation outperforms variational baseline methods on clips containing tool motions which occlude the points of interest, as is routinely observed in intra-operatively recorded surgery videos.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>The results indicate that complex synthetic training datasets can be used to specifically guide optical flow estimation. Our proposed algorithm therefore lays the foundation for a robust system which can assist with intra-operative tracking of moving surgical targets even when occluded.<\/jats:p><\/jats:sec>","DOI":"10.1007\/s11548-020-02160-9","type":"journal-article","created":{"date-parts":[[2020,4,22]],"date-time":"2020-04-22T15:04:07Z","timestamp":1587567847000},"page":"827-836","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Learned optical flow for intra-operative tracking of the retinal fundus"],"prefix":"10.1007","volume":"15","author":[{"given":"Claudio S.","family":"Ravasio","sequence":"first","affiliation":[]},{"given":"Theodoros","family":"Pissas","sequence":"additional","affiliation":[]},{"given":"Edward","family":"Bloch","sequence":"additional","affiliation":[]},{"given":"Blanca","family":"Flores","sequence":"additional","affiliation":[]},{"given":"Sepehr","family":"Jalali","sequence":"additional","affiliation":[]},{"given":"Danail","family":"Stoyanov","sequence":"additional","affiliation":[]},{"given":"Jorge M.","family":"Cardoso","sequence":"additional","affiliation":[]},{"given":"Lyndon","family":"Da Cruz","sequence":"additional","affiliation":[]},{"given":"Christos","family":"Bergeles","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,4,22]]},"reference":[{"issue":"1","key":"2160_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11263-010-0390-2","volume":"92","author":"S Baker","year":"2011","unstructured":"Baker S, Scharstein D, Lewis JP, Roth S, Black MJ, Szeliski R (2011) A database and evaluation methodology for optical flow. Int J Comput Vis 92(1):1\u201331","journal-title":"Int J Comput Vis"},{"issue":"2","key":"2160_CR2","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1007\/BF00128153","volume":"6","author":"R Battiti","year":"1991","unstructured":"Battiti R, Amaldi E, Koch C (1991) Computing optical flow across multiple scales: an adaptive coarse-to-fine strategy. Int J Comput Vis 6(2):133\u2013145","journal-title":"Int J Comput Vis"},{"key":"2160_CR3","doi-asserted-by":"crossref","unstructured":"Black MJ, Yacoob Y, Jepson AD, Fleet DJ (1997) Learning parameterized models of image motion. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 561\u2013567","DOI":"10.1109\/CVPR.1997.609381"},{"issue":"1","key":"2160_CR4","doi-asserted-by":"publisher","first-page":"e1848","DOI":"10.1002\/rcs.1848","volume":"14","author":"D Braun","year":"2018","unstructured":"Braun D, Yang S, Martel JN, Riviere CN, Becker BC (2018) Eyeslam: real-time simultaneous localization and mapping of retinal vessels during intraocular microsurgery. Int J Med Robot Comput Assist Surg 14(1):e1848","journal-title":"Int J Med Robot Comput Assist Surg"},{"key":"2160_CR5","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1038\/nbt.4114","volume":"36","author":"L Da Cruz","year":"2018","unstructured":"Da Cruz L, Fynes K, Georgiadis O, Kerby J, Luo Y, Ahmado A, Vernon A, Daniels J, Nommiste B, Hasan S, Gooljar S, Carr AJ, Vugler A, Ramsden C, Bictash M, Fenster M, Steer J, Harbinson T, Wilbrey A, Tufail A, Feng G, Whitlock M, Robson A, Holder G, Sagoo M, Loudon P, Whiting P, Coffey P (2018) Phase 1 clinical study of an embryonic stem cell-derived retinal pigment epithelium patch in age-related macular degeneration. Nat Biotechnol 36:328","journal-title":"Nat Biotechnol"},{"key":"2160_CR6","doi-asserted-by":"crossref","unstructured":"Dosovitskiy A, Fischer P, Ilg E, Hausser P, Hazirbas C, Golkov V, Van Der\u00a0Smagt P, Cremers D, Brox T (2015) Flownet: Learning optical flow with convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 2758\u20132766","DOI":"10.1109\/ICCV.2015.316"},{"key":"2160_CR7","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1007\/3-540-45103-X_50","volume-title":"Image analysis","author":"G Farneb\u00e4ck","year":"2003","unstructured":"Farneb\u00e4ck G (2003) Two-frame motion estimation based on polynomial expansion. In: Bigun J, Gustavsson T (eds) Image analysis. Springer, Berlin, pp 363\u2013370"},{"key":"2160_CR8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.cviu.2015.02.008","volume":"134","author":"D Fortun","year":"2015","unstructured":"Fortun D, Bouthemy P, Kervrann C (2015) Optical flow modeling and computation: a survey. Comput Vis Image Underst 134:1\u201321","journal-title":"Comput Vis Image Underst"},{"key":"2160_CR9","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1016\/j.cviu.2015.11.020","volume":"145","author":"D Fortun","year":"2016","unstructured":"Fortun D, Bouthemy P, Kervrann C (2016) Aggregation of local parametric candidates with exemplar-based occlusion handling for optical flow. Comput Vis Image Underst 145:81\u201394","journal-title":"Comput Vis Image Underst"},{"key":"2160_CR10","doi-asserted-by":"crossref","unstructured":"Guerre A, Lamard M, Conze P, Cochener B, Quellec G (2018) Optical flow estimation in ocular endoscopy videos using flownet on simulated endoscopy data. In: IEEE international symposium on biomedical imaging, pp 1463\u20131466","DOI":"10.1109\/ISBI.2018.8363848"},{"issue":"1\u20133","key":"2160_CR11","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1016\/0004-3702(81)90024-2","volume":"17","author":"BK Horn","year":"1981","unstructured":"Horn BK, Schunck BG (1981) Determining optical flow. Artif Intell 17(1\u20133):185\u2013203","journal-title":"Artif Intell"},{"key":"2160_CR12","unstructured":"Ilg E, Mayer N, Saikia T, Keuper M, Dosovitskiy A, Brox T (2017) Flownet 2.0: evolution of optical flow estimation with deep networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1647\u20131655"},{"issue":"5","key":"2160_CR13","doi-asserted-by":"publisher","first-page":"978","DOI":"10.1109\/TPAMI.2010.147","volume":"33","author":"C Liu","year":"2010","unstructured":"Liu C, Yuen J, Torralba A (2010) Sift flow: dense correspondence across scenes and its applications. Trans Pattern Anal Mach Intell 33(5):978\u2013994","journal-title":"Trans Pattern Anal Mach Intell"},{"issue":"1183","key":"2160_CR14","first-page":"151","volume":"211","author":"D Marr","year":"1981","unstructured":"Marr D, Ullman S, Brenner S (1981) Directional selectivity and its use in early visual processing. Proc R Soc Lond 211(1183):151\u2013180","journal-title":"Proc R Soc Lond"},{"issue":"9","key":"2160_CR15","doi-asserted-by":"publisher","first-page":"942","DOI":"10.1007\/s11263-018-1082-6","volume":"126","author":"N Mayer","year":"2018","unstructured":"Mayer N, Ilg E, Fischer P, Hazirbas C, Cremers D, Dosovitskiy A, Brox T (2018) What makes good synthetic training data for learning disparity & optical flow estimation? Int J Comput Vis 126(9):942\u2013960","journal-title":"Int J Comput Vis"},{"issue":"5","key":"2160_CR16","doi-asserted-by":"publisher","first-page":"819","DOI":"10.1007\/s11548-019-01925-1","volume":"14","author":"S Mukherjee","year":"2019","unstructured":"Mukherjee S, Kaess M, Martel JN, Riviere CN (2019) Eyesam: graph-based localization and mapping of retinal vasculature during intraocular microsurgery. Int J Comput Assist Radiol Surg 14(5):819\u2013828","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"2","key":"2160_CR17","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1023\/A:1013539930159","volume":"46","author":"E M\u00e9min","year":"2002","unstructured":"M\u00e9min E, P\u00e9rez P (2002) Hierarchical estimation and segmentation of dense motion fields. Int J Comput Vis 46(2):129\u2013155","journal-title":"Int J Comput Vis"},{"key":"2160_CR18","doi-asserted-by":"crossref","unstructured":"Nah S, Hyun\u00a0Kim T, Mu\u00a0Lee K (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE conference on computer vision and pattern recognition","DOI":"10.1109\/CVPR.2017.35"},{"key":"2160_CR19","doi-asserted-by":"crossref","unstructured":"Ranjan A, Black MJ (2017) Optical flow estimation using a spatial pyramid network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4161\u20134170","DOI":"10.1109\/CVPR.2017.291"},{"key":"2160_CR20","doi-asserted-by":"crossref","unstructured":"Ren Z, Yan J, Ni B, Liu B, Yang X, Zha H (2017) Unsupervised deep learning for optical flow estimation. In: AAAI conference on artificial intelligence","DOI":"10.1609\/aaai.v31i1.10723"},{"key":"2160_CR21","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1007\/978-3-642-33415-3_49","volume-title":"Medical Image Computing and Computer-Assisted Intervention \u2013 MICCAI 2012","author":"Rog\u00e9rio Richa","year":"2012","unstructured":"Richa R, V\u00e1gv\u00f6lgyi B, Balicki M, Hager G, Taylor RH (2012) Hybrid tracking and mosaicking for information augmentation in retinal surgery. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 397\u2013404"},{"issue":"4","key":"2160_CR22","doi-asserted-by":"publisher","first-page":"640","DOI":"10.1109\/TPAMI.2016.2572683","volume":"39","author":"E Shelhamer","year":"2017","unstructured":"Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. Trans Pattern Anal Mach Intell 39(4):640\u2013651","journal-title":"Trans Pattern Anal Mach Intell"},{"key":"2160_CR23","unstructured":"Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M (2014) Striving for simplicity: the all convolutional net. CoRR"},{"key":"2160_CR24","doi-asserted-by":"crossref","unstructured":"Sun D, Yang X, Liu MY, Kautz J (2018) PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8934\u201343","DOI":"10.1109\/CVPR.2018.00931"},{"key":"2160_CR25","unstructured":"Terzopoulos D (1984) Efficient multiresolution algorithms for computing lightness, shape-from-shading, and optical flow. In: AAAI conference artificial intelligence, pp 314\u2013343"},{"issue":"2","key":"2160_CR26","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1109\/42.700738","volume":"17","author":"YA Tolias","year":"1998","unstructured":"Tolias YA, Panas SM (1998) A fuzzy vessel tracking algorithm for retinal images based on fuzzy clustering. IEEE Trans Med Imag 17(2):263\u2013273","journal-title":"IEEE Trans Med Imag"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-020-02160-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-020-02160-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-020-02160-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,22]],"date-time":"2022-10-22T01:49:32Z","timestamp":1666403372000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-020-02160-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,22]]},"references-count":26,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2020,5]]}},"alternative-id":["2160"],"URL":"https:\/\/doi.org\/10.1007\/s11548-020-02160-9","relation":{},"ISSN":["1861-6410","1861-6429"],"issn-type":[{"value":"1861-6410","type":"print"},{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,22]]},"assertion":[{"value":"16 November 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 April 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 April 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Human participant data were acquired in accordance with the ethical standards of the institutional research committee, and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"was obtained from all individual participants in the study.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}}]}}