{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,4]],"date-time":"2025-12-04T09:59:22Z","timestamp":1764842362120,"version":"3.37.3"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2020,5,22]],"date-time":"2020-05-22T00:00:00Z","timestamp":1590105600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,5,22]],"date-time":"2020-05-22T00:00:00Z","timestamp":1590105600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Forschungszentrum Medizintechnik Hamburg","award":["04fmthh16"],"award-info":[{"award-number":["04fmthh16"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"published-print":{"date-parts":[[2020,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n<jats:title>Purpose<\/jats:title>\n<jats:p>Localizing structures and estimating the motion of a specific target region are common problems for navigation during surgical interventions. Optical coherence tomography (OCT) is an imaging modality with a high spatial and temporal resolution that has been used for intraoperative imaging and also for motion estimation, for example, in the context of ophthalmic surgery or cochleostomy. Recently, motion estimation between a template and a moving OCT image has been studied with deep learning methods to overcome the shortcomings of conventional, feature-based methods.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Methods<\/jats:title>\n<jats:p>We investigate whether using a temporal stream of OCT image volumes can improve deep learning-based motion estimation performance. For this purpose, we design and evaluate several 3D and 4D deep learning methods and we propose a new deep learning approach. Also, we propose a temporal regularization strategy at the model output.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Results<\/jats:title>\n<jats:p>Using a tissue dataset without additional markers, our deep learning methods using 4D data outperform previous approaches. The best performing 4D architecture achieves an correlation coefficient (aCC) of 98.58% compared to 85.0% of a previous 3D deep learning method. Also, our temporal regularization strategy at the output further improves 4D model performance to an aCC of 99.06%. In particular, our 4D method works well for larger motion and is robust toward image rotations and motion distortions.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Conclusions<\/jats:title>\n<jats:p>We propose 4D spatio-temporal deep learning for OCT-based motion estimation. On a tissue dataset, we find that using 4D information for the model input improves performance while maintaining reasonable inference times. Our regularization strategy demonstrates that additional temporal information is also beneficial at the model output.<\/jats:p>\n<\/jats:sec>","DOI":"10.1007\/s11548-020-02178-z","type":"journal-article","created":{"date-parts":[[2020,5,22]],"date-time":"2020-05-22T14:02:54Z","timestamp":1590156174000},"page":"943-952","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Spatio-temporal deep learning methods for motion estimation using 4D OCT image data"],"prefix":"10.1007","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2229-9547","authenticated-orcid":false,"given":"Marcel","family":"Bengs","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nils","family":"Gessert","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthias","family":"Schl\u00fcter","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexander","family":"Schlaefer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,5,22]]},"reference":[{"key":"2178_CR1","doi-asserted-by":"crossref","unstructured":"Bay H, Tuytelaars T, Van\u00a0Gool L (2006) Surf: speeded up robust features. In: ECCV, Springer, pp 404\u2013417","DOI":"10.1007\/11744023_32"},{"key":"2178_CR2","unstructured":"Bengs M, Gessert N, Schlaefer A (2019) 4d spatio-temporal deep learning with 4d fmri data for autism spectrum disorder classification. In: International conference on medical imaging with deep learning"},{"issue":"8","key":"2178_CR3","doi-asserted-by":"publisher","first-page":"1425","DOI":"10.1007\/s11548-017-1594-5","volume":"12","author":"J Bergmeier","year":"2017","unstructured":"Bergmeier J, Fitzpatrick JM, Daentzer D, Majdani O, Ortmaier T, Kahrs LA (2017) Workflow and simulation of image-to-physical registration of holes inside spongy bone. Int J Comput Assist Radiol Surg 12(8):1425\u20131437","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"2178_CR4","doi-asserted-by":"crossref","unstructured":"Clark D, Badea C (2019) Convolutional regularization methods for 4d, X-ray ct reconstruction. In: Medical imaging 2019: physics of medical imaging, International society for optics and photonics. vol 10948, p 109482A","DOI":"10.1117\/12.2512816"},{"issue":"1","key":"2178_CR5","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1002\/rcs.1462","volume":"9","author":"X Du","year":"2013","unstructured":"Du X, Assadi MZ, Jowitt F, Brett PN, Henshaw S, Dalton J, Proops DW, Coulson CJ, Reid AP (2013) Robustness analysis of a smart surgical drill for cochleostomy. Int J Med Robotics Comput Assist Surg 9(1):119\u2013126","journal-title":"Int J Med Robotics Comput Assist Surg"},{"issue":"8","key":"2178_CR6","doi-asserted-by":"publisher","first-page":"e105224","DOI":"10.1371\/journal.pone.0105224","volume":"9","author":"JP Ehlers","year":"2014","unstructured":"Ehlers JP, Srivastava SK, Feiler D, Noonan AI, Rollins AM, Tao YK (2014) Integrative advances for OCT-guided ophthalmic surgery and intraoperative OCT: microscope integration, surgical instrumentation, and heads-up display surgeon feedback. PLoS ONE 9(8):e105224","journal-title":"PLoS ONE"},{"key":"2178_CR7","doi-asserted-by":"crossref","unstructured":"Eilers H, Baron S, Ortmaier T, Heimann B, Baier C, Rau TS, Leinung M, Majdani O (2009) Navigated, robot assisted drilling of a minimally invasive cochlear access. In: 2009 IEEE international conference on mechatronics, IEEE, pp 1\u20136","DOI":"10.1109\/ICMECH.2009.4957213"},{"issue":"3","key":"2178_CR8","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1002\/rcs.1425","volume":"8","author":"M Finke","year":"2012","unstructured":"Finke M, Kantelhardt S, Schlaefer A, Bruder R, Lankenau E, Giese A, Schweikard A (2012) Automatic scanning of large tissue areas in neurosurgery using optical coherence tomography. Int J Med Robotics Comput Assist Surg 8(3):327\u2013336","journal-title":"Int J Med Robotics Comput Assist Surg"},{"key":"2178_CR9","doi-asserted-by":"crossref","unstructured":"Gessert N, Gromniak M, Schl\u00fcter M, Schlaefer A (2019) Two-path 3d cnns for calibration of system parameters for OCT-based motion compensation. In: Medical imaging 2019: image-guided procedures, robotic interventions, and modeling. International society for optics and photonics, vol 10951, p 1095108","DOI":"10.1117\/12.2512823"},{"key":"2178_CR10","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1016\/j.media.2018.03.002","volume":"46","author":"N Gessert","year":"2018","unstructured":"Gessert N, Schl\u00fcter M, Schlaefer A (2018) A deep learning approach for pose estimation from volumetric OCT data. Med Image Anal 46:162\u2013179","journal-title":"Med Image Anal"},{"key":"2178_CR11","doi-asserted-by":"crossref","unstructured":"Huang G, Liu Z, Van Der\u00a0Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: CVPR, pp 4700\u20134708","DOI":"10.1109\/CVPR.2017.243"},{"key":"2178_CR12","doi-asserted-by":"crossref","unstructured":"Ilg E, Mayer N, Saikia T, Keuper M, Dosovitskiy A, Brox T (2017) Flownet 2.0: evolution of optical flow estimation with deep networks. In: CVPR, pp 2462\u20132470","DOI":"10.1109\/CVPR.2017.179"},{"key":"2178_CR13","doi-asserted-by":"crossref","unstructured":"Irsch K, Lee S, Bose SN, Kang JU (2018) Motion-compensated optical coherence tomography using envelope-based surface detection and kalman-based prediction. In: Advanced biomedical and clinical diagnostic and surgical guidance systems XVI,. International society for optics and photonics, vol 10484, p 104840Q","DOI":"10.1117\/12.2288545"},{"issue":"6","key":"2178_CR14","doi-asserted-by":"publisher","first-page":"1182","DOI":"10.1364\/BOE.3.001182","volume":"3","author":"MF Kraus","year":"2012","unstructured":"Kraus MF, Potsaid B, Mayer MA, Bock R, Baumann B, Liu JJ, Hornegger J, Fujimoto JG (2012) Motion correction in optical coherence tomography volumes on a per a-scan basis using orthogonal scan patterns. Biomed Opt Express 3(6):1182\u20131199","journal-title":"Biomed Opt Express"},{"key":"2178_CR15","doi-asserted-by":"crossref","unstructured":"Lankenau E, Klinger D, Winter C, Malik A, M\u00fcller HH, Oelckers S, Pau HW, Just T, H\u00fcttmann G (2007) Combining optical coherence tomography (OCT) with an operating microscope. In: Advances in medical engineering, Springer, pp 343\u2013348","DOI":"10.1007\/978-3-540-68764-1_57"},{"key":"2178_CR16","doi-asserted-by":"crossref","unstructured":"Laves MH, Ihler S, Kahrs LA, Ortmaier T (2019) Deep-learning-based 2.5 d flow field estimation for maximum intensity projections of 4d optical coherence tomography. In: Medical imaging 2019: image-guided procedures, robotic interventions, and modeling. International society for optics and photonics, vol 10951, p 109510R","DOI":"10.1117\/12.2512952"},{"key":"2178_CR17","doi-asserted-by":"crossref","unstructured":"Laves MH, Schoob A, Kahrs LA, Pfeiffer T, Huber R, Ortmaier T (2017) Feature tracking for automated volume of interest stabilization on 4d-oct images. In: Medical imaging 2017: image-guided procedures, robotic interventions, and modeling . International society for optics and photonics, vol 10135, p 101350W","DOI":"10.1117\/12.2255090"},{"key":"2178_CR18","doi-asserted-by":"crossref","unstructured":"Lowe DG (1999) Object recognition from local scale-invariant features. In: Proceedings of the seventh IEEE international conference on computer vision, vol. 2. IEEE, pp 1150\u20131157","DOI":"10.1109\/ICCV.1999.790410"},{"issue":"9","key":"2178_CR19","doi-asserted-by":"publisher","first-page":"641","DOI":"10.1055\/s-2007-995725","volume":"87","author":"H Pau","year":"2008","unstructured":"Pau H, Lankenau E, Just T, H\u00fcttmann G (2008) Imaging of cochlear structures by optical coherence tomography (OCT). Temporal bone experiments for an OCT-guided cochleostomy technique. Laryngorhinootologie 87(9):641\u2013646","journal-title":"Laryngorhinootologie"},{"key":"2178_CR20","doi-asserted-by":"crossref","unstructured":"Schl\u00fcter M, Otte C, Saathoff T, Gessert N, Schlaefer A (2019) Feasibility of a markerless tracking system based on optical coherence tomography. In: Medical imaging 2019: image-guided procedures, robotic interventions, and modeling . International society for optics and photonics, vol 10951, p 1095107","DOI":"10.1117\/12.2512178"},{"issue":"2","key":"2178_CR21","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1038\/s41566-017-0088-x","volume":"12","author":"M Siddiqui","year":"2018","unstructured":"Siddiqui M, Nam AS, Tozburun S, Lippok N, Blatter C, Vakoc BJ (2018) High-speed optical coherence tomography by circular interferometric ranging. Nat Photonics 12(2):111","journal-title":"Nat Photonics"},{"key":"2178_CR22","doi-asserted-by":"crossref","unstructured":"Tran D, Bourdev L, Fergus R, Torresani L, Paluri M (2015) Learning spatiotemporal features with 3d convolutional networks. In: ICCV, pp 4489\u20134497","DOI":"10.1109\/ICCV.2015.510"},{"key":"2178_CR23","doi-asserted-by":"crossref","unstructured":"van de Leemput SC, Prokop M, van Ginneken B, Manniesing R (2019) Stacked bidirectional convolutional LSTMs for deriving 3D non-contrast ct from spatiotemporal 4D CT. IEEE Trans Med Imaging","DOI":"10.1109\/TMI.2019.2939044"},{"issue":"11","key":"2178_CR24","doi-asserted-by":"publisher","first-page":"2950","DOI":"10.1364\/BOE.3.002950","volume":"3","author":"KV Vienola","year":"2012","unstructured":"Vienola KV, Braaf B, Sheehy CK, Yang Q, Tiruveedhula P, Arathorn DW, de Boer JF, Roorda A (2012) Real-time eye motion compensation for OCT imaging with tracking SLO. Biomed Opt Express 3(11):2950\u20132963","journal-title":"Biomed Opt Express"},{"issue":"5","key":"2178_CR25","first-page":"622","volume":"9","author":"T Wang","year":"2016","unstructured":"Wang T, Pfeiffer T, Regar E, Wieser W, van Beusekom H, Lancee CT, Springeling G, Krabbendam-Peters I, van der Steen AF, Huber R, van Soest G (2016) Heartbeat OCT and motion-free 3D in vivo coronary artery microscopy. JACC: Cardiovascular Imaging 9(5):622\u2013623","journal-title":"JACC: Cardiovascular Imaging"},{"issue":"7","key":"2178_CR26","doi-asserted-by":"publisher","first-page":"1337","DOI":"10.1109\/TMI.2011.2182618","volume":"31","author":"J Xu","year":"2012","unstructured":"Xu J, Ishikawa H, Wollstein G, Kagemann L, Schuman JS (2012) Alignment of 3-D optical coherence tomography scans to correct eye movement using a particle filtering. IEEE Trans Med Imaging 31(7):1337\u20131345","journal-title":"IEEE Trans Med Imaging"},{"key":"2178_CR27","doi-asserted-by":"crossref","unstructured":"Zawadzki RJ, Fuller AR, Choi SS, Wiley DF, Hamann B, Werner JS (2007) Correction of motion artifacts and scanning beam distortions in 3D ophthalmic optical coherence tomography imaging. In: Ophthalmic technologies XVII. International society for optics and photonics, vol 6426, p 642607","DOI":"10.1117\/12.701524"},{"key":"2178_CR28","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1155\/2014\/251814","volume":"2014","author":"Y Zhang","year":"2014","unstructured":"Zhang Y, Pfeiffer T, Weller M, Wieser W, Huber R, Raczkowsky J, Schipper J, W\u00f6rn H, Klenzner T (2014) Optical coherence tomography guided laser cochleostomy: towards the accuracy on tens of micrometer scale. BioMed Res Int 2014:10. https:\/\/doi.org\/10.1155\/2014\/251814","journal-title":"BioMed Res Int"},{"key":"2178_CR29","doi-asserted-by":"crossref","unstructured":"Zhao Y, Li X, Zhang W, Zhao S, Makkie M, Zhang M, Li Q, Liu T (2018) Modeling 4d fmri data via spatio-temporal convolutional neural networks (st-cnn). In: International conference on medical image computing and computer-assisted intervention, Springer, pp 181\u2013189","DOI":"10.1007\/978-3-030-00931-1_21"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-020-02178-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-020-02178-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-020-02178-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,21]],"date-time":"2021-05-21T23:25:02Z","timestamp":1621639502000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-020-02178-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,22]]},"references-count":29,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2020,6]]}},"alternative-id":["2178"],"URL":"https:\/\/doi.org\/10.1007\/s11548-020-02178-z","relation":{},"ISSN":["1861-6410","1861-6429"],"issn-type":[{"type":"print","value":"1861-6410"},{"type":"electronic","value":"1861-6429"}],"subject":[],"published":{"date-parts":[[2020,5,22]]},"assertion":[{"value":"17 January 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 April 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 May 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"This work was partially funded by Forschungszentrum Medizintechnik Hamburg (grants 04fmthh16).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Funding:"}},{"value":"The authors declare that they have no conflict of interest.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest:"}},{"value":"This article does not contain any studies with human participants or animals performed by any of the authors.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval:"}},{"value":"Not applicable","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent:"}}]}}