{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T16:15:18Z","timestamp":1769184918003,"version":"3.49.0"},"reference-count":49,"publisher":"IOP Publishing","issue":"24","license":[{"start":{"date-parts":[[2020,12,8]],"date-time":"2020-12-08T00:00:00Z","timestamp":1607385600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/publishingsupport.iopscience.iop.org\/iop-standard\/v1"},{"start":{"date-parts":[[2020,12,8]],"date-time":"2020-12-08T00:00:00Z","timestamp":1607385600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"National Science Foundation Program of China","award":["61501030"],"award-info":[{"award-number":["61501030"]}]},{"name":"National Science and Technology Major Project of China","award":["2018ZX10723-204-008"],"award-info":[{"award-number":["2018ZX10723-204-008"]}]},{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2017YFC0107900"],"award-info":[{"award-number":["2017YFC0107900"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Phys. Med. Biol."],"published-print":{"date-parts":[[2020,12,21]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Motion compensation can eliminate inconsistencies of respiratory movement during image acquisitions for precise vascular reconstruction in the clinical diagnosis of vascular disease from x-ray angiographic image sequences. In x-ray-based vascular interventional therapy, motion modeling can simulate the process of organ deformation driven by motion signals to display a dynamic organ on angiograms without contrast agent injection. Automatic respiratory signal estimation from x-ray angiographic image sequences is essential for motion compensation and modeling. The effects of respiratory motion, cardiac impulses, and tremors on structures in the chest and abdomen bring difficulty in extracting accurate respiratory signals individually. In this study, an end-to-end deep learning framework based on a motion-flow-guided recurrent network is proposed to address the aforementioned problem. The proposed method utilizes a convolutional neural network to learn the spatial features of every single frame, and a recurrent neural network to learn the temporal features of the entire sequence. The combination of the two networks can effectively analyze the image sequence to realize respiratory signal estimation. In addition, the motion-flow between consecutive frames is introduced to provide a dynamic constraint of spatial features, which enables the recurrent network to learn better temporal features from dynamic spatial features than from static spatial features. We demonstrate the advantages of our approach on designed datasets which contain coronary and hepatic angiographic sequences with diaphragm structures, and coronary angiographic sequences without diaphragm structures. Our method improves over state-of-the-art manifold-learning-based methods by 85.7%, 81.5% and 75.3% in respiratory signal accuracy metric on these datasets. The results demonstrate that the proposed method can effectively estimate respiratory signals from multiple motion patterns.<\/jats:p>","DOI":"10.1088\/1361-6560\/aba087","type":"journal-article","created":{"date-parts":[[2020,6,26]],"date-time":"2020-06-26T21:44:13Z","timestamp":1593207853000},"page":"245020","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Motion-flow-guided recurrent network for respiratory signal estimation of x-ray angiographic image sequences"],"prefix":"10.1088","volume":"65","author":[{"given":"Huihui","family":"Fang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7754-7141","authenticated-orcid":false,"given":"Heng","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuang","family":"Song","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kun","family":"Pang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Danni","family":"Ai","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingfan","family":"Fan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hong","family":"Song","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yang","family":"Yu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1250-6319","authenticated-orcid":false,"given":"Jian","family":"Yang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2020,12,8]]},"reference":[{"key":"pmbaba087bib1","article-title":"An empirical evaluation of generic convolutional and recurrent networks for sequence modeling","author":"Bai","year":"2018","type":"preprint"},{"key":"pmbaba087bib2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11263-010-0390-2","type":"journal-article","article-title":"A database and evaluation methodology for optical flow","volume":"92","author":"Baker","year":"2011","journal-title":"Int. J. Comput. Vis."},{"key":"pmbaba087bib3","first-page":"4","type":"journal-article","article-title":"Pyramidal implementation of the affine Lucas Kanade feature tracker description of the algorithm","volume":"5","author":"Bouguet","year":"2001","journal-title":"Intel Corp."},{"key":"pmbaba087bib4","first-page":"25","type":"conference-proceedings","article-title":"High accuracy optical flow estimation based on a theory for warping","author":"Brox","year":"2004"},{"key":"pmbaba087bib5","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1109\/TPAMI.2010.143","type":"journal-article","article-title":"Large displacement optical flow: descriptor matching in variational motion estimation","volume":"33","author":"Brox","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"pmbaba087bib6","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1016\/j.media.2016.02.007","type":"journal-article","article-title":"Reconstruction of coronary arteries from x-ray angiography: a review","volume":"32","author":"\u00c7imen","year":"2016","journal-title":"Med. Image Anal."},{"key":"pmbaba087bib7","doi-asserted-by":"publisher","first-page":"2079","DOI":"10.1109\/TBME.2015.2408633","type":"journal-article","article-title":"Quantitative analysis of deformable model-based 3-D reconstruction of coronary artery from multiple angiograms","volume":"62","author":"Cong","year":"2015","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"pmbaba087bib8","first-page":"2758","type":"conference-proceedings","article-title":"FlowNet: learning optical flow with convolutional networks","author":"Dosovitskiy","year":"2015"},{"key":"pmbaba087bib9","doi-asserted-by":"publisher","first-page":"44635","DOI":"10.1109\/ACCESS.2018.2864592","type":"journal-article","article-title":"Multichannel fully convolutional network for coronary artery segmentation in x-ray angiograms","volume":"6","author":"Fan","year":"2018","journal-title":"IEEE Access"},{"key":"pmbaba087bib10","doi-asserted-by":"publisher","DOI":"10.1118\/1.4789919","type":"journal-article","article-title":"Integration of cardiac and respiratory motion into MRI roadmaps fused with x\u2010ray","volume":"40","author":"Faranesh","year":"2013","journal-title":"Med. Phys."},{"key":"pmbaba087bib11","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1109\/TMI.2017.2723545","type":"journal-article","article-title":"An MR-based model for cardio-respiratory motion compensation of overlays in x-ray fluoroscopy","volume":"37","author":"Fischer","year":"2017","journal-title":"IEEE Trans. Med. Imaging"},{"key":"pmbaba087bib12","doi-asserted-by":"publisher","first-page":"865","DOI":"10.1109\/TMI.2016.2609888","type":"journal-article","article-title":"Unsupervised learning for robust respiratory signal estimation from x-ray fluoroscopy","volume":"36","author":"Fischer","year":"2016","journal-title":"IEEE Trans. Med. Imaging"},{"key":"pmbaba087bib13","doi-asserted-by":"publisher","first-page":"915","DOI":"10.1109\/ISBI.2014.6868020","type":"conference-proceedings","article-title":"Real-time respiratory signal extraction from x-ray sequences using incremental manifold learning","author":"Fischer","year":"2014"},{"key":"pmbaba087bib14","author":"Gibson","year":"1950","type":"book"},{"key":"pmbaba087bib15","first-page":"5","author":"Graves","year":"2012","type":"book"},{"key":"pmbaba087bib16","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1109\/CVPR.2016.90","type":"conference-proceedings","article-title":"Deep residual learning for image recognition","author":"He","year":"2016"},{"key":"pmbaba087bib17","first-page":"185","type":"book","volume":"17","author":"Horn","year":"1981"},{"key":"pmbaba087bib18","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1016\/j.ijrobp.2012.12.014","type":"journal-article","article-title":"Respiratory amplitude guided 4-dimensional magnetic resonance imaging","volume":"86","author":"Hu","year":"2013","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"pmbaba087bib19","first-page":"481","type":"conference-proceedings","article-title":"Robust interpolation of correspondences for large displacement optical flow","author":"Hu","year":"2017"},{"key":"pmbaba087bib20","doi-asserted-by":"publisher","first-page":"3472","DOI":"10.1088\/0031-9155\/61\/9\/3472","type":"journal-article","article-title":"4D MR imaging using robust internal respiratory signal","volume":"61","author":"Hui","year":"2016","journal-title":"Phys. Med. Biol."},{"key":"pmbaba087bib21","first-page":"2462","type":"conference-proceedings","article-title":"FlowNet 2.0: evolution of optical flow estimation with deep networks","author":"Ilg","year":"2017"},{"key":"pmbaba087bib22","doi-asserted-by":"publisher","first-page":"1313","DOI":"10.1016\/j.jcin.2018.01.274","type":"journal-article","article-title":"Angiography alone versus angiography plus optical coherence tomography to guide percutaneous coronary intervention: outcomes from the Pan-London PCI Cohort","volume":"11","author":"Jones","year":"2018","journal-title":"JACC Cardiovasc. Interventions"},{"key":"pmbaba087bib23","doi-asserted-by":"publisher","DOI":"10.1117\/12.769377","type":"conference-proceedings","article-title":"A technique for respiratory motion correction in image guided cardiac catheterisation procedures","volume":"6918","author":"King","year":"2008"},{"key":"pmbaba087bib24","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1016\/j.media.2009.01.003","type":"journal-article","article-title":"A subject-specific technique for respiratory motion correction in image-guided cardiac catheterisation procedures","volume":"13","author":"King","year":"2009","journal-title":"Med. Image Anal."},{"key":"pmbaba087bib25","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014","type":"preprint"},{"key":"pmbaba087bib26","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.compmedimag.2018.03.003","type":"journal-article","article-title":"Deformable respiratory motion correction for hepatic rotational angiography","volume":"66","author":"Klugmann","year":"2018","journal-title":"Comput. Med. Imaging Graph."},{"key":"pmbaba087bib27","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1118\/1.2147743","type":"journal-article","article-title":"Technical and dosimetric aspects of respiratory gating using a pressure\u2010sensor motion monitoring system","volume":"33","author":"Li","year":"2006","journal-title":"Med. Phys."},{"key":"pmbaba087bib28","doi-asserted-by":"publisher","first-page":"1254","DOI":"10.1118\/1.1576230","type":"journal-article","article-title":"A method for the reconstruction of four\u2010dimensional synchronized CT scans acquired during free breathing","volume":"30","author":"Low","year":"2003","journal-title":"Med. Phys."},{"key":"pmbaba087bib29","first-page":"674","type":"other","article-title":"An iterative image registration technique with an application to stereo vision","volume":"vol 2","author":"Lucas","year":"1981"},{"key":"pmbaba087bib30","doi-asserted-by":"publisher","first-page":"695","DOI":"10.1007\/s11548-015-1185-2","type":"journal-article","article-title":"PCA-derived respiratory motion surrogates from x-ray angiograms for percutaneous coronary interventions","volume":"10","author":"Ma","year":"2015","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"pmbaba087bib31","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2020.101634","type":"journal-article","article-title":"Dynamic coronary roadmapping via catheter tip tracking in x-ray fluoroscopy with deep learning based Bayesian filtering","volume":"61","author":"Ma","year":"2020","journal-title":"Med. Image Anal."},{"key":"pmbaba087bib32","first-page":"391","type":"conference-proceedings","article-title":"Real-time respiratory motion correction for cardiac electrophysiology procedures using image-based coronary sinus catheter tracking","author":"Ma","year":"2010"},{"key":"pmbaba087bib33","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/j.ejmp.2019.02.003","type":"journal-article","article-title":"A clustering approach to 4D MRI retrospective sorting for the investigation of different surrogates","volume":"58","author":"Meschini","year":"2019","journal-title":"Phys. Med."},{"key":"pmbaba087bib34","first-page":"126","type":"conference-proceedings","article-title":"Extraction of cardiac and respiratory motion information from cardiac x-ray fluoroscopy images using hierarchical manifold learning","author":"Panayiotou","year":"2013"},{"key":"pmbaba087bib35","first-page":"1164","type":"conference-proceedings","article-title":"EpicFlow: edge-preserving interpolation of correspondences for optical flow","author":"Revaud","year":"2015"},{"key":"pmbaba087bib36","doi-asserted-by":"publisher","first-page":"2425","DOI":"10.1109\/TMI.2016.2576899","type":"journal-article","article-title":"Fully automated data-driven respiratory signal extraction from SPECT images using Laplacian eigenmaps","volume":"35","author":"Sanders","year":"2016","journal-title":"IEEE Trans. Med. Imaging"},{"key":"pmbaba087bib37","doi-asserted-by":"publisher","first-page":"1046","DOI":"10.1109\/TMI.2004.828676","type":"journal-article","article-title":"Respiratory motion of the heart from free breathing coronary angiograms","volume":"23","author":"Shechter","year":"2004","journal-title":"IEEE Trans. Med. Imaging"},{"key":"pmbaba087bib38","doi-asserted-by":"publisher","first-page":"441","DOI":"10.1109\/TMI.2004.839679","type":"journal-article","article-title":"Prospective motion correction of x-ray images for coronary interventions","volume":"24","author":"Shechter","year":"2005","journal-title":"IEEE Trans. Med. Imaging"},{"key":"pmbaba087bib39","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/aafa06","type":"journal-article","article-title":"Deep motion tracking from multiview angiographic image sequences for synchronization of cardiac phases","volume":"64","author":"Song","year":"2019a","journal-title":"Phys. Med. Biol."},{"key":"pmbaba087bib40","first-page":"381","type":"conference-proceedings","article-title":"Automatic image-based cardiac and respiratory cycle synchronization and gating of image sequences","author":"Sundar","year":"2009"},{"key":"pmbaba087bib41","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1016\/j.jcin.2014.12.232","type":"journal-article","article-title":"Fractional flow reserve and coronary bifurcation anatomy: a novel quantitative model to assess and report the stenosis severity of bifurcation lesions","volume":"8","author":"Tu","year":"2015","journal-title":"JACC Cardiovasc. Interventions"},{"key":"pmbaba087bib42","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1109\/TBME.2015.2463769","type":"journal-article","article-title":"Real-time respiratory motion analysis using 4-D shape priors","volume":"63","author":"Wasza","year":"2015","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"pmbaba087bib43","first-page":"1663","type":"conference-proceedings","article-title":"Structure- and motion-adaptive regularization for high accuracy optic flow","author":"Wedel","year":"2009"},{"key":"pmbaba087bib44","first-page":"1385","type":"conference-proceedings","article-title":"DeepFlow: large displacement optical flow with deep matching","author":"Weinzaepfel","year":"2013"},{"key":"pmbaba087bib45","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1016\/j.media.2017.09.005","type":"journal-article","article-title":"Full left ventricle quantification via deep multitask relationships learning","volume":"43","author":"Xue","year":"2018","journal-title":"Med. Image Anal."},{"key":"pmbaba087bib46","doi-asserted-by":"publisher","first-page":"975","DOI":"10.1088\/0031-9155\/59\/4\/975","type":"journal-article","article-title":"External force back-projective composition and globally deformable optimization for 3-D coronary artery reconstruction","volume":"59","author":"Yang","year":"2014","journal-title":"Phys. Med. Biol."},{"key":"pmbaba087bib47","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1016\/j.bspc.2016.09.012","type":"journal-article","article-title":"A novel structural features-based approach to automatically extract multiple motion parameters from single-arm x-ray angiography","volume":"32","author":"Zhang","year":"2017","journal-title":"Biomed. Signal Process. Control"},{"key":"pmbaba087bib48","first-page":"287","type":"conference-proceedings","article-title":"Image-based respiratory motion compensation for fluoroscopic coronary roadmapping","author":"Zhu","year":"2010"},{"key":"pmbaba087bib49","first-page":"4563","type":"conference-proceedings","article-title":"InterpoNet, a brain inspired neural network for optical flow dense interpolation","author":"Zweig","year":"2017"}],"container-title":["Physics in Medicine &amp; Biology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087","content-type":"text\/html","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,13]],"date-time":"2025-12-13T14:39:15Z","timestamp":1765636755000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/aba087"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,8]]},"references-count":49,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2020,12,8]]},"published-print":{"date-parts":[[2020,12,21]]}},"URL":"https:\/\/doi.org\/10.1088\/1361-6560\/aba087","relation":{},"ISSN":["0031-9155","1361-6560"],"issn-type":[{"value":"0031-9155","type":"print"},{"value":"1361-6560","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,8]]},"assertion":[{"value":"Motion-flow-guided recurrent network for respiratory signal estimation of x-ray angiographic image sequences","name":"article_title","label":"Article Title"},{"value":"Physics in Medicine & Biology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2020 Institute of Physics and Engineering in Medicine. All rights, including for text and data mining, AI training, and similar technologies, are reserved.","name":"copyright_information","label":"Copyright Information"},{"value":"2019-12-20","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2020-06-26","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2020-12-08","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}