{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T11:23:09Z","timestamp":1780053789768,"version":"3.54.0"},"reference-count":83,"publisher":"SAGE Publications","issue":"4-5","license":[{"start":{"date-parts":[[2017,10,16]],"date-time":"2017-10-16T00:00:00Z","timestamp":1508112000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2018,4]]},"abstract":"<jats:p>This paper studies visual odometry (VO) from the perspective of deep learning. After tremendous efforts in the robotics and computer vision communities over the past few decades, state-of-the-art VO algorithms have demonstrated incredible performance. However, since the VO problem is typically formulated as a pure geometric problem, one of the key features still missing from current VO systems is the capability to automatically gain knowledge and improve performance through learning. In this paper, we investigate whether deep neural networks can be effective and beneficial to the VO problem. An end-to-end, sequence-to-sequence probabilistic visual odometry (ESP-VO) framework is proposed for the monocular VO based on deep recurrent convolutional neural networks. It is trained and deployed in an end-to-end manner, that is, directly inferring poses and uncertainties from a sequence of raw images (video) without adopting any modules from the conventional VO pipeline. It can not only automatically learn effective feature representation encapsulating geometric information through convolutional neural networks, but also implicitly model sequential dynamics and relation for VO using deep recurrent neural networks. Uncertainty is also derived along with the VO estimation without introducing much extra computation. Extensive experiments on several datasets representing driving, flying and walking scenarios show competitive performance of the proposed ESP-VO to the state-of-the-art methods, demonstrating a promising potential of the deep learning technique for VO and verifying that it can be a viable complement to current VO systems.<\/jats:p>","DOI":"10.1177\/0278364917734298","type":"journal-article","created":{"date-parts":[[2017,10,16]],"date-time":"2017-10-16T08:22:20Z","timestamp":1508142140000},"page":"513-542","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":184,"title":["End-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks"],"prefix":"10.1177","volume":"37","author":[{"given":"Sen","family":"Wang","sequence":"first","affiliation":[{"name":"Institute of Sensors, Signals and Systems, Heriot\u2013Watt University, Edinburgh, UK"},{"name":"Department of Computer Science, University of Oxford, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronald","family":"Clark","sequence":"additional","affiliation":[{"name":"Department of Computing, Imperial College London, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongkai","family":"Wen","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Warwick, Warwick, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Niki","family":"Trigoni","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Oxford, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2017,10,16]]},"reference":[{"key":"bibr1-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.13"},{"key":"bibr2-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.642"},{"key":"bibr3-0278364917734298","volume-title":"Estimation with Applications to Tracking and Navigation: Theory Algorithms and Software","author":"Bar\u2013Shalom Y","year":"2004"},{"key":"bibr4-0278364917734298","author":"Bishop CM","year":"1994","journal-title":"Aston University"},{"key":"bibr5-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913507326"},{"key":"bibr6-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1177\/0278364915620033"},{"key":"bibr7-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2014.08.001"},{"key":"bibr8-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.284"},{"key":"bibr9-0278364917734298","first-page":"3995","volume-title":"AAAI conference on artificial intelligence (AAAI)","author":"Clark R","year":"2017"},{"key":"bibr10-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.350"},{"key":"bibr11-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2015.2505717"},{"key":"bibr12-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2005.29"},{"key":"bibr13-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.1049"},{"key":"bibr14-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2599174"},{"key":"bibr15-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.316"},{"key":"bibr16-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.304"},{"key":"bibr17-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2013.2279412"},{"key":"bibr18-0278364917734298","volume-title":"arXiv:1607.02565","author":"Engel J","year":"2016"},{"key":"bibr19-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10605-2_54"},{"key":"bibr20-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2015.7353631"},{"key":"bibr21-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.183"},{"key":"bibr22-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6906584"},{"key":"bibr23-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2012.2182810"},{"key":"bibr24-0278364917734298","volume-title":"Proceedings of international conference on machine learning (ICML)","author":"Gal Y","year":"2016"},{"key":"bibr25-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2012.2197158"},{"key":"bibr26-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_45"},{"key":"bibr27-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913491297"},{"key":"bibr28-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"bibr29-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/IVS.2011.5940405"},{"key":"bibr30-0278364917734298","volume-title":"Deep Learning","author":"Goodfellow I","year":"2016"},{"key":"bibr31-0278364917734298","first-page":"1764","volume-title":"Proceedings of the international conference on machine learning (ICML)","volume":"14","author":"Graves A","year":"2014"},{"key":"bibr32-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1177\/0278364912472245"},{"key":"bibr33-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49409-8_9"},{"key":"bibr34-0278364917734298","volume-title":"Multiple View Geometry in Computer Vision","author":"Hartley R","year":"2003"},{"key":"bibr35-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913509675"},{"key":"bibr36-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"bibr37-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.179"},{"key":"bibr38-0278364917734298","unstructured":"Jia D, Su Y, Li C (2016) Deep convolutional neural network for 6-DOF image localization. arXiv preprint arXiv: 1611.02776. Available at: https:\/\/arxiv.org\/abs\/1611.02776"},{"key":"bibr39-0278364917734298","volume-title":"Proceedings of the British Machine Vision Conference (BMVC)","author":"Kendall A","year":"2015"},{"key":"bibr40-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487679"},{"key":"bibr41-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.694"},{"key":"bibr42-0278364917734298","author":"Kendall A","year":"2017","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"bibr43-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.336"},{"key":"bibr44-0278364917734298","volume-title":"Proceedings of the international conference on learning representations (ICLR)","author":"Kingma D","year":"2014"},{"key":"bibr45-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2007.4538852"},{"key":"bibr46-0278364917734298","doi-asserted-by":"publisher","DOI":"10.5220\/0005299304860490"},{"key":"bibr47-0278364917734298","first-page":"3607","volume-title":"Proceedings of IEEE international conference on robotics and automation (ICRA)","author":"K\u00fcmmerle R","year":"2011"},{"key":"bibr48-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1038\/nature14539"},{"key":"bibr49-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1177\/0278364914549607"},{"key":"bibr50-0278364917734298","author":"Levine S","year":"2016","journal-title":"The International Journal of Robotics Research"},{"key":"bibr51-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2505283"},{"key":"bibr52-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/JRA.1987.1087097"},{"key":"bibr53-0278364917734298","first-page":"4628","volume-title":"Proceedings of IEEE international conference on robotics and automation (ICRA)","author":"McCormac J","year":"2016"},{"key":"bibr54-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-70353-4_57"},{"key":"bibr55-0278364917734298","volume-title":"AAAI conference on artificial intelligence (AAAI)","author":"Milan A","year":"2017"},{"key":"bibr56-0278364917734298","volume-title":"Optical flow and deep learning based approach to visual odometry","author":"Muller PM","year":"2016"},{"key":"bibr57-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2015.2463671"},{"key":"bibr58-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2015.7353721"},{"key":"bibr59-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126513"},{"key":"bibr60-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2004.1315094"},{"key":"bibr61-0278364917734298","first-page":"3361","volume-title":"Proceedings of AAAI conference on artificial intelligence (AAAI)","author":"Ondruska P","year":"2016"},{"key":"bibr62-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2008.4543185"},{"key":"bibr63-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2011.943233"},{"key":"bibr64-0278364917734298","volume-title":"Proceedings of robotics: Science and systems (RSS) workshop on visual place recognition: What is it good for","author":"Shahid V","year":"2016"},{"key":"bibr65-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33715-4_54"},{"key":"bibr66-0278364917734298","volume-title":"International conference on learning representations","author":"Simonyan K","year":"2014"},{"key":"bibr67-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1177\/027836498600500404"},{"key":"bibr68-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2013.6631246"},{"key":"bibr69-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2469274"},{"key":"bibr70-0278364917734298","volume-title":"Local accuracy and global consistency for efficient visual SLAM","author":"Strasdat H","year":"2012"},{"key":"bibr71-0278364917734298","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2010.VI.010"},{"key":"bibr72-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2012.02.009"},{"key":"bibr73-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2012.6385773"},{"key":"bibr74-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2015.7353986"},{"key":"bibr75-0278364917734298","author":"S\u00fcnderhauf N","year":"2015","journal-title":"Proceedings of robotics: Science and systems (RSS)"},{"key":"bibr76-0278364917734298","first-page":"3104","author":"Sutskever I","year":"2014","journal-title":"Advances in neural information processing Systems (NIPS)"},{"key":"bibr77-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"bibr78-0278364917734298","volume-title":"Probabilistic Robotics","author":"Thrun S","year":"2005"},{"key":"bibr79-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.596"},{"key":"bibr80-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989236"},{"key":"bibr81-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2010.5509248"},{"key":"bibr82-0278364917734298","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46487-9_33"},{"key":"bibr83-0278364917734298","unstructured":"Zaremba W, Sutskever I (2014) Learning to execute. arXiv preprint arXiv:1410.4615. Available at: https:\/\/arxiv.org\/abs\/1410.4615"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364917734298","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0278364917734298","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364917734298","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:15:47Z","timestamp":1777457747000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0278364917734298"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,10,16]]},"references-count":83,"journal-issue":{"issue":"4-5","published-print":{"date-parts":[[2018,4]]}},"alternative-id":["10.1177\/0278364917734298"],"URL":"https:\/\/doi.org\/10.1177\/0278364917734298","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,10,16]]}}}