{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T18:41:26Z","timestamp":1780598486146,"version":"3.54.1"},"reference-count":38,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2022,7,29]],"date-time":"2022-07-29T00:00:00Z","timestamp":1659052800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100007219","name":"Natural Science Foundation of Shanghai","doi-asserted-by":"publisher","award":["22ZR1441300"],"award-info":[{"award-number":["22ZR1441300"]}],"id":[{"id":"10.13039\/100007219","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>We present a new solution to tracking and mapping with an event camera. The motion of the camera contains both rotation and translation displacements in the plane, and the displacements happen in an arbitrarily structured environment. As a result, the image matching may no longer be represented by a low-dimensional homographic warping, thus complicating an application of the commonly used Image of Warped Events (IWE). We introduce a new solution to this problem by performing contrast maximization in 3D. The 3D location of the rays cast for each event is smoothly varied as a function of a continuous-time motion parametrization, and the optimal parameters are found by maximizing the contrast in a volumetric ray density field. Our method thus performs joint optimization over motion and structure. The practical validity of our approach is supported by an application to AGV motion estimation and 3D reconstruction with a single vehicle-mounted event camera. The method approaches the performance obtained with regular cameras and eventually outperforms in challenging visual conditions.<\/jats:p>","DOI":"10.3390\/s22155687","type":"journal-article","created":{"date-parts":[[2022,8,1]],"date-time":"2022-08-01T23:49:27Z","timestamp":1659397767000},"page":"5687","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Visual Odometry with an Event Camera Using Continuous Ray Warping and Volumetric Contrast Maximization"],"prefix":"10.3390","volume":"22","author":[{"given":"Yifu","family":"Wang","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7038-0913","authenticated-orcid":false,"given":"Jiaqi","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6590-4081","authenticated-orcid":false,"given":"Xin","family":"Peng","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peng","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2294-5155","authenticated-orcid":false,"given":"Ling","family":"Gao","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kun","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiaben","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laurent","family":"Kneip","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"},{"name":"Shanghai Engineering Research Center of Intelligent Vision and Imaging, ShanghaiTech University, Shanghai 201210, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1007\/s10462-012-9365-8","article-title":"Visual simultaneous localization and mapping: A survey","volume":"43","year":"2015","journal-title":"Artif. Rev."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1309","DOI":"10.1109\/TRO.2016.2624754","article-title":"Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age","volume":"32","author":"Cadena","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2333","DOI":"10.1109\/JSSC.2014.2342715","article-title":"A 240 \u00d7 180 130 db 3 \u03bcs latency global shutter spatiotemporal vision sensor","volume":"49","author":"Brandli","year":"2014","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1109\/TPAMI.2020.3008413","article-title":"Event-based vision: A survey","volume":"44","author":"Gallego","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Gallego, G., Rebecq, H., and Scaramuzza, D. (2018, January 18\u201322). A unifying contrast maximization framework for event cameras, with applications to motion, depth, and optical flow estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00407"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Gallego, G., Gehrig, M., and Scaramuzza, D. (2019, January 16\u201320). Focus is all you need: Loss functions for event-based vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01256"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Stoffregen, T., and Kleeman, L. (2019, January 16\u201320). Event cameras, contrast maximization and reward functions: An analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01258"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1109\/TNNLS.2013.2273537","article-title":"Event-based visual flow","volume":"25","author":"Benosman","year":"2013","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhu, A.Z., Atanasov, N., and Daniilidis, K. (June, January 29). Event-based feature tracking with probabilistic data association. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989517"},{"key":"ref_10","unstructured":"Stoffregen, T., and Kleeman, L. (2017, January 11\u201313). Simultaneous optical flow and segmentation (sofas) using dynamic vision sensor. Proceedings of the 2017 Australasian Conference on Robotics and Automation (ACRA), Sydney, Australia."},{"key":"ref_11","unstructured":"Ye, C., Mitrokhin, A., Parameshwara, C., Ferm\u00fcller, C., Yorke, J.A., and Aloimonos, Y. (2018). Unsupervised learning of dense optical flow and depth from sparse event data. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhu, A.Z., Yuan, L., Chaney, K., and Daniilidis, K. (2019, January 16\u201320). Unsupervised event-based learning of optical flow, depth, and egomotion. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00108"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhu, A.Z., Yuan, L., Chaney, K., and Daniilidis, K. (2018). Ev-flownet: Self-supervised optical flow estimation for event-based cameras. arXiv.","DOI":"10.15607\/RSS.2018.XIV.062"},{"key":"ref_14","unstructured":"Stoffregen, T., Gallego, G., Drummond, T., Kleeman, L., and Scaramuzza, D. (November, January 27). Event-based motion segmentation by motion compensation. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mitrokhin, A., Ferm\u00fcller, C., Parameshwara, C., and Aloimonos, Y. (2018, January 1\u20135). Event-based moving object detection and tracking. Proceedings of the RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593805"},{"key":"ref_16","unstructured":"Zhou, Y., Gallego, G., Lu, X., Liu, S., and Shen, S. (2021). Event-based motion segmentation with spatio-temporal graph cuts. IEEE Trans. Neural Netw. Learn. Syst., 1\u201313."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1394","DOI":"10.1007\/s11263-017-1050-6","article-title":"Emvs: Event-based multi-view stereo\u20143d reconstruction with an event camera in real-time","volume":"126","author":"Rebecq","year":"2018","journal-title":"Int. J. Comput. Vis."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhu, A.Z., Chen, Y., and Daniilidis, K. (2018). Realtime time synchronized event-based stereo. European Conference on Computer Vision, Munich, Germany, 8\u201314 September 2018, Springer.","DOI":"10.1007\/978-3-030-01231-1_27"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Gallego, G., Rebecq, H., Kneip, L., Li, H., and Scaramuzza, D. (2018, January 8\u201314). Semi-dense 3d reconstruction with a stereo event camera. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01246-5_15"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1109\/TPAMI.2017.2769655","article-title":"Event-based, 6-dof camera tracking from photometric depth maps","volume":"40","author":"Gallego","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"632","DOI":"10.1109\/LRA.2016.2647639","article-title":"Accurate angular velocity estimation with an event camera","volume":"2","author":"Gallego","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Bryner, S., Gallego, G., Rebecq, H., and Scaramuzza, D. (2019, January 20\u201324). Event-based, direct camera tracking from a photometric 3d map using nonlinear optimization. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794255"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Peng, X., Wang, Y., and Kneip, L. (2020, January 23\u201328). Globally optimal event camera motion estimation. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK.","DOI":"10.1007\/978-3-030-58574-7_4"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Liu, D., Parra, A., and Chin, T.-J. (2020, January 14\u201319). Globally optimal contrast maximisation for event-based motion estimation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00638"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Liu, D., Parra, A., and Chin, T.-J. (2021, January 19\u201325). Spatiotemporal registration for event-based visual odometry. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, virtual.","DOI":"10.1109\/CVPR46437.2021.00490"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jiao, J., Huang, H., Li, L., He, Z., Zhu, Y., and Liu, M. (2021, January 19\u201325). Comparing Representations in Tracking for Event Camera-based SLAM. In Proceeding of the IEEE\/CVF Conference On Computer Vision And Pattern Recognition Workshops (CVPRW), virtual.","DOI":"10.1109\/CVPRW53098.2021.00151"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1433","DOI":"10.1109\/TRO.2021.3062252","article-title":"Event-based stereo visual odometry","volume":"37","author":"Zhou","year":"2021","journal-title":"IEEE Trans. Robot."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Kim, H., Leutenegger, S., and Davison, A.J. (2016, January 11\u201314). Real-time 3d reconstruction and 6-dof tracking with an event camera. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46466-4_21"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1109\/LRA.2016.2645143","article-title":"Evo: A geometric approach to event-based 6-dof parallel tracking and mapping in real time","volume":"2","author":"Rebecq","year":"2016","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhu, D., Xu, Z., Dong, J., Ye, C., Hu, Y., Su, H., Liu, Z., and Chen, G. (2019, January 6\u20138). Neuromorphic visual odometry system for intelligent vehicle application with bio-inspired vision sensor. Proceedings of the 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), Dali, China.","DOI":"10.1109\/ROBIO49542.2019.8961878"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1688","DOI":"10.1177\/0278364915585860","article-title":"Continuous-time batch trajectory estimation using temporal basis functions","volume":"34","author":"Furgale","year":"2015","journal-title":"Int. J. Robot. Res."},{"key":"ref_32","unstructured":"Nist\u00e9r, D., Naroditsky, O., and Bergen, J. (July, January 27). Visual odometry. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), Washington, DC, USA."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"ORB-SLAM2: An open-source slam system for monocular, stereo, and rgb-d cameras","volume":"33","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Scaramuzza, D., Fraundorfer, F., and Siegwart, R. (2009, January 12\u201317). Real-time monocular visual odometry for on-road vehicles with 1-point ransac. Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan.","DOI":"10.1109\/ROBOT.2009.5152255"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Huang, K., Wang, Y., and Kneip, L. (2019, January 16\u201320). Motion estimation of non-holonomic ground vehicles from a single feature correspondence measured over n views. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01299"},{"key":"ref_36","unstructured":"Piegl, L., and Tiller, W. (2012). The NURBS Book, Springer Science & Business Media."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The kitti dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res. (IJRR)"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Gehrig, D., Gehrig, M., Hidalgo-Carri\u00f3, J., and Scaramuzza, D. (2020, January 14\u201319). Video to events: Recycling video datasets for event cameras. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2020, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00364"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/15\/5687\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:59:16Z","timestamp":1760140756000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/15\/5687"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,29]]},"references-count":38,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["s22155687"],"URL":"https:\/\/doi.org\/10.3390\/s22155687","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,29]]}}}