{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T16:44:21Z","timestamp":1777653861974,"version":"3.51.4"},"reference-count":31,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,2,6]],"date-time":"2021-02-06T00:00:00Z","timestamp":1612569600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this work, we propose a novel method of estimating optical flow from event-based cameras by matching the time surface of events. The proposed loss function measures the timestamp consistency between the time surface formed by the latest timestamp of each pixel and the one that is slightly shifted in time. This makes it possible to estimate dense optical flows with high accuracy without restoring luminance or additional sensor information. In the experiment, we show that the gradient was more correct and the loss landscape was more stable than the variance loss in the motion compensation approach. In addition, we show that the optical flow can be estimated with high accuracy by optimization with L1 smoothness regularization using publicly available datasets.<\/jats:p>","DOI":"10.3390\/s21041150","type":"journal-article","created":{"date-parts":[[2021,2,10]],"date-time":"2021-02-10T04:33:46Z","timestamp":1612931626000},"page":"1150","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Optical Flow Estimation by Matching Time Surface with Event-Based Cameras"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6231-7490","authenticated-orcid":false,"given":"Jun","family":"Nagata","sequence":"first","affiliation":[{"name":"Department of Electronics and Electrical Engineering, Faculty of Science and Technology, Keio University, 3-14-1, Hiyoshi, Kohoku-ku, Yokohama, Kanagawa 223-8522, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yusuke","family":"Sekikawa","sequence":"additional","affiliation":[{"name":"Denso IT Laboratory, 2-15-1, Shibuya, Shibuya-ku, Tokyo 150-0002, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yoshimitsu","family":"Aoki","sequence":"additional","affiliation":[{"name":"Department of Electronics and Electrical Engineering, Faculty of Science and Technology, Keio University, 3-14-1, Hiyoshi, Kohoku-ku, Yokohama, Kanagawa 223-8522, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,6]]},"reference":[{"key":"ref_1","unstructured":"Gallego, G., Delbruck, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A., Conradt, J., and Daniilidis, K. (2019). Event-based Vision: A Survey. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kim, H., Handa, A., Benosman, R., Ieng, S.H., and Davison, A. (2014, January 1\u20135). Simultaneous Mosaicing and Tracking with an Event Camera. Proceedings of the British Machine Vision Conference (BMVC), Nottingham, UK.","DOI":"10.5244\/C.28.26"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Kim, H., Leutenegger, S., and Davison, A.J. (2016, January 11\u201314). Real-time 3D reconstruction and 6-DoF tracking with an event camera. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46466-4_21"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Rebecq, H., Ranftl, R., Koltun, V., and Scaramuzza, D. (2019, January 16\u201320). Events-to-Video: Bringing Modern Computer Vision to Event Cameras. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00398"},{"key":"ref_5","unstructured":"Lucas, B.D., and Kanade, T. (2021, January 20). An Iterative Image Registration Technique with an Application to Stereo Vision. Available online: https:\/\/ri.cmu.edu\/pub_files\/pub3\/lucas_bruce_d_1981_2\/lucas_bruce_d_1981_2.pdf."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/0004-3702(81)90024-2","article-title":"Determining Optical Flow","volume":"17","author":"Horn","year":"1981","journal-title":"Artif. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1016\/j.neunet.2011.11.001","article-title":"Asynchronous frameless event-based optical flow","volume":"27","author":"Benosman","year":"2012","journal-title":"Neural Netw."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Brosch, T., Tschechne, S., and Neumann, H. (2015). On event-based optical flow detection. Front. Neurosci.","DOI":"10.3389\/fnins.2015.00137"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Bardow, P., Davison, A.J., and Leutenegger, S. (2016, January 27\u201330). Simultaneous Optical Flow and Intensity Estimation from an Event Camera. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.102"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Gallego, G., Rebecq, H., and Scaramuzza, D. (2018, January 18\u201322). A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00407"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhu, A.Z., Yuan, L., Chaney, K., and Daniilidis, K. (2019, January 16\u201320). Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00108"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Gallego, G., Gehrig, M., and Scaramuzza, D. (2019, January 16\u201320). Focus Is All You Need: Loss Functions for Event-Based Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01256"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1177\/0278364917691115","article-title":"The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM","volume":"36","author":"Mueggler","year":"2017","journal-title":"Int. J. Robot. Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2333","DOI":"10.1109\/JSSC.2014.2342715","article-title":"A 240 \u00d7 180 130 dB 3 \u03bcs latency global shutter spatiotemporal vision sensor","volume":"49","author":"Brandli","year":"2014","journal-title":"IEEE J. Solid State Circuits"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1109\/TNNLS.2013.2273537","article-title":"Event-Based Visual Flow","volume":"25","author":"Benosman","year":"2014","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Rueckauer, B., and Delbruck, T. (2016). Evaluation of Event-Based Algorithms for Optical Flow with Ground-Truth from Inertial Measurement Sensor. Front. Neurosci., 10.","DOI":"10.3389\/fnins.2016.00176"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhu, A., Yuan, L., Chaney, K., and Daniilidis, K. (2018). EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras. Robot. Sci. Syst.","DOI":"10.15607\/RSS.2018.XIV.062"},{"key":"ref_18","unstructured":"Ye, C., Mitrokhin, A., Ferm\u00fcller, C., Yorke, J.A., and Aloimonos, Y. (2018). Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"632","DOI":"10.1109\/LRA.2016.2647639","article-title":"Accurate Angular Velocity Estimation With an Event Camera","volume":"2","author":"Gallego","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_20","unstructured":"Zhu, A.Z., Atanasov, N., and Daniilidis, K. (June, January 29). Event-based feature tracking with probabilistic data association. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Singapore."},{"key":"ref_21","unstructured":"Stoffregen, T., and Kleeman, L. (2017, January 11\u201313). Simultaneous Optical Flow and Segmentation (SOFAS) using Dynamic Vision Sensor. Proceedings of the Australasian Conference on Robotics and Automation (ACRA), Sydney, Australia."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Mitrokhin, A., Fermuller, C., Parameshwara, C., and Aloimonos, Y. (2018, January 1\u20135). Event-Based Moving Object Detection and Tracking. Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593805"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Stoffregen, T., and Kleeman, L. (2019, January 15\u201320). Event Cameras, Contrast Maximization and Reward Functions: An Analysis. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01258"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1346","DOI":"10.1109\/TPAMI.2016.2574707","article-title":"HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern Recognition","volume":"39","author":"Lagorce","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Manderscheid, J., Sironi, A., Bourdis, N., Migliore, D., and Lepetit, V. (2019, January 16\u201320). Speed Invariant Time Surface for Learning to Detect Corner Points With Event-Based Cameras. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01049"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zach, C., Pock, T., and Bischof, H. (2007). A Duality Based Approach for Realtime TV-L1 Optical Flow. Pattern Recognit., 214\u2013223.","DOI":"10.1007\/978-3-540-74936-3_22"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"137","DOI":"10.5201\/ipol.2013.26","article-title":"TV-L1 Optical Flow Estimation","volume":"3","author":"Facciolo","year":"2013","journal-title":"Image Process. Line"},{"key":"ref_28","first-page":"969","article-title":"ESIM: An Open Event Camera Simulator","volume":"87","author":"Rebecq","year":"2018","journal-title":"Conf. Robot. Learn."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2032","DOI":"10.1109\/LRA.2018.2800793","article-title":"The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception","volume":"3","author":"Zhu","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1470","DOI":"10.1109\/JPROC.2014.2346153","article-title":"Retinomorphic event-based vision sensors: Bioinspired cameras with spiking output","volume":"102","author":"Posch","year":"2014","journal-title":"Proc. IEEE"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1007\/s10851-010-0251-1","article-title":"A first-order primal-dual algorithm for convex problems with applications to imaging","volume":"40","author":"Chambolle","year":"2011","journal-title":"J. Math. Imaging Vis."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1150\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:20:41Z","timestamp":1760160041000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1150"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,6]]},"references-count":31,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["s21041150"],"URL":"https:\/\/doi.org\/10.3390\/s21041150","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,6]]}}}