{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T18:14:26Z","timestamp":1775326466838,"version":"3.50.1"},"reference-count":86,"publisher":"Springer Science and Business Media LLC","issue":"8014","license":[{"start":{"date-parts":[[2024,5,29]],"date-time":"2024-05-29T00:00:00Z","timestamp":1716940800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,5,29]],"date-time":"2024-05-29T00:00:00Z","timestamp":1716940800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Nature"],"published-print":{"date-parts":[[2024,5,30]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The computer vision algorithms used currently in advanced driver assistance systems rely on image-based RGB cameras, leading to a critical bandwidth\u2013latency trade-off for delivering safe driving experiences. To address this, event cameras have emerged as alternative vision sensors. Event cameras measure the changes in intensity asynchronously, offering high temporal resolution and sparsity, markedly reducing bandwidth and latency requirements<jats:sup>1<\/jats:sup>. Despite these advantages, event-camera-based algorithms are either highly efficient but lag behind image-based ones in terms of accuracy or sacrifice the sparsity and efficiency of events to achieve comparable results. To overcome this, here we propose a hybrid event- and frame-based object detector that preserves the advantages of each modality and thus does not suffer from this trade-off. Our method exploits the high temporal resolution and sparsity of events and the rich but low temporal resolution information in standard images to generate efficient, high-rate object detections, reducing perceptual and computational latency. We show that the use of a 20\u00a0frames\u00a0per second (fps) RGB camera plus an event camera can achieve the same latency as a 5,000-fps camera with the bandwidth of a 45-fps camera without compromising accuracy. Our approach paves the way for efficient and robust perception in edge-case scenarios by uncovering the potential of event cameras<jats:sup>2<\/jats:sup>.<\/jats:p>","DOI":"10.1038\/s41586-024-07409-w","type":"journal-article","created":{"date-parts":[[2024,5,29]],"date-time":"2024-05-29T16:02:05Z","timestamp":1716998525000},"page":"1034-1040","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":170,"title":["Low-latency automotive vision with event cameras"],"prefix":"10.1038","volume":"629","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9952-3335","authenticated-orcid":false,"given":"Daniel","family":"Gehrig","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3831-6778","authenticated-orcid":false,"given":"Davide","family":"Scaramuzza","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,5,29]]},"reference":[{"key":"7409_CR1","doi-asserted-by":"crossref","unstructured":"Gallego, G. et al. Event-based vision: a survey. In Proc. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 44, 154\u2013180 (IEEE, 2020).","DOI":"10.1109\/TPAMI.2020.3008413"},{"key":"7409_CR2","doi-asserted-by":"crossref","unstructured":"Big data needs a hardware revolution. Nature 554, 145\u2013146 (2018).","DOI":"10.1038\/d41586-018-01683-1"},{"key":"7409_CR3","doi-asserted-by":"publisher","first-page":"eaaz9712","DOI":"10.1126\/scirobotics.aaz9712","volume":"5","author":"D Falanga","year":"2020","unstructured":"Falanga, D., Kleber, K. & Scaramuzza, D. Dynamic obstacle avoidance for quadrotors with event cameras. Sci. Robot. 5, eaaz9712 (2020).","journal-title":"Sci. Robot."},{"key":"7409_CR4","unstructured":"Cruise. Cruise 101: Learn the Basics of How a Cruise Car Navigates City Streets Safely and Efficiently. https:\/\/getcruise.com\/technology (2023)."},{"key":"7409_CR5","unstructured":"Cristovao, N. Tesla\u2019s FSD hardware 4.0 to use cameras with LED flicker mitigation. Not a Tesla App. https:\/\/www.notateslaapp.com\/news\/679\/tesla-s-fsd-hardware-4-0-to-use-new-cameras (2022)."},{"key":"7409_CR6","unstructured":"Sony. Image Sensors for Automotive Use. https:\/\/www.sony-semicon.com\/en\/products\/is\/automotive\/automotive.html (2023)."},{"key":"7409_CR7","unstructured":"Bosch. Multi Purpose Camera: Combination of Classic Cutting Edge Computer Vision Algorithms and Artificial Intelligence Methods. https:\/\/www.bosch-mobility.com\/media\/global\/solutions\/passenger-cars-and-light-commercial-vehicles\/driver-assistance-systems\/multi-camera-system\/multi-purpose-camera\/summary_multi-purpose-camera_en.pdf (2023)."},{"key":"7409_CR8","unstructured":"OmniVision. OX08B4C 8.3 MP Product Brief. https:\/\/www.ovt.com\/wp-content\/uploads\/2022\/01\/OX08B4C-PB-v1.0-WEB.pdf (2023)."},{"key":"7409_CR9","unstructured":"Mobileye. EyeQ: Vision System on a Chip. https:\/\/www.mobileye-vision.com\/uploaded\/eyeq.pdf (2023)."},{"key":"7409_CR10","doi-asserted-by":"crossref","unstructured":"Cui, A., Casas, S., Wong, K., Suo, S. & Urtasun, R. GoRela: go relative for viewpoint-invariant motion forecasting. In Proc. 2023 IEEE International Conference on Robotics and Automation (ICRA) 7801\u20137807 (IEEE, 2022).","DOI":"10.1109\/ICRA48891.2023.10160984"},{"key":"7409_CR11","doi-asserted-by":"crossref","unstructured":"Wang, X., Su, T., Da, F. & Yang, X. ProphNet: efficient agent-centric motion forecasting with anchor-informed proposals. In Proc. 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 21995\u201322003 (IEEE, 2023).","DOI":"10.1109\/CVPR52729.2023.02106"},{"key":"7409_CR12","doi-asserted-by":"crossref","unstructured":"Zhou, Z., Wang, J., Li, Y.-H. & Huang, Y.-K. Query-centric trajectory prediction. In Proc. IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 17863\u201317873 (IEEE, 2023).","DOI":"10.1109\/CVPR52729.2023.01713"},{"key":"7409_CR13","unstructured":"Zeng, W., Liang, M., Liao, R. & Urtasun, R. Systems and methods for actor motion forecasting within a surrounding environment of an autonomous vehicle, US Patent 0347941 (2023)."},{"key":"7409_CR14","unstructured":"Shashua, A., Shalev-Shwartz, S. & Shammah, S. Systems and methods for navigating with sensing uncertainty. US patent 0269277 (2022)."},{"key":"7409_CR15","unstructured":"Naughton, K. Driverless cars\u2019 need for data is sparking a new space race. Bloomberg (17 September 2021)."},{"key":"7409_CR16","doi-asserted-by":"publisher","first-page":"566","DOI":"10.1109\/JSSC.2007.914337","volume":"43","author":"P Lichtsteiner","year":"2008","unstructured":"Lichtsteiner, P., Posch, C. & Delbruck, T. A 128\u2009\u00d7\u2009128 120\u2009dB 15\u2009\u03bcs latency asynchronous temporal contrast vision sensor. IEEE J. Solid State Circuits 43, 566\u2013576 (2008).","journal-title":"IEEE J. Solid State Circuits"},{"key":"7409_CR17","doi-asserted-by":"publisher","first-page":"2333","DOI":"10.1109\/JSSC.2014.2342715","volume":"49","author":"C Brandli","year":"2014","unstructured":"Brandli, C., Berner, R., Yang, M., Liu, Shih-Chii. & Delbruck, T. A 240\u2009\u00d7\u2009180 130\u2009dB 3\u2009\u03bcs latency global shutter spatiotemporal vision sensor. IEEE J. Solid State Circuits 49, 2333\u20132341 (2014).","journal-title":"IEEE J. Solid State Circuits"},{"key":"7409_CR18","doi-asserted-by":"crossref","unstructured":"Sun, Z., Messikommer, N., Gehrig, D. & Scaramuzza, D. ESS: learning event-based semantic segmentation from still images. In Proc. 17th European Conference of Computer Vision (ECCV) 341\u2013357 (ACM, 2022).","DOI":"10.1007\/978-3-031-19830-4_20"},{"key":"7409_CR19","unstructured":"Perot, E., de Tournemire, P., Nitti, D., Masci, J. & Sironi, A. Learning to detect objects with a 1 megapixel event camera. In Proc. Advances in Neural Information Processing Systems 33 (NeurIPS) 16639\u201316652 (eds Larochelle, H. et al.) (2020)."},{"key":"7409_CR20","doi-asserted-by":"crossref","unstructured":"Alonso, I\u00f1igo and Murillo, A. C. EV-SegNet: semantic segmentation for event-based cameras. In Proc. 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 1624\u20131633 (IEEE, 2019).","DOI":"10.1109\/CVPRW.2019.00205"},{"key":"7409_CR21","doi-asserted-by":"crossref","unstructured":"Tulyakov, S., Fleuret, F., Kiefel, M., Gehler, P. & Hirsch, M. Learning an event sequence embedding for dense event-based deep stereo. In Proc. 2019 IEEE\/CVF International Conference on Computer Vision (ICCV) 1527\u20131537 (IEEE, 2019).","DOI":"10.1109\/ICCV.2019.00161"},{"key":"7409_CR22","doi-asserted-by":"crossref","unstructured":"Tulyakov, S. et al. Time lens: event-based video frame interpolation. In Proc. 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 16150\u201316159 (IEEE, 2021).","DOI":"10.1109\/CVPR46437.2021.01589"},{"key":"7409_CR23","doi-asserted-by":"crossref","unstructured":"Gehrig, D., Loquercio, A., Derpanis, K. G. & Scaramuzza, D. End-to-end learning of representations for asynchronous event-based data. In Proc. 2019 IEEE\/CVF International Conference on Computer Vision (ICCV) 5632\u20135642 (IEEE, 2019).","DOI":"10.1109\/ICCV.2019.00573"},{"key":"7409_CR24","doi-asserted-by":"crossref","unstructured":"Zhu, A. Z., Yuan, L., Chaney, K. & Daniilidis, K. Unsupervised event-based learning of optical flow, depth, and egomotion. In Proc. 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 989\u2013997 (IEEE, 2019).","DOI":"10.1109\/CVPR.2019.00108"},{"key":"7409_CR25","doi-asserted-by":"crossref","unstructured":"Rebecq, H., Ranftl, R., Koltun, V. & Scaramuzza, D. Events-to-video: bringing modern computer vision to event cameras. In Proc. 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 3852\u20133861 (IEEE, 2019).","DOI":"10.1109\/CVPR.2019.00398"},{"key":"7409_CR26","doi-asserted-by":"crossref","unstructured":"Iacono, M., Weber, S., Glover, A. & Bartolozzi, C. Towards event-driven object detection with off-the-shelf deep learning. In Proc. 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) 1\u20139 (IEEE, 2018).","DOI":"10.1109\/IROS.2018.8594119"},{"key":"7409_CR27","doi-asserted-by":"crossref","unstructured":"Jian, Z. et al. Mixed frame-\/event-driven fast pedestrian detection. In Proc. 2019 International Conference on Robotics and Automation (ICRA) 8332\u20138338 (IEEE, 2019).","DOI":"10.1109\/ICRA.2019.8793924"},{"key":"7409_CR28","doi-asserted-by":"publisher","first-page":"2975","DOI":"10.1109\/TIP.2022.3162962","volume":"31","author":"J Li","year":"2022","unstructured":"Li, J. et al. Asynchronous spatio-temporal memory network for continuous event-based object detection. IEEE Trans. Image Process. 31, 2975\u20132987 (2022).","journal-title":"IEEE Trans. Image Process."},{"key":"7409_CR29","doi-asserted-by":"crossref","unstructured":"Cannici, M., Ciccone, M., Romanoni, A. & Matteucci, M. A differentiable recurrent surface for asynchronous event-based data. In Proc. European Conference of Computer Vision (ECCV) (eds Vedaldi, A. et al.) Vol. 12365, 136\u2013152 (Springer, 2020).","DOI":"10.1007\/978-3-030-58565-5_9"},{"key":"7409_CR30","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770\u2013778 (IEEE, 2016).","DOI":"10.1109\/CVPR.2016.90"},{"key":"7409_CR31","doi-asserted-by":"crossref","unstructured":"Schaefer, S., Gehrig, D. & Scaramuzza, D. AEGNN: asynchronous event-based graph neural networks. In Proc. Conference of Computer Vision and Pattern Recognition (CVPR) 12371\u201312381 (CVF, 2022).","DOI":"10.1109\/CVPR52688.2022.01205"},{"key":"7409_CR32","doi-asserted-by":"crossref","unstructured":"Li, Y. et al. Graph-based asynchronous event processing for rapid object recognition. In Proc. 2021 IEEE\/CVF International Conference on Computer Vision (ICCV) 914\u2013923 (IEEE, 2021).","DOI":"10.1109\/ICCV48922.2021.00097"},{"key":"7409_CR33","doi-asserted-by":"crossref","unstructured":"Simonovsky, M. & Komodakis, N. Dynamic edge-conditioned filters in convolutional neural networks on graphs. In Proc. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 29\u201338 (IEEE, 2017).","DOI":"10.1109\/CVPR.2017.11"},{"key":"7409_CR34","unstructured":"Ge, Z., Liu, S., Wang, F., Li, Z. & Sun, J. YOLOX: exceeding YOLO series in 2021. Preprint at https:\/\/arxiv.org\/abs\/2107.08430 (2021)."},{"key":"7409_CR35","doi-asserted-by":"crossref","unstructured":"Fey, M., Lenssen, J. E., Weichert, F. & M\u00fcller, H. SplineCNN: fast geometric deep learning with continuous b-spline kernels. In Proc. 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition 869\u2013877 (2018).","DOI":"10.1109\/CVPR.2018.00097"},{"key":"7409_CR36","doi-asserted-by":"crossref","unstructured":"Messikommer, N. A., Gehrig, D., Loquercio, A. & Scaramuzza, D. Event-based asynchronous sparse convolutional networks. In Proc. 16th European Conference of Computer Vision (ECCV) 415\u2013431 (ACM, 2020).","DOI":"10.1007\/978-3-030-58598-3_25"},{"key":"7409_CR37","doi-asserted-by":"crossref","unstructured":"Jouppi, N. P. et al. Ten lessons from three generations shaped Google\u2019s TPUv4i: industrial product. In Proc. 2021 ACM\/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) 1\u201314 (IEEE, 2021).","DOI":"10.1109\/ISCA52012.2021.00010"},{"key":"7409_CR38","doi-asserted-by":"crossref","unstructured":"Chen, Nicholas F. Y. Pseudo-labels for supervised learning on dynamic vision sensor data, applied to object detection under ego-motion. In Proc. 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 757\u2013709 (IEEE, 2018).","DOI":"10.1109\/CVPRW.2018.00107"},{"key":"7409_CR39","doi-asserted-by":"crossref","unstructured":"Cordone, L., Miramond, B. & Thierion, P. Object detection with spiking neural networks on automotive event data. In Proc. 2022 International Joint Conference on Neural Networks (IJCNN) 1\u20138 (IEEE, 2022).","DOI":"10.1109\/IJCNN55064.2022.9892618"},{"key":"7409_CR40","doi-asserted-by":"publisher","first-page":"4947","DOI":"10.1109\/LRA.2021.3068942","volume":"6","author":"M Gehrig","year":"2021","unstructured":"Gehrig, M., Aarents, W., Gehrig, D. & Scaramuzza,D. DSEC: a stereo event camera dataset for driving scenarios. IEEE Robot. Automat.Lett. 6,4947\u20134954 (2021).","journal-title":"IEEE Robot. Automat.Lett."},{"key":"7409_CR41","unstructured":"de Tournemire, P., Nitti, D., Perot, E., Migliore, D. & Sironi, A. A large scale event-based detection dataset for automotive. Preprint at https:\/\/arxiv.org\/abs\/2001.08499 (2020)."},{"key":"7409_CR42","doi-asserted-by":"publisher","first-page":"437","DOI":"10.3389\/fnins.2015.00437","volume":"9","author":"G Orchard","year":"2015","unstructured":"Orchard, G., Jayawant, A., Cohen, G. K. & Thakor, N. Converting static image datasets to spiking neuromorphic datasets using saccades. Front. Neurosci. 9, 437 (2015).","journal-title":"Front. Neurosci."},{"key":"7409_CR43","doi-asserted-by":"crossref","unstructured":"Bi, Y., Chadha, A., Abbas, A., Bourtsoulatze, E. & Andreopoulos, Y. Graph-based object classification for neuromorphic vision sensing. In Proc. 2019 IEEE\/CVF International Conference on Computer Vision (ICCV) 491\u2013501 (IEEE, 2019).","DOI":"10.1109\/ICCV.2019.00058"},{"key":"7409_CR44","doi-asserted-by":"crossref","unstructured":"Deng, Y., Chen, H., Liu, H. & Li, Y. A voxel graph CNN for object classification with event cameras. In Proc. 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 1162\u20131171 (IEEE, 2022).","DOI":"10.1109\/CVPR52688.2022.00124"},{"key":"7409_CR45","doi-asserted-by":"crossref","unstructured":"Mitrokhin, A., Hua, Z., Fermuller, C. & Aloimonos, Y. Learning visual motion segmentation using event surfaces. In Proc. 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 14402\u201314411 (IEEE, 2020).","DOI":"10.1109\/CVPR42600.2020.01442"},{"key":"7409_CR46","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky, O. et al. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211\u2013252 (2015).","journal-title":"Int. J. Comput. Vis."},{"key":"7409_CR47","unstructured":"Fei-Fei, L., Fergus, R. & Perona, P. Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In Proc. 2004 Conference on Computer Vision and Pattern Recognition Workshop 178 (IEEE, 2004)."},{"key":"7409_CR48","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1109\/JSSC.2010.2085952","volume":"46","author":"C Posch","year":"2011","unstructured":"Posch, C., Matolin, D. & Wohlgenannt, R. A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS. IEEE J. Solid State Circuits 46, 259\u2013275 (2011).","journal-title":"IEEE J. Solid State Circuits"},{"key":"7409_CR49","doi-asserted-by":"crossref","unstructured":"Fischer, T. et al. QDTrack: quasi-dense similarity learning for appearance-only multiple object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 45, 15380\u201315393 (2023).","DOI":"10.1109\/TPAMI.2023.3301975"},{"key":"7409_CR50","doi-asserted-by":"crossref","unstructured":"Pang, J. et al. Quasi-dense similarity learning for multiple object tracking. In Proc. 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 164\u2013173 (IEEE, 2021).","DOI":"10.1109\/CVPR46437.2021.00023"},{"key":"7409_CR51","doi-asserted-by":"crossref","unstructured":"Zhou, Z. et al. RGB-event fusion for moving object detection in autonomous driving. In Proc. 2023 IEEE International Conference on Robotics and Automation (ICRA) 7808\u20137815 (IEEE, 2023).","DOI":"10.1109\/ICRA48891.2023.10161563"},{"key":"7409_CR52","unstructured":"Prophesee Evaluation Kit - 2 HD. https:\/\/www.prophesee.ai\/event-based-evk (2023)."},{"key":"7409_CR53","unstructured":"Prophesee. Transfer latency. https:\/\/support.prophesee.ai\/portal\/en\/kb\/articles\/evk-latency (2023)."},{"key":"7409_CR54","doi-asserted-by":"crossref","unstructured":"Cho, H., Cho, J. & Yoon, K.-J. Learning adaptive dense event stereo from the image domain. In Proc. 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 17797\u201317807 (IEEE, 2023).","DOI":"10.1109\/CVPR52729.2023.01707"},{"key":"7409_CR55","unstructured":"Loshchilov, I. & Hutter, F. Decoupled weight decay regularization. In Proc. 2019 International Conference on Learning Representations (OpenReview.net, 2019)."},{"key":"7409_CR56","unstructured":"Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D. & Wilson, A. G. Averaging weights leads to wider optima and better generalization. In Proc. 34th Conference on Uncertainty in Artificial Intelligence (UAI) Vol. 2 (eds Silva, R. et al.) 876\u2013885 (Association For Uncertainty in Artificial Intelligence, 2018)."},{"key":"7409_CR57","doi-asserted-by":"crossref","unstructured":"Graham, B., Engelcke, M. & van der Maaten, L. 3D semantic segmentation with submanifold sparse convolutional networks. In Proc. 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition 9224\u20139232 (IEEE, 2018).","DOI":"10.1109\/CVPR.2018.00961"},{"key":"7409_CR58","doi-asserted-by":"crossref","unstructured":"Cannici, M., Ciccone, M., Romanoni, A. & Matteucci, M. Asynchronous convolutional networks for object detection in neuromorphic cameras. In Proc. 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 1656\u20131665 (IEEE, 2019).","DOI":"10.1109\/CVPRW.2019.00209"},{"key":"7409_CR59","doi-asserted-by":"crossref","unstructured":"Maqueda, A. I., Loquercio, A., Gallego, G., Garc\u00eda, N. & Scaramuzza, D. Event-based vision meets deep learning on steering prediction for self-driving cars. In Proc. 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition 5419\u20135427 (IEEE, 2018).","DOI":"10.1109\/CVPR.2018.00568"},{"key":"7409_CR60","doi-asserted-by":"crossref","unstructured":"Forrai, B., Miki, T., Gehrig, D., Hutter, M. & Scaramuzza, D. Event-based agile object catching with a quadrupedal robot. In Proc. 2023 IEEE International Conference on Robotics and Automation (ICRA) 12177\u201312183 (IEEE, 2023).","DOI":"10.1109\/ICRA48891.2023.10161392"},{"key":"7409_CR61","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T. & Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proc. 2014 IEEE Conference on Computer Vision and Pattern Recognition 580\u2013587 (IEEE, 2014).","DOI":"10.1109\/CVPR.2014.81"},{"key":"7409_CR62","doi-asserted-by":"crossref","unstructured":"Girshick, R. Fast R-CNN. In Proc. 2015 IEEE International Conference on Computer Vision (ICCV) 1440\u20131448 (IEEE, 2015).","DOI":"10.1109\/ICCV.2015.169"},{"key":"7409_CR63","unstructured":"Ren, S., He, K., Girshick, R. & Sun, J. in Advances in Neural Information Processing Systems Vol. 28. (eds Cortes, C. et al.) 91\u201399 (Curran Associates, 2015)."},{"key":"7409_CR64","doi-asserted-by":"crossref","unstructured":"Liu, W. et al. SSD: single shot multibox detector. In Proc. 2016 European Conference of Computer Vision (ECCV) Vol. 9905, 21\u201337 (eds Leibe, B. et al.) (Springer, 2016).","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"7409_CR65","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R. & Farhadi, A. You Only Look Once: unified, real-time object detection. In Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 779\u2013788 (IEEE, 2016).","DOI":"10.1109\/CVPR.2016.91"},{"key":"7409_CR66","unstructured":"Redmon, J. & Farhadi, A. YOLOv3: an incremental improvement. Preprint at https:\/\/arxiv.org\/abs\/1804.02767 (2018)."},{"key":"7409_CR67","doi-asserted-by":"publisher","first-page":"73","DOI":"10.3389\/fnins.2011.00073","volume":"5","author":"G Indiveri","year":"2011","unstructured":"Indiveri, G. et al. Neuromorphic silicon neuron circuits. Front. Neurosci. 5, 73 (2011).","journal-title":"Front. Neurosci."},{"key":"7409_CR68","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1109\/TBCAS.2008.2005781","volume":"3","author":"S Mitra","year":"2009","unstructured":"Mitra, S., Fusi, S. & Indiveri, G. Real-time classification of complex patterns using spike-based learning in neuromorphic vlsi. IEEE Trans. Biomed. Circuits Syst. 3, 32\u201342 (2009).","journal-title":"IEEE Trans. Biomed. Circuits Syst."},{"key":"7409_CR69","doi-asserted-by":"crossref","unstructured":"Sanket, N. et al. EVDodgeNet: deep dynamic obstacle dodging with event cameras. In Proc. 2020 IEEE International Conference on Robotics and Automation (ICRA) 10651\u201310657 (IEEE, 2020).","DOI":"10.1109\/ICRA40945.2020.9196877"},{"key":"7409_CR70","doi-asserted-by":"crossref","unstructured":"Gehrig, M., Shrestha, S. B., Mouritzen, D. & Scaramuzza, D. Event-based angular velocity regression with spiking networks. In Proc. 2020 IEEE International Conference on Robotics and Automation (ICRA) 4195-4202 (IEEE, 2020).","DOI":"10.1109\/ICRA40945.2020.9197133"},{"key":"7409_CR71","doi-asserted-by":"publisher","first-page":"508","DOI":"10.3389\/fnins.2016.00508","volume":"10","author":"JH Lee","year":"2016","unstructured":"Lee, J. H., Delbruck, T. & Pfeiffer, M. Training deep spiking neural networks using backpropagation. Front. Neurosci. 10, 508 (2016).","journal-title":"Front. Neurosci."},{"key":"7409_CR72","doi-asserted-by":"crossref","unstructured":"Amir, A. et al. A low power, fully event-based gesture recognition system. In Proc. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 7388\u20137397 (IEEE, 2017).","DOI":"10.1109\/CVPR.2017.781"},{"key":"7409_CR73","doi-asserted-by":"publisher","first-page":"2706","DOI":"10.1109\/TPAMI.2013.71","volume":"35","author":"JA Perez-Carrasco","year":"2013","unstructured":"Perez-Carrasco, J. A. et al. Mapping from frame-driven to frame-free event-driven vision systems by low-rate rate coding and coincidence processing\u2013application to feedforward ConvNets. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2706\u20132719 (2013).","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"7409_CR74","doi-asserted-by":"crossref","unstructured":"Sironi, A., Brambilla, M., Bourdis, N., Lagorce, X. & Benosman, R. HATS: histograms of averaged time surfaces for robust event-based object classification. In Proc. 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition 1731\u20131740 (IEEE, 2018).","DOI":"10.1109\/CVPR.2018.00186"},{"key":"7409_CR75","doi-asserted-by":"crossref","unstructured":"Sekikawa, Y., Hara, K. & Saito, H. EventNet: asynchronous recursive event processing. In 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 3882\u20133891 (IEEE, 2019).","DOI":"10.1109\/CVPR.2019.00401"},{"key":"7409_CR76","doi-asserted-by":"crossref","unstructured":"Mitrokhin, A., Fermuller, C., Parameshwara, C. & Aloimonos, Y. Event-based moving object detection and tracking. In Proc. 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2018).","DOI":"10.1109\/IROS.2018.8593805"},{"key":"7409_CR77","unstructured":"Qi, C. R., Yi, L., Su, H. & Guibas, L. J. in Advances in Neural Information Processing Systems pages 5099\u20135108 (MIT, 2017)."},{"key":"7409_CR78","doi-asserted-by":"crossref","unstructured":"Groh, F., Wieschollek, P. & Lensch, H. P. A. Flex-convolution (million-scale point-cloud learning beyond grid-worlds). In Proc. Computer Vision \u2013 ACCV 2018 Vol. 11361 (eds Jawahar, C. et al.) 105\u2013122 (Springer, 2018).","DOI":"10.1007\/978-3-030-20887-5_7"},{"key":"7409_CR79","doi-asserted-by":"publisher","first-page":"31","DOI":"10.3390\/biomimetics7010031","volume":"7","author":"J Zhao","year":"2022","unstructured":"Zhao, J., Ji, S., Cai, Z., Zeng, Y. & Wang, Y. Moving object detection and tracking by event frame from neuromorphic vision sensors. Biomimetics 7, 31 (2022).","journal-title":"Biomimetics"},{"key":"7409_CR80","doi-asserted-by":"publisher","first-page":"601","DOI":"10.1007\/s11263-019-01209-w","volume":"128","author":"D Gehrig","year":"2019","unstructured":"Gehrig, D., Rebecq, H., Gallego, G. & Scaramuzza, D. EKLT: asynchronous photometric feature tracking using events and frames. Int. J. Comput. Vis. 128, 601\u2013618 (2019).","journal-title":"Int. J. Comput. Vis."},{"key":"7409_CR81","doi-asserted-by":"publisher","first-page":"148075","DOI":"10.1109\/ACCESS.2020.3015759","volume":"8","author":"L Zhang","year":"2020","unstructured":"Zhang, L., Zhang, H., Chen, J. & Wang, L. Hybrid deblur net: deep non-uniform deblurring with event camera. IEEE Access 8, 148075\u2013148083 (2020).","journal-title":"IEEE Access"},{"key":"7409_CR82","doi-asserted-by":"publisher","first-page":"7489","DOI":"10.1109\/TCSVT.2022.3189480","volume":"32","author":"SMNadim Uddin","year":"2022","unstructured":"Uddin, S. M. Nadim, Ahmed, SoikatHasan & Jung, YongJu Unsupervised deep event stereo for depth estimation. IEEE Trans. Circuits Syst. Video Technol. 32, 7489\u20137504 (2022).","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"7409_CR83","doi-asserted-by":"crossref","unstructured":"Tulyakov, S. et al. Time lens++: event-based frame interpolation with parametric nonlinear flow and multi-scale fusion. In Proc. 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 17734\u201317743 (IEEE, 2022).","DOI":"10.1109\/CVPR52688.2022.01723"},{"key":"7409_CR84","doi-asserted-by":"publisher","first-page":"1330","DOI":"10.1109\/34.888718","volume":"22","author":"Z Zhang","year":"2000","unstructured":"Zhang, Z. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330\u20131334 (2000).","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"7409_CR85","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y. et al. Microsoft COCO: common objects in context. In Proc. 2014 European Conference of Computer Vision (ECCV), 740\u2013755 (Springer, 2014).","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"7409_CR86","unstructured":"Fey, M. & Lenssen, J. E. Fast graph representation learning with PyTorch geometric. In Proc. ICLR 2019 Workshop on Representation Learning on Graphs and Manifolds (ICLR, 2019)."}],"container-title":["Nature"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41586-024-07409-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41586-024-07409-w","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41586-024-07409-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,27]],"date-time":"2024-06-27T14:08:54Z","timestamp":1719497334000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41586-024-07409-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,29]]},"references-count":86,"journal-issue":{"issue":"8014","published-print":{"date-parts":[[2024,5,30]]}},"alternative-id":["7409"],"URL":"https:\/\/doi.org\/10.1038\/s41586-024-07409-w","relation":{},"ISSN":["0028-0836","1476-4687"],"issn-type":[{"value":"0028-0836","type":"print"},{"value":"1476-4687","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,29]]},"assertion":[{"value":"24 August 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 April 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 May 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}