{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T16:32:51Z","timestamp":1779381171810,"version":"3.53.1"},"reference-count":44,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T00:00:00Z","timestamp":1658880000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Event-based vision is an emerging field of computer vision that offers unique properties, such as asynchronous visual output, high temporal resolutions, and dependence on brightness changes, to generate data. These properties can enable robust high-temporal-resolution object detection and tracking when combined with frame-based vision. In this paper, we present a hybrid, high-temporal-resolution object detection and tracking approach that combines learned and classical methods using synchronized images and event data. Off-the-shelf frame-based object detectors are used for initial object detection and classification. Then, event masks, generated per detection, are used to enable inter-frame tracking at varying temporal resolutions using the event data. Detections are associated across time using a simple, low-cost association metric. Moreover, we collect and label a traffic dataset using the hybrid sensor DAVIS 240c. This dataset is utilized for quantitative evaluation using state-of-the-art detection and tracking metrics. We provide ground truth bounding boxes and object IDs for each vehicle annotation. Further, we generate high-temporal-resolution ground truth data to analyze tracking performance at different temporal rates. Our approach shows promising results, with minimal performance deterioration at higher temporal resolutions (48\u2013384 Hz) when compared with the baseline frame-based performance at 24 Hz.<\/jats:p>","DOI":"10.3390\/jimaging8080210","type":"journal-article","created":{"date-parts":[[2022,7,28]],"date-time":"2022-07-28T03:21:16Z","timestamp":1658978476000},"page":"210","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["High-Temporal-Resolution Object Detection and Tracking Using Images and Events"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9518-2828","authenticated-orcid":false,"given":"Zaid","family":"El Shair","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Michigan-Dearborn, Dearborn, MI 48128, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samir A.","family":"Rawashdeh","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Michigan-Dearborn, Dearborn, MI 48128, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"13-es","DOI":"10.1145\/1177352.1177355","article-title":"Object tracking: A survey","volume":"38","author":"Yilmaz","year":"2006","journal-title":"Acm Comput. Surv."},{"key":"ref_2","first-page":"346","article-title":"A survey on moving object tracking in video","volume":"3","author":"Deori","year":"2014","journal-title":"Int. J. Inf. Theory"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1007\/s11263-020-01393-0","article-title":"MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking","volume":"129","author":"Dendorfer","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"566","DOI":"10.1109\/JSSC.2007.914337","article-title":"A 128 \u00d7 128 120 dB 15\u03bcs Latency Asynchronous Temporal Contrast Vision Sensor","volume":"43","author":"Lichtsteiner","year":"2008","journal-title":"IEEE J. Solid State Circuits"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2333","DOI":"10.1109\/JSSC.2014.2342715","article-title":"A 240 \u00d7 180 130 db 3 \u03bcs latency global shutter spatiotemporal vision sensor","volume":"49","author":"Brandli","year":"2014","journal-title":"IEEE J. Solid State Circuits"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1109\/TPAMI.2020.3008413","article-title":"Event-based vision: A survey","volume":"44","author":"Gallego","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_7","unstructured":"Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"53","DOI":"10.14198\/jhse.2016.111.05","article-title":"High speed cameras for motion analysis in sports science","volume":"11","author":"Pueo","year":"2016","journal-title":"J. Hum. Sport Exerc."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Rebecq, H., Ranftl, R., Koltun, V., and Scaramuzza, D. (2019, January 15\u201320). Events-to-video: Bringing modern computer vision to event cameras. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00398"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1040","DOI":"10.1109\/34.61704","article-title":"Estimating 3D egomotion from perspective image sequence","volume":"12","author":"Burger","year":"1990","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"11714","DOI":"10.1109\/JSEN.2019.2937304","article-title":"A Combined Vision-Based Multiple Object Tracking and Visual Odometry System","volume":"19","author":"Aladem","year":"2019","journal-title":"IEEE Sensors J."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wojke, N., Bewley, A., and Paulus, D. (2017, January 17\u201320). Simple online and realtime tracking with a deep association metric. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.","DOI":"10.1109\/ICIP.2017.8296962"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zheng, L., Tang, M., Chen, Y., Zhu, G., Wang, J., and Lu, H. (2021, January 20\u201325). Improving multiple object tracking with single object tracking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00248"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"107480","DOI":"10.1016\/j.patcog.2020.107480","article-title":"TPM: Multiple object tracking with tracklet-plane matching","volume":"107","author":"Peng","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"104091","DOI":"10.1016\/j.imavis.2020.104091","article-title":"ReMOT: A model-agnostic refinement for multiple object tracking","volume":"106","author":"Yang","year":"2020","journal-title":"Image Vis. Comput."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A New Approach to Linear Filtering and Prediction Problems","volume":"82","author":"Kalman","year":"1960","journal-title":"J. Basic Eng."},{"key":"ref_17","first-page":"1137","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"28","author":"Ren","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yu, F., Wang, D., Shelhamer, E., and Darrell, T. (2018, January 18\u201323). Deep layer aggregation. Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00255"},{"key":"ref_19","unstructured":"Dendorfer, P., Rezatofighi, H., Milan, A., Shi, J., Cremers, D., Reid, I., Roth, S., Schindler, K., and Leal-Taix\u00e9, L. (2020). Mot20: A benchmark for multi object tracking in crowded scenes. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/0146-664X(80)90054-4","article-title":"Euclidean distance mapping","volume":"14","author":"Danielsson","year":"1980","journal-title":"Comput. Graph. Image Process."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Y\u0131lmaz, \u00d6., Simon-Chane, C., and Histace, A. (2021). Evaluation of Event-Based Corner Detectors. J. Imaging, 7.","DOI":"10.3390\/jimaging7020025"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Tedaldi, D., Gallego, G., Mueggler, E., and Scaramuzza, D. (2016, January 13\u201315). Feature detection and tracking with the dynamic and active-pixel vision sensor (DAVIS). Proceedings of the 2016 Second International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP), Krakow, Poland.","DOI":"10.1109\/EBCCSP.2016.7605086"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1007\/s11263-019-01209-w","article-title":"EKLT: Asynchronous, Photometric Feature Tracking using Events and Frames","volume":"128","author":"Gehrig","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Liu, H., Moeys, D.P., Das, G., Neil, D., Liu, S.-C., and Delbr\u00fcck, T. (2016, January 22\u201325). Combined frame-and event-based detection and tracking. Proceedings of the 2016 IEEE International Symposium on Circuits and systems (ISCAS), Montr\u00e9al, QC, Canada.","DOI":"10.1109\/ISCAS.2016.7539103"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"166588","DOI":"10.1109\/ACCESS.2021.3133533","article-title":"Event Camera Based Real-Time Detection and Tracking of Indoor Ground Robots","volume":"9","author":"Iaboni","year":"2021","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Mondal, A., Giraldo, J.H., Bouwmans, T., and Chowdhury, A.S. (2021, January 11\u201317). Moving Object Detection for Event-based Vision using Graph Spectral Clustering. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCVW54120.2021.00103"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Mitrokhin, A., Ferm\u00fcller, C., Parameshwara, C., and Aloimonos, Y. (2018, January 1\u20135). Event-based moving object detection and tracking. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593805"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3996","DOI":"10.1109\/TCSVT.2020.3044287","article-title":"e-tld: Event-based framework for dynamic object tracking","volume":"31","author":"Ramesh","year":"2020","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chen, H., Wu, Q., Liang, Y., Gao, X., and Wang, H. (2019, January 21\u201325). Asynchronous Tracking-by-Detection on Adaptive Time Surfaces for Event-based Object Tracking. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.","DOI":"10.1145\/3343031.3350975"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"82","DOI":"10.3389\/fnbot.2019.00082","article-title":"Robust Event-Based Object Tracking Combining Correlation Filter and CNN Representation","volume":"13","author":"Li","year":"2019","journal-title":"Front. Neurorobotics"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"24895","DOI":"10.1109\/ACCESS.2022.3154895","article-title":"Unsupervised Adaptive Multi-Object Tracking-by-Clustering Algorithm With a Bio-Inspired System","volume":"10","author":"Cabello","year":"2022","journal-title":"IEEE Access"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhang, J., Yang, X., Fu, Y., Wei, X., Yin, B., and Dong, B. (2021, January 11\u201317). Object tracking by jointly exploiting frame and event domain. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.01280"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhao, J., Ji, S., Cai, Z., Zeng, Y., and Wang, Y. (2022). Moving Object Detection and Tracking by Event Frame from Neuromorphic Vision Sensors. Biomimetics, 7.","DOI":"10.3390\/biomimetics7010031"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Barranco, F., Fermuller, C., and Ros, E. (2018, January 1\u20135). Real-time clustering and multi-target tracking using event-based sensors. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593380"},{"key":"ref_35","unstructured":"Ramesh, B., Zhang, S., Lee, Z.W., Gao, Z., Orchard, G., and Xiang, C. (2018, January 3\u20136). Long-term object tracking with a moving event camera. Proceedings of the 29th British Machine Vision Conference, Newcastle, UK."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1177\/0278364917691115","article-title":"The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM","volume":"36","author":"Mueggler","year":"2017","journal-title":"Int. J. Robot. Res."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"548","DOI":"10.1007\/s11263-020-01375-2","article-title":"HOTA: A Higher Order Metric for Evaluating Multi-object Tracking","volume":"129","author":"Luiten","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"ref_38","unstructured":"Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11\u201314). Ssd: Single shot multibox detector. Proceedings of the European conference on computer vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_40","first-page":"54","article-title":"A Variational Approach to Edge Detection","volume":"1983","author":"Canny","year":"1983","journal-title":"AAAI"},{"key":"ref_41","unstructured":"Rosebrock, A. (2021, October 01). Simple Object Tracking with OpenCV. PyImageSearch. Available online: https:\/\/www.pyimagesearch.com\/2018\/07\/23\/simple-object-tracking-with-opencv\/."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Mueggler, E., Huber, B., and Scaramuzza, D. (2014, January 14\u201318). Event-based, 6-DOF pose tracking for high-speed maneuvers. Proceedings of the 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA.","DOI":"10.1109\/IROS.2014.6942940"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"246309","DOI":"10.1155\/2008\/246309","article-title":"Evaluating Multiple Object Tracking Performance: The Clear Mot Metrics","volume":"2008","author":"Bernardin","year":"2008","journal-title":"EURASIP J. Image Video Process."},{"key":"ref_44","unstructured":"Luiten, J., and Hoffhues, A. (2022, June 29). TrackEval. Available online: https:\/\/github.com\/JonathonLuiten\/TrackEval."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/8\/210\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:57:37Z","timestamp":1760140657000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/8\/210"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,27]]},"references-count":44,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["jimaging8080210"],"URL":"https:\/\/doi.org\/10.3390\/jimaging8080210","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202206.0426.v1","asserted-by":"object"}]},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,27]]}}}