{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T05:31:22Z","timestamp":1779168682869,"version":"3.51.4"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,11,22]],"date-time":"2024-11-22T00:00:00Z","timestamp":1732233600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Sen. Netw."],"published-print":{"date-parts":[[2024,11,30]]},"abstract":"<jats:p>\n            Drones have witnessed extensive popularity among diverse smart applications, and visual Simultaneous Localization and Mapping (SLAM) technology is commonly used to estimate the six-degrees-of-freedom pose for drone flight control systems. However, traditional image-based SLAM cannot ensure the flight safety of drones, especially in challenging environments such as high-speed flight and high dynamic range scenarios. The event camera, a new vision sensor, holds the potential to enable drones to overcome these challenging scenarios if fused with the image-based SLAM. Unfortunately, the computational demands of event-image fusion SLAM have grown manifold compared with image-based SLAM. Existing research on visual SLAM acceleration cannot achieve real-time operation of event-image fusion SLAM on on-board computing platforms for drones. To fill this gap, we present\n            <jats:italic>TrinitySLAM<\/jats:italic>\n            , a high-accuracy, real-time, low-energy consumption event-image fusion SLAM acceleration framework utilizing Xilinx Zynq, an on-board heterogeneous computing platform. The key innovations of\n            <jats:italic>TrinitySLAM<\/jats:italic>\n            \u00a0 include a fine-grained computation allocation strategy, several novel hardware\u2013software co-acceleration designs, and an efficient data exchange mechanism. We fully implement\n            <jats:italic>TrinitySLAM<\/jats:italic>\n            \u00a0 on the latest Zynq UltraScale+ platform and evaluate its performance on one custom-made drone dataset and four official datasets covering various scenarios. Comprehensive experiments show that\n            <jats:italic>TrinitySLAM<\/jats:italic>\n            \u00a0 improves the pose estimation accuracy by 28% with half end-to-end latency and 1.2\u00d7 energy consumption reduction compared with the most comparable state-of-the-art heterogeneous computing platform acceleration baseline.\n          <\/jats:p>","DOI":"10.1145\/3696420","type":"journal-article","created":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T09:35:53Z","timestamp":1726824953000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["TrinitySLAM: On-board Real-time Event-image Fusion SLAM System for Drones"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3468-9403","authenticated-orcid":false,"given":"Xinjun","family":"Cai","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8347-2657","authenticated-orcid":false,"given":"Jingao","family":"Xu","sequence":"additional","affiliation":[{"name":"School of softwore, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-6203-5685","authenticated-orcid":false,"given":"Kuntian","family":"Deng","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-7333-1863","authenticated-orcid":false,"given":"Hongbo","family":"Lan","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8306-2252","authenticated-orcid":false,"given":"Yue","family":"Wu","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-4256-438X","authenticated-orcid":false,"given":"Xiangwen","family":"Zhuge","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4048-2684","authenticated-orcid":false,"given":"Zheng","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Software, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,11,22]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"5370","volume-title":"2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS\u201919)","author":"Aldegheri Stefano","year":"2019","unstructured":"Stefano Aldegheri, Nicola Bombieri, Domenico D. Bloisi, and Alessandro Farinelli. 2019. Data flow ORB-SLAM for real-time performance on embedded GPU boards. In 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS\u201919). IEEE, 5370\u20135375."},{"issue":"1","key":"e_1_3_1_3_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3561972","article-title":"Edge-SLAM: Edge-assisted visual simultaneous localization and mapping","volume":"22","author":"Ali Ali J. Ben","year":"2022","unstructured":"Ali J. Ben Ali, Marziye Kouroshli, Sofiya Semenova, Zakieh Sadat Hashemifar, Steven Y. Ko, and Karthik Dantu. 2022. Edge-SLAM: Edge-assisted visual simultaneous localization and mapping. ACM Transactions on Embedded Computing Systems 22, 1 (2022), 1\u201331.","journal-title":"ACM Transactions on Embedded Computing Systems"},{"key":"e_1_3_1_4_2","first-page":"1","volume-title":"2016 26th International Conference on Field Programmable Logic and Applications (FPL\u201916)","author":"Boikos Konstantinos","year":"2016","unstructured":"Konstantinos Boikos and Christos-Savvas Bouganis. 2016. Semi-dense SLAM on an FPGA SoC. In 2016 26th International Conference on Field Programmable Logic and Applications (FPL\u201916). IEEE, 1\u20134."},{"issue":"6","key":"e_1_3_1_5_2","article-title":"The comparison and application of corner detection algorithms.","volume":"4","author":"Chen Jie","year":"2009","unstructured":"Jie Chen, Li-hui Zou, Juan Zhang, and Li-hua Dou. 2009. The comparison and application of corner detection algorithms. Journal of Multimedia 4, 6 (2009).","journal-title":"Journal of Multimedia"},{"key":"e_1_3_1_6_2","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1007\/978-3-319-65482-9_5","volume-title":"Algorithms and Architectures for Parallel Processing: 17th International Conference (ICA3PP 2017), Helsinki, Finland, August 21\u201323, 2017, Proceedings","author":"Chen Wenjie","year":"2017","unstructured":"Wenjie Chen, Yangyang Ma, Zhilei Chai, Mingsong Chen, and Daojing He. 2017. An FPGA-based real-time moving object tracking approach. In Algorithms and Architectures for Parallel Processing: 17th International Conference (ICA3PP 2017), Helsinki, Finland, August 21\u201323, 2017, Proceedings. Springer, 65\u201380."},{"key":"e_1_3_1_7_2","unstructured":"Debian.2023. Debian. Retrieved from https:\/\/www.debian.org\/"},{"key":"e_1_3_1_8_2","first-page":"3867","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Gallego Guillermo","year":"2018","unstructured":"Guillermo Gallego, Henri Rebecq, and Davide Scaramuzza. 2018. A unifying contrast maximization framework for event cameras, with applications to motion, depth, and optical flow estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3867\u20133876."},{"key":"e_1_3_1_9_2","first-page":"83","volume-title":"2019 IEEE 30th International Conference on Application-specific Systems, Architectures and Processors (ASAP\u201919)","volume":"2160","author":"Gautier Quentin","year":"2019","unstructured":"Quentin Gautier, Alric Althoff, and Ryan Kastner. 2019. FPGA architectures for real-time dense SLAM. In 2019 IEEE 30th International Conference on Application-specific Systems, Architectures and Processors (ASAP\u201919), Vol. 2160. IEEE, 83\u201390."},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.measurement.2019.03.038"},{"key":"e_1_3_1_11_2","doi-asserted-by":"crossref","first-page":"828","DOI":"10.23919\/DATE51398.2021.9473983","volume-title":"2021 Design, Automation & Test in Europe Conference & Exhibition (DATE\u201921)","author":"Gkeka Maria Rafaela","year":"2021","unstructured":"Maria Rafaela Gkeka, Alexandros Patras, Christos D. Antonopoulos, Spyros Lalis, and Nikolaos Bellas. 2021. FPGA architectures for approximate dense SLAM computing. In 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE\u201921). IEEE, 828\u2013833."},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3570210"},{"key":"e_1_3_1_13_2","first-page":"5781","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Hidalgo-Carri\u00f3 Javier","year":"2022","unstructured":"Javier Hidalgo-Carri\u00f3, Guillermo Gallego, and Davide Scaramuzza. 2022. Event-aided direct sparse odometry. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 5781\u20135790."},{"issue":"4","key":"e_1_3_1_14_2","doi-asserted-by":"crossref","first-page":"1014","DOI":"10.3390\/s18041014","article-title":"A new FPGA architecture of FAST and BRIEF algorithm for on-board corner detection and matching","volume":"18","author":"Huang Jingjin","year":"2018","unstructured":"Jingjin Huang, Guoqing Zhou, Xiang Zhou, and Rongting Zhang. 2018. A new FPGA architecture of FAST and BRIEF algorithm for on-board corner detection and matching. Sensors 18, 4 (2018), 1014.","journal-title":"Sensors"},{"key":"e_1_3_1_15_2","unstructured":"Inivation Inc.2022. DAVIS 346. Retrieved fromhttps:\/\/inivation.com\/wp-content\/uploads\/2019\/08\/DAVIS346.pdf"},{"key":"e_1_3_1_16_2","unstructured":"Xilinx Inc.2020. Petalinux. Retrieved fromhttps:\/\/www.xilinx.com\/products\/design-tools\/embedded-software\/petalinux-sdk.html"},{"key":"e_1_3_1_17_2","unstructured":"Xilinx Inc.2022. MPSoC PS and PL Ethernet Example Projects. Retrieved fromhttps:\/\/xilinx-wiki.atlassian.net\/wiki\/spaces\/A\/pages\/478937213\/MPSoC+PS+and+PL+Ethernet+Example+Projects"},{"key":"e_1_3_1_18_2","unstructured":"Xilinx Inc.2022. Vivado HLS. Retrieved fromhttps:\/\/docs.xilinx.com\/v\/u\/en-US\/dh0012-vivado-high-level-synthesis-hub"},{"key":"e_1_3_1_19_2","unstructured":"Xilinx Inc.2022. Zynq UltraScale+ MPSoC. Retrieved fromhttps:\/\/www.xilinx.com\/products\/silicon-devices\/soc\/zynq-ultrascale-mpsoc.html"},{"key":"e_1_3_1_20_2","first-page":"349","volume-title":"Computer Vision\u2013ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11\u201314, 2016, Proceedings, Part VI 14","author":"Kim Hanme","year":"2016","unstructured":"Hanme Kim, Stefan Leutenegger, and Andrew . Davison. 2016. Real-time 3D reconstruction and 6-DoF tracking with an event camera. In Computer Vision\u2013ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11\u201314, 2016, Proceedings, Part VI 14. Springer, 349\u2013364."},{"key":"e_1_3_1_21_2","first-page":"1","volume-title":"2021 International Conference on Field-Programmable Technology (ICFPT\u201921)","author":"Kojima Akira","year":"2021","unstructured":"Akira Kojima. 2021. Autonomous driving system implemented on robot car using SoC FPGA. In 2021 International Conference on Field-Programmable Technology (ICFPT\u201921). IEEE, 1\u20134."},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/SSRR53300.2021.9597857"},{"key":"e_1_3_1_23_2","first-page":"1","volume-title":"2017 IEEE International Symposium on Circuits and Systems (ISCAS\u201917)","author":"Liu Min","year":"2017","unstructured":"Min Liu and Tobi Delbruck. 2017. Block-matching optical flow for dynamic vision sensors: Algorithm and FPGA implementation. In 2017 IEEE International Symposium on Circuits and Systems (ISCAS\u201917). IEEE, 1\u20134."},{"issue":"9","key":"e_1_3_1_24_2","doi-asserted-by":"crossref","first-page":"5776","DOI":"10.1109\/TCSVT.2022.3156653","article-title":"EDFLOW: Event driven optical flow camera with keypoint detection and adaptive block matching","volume":"32","author":"Liu Min","year":"2022","unstructured":"Min Liu and Tobi Delbruck. 2022. EDFLOW: Event driven optical flow camera with keypoint detection and adaptive block matching. IEEE Transactions on Circuits and Systems for Video Technology 32, 9 (2022), 5776\u20135789.","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"e_1_3_1_25_2","first-page":"1","volume-title":"Proceedings of the 56th Annual Design Automation Conference 2019","author":"Liu Runze","year":"2019","unstructured":"Runze Liu, Jianlei Yang, Yiran Chen, and Weisheng Zhao. 2019. eSLAM: An energy-efficient accelerator for real-time ORB-SLAM on FPGA platform. In Proceedings of the 56th Annual Design Automation Conference 2019. 1\u20136."},{"key":"e_1_3_1_26_2","first-page":"674","volume-title":"7th International Joint Conference on Artificial Intelligence (IJCAI\u201981)","volume":"2","author":"Lucas Bruce D.","year":"1981","unstructured":"Bruce D. Lucas and Takeo Kanade. 1981. An iterative image registration technique with an application to stereo vision. In 7th International Joint Conference on Artificial Intelligence (IJCAI\u201981), Vol. 2. 674\u2013679."},{"key":"e_1_3_1_27_2","first-page":"2408","volume-title":"2017 IEEE International Conference on Computer Vision Workshop (ICCVW)","author":"Maity Soumyadip","year":"2017","unstructured":"Soumyadip Maity, Arindam Saha, and Brojeshwar Bhowmick. 2017. Edge SLAM: Edge points based monocular visual SLAM. In 2017 IEEE International Conference on Computer Vision Workshop (ICCVW). IEEE Computer Society, 2408\u20132417."},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2021.114854"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364917691115"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2017.2705103"},{"key":"e_1_3_1_31_2","unstructured":"NVIDIA. 2022. Jetson TX2 Module. Retrieved from https:\/\/developer.nvidia.com\/embedded\/jetson-tx2"},{"key":"e_1_3_1_32_2","unstructured":"NVIDIA.2022. Tegra. Retrieved from https:\/\/developer.nvidia.com\/tegra-hardware-sales-inquiries"},{"key":"e_1_3_1_33_2","unstructured":"OpenAMP.2022. OpenAMP Project. Retrieved from https:\/\/www.openampproject.org\/"},{"key":"e_1_3_1_34_2","unstructured":"OptiTrack.2022. Motion Capture Cameras. Retrieved from https:\/\/optitrack.com\/cameras\/"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2018.2853729"},{"key":"e_1_3_1_36_2","doi-asserted-by":"crossref","unstructured":"Henri Rebecq Timo Horstschaefer and Davide Scaramuzza. 2017. Real-time visual-inertial odometry for event cameras using keyframe-based nonlinear optimization. (2017).","DOI":"10.5244\/C.31.16"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2022.3218774"},{"issue":"6","key":"e_1_3_1_38_2","doi-asserted-by":"crossref","first-page":"789","DOI":"10.1587\/transinf.2020EDP7174","article-title":"An FPGA acceleration and optimization techniques for 2D LiDAR SLAM algorithm","volume":"104","author":"Sugiura Keisuke","year":"2021","unstructured":"Keisuke Sugiura and Hiroki Matsutani. 2021. An FPGA acceleration and optimization techniques for 2D LiDAR SLAM algorithm. IEICE Transactions on Information and Systems 104, 6 (2021), 789\u2013800.","journal-title":"IEICE Transactions on Information and Systems"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2018.8594181"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpls.2020.00510"},{"key":"e_1_3_1_41_2","article-title":"ORB-based SLAM accelerator on SoC FPGA","author":"Vemulapati Vibhakar","year":"2022","unstructured":"Vibhakar Vemulapati and Deming Chen. 2022. ORB-based SLAM accelerator on SoC FPGA. arXiv preprint arXiv:2207.08405 (2022).","journal-title":"arXiv preprint arXiv:2207.08405"},{"key":"e_1_3_1_42_2","unstructured":"Antoni Rosinol Vidal Henri Rebecq Timo Horstschaefer and Davide Scaramuzza. 2018. A C++ implementation of UltimateSLAM. Retrieved from https:\/\/github.com\/uzh-rpg\/rpg_ultimate_slam_open (2018)."},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2018.2793357"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2007.899191"},{"key":"e_1_3_1_45_2","unstructured":"Xenomai.2023. Xenomai. Retrieved fromhttps:\/\/www.xenomai.org\/"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM41043.2020.9155438"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2021.3062252"}],"container-title":["ACM Transactions on Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696420","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3696420","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:55Z","timestamp":1750295935000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696420"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,22]]},"references-count":46,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,11,30]]}},"alternative-id":["10.1145\/3696420"],"URL":"https:\/\/doi.org\/10.1145\/3696420","relation":{},"ISSN":["1550-4859","1550-4867"],"issn-type":[{"value":"1550-4859","type":"print"},{"value":"1550-4867","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,22]]},"assertion":[{"value":"2023-09-08","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-27","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}