{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T14:54:29Z","timestamp":1781621669256,"version":"3.54.5"},"reference-count":26,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2023,6,28]],"date-time":"2023-06-28T00:00:00Z","timestamp":1687910400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"publisher","award":["348480"],"award-info":[{"award-number":["348480"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>LiDAR-based simultaneous localization and mapping (SLAM) approaches have obtained considerable success in autonomous robotic systems. This is in part owing to the high accuracy of robust SLAM algorithms and the emergence of new and lower-cost LiDAR products. This study benchmarks the current state-of-the-art LiDAR SLAM algorithms with a multi-modal LiDAR sensor setup, showcasing diverse scanning modalities (spinning and solid state) and sensing technologies, and LiDAR cameras, mounted on a mobile sensing and computing platform. We extend our previous multi-modal multi-LiDAR dataset with additional sequences and new sources of ground truth data. Specifically, we propose a new multi-modal multi-LiDAR SLAM-assisted and ICP-based sensor fusion method for generating ground truth maps. With these maps, we then match real-time point cloud data using a normal distributions transform (NDT) method to obtain the ground truth with a full six-degrees-of-freedom (DOF) pose estimation. These novel ground truth data leverage high-resolution spinning and solid-state LiDARs. We also include new open road sequences with GNSS-RTK data and additional indoor sequences with motion capture (MOCAP) ground truth, complementing the previous forest sequences with MOCAP data. We perform an analysis of the positioning accuracy achieved, comprising ten unique configurations generated by pairing five distinct LiDAR sensors with five SLAM algorithms, to critically compare and assess their respective performance characteristics. We also report the resource utilization in four different computational platforms and a total of five settings (Intel and Jetson ARM CPUs). Our experimental results show that the current state-of-the-art LiDAR SLAM algorithms perform very differently for different types of sensors. More results, code, and the dataset can be found at GitHub.<\/jats:p>","DOI":"10.3390\/rs15133314","type":"journal-article","created":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T01:15:47Z","timestamp":1688001347000},"page":"3314","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["A Benchmark for Multi-Modal LiDAR SLAM with Ground Truth in GNSS-Denied Environments"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-3617-107X","authenticated-orcid":false,"given":"Ha","family":"Sier","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, Fudan University, Shanghai 200433, China"},{"name":"Turku Intelligent Embedded and Robotic Systems Lab, Faculty of Technology, University of Turku, 20014 Turuku, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6556-2213","authenticated-orcid":false,"given":"Qingqing","family":"Li","sequence":"additional","affiliation":[{"name":"Turku Intelligent Embedded and Robotic Systems Lab, Faculty of Technology, University of Turku, 20014 Turuku, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9042-3730","authenticated-orcid":false,"given":"Xianjia","family":"Yu","sequence":"additional","affiliation":[{"name":"Turku Intelligent Embedded and Robotic Systems Lab, Faculty of Technology, University of Turku, 20014 Turuku, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3091-3217","authenticated-orcid":false,"given":"Jorge","family":"Pe\u00f1a Queralta","sequence":"additional","affiliation":[{"name":"Turku Intelligent Embedded and Robotic Systems Lab, Faculty of Technology, University of Turku, 20014 Turuku, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8546-1329","authenticated-orcid":false,"given":"Zhuo","family":"Zou","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, Fudan University, Shanghai 200433, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1793-2694","authenticated-orcid":false,"given":"Tomi","family":"Westerlund","sequence":"additional","affiliation":[{"name":"Turku Intelligent Embedded and Robotic Systems Lab, Faculty of Technology, University of Turku, 20014 Turuku, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1142\/S2301385020500168","article-title":"Multi-sensor fusion for navigation and mapping in autonomous vehicles: Accurate localization in urban environments","volume":"8","author":"Li","year":"2020","journal-title":"Unmanned Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Varney, N., Asari, V.K., and Graehling, Q. (2020, January 13\u201319). DALES: A large-scale aerial LiDAR data set for semantic segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00101"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1109\/JSTARS.2020.2979369","article-title":"An individual tree segmentation method based on watershed algorithm and three-dimensional spatial distribution analysis from airborne LiDAR point clouds","volume":"13","author":"Yang","year":"2020","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Van Nam, D., and Gon-Woo, K. (2021, January 17\u201320). Solid-state LiDAR based-SLAM: A concise review and application. Proceedings of the 2021 IEEE International Conference on Big Data and Smart Computing (BigComp), Jeju Island, Republic of Korea.","DOI":"10.1109\/BigComp51126.2021.00064"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Qingqing, L., Xianjia, Y., Queralta, J.P., and Westerlund, T. (2021, January 6\u201310). Adaptive lidar scan frame integration: Tracking known mavs in 3d point clouds. Proceedings of the 2021 20th International Conference on Advanced Robotics (ICAR), Ljubljana, Slovenia.","DOI":"10.1109\/ICAR53236.2021.9659483"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"5167","DOI":"10.1109\/LRA.2021.3070251","article-title":"Towards high-performance solid-state-lidar-inertial odometry and mapping","volume":"6","author":"Li","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Queralta, J.P., Qingqing, L., Schiano, F., and Westerlund, T. (2022, January 3\u20135). VIO-UWB-based collaborative localization and dense scene reconstruction within heterogeneous multi-robot systems. Proceedings of the 2022 International Conference on Advanced Robotics and Mechatronics (ICARM), IEEE, Guilin, Guangxi, China.","DOI":"10.1109\/ICARM54641.2022.9959470"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Lin, J., and Zhang, F. (August, January 31). Loam livox: A fast, robust, high-precision LiDAR odometry and mapping package for LiDARs of small FoV. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197440"},{"key":"ref_9","unstructured":"Li, Q., Yu, X., Queralta, J.P., and Westerlund, T. (2022, January 23\u201327). Multi-Modal Lidar Dataset for Benchmarking General-Purpose Localization and Mapping Algorithms. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1309","DOI":"10.1109\/TRO.2016.2624754","article-title":"Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age","volume":"32","author":"Cadena","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Rozenberszki, D., and Majdik, A.L. (August, January 31). LOL: Lidar-only odometry and localization in 3D point cloud maps. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197450"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhen, W., Zeng, S., and Soberer, S. (June, January 29). Robust localization and localizability estimation with a rotating laser scanner. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989739"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ye, H., Chen, Y., and Liu, M. (2019, January 20\u201324). Tightly coupled 3d lidar inertial odometry and mapping. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793511"},{"key":"ref_14","first-page":"1","article-title":"LOAM: Lidar Odometry and Mapping in Real-time","volume":"2","author":"Zhang","year":"2014","journal-title":"Robot. Sci. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Shan, T., and Englot, B. (2018, January 1\u20135). Lego-loam: Lightweight and ground-optimized lidar odometry and mapping on variable terrain. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594299"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Li, Q., Nevalainen, P., Pe\u00f1a Queralta, J., Heikkonen, J., and Westerlund, T. (2020). Localization in Unstructured Environments: Towards Autonomous Robots in Forests with Delaunay Triangulation. Remote Sens., 12.","DOI":"10.3390\/rs12111870"},{"key":"ref_17","unstructured":"Nevalainen, P., Movahedi, P., Queralta, J.P., Westerlund, T., and Heikkonen, J. (2022). New Developments and Environmental Applications of Drones, Springer."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3317","DOI":"10.1109\/LRA.2021.3064227","article-title":"Fast-lio: A fast, robust lidar-inertial odometry package by tightly-coupled iterated kalman filter","volume":"6","author":"Xu","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2053","DOI":"10.1109\/TRO.2022.3141876","article-title":"Fast-lio2: Fast direct lidar-inertial odometry","volume":"38","author":"Xu","year":"2022","journal-title":"IEEE Trans. Robot."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lin, J., and Zhang, F. (2022, January 23\u201327). R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9811935"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"958","DOI":"10.1109\/TRO.2021.3094157","article-title":"Viral-fusion: A visual-inertial-ranging-lidar sensor fusion approach","volume":"38","author":"Nguyen","year":"2021","journal-title":"IEEE Trans. Robot."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The kitti dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1476","DOI":"10.1109\/TIM.2011.2180973","article-title":"A software-only PTP synchronization for power system state estimation with PMUs","volume":"61","author":"Lixia","year":"2012","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ramezani, M., Wang, Y., Camurri, M., Wisth, D., Mattamala, M., and Fallon, M. (2020\u201324, January 24). The newer college dataset: Handheld lidar, inertial and vision with ground truth. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9340849"},{"key":"ref_25","unstructured":"Biber, P., and Stra\u00dfer, W. (2003, January 27\u201331). The normal distributions transform: A new approach to laser scan matching. Proceedings of the 2003 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No. 03CH37453), Las Vegas, NV, USA."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012, January 7\u201312). A benchmark for the evaluation of RGB-D SLAM systems. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems(IROS), Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6385773"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/13\/3314\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:02:59Z","timestamp":1760126579000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/13\/3314"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,28]]},"references-count":26,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2023,7]]}},"alternative-id":["rs15133314"],"URL":"https:\/\/doi.org\/10.3390\/rs15133314","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,28]]}}}