{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T15:28:57Z","timestamp":1774538937906,"version":"3.50.1"},"reference-count":40,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,3,13]],"date-time":"2025-03-13T00:00:00Z","timestamp":1741824000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>The online calibration algorithm for camera and LiDAR helps solve the problem of multi-sensor fusion and is of great significance in autonomous driving perception algorithms. Existing online calibration algorithms fail to account for both real-time performance and accuracy. High-precision calibration algorithms require high hardware requirements, while it is difficult for lightweight calibration algorithms to meet the accuracy requirements. Secondly, sensor noise, vibration, and changes in environmental conditions may reduce calibration accuracy. In addition, due to the large domain differences between different public datasets, the existing online calibration algorithms are unstable for various datasets and have poor algorithm robustness. To solve the above problems, we propose an online calibration algorithm based on multi-scale cost volume fusion. First, a multi-layer convolutional network is used to downsample and concatenate the camera RGB data and LiDAR point cloud data to obtain three-scale feature maps. The latter is then subjected to feature concatenation and group-wise correlation processing to generate three sets of cost volumes of different scales. After that, all the cost volumes are spliced and sent to the pose estimation module. After post-processing, the translation and rotation matrix between the camera and LiDAR coordinate systems can be obtained. We tested and verified this method on the KITTI odometry dataset and measured the average translation error of the calibration results to be 0.278 cm, the average rotation error to be 0.020\u00b0, and the single frame took 23 ms, reaching the advanced level.<\/jats:p>","DOI":"10.3390\/info16030223","type":"journal-article","created":{"date-parts":[[2025,3,13]],"date-time":"2025-03-13T12:06:59Z","timestamp":1741867619000},"page":"223","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Online Calibration Method of LiDAR and Camera Based on Fusion of Multi-Scale Cost Volume"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-1063-0804","authenticated-orcid":false,"given":"Xiaobo","family":"Han","sequence":"first","affiliation":[{"name":"School of Automation, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Automation, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-2890-4909","authenticated-orcid":false,"given":"Xiaoxu","family":"Wei","sequence":"additional","affiliation":[{"name":"School of Automotive Engineering, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yongsheng","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Yeong, D.J., Velasco-Hernandez, G., Barry, J., and Walsh, J. (2021). Sensor and sensor fusion technology in autonomous vehicles: A review. Sensors, 21.","DOI":"10.20944\/preprints202102.0459.v1"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1109\/TETCI.2022.3141105","article-title":"A survey of embodied ai: From simulators to research tasks","volume":"6","author":"Duan","year":"2022","journal-title":"IEEE Trans. Emerg. Top. Comput. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.inffus.2023.01.025","article-title":"Multi-sensor integrated navigation\/positioning systems using data fusion: From analytics-based to learning-based approaches","volume":"95","author":"Zhuang","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1016\/j.procs.2021.02.100","article-title":"A survey of LiDAR and camera fusion enhancement","volume":"183","author":"Zhong","year":"2021","journal-title":"Procedia Comput. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wang, P. (2021). Research on comparison of LiDAR and camera in autonomous driving. J. Phys. Conf. Ser., 2093.","DOI":"10.1088\/1742-6596\/2093\/1\/012032"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Grammatikopoulos, L., Papanagnou, A., Venianakis, A., Kalisperakis, I., and Stentoumis, C. (2022). An effective camera-to-LiDAR spatiotemporal calibration based on a simple calibration target. Sensors, 22.","DOI":"10.3390\/s22155576"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Li, Y., Yu, A.W., Meng, T., Caine, B., Ngiam, J., Peng, D., Shen, J., Lu, Y., Zhou, D., and Le, Q.V. (2022, January 18\u201324). Deepfusion: LiDAR-camera deep fusion for multi-modal 3d object detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01667"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"9949","DOI":"10.1007\/s10462-022-10317-y","article-title":"Automatic targetless LiDAR\u2013camera calibration: A survey","volume":"56","author":"Li","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wang, L., and Huang, Y. (2022). LiDAR\u2013camera fusion for road detection using a recurrent conditional random field model. Sci. Rep., 12.","DOI":"10.1038\/s41598-022-14438-w"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"6282","DOI":"10.1109\/TITS.2021.3086804","article-title":"Automotive LiDAR technology: A survey","volume":"23","author":"Roriz","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yan, G., He, F., Shi, C., Wei, P., Cai, X., and Li, Y. (June, January 29). Joint camera intrinsic and LiDAR-camera extrinsic calibration. Proceedings of the 2023 IEEE International Conference On Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10160542"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhu, J., Xue, J., and Zhang, P. (June, January 29). Calibdepth: Unifying depth map representation for iterative LiDAR-camera online calibration. Proceedings of the 2023 IEEE International Conference On Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10161575"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"415","DOI":"10.26599\/TST.2023.9010010","article-title":"Camera, LiDAR, and imu based multi-sensor fusion slam: A survey","volume":"29","author":"Zhu","year":"2023","journal-title":"Tsinghua Sci. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"110417","DOI":"10.1109\/ACCESS.2023.3322229","article-title":"External Extrinsic Calibration of Multi-modal Imaging Sensors: A Review","volume":"11","author":"Liu","year":"2023","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1515\/nleng-2021-0038","article-title":"Research on online calibration of LiDAR and camera for intelligent connected vehicles based on depth-edge matching","volume":"10","author":"Guo","year":"2021","journal-title":"Nonlinear Eng."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yan, G., Liu, Z., Wang, C., Shi, C., Wei, P., Cai, X., Ma, T., Liu, Z., Zhong, Z., and Liu, Y. (2022). Opencalib: A multi-sensor calibration toolbox for autonomous driving. Softw. Impacts, 14.","DOI":"10.1016\/j.simpa.2022.100393"},{"key":"ref_17","first-page":"1","article-title":"Keypoint-based LiDAR-camera online calibration with robust geometric network","volume":"71","author":"Ye","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Shen, Z., Dai, Y., Song, X., Rao, Z., Zhou, D., and Zhang, L. (2022, January 23\u201327). Pcw-net: Pyramid combination and warping cost volume for stereo matching. Proceedings of the European Conference On Computer Vision, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-19824-3_17"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"17677","DOI":"10.1109\/TITS.2022.3155228","article-title":"Automatic extrinsic calibration method for LiDAR and camera sensor setups","volume":"23","author":"Guindel","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ou, J., Huang, P., Zhou, J., Zhao, Y., and Lin, L. (2022). Automatic extrinsic calibration of 3D LiDAR and multi-cameras based on graph optimization. Sensors, 22.","DOI":"10.3390\/s22062221"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3648","DOI":"10.1109\/TASE.2024.3398623","article-title":"GTSCalib: Generalized Target Segmentation for Target-Based Extrinsic Calibration of Non-Repetitive Scanning LiDAR and Camera","volume":"22","author":"Huang","year":"2024","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_22","first-page":"2301","article-title":"Extrinsic calibration of a camera and laser range finder (improves camera calibration)","volume":"Volume 3","author":"Zhang","year":"2004","journal-title":"Proceedings of the 2004 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)"},{"key":"ref_23","unstructured":"Unnikrishnan, R., and Hebert, M. (2005). Fast Extrinsic Calibration of a Laser Rangefinder to a Camera, Robotics Institute. Tech. Rep. CMU-RI-TR-05-09."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"336","DOI":"10.3182\/20100906-3-IT-2019.00059","article-title":"Extrinsic calibration of a 3d laser scanner and an omnidirectional camera","volume":"43","author":"Pandey","year":"2010","journal-title":"IFAC Proc. Vol."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kwak, K., Huber, D.F., Badino, H., and Kanade, T. (2011, January 25\u201330). Extrinsic calibration of a single line scanning LiDAR and a camera. Proceedings of the 2011 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6094490"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"5333","DOI":"10.3390\/s140305333","article-title":"Calibration between color camera and 3d LiDAR instruments with a polygonal planar board","volume":"14","author":"Park","year":"2014","journal-title":"Sensors"},{"key":"ref_27","unstructured":"Dhall, A., Chelani, K., Radhakrishnan, V., and Krishna, K. (2017). LiDAR-camera calibration using 3d-3d point correspondences. arXiv."},{"key":"ref_28","unstructured":"Velas, M., \u0160pan\u011bl, M., Materna, Z., and Herout, A. (2025, February 08). Calibration of Rgb Camera with Velodyne LiDAR. Available online: https:\/\/www.fit.vut.cz\/research\/publication-file\/10578\/Calibration_of_RGB_Camera_With_Velodyne_LiDAR.pdf."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Scaramuzza, D., Harati, A., and Siegwart, R. (November, January 29). Extrinsic self calibration of a camera and a 3d laser range finder from natural scenes. Proceedings of the 2007 IEEE\/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA, USA.","DOI":"10.1109\/IROS.2007.4399276"},{"key":"ref_30","unstructured":"Pandey, G., McBride, J., Savarese, S., and Eustice, R. (2012, January 22\u201323). Automatic targetless extrinsic calibration of a 3d LiDAR and camera by maximizing mutual information. Proceedings of the AAAI Conference on Artificial Intelligence, Toronto, ON, Canada."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1215","DOI":"10.1109\/TRO.2016.2596771","article-title":"Motion-based calibration of multimodal sensor extrinsics and timing offset estimation","volume":"32","author":"Taylor","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Jiang, J., Xue, P., Chen, S., Liu, Z., Zhang, X., and Zheng, N. (2018, January 12\u201314). Line feature based extrinsic calibration of LiDAR and camera. Proceedings of the 2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES), Madrid, Spain.","DOI":"10.1109\/ICVES.2018.8519493"},{"key":"ref_33","unstructured":"Li, L., Li, H., Liu, X., He, D., Miao, Z., Kong, F., Li, R., Liu, Z., and Zhang, F. (2023). Joint intrinsic and extrinsic lidar-camera calibration in targetless environments using plane-constrained bundle adjustment. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Schneider, N., Piewak, F., Stiller, C., and Franke, U. (2017, January 11\u201314). Regnet: Multimodal sensor registration using deep neural networks. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995968"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Iyer, G., Ram, R.K., Murthy, J.K., and Krishna, K.M. (2018, January 1\u20135). Calibnet: Geometrically supervised extrinsic calibration using 3d spatial transformer networks. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593693"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Cattaneo, D., Vaghi, M., Ballardini, A.L., Fontana, S., Sorrenti, D.G., and Burgard, W. (2019, January 27\u201330). Cmrnet: Camera to LiDAR-map registration. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.","DOI":"10.1109\/ITSC.2019.8917470"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Shen, Z., Dai, Y., and Rao, Z. (2021, January 19\u201325). Cfnet: Cascade and fused cost volume for robust stereo matching. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01369"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Lv, X., Wang, B., Dou, Z., Ye, D., and Wang, S. (2021, January 19\u201325). Lccnet: LiDAR and camera self-calibration using cost volume network. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPRW53098.2021.00324"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Luo, Z., Yan, G., and Li, Y. (2023). Calib-anything: Zero-training lidar-camera extrinsic calibration method using segment anything. arXiv.","DOI":"10.1109\/ICRA57147.2024.10610983"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Xiao, Y., Li, Y., Meng, C., Li, X., Ji, J., and Zhang, Y. (2024, January 13\u201317). Calibformer: A transformer-based automatic lidar-camera calibration network. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan.","DOI":"10.1109\/ICRA57147.2024.10610018"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/3\/223\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:53:16Z","timestamp":1760028796000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/3\/223"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,13]]},"references-count":40,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,3]]}},"alternative-id":["info16030223"],"URL":"https:\/\/doi.org\/10.3390\/info16030223","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,13]]}}}