{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T17:38:28Z","timestamp":1772041108689,"version":"3.50.1"},"reference-count":36,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2024,1,21]],"date-time":"2024-01-21T00:00:00Z","timestamp":1705795200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Defense Science and Technology Foundation Strengthening Plan","award":["2021-JCJQ-JJ-1020"],"award-info":[{"award-number":["2021-JCJQ-JJ-1020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the continuous evolution of autonomous driving and unmanned driving systems, traditional limitations such as a limited field-of-view, poor ranging accuracy, and real-time display are becoming inadequate to satisfy the requirements of binocular stereo-perception systems. Firstly, we designed a binocular stereo-imaging-perception system with a wide-field-of-view and infrared- and visible light-dual-band fusion. Secondly we proposed a binocular stereo-perception optical imaging system with a wide field-of-view of 120.3\u00b0, which solves the small field-of-view of current binocular stereo-perception systems. Thirdly, For image aberration caused by the wide-field-of-view system design, we propose an ellipsoidal-image-aberration algorithm with a low consumption of memory resources and no loss of field-of-view. This algorithm simultaneously solves visible light and infrared images with an aberration rate of 45% and 47%, respectively. Fourthly, a multi-scale infrared- and visible light-image-fusion algorithm is used, which improves the situational-awareness capabilities of a binocular stereo-sensing system in a scene and enhances image details to improve ranging accuracy. Furthermore, this paper is based on the Taylor model-calibration binocular stereo-sensing system of internal and external parameters for limit correction; the implemented algorithms are integrated into an NVIDIA Jetson TX2 + FPGA hardware framework, enabling near-distance ranging experiments. The fusion-ranging accuracy within 20 m achieved an error of 0.02 m, outperforming both visible light- and infrared-ranging methods. It generates the fusion-ranging-image output with a minimal delay of only 22.31 ms at a frame rate of 50 Hz.<\/jats:p>","DOI":"10.3390\/s24020676","type":"journal-article","created":{"date-parts":[[2024,1,22]],"date-time":"2024-01-22T11:36:41Z","timestamp":1705923401000},"page":"676","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["A Binocular Stereo-Imaging-Perception System with a Wide Field-of-View and Infrared- and Visible Light-Dual-Band Fusion"],"prefix":"10.3390","volume":"24","author":[{"given":"Youpan","family":"Zhu","sequence":"first","affiliation":[{"name":"MOE Key Laboratory of Optoelectronic Imaging Technology and System, Beijing Institute of Technology, Beijing 100081, China"},{"name":"Kunming Institute of Physics, No. 31, Jiaochang East Road, Wuhua District, Kunming 650223, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Yunnan North Optical & Electronic Instrument Co., Ltd., No. 300, Haikou Town, Xishan District, Kunming 650114, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yongkang","family":"Zhou","sequence":"additional","affiliation":[{"name":"Kunming Institute of Physics, No. 31, Jiaochang East Road, Wuhua District, Kunming 650223, China"},{"name":"School of Life Science, Beijing Institute of Technology, 5 South Zhongguancun Street, Haidian District, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weiqi","family":"Jin","sequence":"additional","affiliation":[{"name":"MOE Key Laboratory of Optoelectronic Imaging Technology and System, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lingling","family":"Zhou","sequence":"additional","affiliation":[{"name":"Yunnan North Optical & Electronic Instrument Co., Ltd., No. 300, Haikou Town, Xishan District, Kunming 650114, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guanlin","family":"Wu","sequence":"additional","affiliation":[{"name":"MOE Key Laboratory of Optoelectronic Imaging Technology and System, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yong","family":"Li","sequence":"additional","affiliation":[{"name":"Chengdu Zhongke Information Technology Co., Ltd., No. 1369 Kezhi Road, Xinglong Street, Tianfu New District, Chengdu 610042, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,1,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Srivastav, A., and Mandal, S. (2023, January 7). Radars for Autonomous Driving: A Review of Deep Learning Methods and Challenges. Proceedings of the Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/ACCESS.2023.3312382"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1109\/TITS.2020.2972974","article-title":"Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges","volume":"22","author":"Feng","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1016\/j.inffus.2022.06.009","article-title":"Radar Sensor Network Resource Allocation for Fused Target Tracking: A Brief Review","volume":"86\u201387","author":"Yan","year":"2022","journal-title":"Inf. Fusion"},{"key":"ref_4","unstructured":"Fernandez Llorca, D., Hernandez Martinez, A., and Garcia Daza, I. (2021, January 26). Vision-based Vehicle Speed Estimation for ITS: A Survey. Proceedings of the Computer Vision and Pattern Recognition, Nashville, TN, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","article-title":"ORB-SLAM: A Versatile and Accurate Monocular SLAM System","volume":"31","author":"Montiel","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras","volume":"33","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_7","first-page":"37","article-title":"ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual\u2013Inertial, and Multimap SLAM","volume":"6","author":"Campos","year":"2021","journal-title":"IEEE Trans. Robot."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1126\/science.968482","article-title":"Cooperative computation of stereo disparity","volume":"194","author":"Marr","year":"1976","journal-title":"Science"},{"key":"ref_9","first-page":"301","article-title":"A Computational Theory of Human Stereo Vision","volume":"204","author":"Marr","year":"1979","journal-title":"R. Soc. B Biol. Sci."},{"key":"ref_10","first-page":"5314","article-title":"On the Synergies between Machine Learning and Binocular Stereo for Depth Estimation from Images: A Survey","volume":"44","author":"Poggi","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1109\/TSMC.2017.2779474","article-title":"Enhancing Binocular Depth Estimation Based on Proactive Perception and Action Cyclic Learning for an Autonomous Developmental Robot","volume":"49","author":"Jin","year":"2019","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2794","DOI":"10.1109\/TED.2021.3131430","article-title":"Direct Time-of-Flight Single-Photon Imaging","volume":"69","author":"Gyongy","year":"2022","journal-title":"IEEE Trans. Electron Devices"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"36717","DOI":"10.1364\/OE.27.036717","article-title":"Depth range enhancement of binary defocusing technique based on multi-frequency phase merging","volume":"27","author":"Zhang","year":"2019","journal-title":"Opt. Express"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Real-Moreno, O., Rodr\u00edguez-Qui\u00f1onez, J.C., Sergiyenko, O., Flores-Fuentes, W., Mercorelli, P., and Ram\u00edrez-Hern\u00e1ndez, L.R. (2021, January 20\u201323). Obtaining Object Information from Stereo Vision System for Autonomous Vehicles. Proceedings of the 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE), Kyoto, Japan.","DOI":"10.1109\/ISIE45552.2021.9576262"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2050020","DOI":"10.1142\/S0218001420500202","article-title":"Vision-Based Intelligent Vehicle Road Recognition and Obstacle Detection Method","volume":"34","author":"Yang","year":"2020","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Montemurro, N., Scerrati, A., Ricciardi, L., and Trevisi, G. (2021). The Exoscope in Neurosurgery: An Overview of the Current Literature of Intraoperative Use in Brain and Spine Surgery. J. Clin. Med., 11.","DOI":"10.3390\/jcm11010223"},{"key":"ref_17","first-page":"1209001","article-title":"3D Imaging Using Geometric Light Field: A Review","volume":"48","author":"Yin","year":"2021","journal-title":"Chin. J. Lasers"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Xu, Y., Liu, K., Ni, J., and Li, Q. (2023). 3D reconstruction method based on second-order semiglobal stereo matching and fast point positioning Delaunay triangulation. PLoS ONE, 17.","DOI":"10.1371\/journal.pone.0260466"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"754","DOI":"10.1016\/j.visres.2010.10.009","article-title":"Binocular vision","volume":"51","author":"Blake","year":"2011","journal-title":"Vis. Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1162\/neco.1991.3.1.88","article-title":"Efficient Training of Artificial Neural Networks for Autonomous Navigation","volume":"3","author":"Pomerleau","year":"2014","journal-title":"Neural Comput."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Shao, N., Li, H.G., Liu, L., and Zhang, Z.L. (2010, January 16\u201317). Stereo Vision Robot Obstacle Detection Based on the SIFT. Proceedings of the 2010 Second WRI Global Congress on Intelligent Systems, Wuhan, China.","DOI":"10.1109\/GCIS.2010.168"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sivaraman, S., and Trivedi, M.M. (2012, January 16\u201319). Real-time vehicle detection using parts at intersections. Proceedings of the IEEE Conference on Intelligent Transportation Systems, Anchorage, AK, USA.","DOI":"10.1109\/ITSC.2012.6338886"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Bontar, J., and Lecun, Y. (2015, January 7\u201312). Computing the Stereo Matching Cost with a Convolutional Neural Network. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298767"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, P., Chen, X., and Shen, S. (2019, January 15\u201320). Stereo R-CNN based 3D Object Detection for Autonomous Driving. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00783"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"118573","DOI":"10.1016\/j.eswa.2022.118573","article-title":"Fruit detection and positioning technology for a Camellia oleifera C. Abel orchard based on improved YOLOv4-tiny model and binocular stereo vision","volume":"211","author":"Tang","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"7992","DOI":"10.1021\/acs.cgd.3c00780","article-title":"Free Field of View Infrared Digital Holography for Mineral Crystallization","volume":"23","author":"Huang","year":"2023","journal-title":"Cryst. Growth Des."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ma, K., Zhou, H., Li, J., and Liu, H. (2019, January 1\u20134). Design of Binocular Stereo Vision System with Parallel Optical Axesand Image 3D Reconstruction. Proceedings of the 2019 China-Qatar International Workshop on Artificial Intelligence and Applications to Intelligent Manufacturing (AIAIM), Doha, Qatar.","DOI":"10.1109\/AIAIM.2019.8632788"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.optlaseng.2018.09.011","article-title":"A new microscopic telecentric stereo vision system\u2014Calibration, rectification, and three-dimensional reconstruction","volume":"113","author":"Hu","year":"2019","journal-title":"Opt. Lasers Eng."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2856","DOI":"10.1109\/TIP.2018.2813092","article-title":"Generalization of the Dark Channel Prior for Single Image Restoration","volume":"27","author":"Peng","year":"2018","journal-title":"IEEE Trans. Image Process"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"105013","DOI":"10.1088\/0957-0233\/23\/10\/105013","article-title":"Noniterative calibration of a camera lens with radial distortion","volume":"23","author":"Wu","year":"2012","journal-title":"Meas. Sci. Technol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"113745","DOI":"10.1016\/j.measurement.2023.113745","article-title":"A low delay highly dynamic range infrared imaging system for complex scenes based on FPGA","volume":"223","author":"Zhou","year":"2023","journal-title":"Measurement"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Li, H., Li, L., Jin, W., Song, J., and Zhou, Y. (2023, January 1\u20133). A stereo vision depth estimation method of binocular wide-field infrared camera. Proceedings of the Third International Computing Imaging Conference, Sydney, Australia.","DOI":"10.1117\/12.2688115"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cviu.2007.09.014","article-title":"SURF: Speeded up robust features","volume":"110","author":"Bay","year":"2008","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"18839","DOI":"10.1007\/s11042-021-10646-0","article-title":"2D object recognition: A comparative analysis of SIFT, SURF, and ORB feature descriptors","volume":"80","author":"Bansal","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.infrared.2017.02.005","article-title":"Infrared and visible image fusion based on visual saliency map and weighted least square optimization","volume":"82","author":"Ma","year":"2017","journal-title":"Infrared Phys. Technol."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Mu, Q., Wei, J., Yuan, Z., and Yin, Y. (2021, January 25\u201327). Research on Target Ranging Method Based on Binocular Stereo Vision. Proceedings of the 2021 International Conference on Intelligent Computing, Automation and Applications (ICAA), Nanjing, China.","DOI":"10.1109\/ICAA53760.2021.00023"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/2\/676\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:46:38Z","timestamp":1760103998000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/2\/676"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,21]]},"references-count":36,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["s24020676"],"URL":"https:\/\/doi.org\/10.3390\/s24020676","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,21]]}}}