{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:14:35Z","timestamp":1760148875799,"version":"build-2065373602"},"reference-count":31,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2023,6,14]],"date-time":"2023-06-14T00:00:00Z","timestamp":1686700800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science and Technology Council (NSTC)","award":["111-2218-E-A49-028-","111-2218-E-002-039-","111-2622-8-A49-023-","111-2221-E-A49-126-MY3","111-2634-F-A49-013-","110-2221-E-A49-145-MY3"],"award-info":[{"award-number":["111-2218-E-A49-028-","111-2218-E-002-039-","111-2622-8-A49-023-","111-2221-E-A49-126-MY3","111-2634-F-A49-013-","110-2221-E-A49-145-MY3"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper proposes the design of a 360\u00b0 map establishment and real-time simultaneous localization and mapping (SLAM) algorithm based on equirectangular projection. All equirectangular projection images with an aspect ratio of 2:1 are supported for input image types of the proposed system, allowing an unlimited number and arrangement of cameras. Firstly, the proposed system uses dual back-to-back fisheye cameras to capture 360\u00b0 images, followed by the adoption of the perspective transformation with any yaw degree given to shrink the feature extraction area in order to reduce the computational time, as well as retain the 360\u00b0 field of view. Secondly, the oriented fast and rotated brief (ORB) feature points extracted from perspective images with a GPU acceleration are used for tracking, mapping, and camera pose estimation in the system. The 360\u00b0 binary map supports the functions of saving, loading, and online updating to enhance the flexibility, convenience, and stability of the 360\u00b0 system. The proposed system is also implemented on an nVidia Jetson TX2 embedded platform with 1% accumulated RMS error of 250 m. The average performance of the proposed system achieves 20 frames per second (FPS) in the case with a single-fisheye camera of resolution 1024 \u00d7 768, and the system performs panoramic stitching and blending under 1416 \u00d7 708 resolution from a dual-fisheye camera at the same time.<\/jats:p>","DOI":"10.3390\/s23125560","type":"journal-article","created":{"date-parts":[[2023,6,14]],"date-time":"2023-06-14T02:26:56Z","timestamp":1686709616000},"page":"5560","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["360\u00b0 Map Establishment and Real-Time Simultaneous Localization and Mapping Based on Equirectangular Projection for Autonomous Driving Vehicles"],"prefix":"10.3390","volume":"23","author":[{"given":"Bo-Hong","family":"Lin","sequence":"first","affiliation":[{"name":"Institute of Electronics, National Yang Ming Chiao Tung University, Hsinchu 30010, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9434-5899","authenticated-orcid":false,"given":"Vinay M.","family":"Shivanna","sequence":"additional","affiliation":[{"name":"Institute of Electronics, National Yang Ming Chiao Tung University, Hsinchu 30010, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-3367-8678","authenticated-orcid":false,"given":"Jiun-Shiung","family":"Chen","sequence":"additional","affiliation":[{"name":"Lite-On Technology Corporation, Taipei 11492, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0402-2621","authenticated-orcid":false,"given":"Jiun-In","family":"Guo","sequence":"additional","affiliation":[{"name":"Institute of Electronics, National Yang Ming Chiao Tung University, Hsinchu 30010, Taiwan"},{"name":"Pervasive Artificial Intelligence Research (PAIR) Labs, National Yang Ming Chiao Tung University, Hsinchu 30010, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1109\/MRA.2006.1678144","article-title":"Simultaneous localization and mapping: Part I","volume":"Volume 13","author":"Bailey","year":"2006","journal-title":"IEEE Robotics & Automation Magazine"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1109\/MRA.2006.1678144","article-title":"Simultaneous localization and mapping (SLAM): Part II","volume":"Volume 13","author":"Bailey","year":"2006","journal-title":"IEEE Robotics & Automation Magazine"},{"key":"ref_3","unstructured":"Stachniss, C., Leonard, J.J., and Thrun, S. (2016). Springer Handbook of Robotics, Springer."},{"key":"ref_4","unstructured":"Aulinas, J., Petillot, Y.R., Salvi, J., and Llad\u00f3, X. (2008). The SLAM Problem: A Survey, IOS Press."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1561\/1100000049","article-title":"A survey of augmented reality","volume":"8","author":"Billinghurst","year":"2015","journal-title":"Found. Trends\u00ae Hum. Comput. Interact."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Engel, J., Sturm, J., and Cremers, D. (2012, January 7\u201312). Camera-based navigation of a low-cost quadrocopter. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6385458"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Klein, G., and Murray, D. (2007, January 13\u201316). Parallel tracking and mapping for small AR workspaces. Proceedings of the 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2007), Nara, Japan.","DOI":"10.1109\/ISMAR.2007.4538852"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Engel, J., Sch\u00f6ps, T., and Cremers, D. (2014, January 6\u201312). LSD-SLAM: Large-scale direct monocular SLAM. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10605-2_54"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Newcombe, R.A., Lovegrove, S.J., and Davison, A.J. (2011, January 6\u201313). DTAM: Dense tracking and mapping in real-time. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126513"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Bay, H., Tuytelaars, T., and Van Gool, L. (2006, January 7\u201313). Surf: Speeded up robust features. Proceedings of the European Conference on Computer Vision, Graz, Austria.","DOI":"10.1007\/11744023_32"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6\u201313). ORB: An efficient alternative to SIFT or SURF. Proceedings of the IEEE international conference on Computer Vision (ICCV), Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Rosten, E., and Drummond, T. (2006, January 7\u201313). Machine learning for high-speed corner detection. Proceedings of the European Conference on Computer Vision, Graz, Austria.","DOI":"10.1007\/11744023_34"},{"key":"ref_14","unstructured":"Chatila, R., and Laumond, J.P. (1985, January 25\u201328). Position referencing and consistent world modeling for mobile robots. Proceedings of the IEEE International Conference on Robotics and Automation, St. Louis, MO, USA."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","article-title":"ORB-SLAM: A versatile and accurate monocular SLAM system","volume":"31","author":"Tardos","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Triggs, B., McLauchlan, P.F., Hartley, R.I., and Fitzgibbon, A.W. (1999, January 21\u201322). Bundle adjustment\u2014A modern synthesis. Proceedings of the International Workshop on Vision Algorithms, Corfu, Greece.","DOI":"10.1007\/3-540-44480-7_21"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Davison, A.J. (2003, January 13\u201316). Real-time simultaneous localisation and mapping with a single camera. Proceedings of the IEEE International Conference on Computer Vision, Nice, France.","DOI":"10.1109\/ICCV.2003.1238654"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1052","DOI":"10.1109\/TPAMI.2007.1049","article-title":"MonoSLAM: Real-time single camera SLAM","volume":"6","author":"Davison","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"756","DOI":"10.1109\/TPAMI.2004.17","article-title":"An efficient solution to the five-point relative pose problem","volume":"26","year":"2004","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Williams, B., Klein, G., and Reid, I. (2007, January 14\u201321). Real-time SLAM relocalisation. Proceedings of the IEEE International Conference on Computer Vision (ICCV 2007), Rio de Janeiro, Brazil.","DOI":"10.1109\/ICCV.2007.4409115"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1109\/34.206955","article-title":"A multiple-baseline stereo","volume":"15","author":"Okutomi","year":"1993","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/0167-2789(92)90242-F","article-title":"Nonlinear total variation based noise removal algorithms","volume":"60","author":"Rudin","year":"1992","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Dignadice, S.J., Red, J.R., Bautista, A.J., Perol, A., Ollanda, A., and Santos, R. (2022, January 8\u201310). Application of Simultaneous Localization and Mapping in the Development of an Autonomous Robot. Proceedings of the 2022 8th International Conference on Control, Automation and Robotics (ICCAR), Xiamen, China.","DOI":"10.1109\/ICCAR55106.2022.9782658"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Gobhinath, S., Anandapoorani, K., Anitha, K., Sri, D.D., and DivyaDharshini, R. (2021, January 19\u201320). Simultaneous Localization and Mapping [SLAM] of Robotic Operating System for Mobile Robots. Proceedings of the 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India.","DOI":"10.1109\/ICACCS51430.2021.9441758"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Chen, C.H., Wang, C.C., and Lin, S.F. (2020, January 28\u201330). A Navigation Aid for Blind People Based on Visual Simultaneous Localization and Mapping. Proceedings of the 2020 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-Taiwan), Taoyuan, Taiwan.","DOI":"10.1109\/ICCE-Taiwan49838.2020.9258263"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Reynolds, S., Fan, D., Taha, T.M., DeMange, A., and Jenkins, T. (2021, January 16\u201319). An Implementation of Simultaneous Localization and Mapping Using Dynamic Field Theory. Proceedings of the NAECON 2021-IEEE National Aerospace and Electronics Conference, Dayton, OH, USA.","DOI":"10.1109\/NAECON49338.2021.9696440"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Caracciolo, M., Casciotti, O., Lloyd, C., Sola-Thomas, E., Weaver, M., Bielby, K., Sarker, M.A.B., and Imtiaz, M.H. (2022, January 23\u201326). Autonomous Navigation System from Simultaneous Localization and Mapping. Proceedings of the 2022 IEEE 31st Microelectronics Design & Test Symposium (MDTS), Albany, NY, USA.","DOI":"10.1109\/MDTS54894.2022.9826969"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhao, X., Hu, Q., Zhang, X., and Wang, H. (2022, January 15\u201318). An ORB-SLAM3 Autonomous Positioning and Orientation Approach using 360-degree Panoramic Video. Proceedings of the 2022 29th International Conference on Geoinformatics, Beijing, China.","DOI":"10.1109\/Geoinformatics57846.2022.9963855"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Roil, M.K., and Prakash, A. (2023, January 7\u20139). Exploring Possible Applications of ORB SLAM 2 in Education, Healthcare, and Industry: Insights into the Challenges, Features, and Effects. Proceedings of the 2023 IEEE 8th International Conference for Convergence in Technology (I2CT), Lonavla, India.","DOI":"10.1109\/I2CT57861.2023.10126239"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1177\/0278364909103911","article-title":"The new college vision and laser data set","volume":"28","author":"Smith","year":"2009","journal-title":"Int. J. Robot. Res."},{"key":"ref_31","unstructured":"Zhang, Z., Rebecq, H., Forster, C., and Scaramuzza, D. (2016, January 16\u201321). Benefit of large field-of-view cameras for visual odometry. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/12\/5560\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:54:28Z","timestamp":1760126068000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/12\/5560"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,14]]},"references-count":31,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["s23125560"],"URL":"https:\/\/doi.org\/10.3390\/s23125560","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,6,14]]}}}