{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T21:10:06Z","timestamp":1774645806628,"version":"3.50.1"},"reference-count":45,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2023,12,13]],"date-time":"2023-12-13T00:00:00Z","timestamp":1702425600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62163014"],"award-info":[{"award-number":["62163014"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Although numerous effective Simultaneous Localization and Mapping (SLAM) systems have been developed, complex dynamic environments continue to present challenges, such as managing moving objects and enabling robots to comprehend environments. This paper focuses on a visual SLAM method specifically designed for complex dynamic environments. Our approach proposes a dynamic feature removal module based on the tight coupling of instance segmentation and multi-view geometric constraints (TSG). This method seamlessly integrates semantic information with geometric constraint data, using the fundamental matrix as a connecting element. In particular, instance segmentation is performed on frames to eliminate all dynamic and potentially dynamic features, retaining only reliable static features for sequential feature matching and acquiring a dependable fundamental matrix. Subsequently, based on this matrix, true dynamic features are identified and removed by capitalizing on multi-view geometry constraints while preserving reliable static features for further tracking and mapping. An instance-level semantic map of the global scenario is constructed to enhance the perception and understanding of complex dynamic environments. The proposed method is assessed on TUM datasets and in real-world scenarios, demonstrating that TSG-SLAM exhibits superior performance in detecting and eliminating dynamic feature points and obtains good localization accuracy in dynamic environments.<\/jats:p>","DOI":"10.3390\/s23249807","type":"journal-article","created":{"date-parts":[[2023,12,13]],"date-time":"2023-12-13T12:00:37Z","timestamp":1702468837000},"page":"9807","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["TSG-SLAM: SLAM Employing Tight Coupling of Instance Segmentation and Geometric Constraints in Complex Dynamic Environments"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-4255-3690","authenticated-orcid":false,"given":"Yongchao","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Intelligent Manufacturing, Taizhou University, Taizhou 318000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuanming","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Ganzhou Polytechnic, Ganzhou 341000, China"},{"name":"School of Electrical and Automation Engineering, East China Jiaotong University, Nanchang 330013, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7188-362X","authenticated-orcid":false,"given":"Pengzhan","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Intelligent Manufacturing, Taizhou University, Taizhou 318000, China"},{"name":"School of Electrical and Automation Engineering, East China Jiaotong University, Nanchang 330013, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1052","DOI":"10.1109\/TPAMI.2007.1049","article-title":"MonoSLAM: Real-Time Single Camera SLAM","volume":"29","author":"Davison","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Klein, G., and Murray, D. (2007, January 13\u201316). Parallel Tracking and Mapping for Small AR Workspaces. Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan.","DOI":"10.1109\/ISMAR.2007.4538852"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1177\/0278364914554813","article-title":"Keyframe-Based Visual\u2013Inertial Odometry Using Nonlinear Optimization","volume":"34","author":"Leutenegger","year":"2015","journal-title":"Int. J. Robot. Res."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","article-title":"ORB-SLAM: A Versatile and Accurate Monocular SLAM System","volume":"31","author":"Montiel","year":"2015","journal-title":"Trans. Rob."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras","volume":"33","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1874","DOI":"10.1109\/TRO.2021.3075644","article-title":"ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual\u2013Inertial, and Multimap SLAM","volume":"37","author":"Campos","year":"2021","journal-title":"IEEE Trans. Robot."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Mccormac, J., Clark, R., Bloesch, M., Davison, A., and Leutenegger, S. (2018, January 5\u20138). Fusion++: Volumetric Object-Level SLAM. Proceedings of the 2018 International Conference on 3D Vision (3DV), Verona, Italy.","DOI":"10.1109\/3DV.2018.00015"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"103632","DOI":"10.1016\/j.robot.2020.103632","article-title":"Object-RPE: Dense 3D Reconstruction and Pose Estimation with Convolutional Neural Networks","volume":"133","author":"Hoang","year":"2020","journal-title":"Robot. Auton. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Hosseinzadeh, M., Li, K., Latif, Y., and Reid, I. (2019, January 20\u201324). Real-Time Monocular Object-Model Aware Sparse SLAM. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793728"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Oberlander, J., Uhl, K., Zollner, J.M., and Dillmann, R. (2008, January 19\u201323). A Region-Based SLAM Algorithm Capturing Metric, Topological, and Semantic Properties. Proceedings of the 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, USA.","DOI":"10.1109\/ROBOT.2008.4543482"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.robot.2014.12.006","article-title":"Semantic Mapping for Mobile Robotics Tasks: A Survey","volume":"66","author":"Kostavelis","year":"2015","journal-title":"Robot. Auton. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"61287","DOI":"10.1109\/ACCESS.2018.2873597","article-title":"Hierarchical Semantic Mapping Using Convolutional Neural Networks for Intelligent Service Robotics","volume":"6","author":"Luo","year":"2018","journal-title":"IEEE Access"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"7041","DOI":"10.1109\/LRA.2021.3097242","article-title":"Topology Aware Object-Level Semantic Mapping Towards More Robust Loop Closure","volume":"6","author":"Lin","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"7809","DOI":"10.1109\/JSTARS.2022.3205746","article-title":"Automated Semantics and Topology Representation of Residential-Building Space Using Floor-Plan Raster Maps","volume":"15","author":"Yang","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"19418","DOI":"10.1007\/s10489-023-04531-6","article-title":"Dynamic Visual Simultaneous Localization and Mapping Based on Semantic Segmentation Module","volume":"53","author":"Jin","year":"2023","journal-title":"Appl. Intell."},{"key":"ref_16","first-page":"715","article-title":"Review of Visual SLAM in Dynamic Environment","volume":"43","author":"Wang","year":"2021","journal-title":"Robot"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"882","DOI":"10.1108\/IR-01-2019-0001","article-title":"Fast and Robust Visual Odometry with a Low-Cost IMU in Dynamic Environments","volume":"46","author":"Yao","year":"2019","journal-title":"Ind. Robot."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yang, D., Bi, S., Wang, W., Yuan, C., Wang, W., Qi, X., and Cai, Y. (2019). DRE-SLAM: Dynamic RGB-D Encoder SLAM for a Differential-Drive Robot. Remote Sens., 11.","DOI":"10.3390\/rs11040380"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.robot.2016.11.012","article-title":"Improving RGB-D SLAM in Dynamic Environments: A Motion Removal Approach","volume":"89","author":"Sun","year":"2017","journal-title":"Robot. Auton. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1016\/j.robot.2018.07.002","article-title":"Motion Removal for Reliable RGB-D SLAM in Dynamic Environments","volume":"108","author":"Sun","year":"2018","journal-title":"Robot. Auton. Syst."},{"key":"ref_22","unstructured":"Tan, W., Liu, H., Dong, Z., Zhang, G., and Bao, H. (2013, January 1\u20134). Robust Monocular SLAM in Dynamic Environments. Proceedings of the 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Adelaide, SA, Australia."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, Y., and Huang, S. (2014, January 10\u201312). Towards Dense Moving Object Segmentation Based Robust Dense RGB-D SLAM in Dynamic Scenarios. Proceedings of the 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV), Singapore.","DOI":"10.1109\/ICARCV.2014.7064596"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Kerl, C., Sturm, J., and Cremers, D. (2013, January 3\u20137). Dense Visual SLAM for RGB-D Cameras. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696650"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Yu, C., Liu, Z., Liu, X.-J., Xie, F., Yang, Y., Wei, Q., and Fei, Q. (2018, January 1\u20135). DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593691"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4076","DOI":"10.1109\/LRA.2018.2860039","article-title":"DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes","volume":"3","author":"Bescos","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"5191","DOI":"10.1109\/LRA.2021.3068640","article-title":"DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM","volume":"6","author":"Bescos","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e7600669","DOI":"10.1155\/2022\/7600669","article-title":"MISD-SLAM: Multimodal Semantic SLAM for Dynamic Environments","volume":"2022","author":"You","year":"2022","journal-title":"Wirel. Commun. Mob. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"23772","DOI":"10.1109\/ACCESS.2021.3050617","article-title":"RDS-SLAM: Real-Time Dynamic SLAM Using Semantic Segmentation Methods","volume":"9","author":"Liu","year":"2021","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s10846-022-01613-4","article-title":"KSF-SLAM: A Key Segmentation Frame Based Semantic SLAM in Dynamic Environments","volume":"105","author":"Zhao","year":"2022","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"6846","DOI":"10.1109\/LRA.2022.3178150","article-title":"TwistSLAM: Constrained SLAM in Dynamic Environment","volume":"7","author":"Gonzalez","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"015402","DOI":"10.1088\/1361-6501\/ac92a0","article-title":"A Robust RGB-D SLAM Based on Multiple Geometric Features and Semantic Segmentation in Dynamic Environments","volume":"34","author":"Kuang","year":"2022","journal-title":"Meas. Sci. Technol."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Runz, M., Buffier, M., and Agapito, L. (2018, January 16\u201320). MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects. Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Munich, Germany.","DOI":"10.1109\/ISMAR.2018.00024"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Xu, B., Li, W., Tzoumanikas, D., Bloesch, M., Davison, A., and Leutenegger, S. (2019, January 20\u201324). MID-Fusion: Octree-Based Object-Level Multi-Instance Dynamic SLAM. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794371"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.ins.2020.12.019","article-title":"DP-SLAM: A Visual SLAM with Moving Probability towards Dynamic Environments","volume":"556","author":"Li","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"6011","DOI":"10.1007\/s00521-021-06764-3","article-title":"YOLO-SLAM: A Semantic SLAM System towards Dynamic Environment with Geometric Constraint","volume":"34","author":"Wu","year":"2022","journal-title":"Neural Comput. Appl."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Li, J., Zhang, R., Liu, Y., Zhang, Z., Fan, R., and Liu, W. (2021). The Method of Static Semantic Map Construction Based on Instance Segmentation and Dynamic Point Elimination. Electronics, 10.","DOI":"10.3390\/electronics10161883"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1002\/rob.22062","article-title":"DE-SLAM: SLAM for Highly Dynamic Environment","volume":"39","author":"Xing","year":"2022","journal-title":"J. Field Robot."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_41","unstructured":"Wang, X., Zhang, R., Kong, T., Li, L., and Shen, C. (2020, January 6\u201312). SOLOv2: Dynamic and Fast Instance Segmentation. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Doll\u00e1r, P. (2015). Microsoft COCO: Common Objects in Context. arXiv.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Rusu, R.B., and Cousins, S. (2011, January 9\u201313). 3D Is Here: Point Cloud Library (PCL). Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980567"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012, January 7\u201312). A Benchmark for the Evaluation of RGB-D SLAM Systems. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6385773"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/s10514-012-9321-0","article-title":"OctoMap: An Efficient Probabilistic 3D Mapping Framework Based on Octrees","volume":"34","author":"Hornung","year":"2013","journal-title":"Auton. Robot."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/24\/9807\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:38:17Z","timestamp":1760132297000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/24\/9807"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,13]]},"references-count":45,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["s23249807"],"URL":"https:\/\/doi.org\/10.3390\/s23249807","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,13]]}}}