{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,26]],"date-time":"2025-12-26T05:16:10Z","timestamp":1766726170364,"version":"3.48.0"},"reference-count":50,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2025,12,24]],"date-time":"2025-12-24T00:00:00Z","timestamp":1766534400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Coupled simultaneous localization and mapping (SLAM) and multi-object tracking have been studied in recent years. Although these tasks achieve promising results, they mainly associate keypoints and objects across frames separately, which limits their robustness in complex dynamic scenes. To overcome this limitation, we propose KOM-SLAM, a tightly coupled SLAM and multi-object tracking framework based on a Graph Neural Network (GNN), which jointly learns keypoint and object associations across frames while estimating ego-poses in a differentiable manner. The framework constructs a spatiotemporal graph over keypoints and object detections for association, and employs a multilayer perceptron (MLP) followed by a sigmoid activation that adaptively adjusts association thresholds based on ego-motion and spatial context. We apply a soft assignment on keypoints to ensure differentiable pose estimation, enabling the pose loss to supervise the association learning directly. Experiments on the KITTI Tracking demonstrate that our method achieves improved performance in both localization and object tracking.<\/jats:p>","DOI":"10.3390\/s26010128","type":"journal-article","created":{"date-parts":[[2025,12,24]],"date-time":"2025-12-24T16:41:19Z","timestamp":1766594479000},"page":"128","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["KOM-SLAM: A GNN-Based Tightly Coupled SLAM and Multi-Object Tracking Framework"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-5244-5779","authenticated-orcid":false,"given":"Jinze","family":"Liu","sequence":"first","affiliation":[{"name":"Graduate School of Information Science and Technology, The University of Tokyo, Tokyo 113-0033, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0295-7204","authenticated-orcid":false,"given":"Ye","family":"Tian","sequence":"additional","affiliation":[{"name":"Institute of Industrial Science, The University of Tokyo, Tokyo 153-8505, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanlei","family":"Gu","sequence":"additional","affiliation":[{"name":"Graduate School of Advanced Science and Engineering, Hiroshima University, Hiroshima 739-8527, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shunsuke","family":"Kamijo","sequence":"additional","affiliation":[{"name":"Institute of Industrial Science, The University of Tokyo, Tokyo 153-8505, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,12,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras","volume":"33","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Shan, T., Englot, B., Meyers, D., Wang, W., Ratti, C., and Rus, D. (2020, January 25\u201329). LIO-SAM: Tightly-Coupled Lidar Inertial Odometry via Smoothing and Mapping. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341176"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Weng, X., Wang, Y., Man, Y., and Kitani, K.M. (2020, January 13\u201319). GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with 2D-3D Multi-Feature Learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00653"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"16047","DOI":"10.1109\/TITS.2025.3581980","article-title":"RobMOT: 3D Multi-Object Tracking Enhancement Through Observational Noise and State Estimation Drift Mitigation in LiDAR Point Clouds","volume":"26","author":"Nagy","year":"2025","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5191","DOI":"10.1109\/LRA.2021.3068640","article-title":"DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM","volume":"6","author":"Bescos","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"17234","DOI":"10.1109\/TITS.2024.3418949","article-title":"DynaQuadric: Dynamic Quadric SLAM for Quadric Initialization, Mapping, and Tracking","volume":"25","author":"Tian","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shen, Y., Li, H., Yi, S., Chen, D., and Wang, X. (2018, January 8\u201314). Person Re-Identification with Deep Similarity-Guided Graph Neural Network. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01267-0_30"},{"key":"ref_8","unstructured":"Li, Y., Gu, C., Dullien, T., Vinyals, O., and Kohli, P. (2019, January 9\u201315). Graph Matching Networks for Learning the Similarity of Graph Structured Objects. Proceedings of the International Conference on Machine Learning (ICML), Long Beach, CA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Sarlin, P.-E., DeTone, D., Malisiewicz, T., and Rabinovich, A. (2020, January 13\u201319). SuperGlue: Learning Feature Matching with Graph Neural Networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00499"},{"key":"ref_10","unstructured":"Cetintas, O., Bras\u00f3, G., and Leal-Taix\u00e9, L. (, January 18\u201322). Unifying Short and Long-Term Tracking with Graph Hierarchies. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"DeTone, D., Malisiewicz, T., and Rabinovich, A. (2018, January 18\u201322). SuperPoint: Self-Supervised Interest Point Detection and Description. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00060"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhang, J., and Singh, S. (2014, January 12\u201316). LOAM: Lidar Odometry and Mapping in Real-Time. Proceedings of the Robotics: Science and Systems, Berkeley, CA, USA.","DOI":"10.15607\/RSS.2014.X.007"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kannapiran, S., Bendapudi, N., Yu, M.-Y., Parikh, D., Berman, S., Vora, A., and Pandey, G. (2023, January 1\u20135). Stereo Visual Odometry with Deep Learning-Based Point and Line Feature Matching Using an Attention Graph Neural Network. Proceedings of the 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA.","DOI":"10.1109\/IROS55552.2023.10341872"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Cui, J., Chen, J., and Li, L. (2024, January 13\u201317). SAGE-ICP: Semantic Information-Assisted ICP. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan.","DOI":"10.1109\/ICRA57147.2024.10610280"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Koledi\u0107, K., Cvi\u0161i\u0107, I., Markovi\u0107, I., and Petrovi\u0107, I. (June, January 29). MOFT: Monocular Odometry Based on Deep Depth and Careful Feature Selection and Tracking. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10160588"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zunair, H., Khan, S., and MHamza, A.B. (2024, January 27\u201330). RSUD20K: A dataset for road scene understanding in autonomous driving. Proceedings of the 2024 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/ICIP51287.2024.10648203"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Runz, M., Buffier, M., and Agapito, L. (2018, January 16\u201320). MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects. Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Munich, Germany.","DOI":"10.1109\/ISMAR.2018.00024"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ballester, I., Font\u00e1n, A., Civera, J., Strobl, K.H., and Triebel, R. (June, January 30). DOT: Dynamic Object Tracking for Visual SLAM. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561452"},{"key":"ref_19","unstructured":"Huang, J., Yang, S., Zhao, Z., Lai, Y.-K., and Hu, S.-M. (November, January 27). ClusterSLAM: A SLAM Backend for Simultaneous Rigid Body Clustering and Motion Estimation. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Huang, J., Yang, S., Mu, T.-J., and Hu, S.-M. (2020, January 13\u201319). ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00224"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Moosmann, F., and Stiller, C. (2013, January 6\u201310). Joint Self-Localization and Tracking of Generic Objects in 3D Range Data. Proceedings of the 2013 IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6630716"},{"key":"ref_22","first-page":"1","article-title":"DYNAM-LVIO: A Dynamic-Object-Aware Lidar Visual Inertial Odometry in Dynamic Urban Environments","volume":"73","author":"Shi","year":"2024","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"925","DOI":"10.1109\/TRO.2019.2909168","article-title":"CubeSLAM: Monocular 3-D Object SLAM","volume":"35","author":"Yang","year":"2019","journal-title":"IEEE Trans. Robot."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"6846","DOI":"10.1109\/LRA.2022.3178150","article-title":"TwistSLAM: Constrained SLAM in Dynamic Environment","volume":"7","author":"Gonzalez","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Qiu, Y., Wang, C., Wang, W., Henein, M., and Scherer, S. (2022, January 23\u201327). AIRDOS: Dynamic SLAM Benefits from Articulated Objects. Proceedings of the 2022 IEEE International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9811667"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1017","DOI":"10.1109\/TIV.2023.3317308","article-title":"DL-SLOT: Tightly-Coupled Dynamic Lidar SLAM and 3D Object Tracking Based on Collaborative Graph Optimization","volume":"9","author":"Tian","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"6600","DOI":"10.1109\/LRA.2024.3405385","article-title":"LIMOT: A Tightly-Coupled System for Lidar-Inertial Odometry and Multi-Object Tracking","volume":"9","author":"Zhu","year":"2024","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"742","DOI":"10.1109\/TITS.2024.3488288","article-title":"LIO-LOT: Tightly-Coupled Multi-Object Tracking and Lidar-Inertial Odometry","volume":"26","author":"Li","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3964","DOI":"10.1109\/TIV.2023.3346040","article-title":"IMM-SLAMMOT: Tightly-Coupled SLAM and IMM-Based Multi-Object Tracking","volume":"9","author":"Ying","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1296","DOI":"10.1109\/LRA.2021.3056072","article-title":"A Switching-Coupled Backend for Simultaneous Localization and Dynamic Object Tracking","volume":"6","author":"Liu","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Lin, Y.-K., Lin, W.-C., and Wang, C.-C. (June, January 29). Asynchronous State Estimation of Simultaneous Ego-Motion Estimation and Multiple Object Tracking for Lidar-Inertial Odometry. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10161269"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wang, Y., Kitani, K., and Weng, X. (June, January 30). Joint Object Detection and Multi-Object Tracking with Graph Neural Networks. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561110"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4640","DOI":"10.1109\/LRA.2021.3068925","article-title":"PTP: Parallelized Tracking and Prediction with Graph Neural Networks and Diversity Sampling","volume":"6","author":"Weng","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Bras\u00f3, G., and Leal-Taix\u00e9, L. (2020, January 13\u201319). Learning a Neural Solver for Multiple Object Tracking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00628"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Bilgi, H.\u00c7., and Alatan, A.A. (2024, January 27\u201330). Bi-Directional Tracklet Embedding for Multi-Object Tracking. Proceedings of the 2024 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/ICIP51287.2024.10648019"},{"key":"ref_36","first-page":"5639914","article-title":"Multiple Object Tracking in Satellite Video with Graph-Based Multi-Clue Fusion Tracker","volume":"62","author":"Chen","year":"2024","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Gao, Y., Xu, H., Li, J., Wang, N., and Gao, X. (2024, January 20\u201327). Multi-Scene Generalized Trajectory Global Graph Solver with Composite Nodes for Multiple Object Tracking. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i3.27953"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Lindenberger, P., Sarlin, P.-E., and Pollefeys, M. (2023, January 4\u20136). LightGlue: Local Feature Matching at Light Speed. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.01616"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Chen, H., Luo, Z., Zhang, J., Zhou, L., Bai, X., Hu, Z., Tai, C.-L., and Quan, L. (2021, January 11\u201317). Learning to Match Features with Seeded Graph Matching Network. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00624"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Shi, Y., Cai, J.-X., Shavit, Y., Mu, T.-J., Feng, W., and Zhang, K. (2022, January 18\u201324). ClusterGNN: Cluster-Based Coarse-to-Fine Graph Neural Network for Efficient Feature Matching. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01219"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Sun, J., Shen, Z., Wang, Y., Bao, H., and Zhou, X. (2021, January 20\u201325). LoFTR: Detector-Free Local Feature Matching with Transformers. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00881"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Shi, S., Wang, X., and Li, H. (2019, January 15\u201320). PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00086"},{"key":"ref_43","unstructured":"Mor\u00e9, J.J. (2006). The Levenberg-Marquardt Algorithm: Implementation and Theory. Numerical Analysis: Proceedings of the Biennial Conference Held at Dundee, Dundee, UK, 28 June\u20131 July 1977, Springer."},{"key":"ref_44","first-page":"3801","article-title":"Theseus: A Library for Differentiable Nonlinear Optimization","volume":"35","author":"Pineda","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision Meets Robotics: The KITTI Dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"4076","DOI":"10.1109\/LRA.2018.2860039","article-title":"DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes","volume":"3","author":"Bescos","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., and Beijbom, O. (2020, January 13\u201319). nuScenes: A multimodal dataset for autonomous driving. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01164"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., and Caine, B. (2020, January 13\u201319). Scalability in perception for autonomous driving: Waymo open dataset. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00252"},{"key":"ref_49","first-page":"16558","article-title":"Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras","volume":"34","author":"Teed","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_50","unstructured":"Teed, Z., and Deng, J. (2018). Deepv2d: Video to depth with differentiable structure from motion. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/1\/128\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,26]],"date-time":"2025-12-26T05:13:21Z","timestamp":1766726001000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/1\/128"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,24]]},"references-count":50,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,1]]}},"alternative-id":["s26010128"],"URL":"https:\/\/doi.org\/10.3390\/s26010128","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2025,12,24]]}}}