{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T07:28:02Z","timestamp":1740122882455,"version":"3.37.3"},"reference-count":51,"publisher":"Springer Science and Business Media LLC","issue":"19","license":[{"start":{"date-parts":[[2022,1,8]],"date-time":"2022-01-08T00:00:00Z","timestamp":1641600000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,8]],"date-time":"2022-01-08T00:00:00Z","timestamp":1641600000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100006302","name":"Universidad de Alcal\u00e1","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100006302","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Multimed Tools Appl"],"published-print":{"date-parts":[[2022,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Autonomous Driving (AD) promises an efficient, comfortable and safe driving experience. Nevertheless, fatalities involving vehicles equipped with Automated Driving Systems (ADSs) are on the rise, especially those related to the perception module of the vehicle. This paper presents a real-time and power-efficient 3D Multi-Object Detection and Tracking (DAMOT) method proposed for Intelligent Vehicles (IV) applications, allowing the vehicle to track <jats:inline-formula><jats:alternatives><jats:tex-math>$$360^{\\circ }$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:msup>\n                    <mml:mn>360<\/mml:mn>\n                    <mml:mo>\u2218<\/mml:mo>\n                  <\/mml:msup>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> surrounding objects as a preliminary stage to perform trajectory forecasting to prevent collisions and anticipate the ego-vehicle to future traffic scenarios. First, we present our DAMOT pipeline based on Fast Encoders for object detection and a combination of a 3D Kalman Filter and Hungarian Algorithm, used for state estimation and data association respectively. We extend our previous work ellaborating a preliminary version of sensor fusion based DAMOT, merging the extracted features by a Convolutional Neural Network (CNN) using camera information for long-term re-identification and obstacles retrieved by the 3D object detector. Both pipelines exploit the concepts of lightweight Linux containers using the Docker approach to provide the system with isolation, flexibility and portability, and standard communication in robotics using the Robot Operating System (ROS). Second, both pipelines are validated using the recently proposed KITTI-3DMOT evaluation tool that demonstrates the full strength of 3D localization and tracking of a MOT system. Finally, the most efficient architecture is validated in some interesting traffic scenarios implemented in the CARLA (Car Learning to Act) open-source driving simulator and in our real-world autonomous electric car using the NVIDIA AGX Xavier, an AI embedded system for autonomous machines, studying its performance in a controlled but realistic urban environment with real-time execution (results).<\/jats:p>","DOI":"10.1007\/s11042-021-11624-2","type":"journal-article","created":{"date-parts":[[2022,1,8]],"date-time":"2022-01-08T20:02:40Z","timestamp":1641672160000},"page":"26915-26940","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["$$360^{\\circ }$$ real-time and power-efficient 3D DAMOT for autonomous driving applications"],"prefix":"10.1007","volume":"81","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3819-3747","authenticated-orcid":false,"given":"Carlos","family":"G\u00f3mez-Hu\u00e9lamo","sequence":"first","affiliation":[]},{"given":"Javier","family":"Del Egido","sequence":"additional","affiliation":[]},{"given":"Luis Miguel","family":"Bergasa","sequence":"additional","affiliation":[]},{"given":"Rafael","family":"Barea","sequence":"additional","affiliation":[]},{"given":"Elena","family":"L\u00f3pez-Guill\u00e9n","sequence":"additional","affiliation":[]},{"given":"Javier","family":"Araluce","sequence":"additional","affiliation":[]},{"given":"Miguel","family":"Antunes","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,1,8]]},"reference":[{"key":"11624_CR1","doi-asserted-by":"crossref","unstructured":"Baser E, Balasubramanian V, Bhattacharyya P, Czarnecki K (2019) Fantrack: 3d multi-object tracking with feature association network. In: 2019 IEEE Intelligent Vehicles Symposium (IV). IEEE 1426\u20131433","DOI":"10.1109\/IVS.2019.8813779"},{"key":"11624_CR2","doi-asserted-by":"publisher","unstructured":"Bernardin K, Stiefelhagen R (2008) Evaluating multiple object tracking performance: The clear mot metrics. EURASIP J Ima Video Proc\u00a02008. https:\/\/doi.org\/10.1155\/2008\/246309","DOI":"10.1155\/2008\/246309"},{"key":"11624_CR3","doi-asserted-by":"crossref","unstructured":"Bewley A, Ge Z, Ott L, Ramos F, Upcroft B (2016) Simple online and realtime tracking. 1602.00763","DOI":"10.1109\/ICIP.2016.7533003"},{"key":"11624_CR4","doi-asserted-by":"crossref","unstructured":"Chiu Hk, Prioletti A, Li J, Bohg J (2020) Probabilistic 3d multi-object tracking for autonomous driving. arXiv preprint\u00a0arXiv:200105673","DOI":"10.1109\/ICRA48506.2021.9561754"},{"key":"11624_CR5","doi-asserted-by":"crossref","unstructured":"Choi W (2015) Near-online multi-target tracking with aggregated local flow descriptor. In: Proceedings of the IEEE international conference on computer vision. 3029\u20133037","DOI":"10.1109\/ICCV.2015.347"},{"key":"11624_CR6","doi-asserted-by":"crossref","unstructured":"Dao MQ, Fr\u00e9mont V (2021) A two-stage data association approach for 3d multi-object tracking. arXiv preprint\u00a0arXiv:210108684","DOI":"10.3390\/s21092894"},{"key":"11624_CR7","doi-asserted-by":"crossref","unstructured":"Del\u00a0Egido J, G\u00f3mez-Hu\u00e9lamo C, Bergasa LM, Barea R, L\u00f3pez-Guill\u00e9n E, Araluce J, Guti\u00e9rrez R, Antunes M (2020) 360 real-time 3d multi-object detection and tracking for autonomous vehicle navigation. In: Workshop of Physical Agents, Springer 241\u2013255","DOI":"10.1007\/978-3-030-62579-5_17"},{"key":"11624_CR8","unstructured":"Dosovitskiy A, Ros G, Codevilla F, Lopez A, Koltun V (2017) Carla: An open urban driving simulator. 1711.03938"},{"key":"11624_CR9","unstructured":"Fan R, Wang L, Bocus MJ, Pitas I (2020) Computer stereo vision for autonomous driving. arXiv preprint\u00a0arXiv:201203194"},{"key":"11624_CR10","doi-asserted-by":"crossref","unstructured":"Frossard D, Urtasun R (2018) End-to-end learning of multi-sensor 3d tracking by detection. In: 2018 IEEE international conference on robotics and automation (ICRA). IEEE 635\u2013642","DOI":"10.1109\/ICRA.2018.8462884"},{"key":"11624_CR11","doi-asserted-by":"crossref","unstructured":"Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? the kitti vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE 3354\u20133361","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"11624_CR12","doi-asserted-by":"crossref","unstructured":"G\u00f3mez-Huelamo C, Bergasa LM, Barea R, L\u00f3pez-Guill\u00e9n E, Arango F, S\u00e1nchez P (2019) Simulating use cases for the uah autonomous electric car. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC). IEEE. 2305\u20132311","DOI":"10.1109\/ITSC.2019.8917017"},{"key":"11624_CR13","doi-asserted-by":"crossref","unstructured":"G\u00f3mez-Hu\u00e9lamo C, Del\u00a0Egido J, Bergasa LM, Barea R, L\u00f3pez-Guill\u00e9n E, Arango F, Araluce J, L\u00f3pez J (2020) Train here, drive there: Simulating real-world use cases with fully-autonomous driving architecture in carla simulator. In: Workshop of Physical Agents, Springer. 44\u201359","DOI":"10.1007\/978-3-030-62579-5_4"},{"key":"11624_CR14","doi-asserted-by":"crossref","unstructured":"G\u00f3mez-Huelamo C, del Egido J, Bergasa LM, Barea R, Ocaa M, Arango F, G\u00fatierrez R (2020) Real-time bird\u2019s eye view multi-object tracking system based on fast encoders for object detection. In: 2020 IEEE Intelligent Transportation Systems Conference (ITSC). IEEE","DOI":"10.1109\/ITSC45102.2020.9294737"},{"key":"11624_CR15","doi-asserted-by":"crossref","unstructured":"Jullien JM, Martel C, Vignollet L, Wentland M (2009) Openscenario: a flexible integrated environment to develop educational activities based on pedagogical scenarios. In: 2009 Ninth IEEE International Conference on Advanced Learning Technologies. IEEE 509\u2013513","DOI":"10.1109\/ICALT.2009.24"},{"issue":"1","key":"11624_CR16","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1115\/1.3662552","volume":"82","author":"RE Kalman","year":"1960","unstructured":"Kalman RE et al (1960) A new approach to linear filtering and prediction problems. J basic Eng 82(1):35\u201345","journal-title":"J basic Eng"},{"key":"11624_CR17","doi-asserted-by":"publisher","unstructured":"Konigshof H, Salscheider N, Stiller C (2019) Realtime 3d object detection for automated driving using stereo vision and semantic information. 1405\u20131410.\u00a0https:\/\/doi.org\/10.1109\/ITSC.2019.8917330","DOI":"10.1109\/ITSC.2019.8917330"},{"key":"11624_CR18","doi-asserted-by":"crossref","unstructured":"Kuhn HW, Yaw B (1955) The hungarian method for the assignment problem. Naval Res Logist Quart 83\u201397","DOI":"10.1002\/nav.3800020109"},{"key":"11624_CR19","doi-asserted-by":"crossref","unstructured":"Lang AH, Vora S, Caesar H, Zhou L, Yang J, Beijbom O (2019) Pointpillars: Fast encoders for object detection from point clouds. 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR).\u00a0http:\/\/dx.doi.org\/10.1109\/CVPR.2019.01298","DOI":"10.1109\/CVPR.2019.01298"},{"key":"11624_CR20","doi-asserted-by":"crossref","unstructured":"Law H, Deng J (2018) Cornernet: Detecting objects as paired keypoints. In: Proceedings of the European conference on computer vision (ECCV). 734\u2013750","DOI":"10.1007\/978-3-030-01264-9_45"},{"issue":"2","key":"11624_CR21","doi-asserted-by":"publisher","first-page":"548","DOI":"10.1007\/s11263-020-01375-2","volume":"129","author":"J Luiten","year":"2021","unstructured":"Luiten J, Osep A, Dendorfer P, Torr P, Geiger A, Leal-Taix\u00e9 L, Leibe B (2021) Hota: A higher order metric for evaluating multi-object tracking. Int j comp vision 129(2):548\u2013578","journal-title":"Int j comp vision"},{"issue":"239","key":"11624_CR22","first-page":"2","volume":"2014","author":"D Merkel","year":"2014","unstructured":"Merkel D (2014) Docker: lightweight linux containers for consistent development and deployment. Linux j 2014(239):2","journal-title":"Linux j"},{"key":"11624_CR23","doi-asserted-by":"crossref","unstructured":"Mousavian A, Anguelov D, Flynn J, Kosecka J (2017) 3d bounding box estimation using deep learning and geometry. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 7074\u20137082","DOI":"10.1109\/CVPR.2017.597"},{"key":"11624_CR24","doi-asserted-by":"crossref","unstructured":"Osep A, Mehner W, Mathias M, Leibe B (2017) Combined image-and world-space tracking in traffic scenes. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE. 1988\u20131995","DOI":"10.1109\/ICRA.2017.7989230"},{"key":"11624_CR25","doi-asserted-by":"crossref","unstructured":"Patil A, Malla S, Gang H, Chen YT (2019) The h3d dataset for full-surround 3d multi-object detection and tracking in crowded urban scenes. 1903.01568","DOI":"10.1109\/ICRA.2019.8793925"},{"key":"11624_CR26","doi-asserted-by":"crossref","unstructured":"Pirsiavash H, Ramanan D, Fowlkes CC (2011) Globally-optimal greedy algorithms for tracking a variable number of objects. In: CVPR 2011.\u00a0IEEE 1201\u20131208","DOI":"10.1109\/CVPR.2011.5995604"},{"key":"11624_CR27","unstructured":"Qi CR, Yi L, Su H, Guibas LJ (2017) Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In: Advances in neural information processing systems. 5099\u20135108"},{"key":"11624_CR28","doi-asserted-by":"crossref","unstructured":"Qin Z, Wang J, Lu Y (2019) Triangulation learning network: From monocular to stereo 3d object detection. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","DOI":"10.1109\/CVPR.2019.00780"},{"key":"11624_CR29","unstructured":"Quigley M, Conley K, Gerkey B, Faust J, Foote T, Leibs J, Wheeler R, Ng A (2009) Ros: an open-source robot operating system. vol\u00a03"},{"key":"11624_CR30","doi-asserted-by":"crossref","unstructured":"Sanders A (2016) An introduction to unreal engine 4. AK Peters\/CRC Press","DOI":"10.1201\/9781315382555"},{"key":"11624_CR31","doi-asserted-by":"crossref","unstructured":"Scheidegger S, Benjaminsson J, Rosenberg E, Krishnan A, Granstr\u00f6m K (2018) Mono-camera 3d multi-object tracking using deep learning detections and pmbm filtering. In: 2018 IEEE Intelligent Vehicles Symposium (IV). IEEE 433\u2013440","DOI":"10.1109\/IVS.2018.8500454"},{"key":"11624_CR32","doi-asserted-by":"crossref","unstructured":"Sch\u00f6ner H (2017) The role of simulation in development and testing of autonomous vehicles. In: Driving Simulation Conference, Stuttgart","DOI":"10.1007\/978-3-658-21194-3_82"},{"key":"11624_CR33","doi-asserted-by":"crossref","unstructured":"Schulter S, Vernaza P, Choi W, Chandraker M (2017) Deep network flow for multi-object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6951\u20136960","DOI":"10.1109\/CVPR.2017.292"},{"key":"11624_CR34","doi-asserted-by":"crossref","unstructured":"Shi S, Wang X, Li H (2018) Pointrcnn: 3d object proposal generation and detection from point cloud. 1812.04244","DOI":"10.1109\/CVPR.2019.00086"},{"key":"11624_CR35","doi-asserted-by":"crossref","unstructured":"Shi S, Guo C, Jiang L, Wang Z, Shi J, Wang X, Li H (2020a) Pv-rcnn: Point-voxel feature set abstraction for 3d object detection. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 10529\u201310538","DOI":"10.1109\/CVPR42600.2020.01054"},{"key":"11624_CR36","doi-asserted-by":"crossref","unstructured":"Shi S, Wang Z, Shi J, Wang X, Li H (2020b) From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network. IEEE transactions on pattern analysis and machine intelligence","DOI":"10.1109\/TPAMI.2020.2977026"},{"key":"11624_CR37","doi-asserted-by":"crossref","unstructured":"Simon M, Amende K, Kraus A, Honer J, Samann T, Kaulbersch H, Milz S, Michael\u00a0Gross H (2019) Complexer-yolo: Real-time 3d object detection and tracking on semantic point clouds. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 0\u20130","DOI":"10.1109\/CVPRW.2019.00158"},{"key":"11624_CR38","doi-asserted-by":"crossref","unstructured":"Song S, Xiao J (2014) Sliding shapes for 3d object detection in depth images. In: European conference on computer vision, Springer. 634\u2013651","DOI":"10.1007\/978-3-319-10599-4_41"},{"key":"11624_CR39","volume-title":"Definitions for terms related to driving automation systems for on-road motor vehicles (j3016)","author":"S Taxonomy","year":"2016","unstructured":"Taxonomy S (2016) Definitions for terms related to driving automation systems for on-road motor vehicles (j3016). Tech. rep., Technical report, Society for Automotive Engineering"},{"key":"11624_CR40","unstructured":"Team OD (2020) Openpcdet: An open-source toolbox for 3d object detection from point clouds. https:\/\/github.com\/open-mmlab\/OpenPCDet"},{"key":"11624_CR41","doi-asserted-by":"crossref","unstructured":"Voigtlaender P, Krause M, Osep A, Luiten J, Sekar BBG, Geiger A, Leibe B (2019) Mots: Multi-object tracking and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7942\u20137951","DOI":"10.1109\/CVPR.2019.00813"},{"key":"11624_CR42","unstructured":"Weng X, Kitani K (2019a) A baseline for 3d multi-object tracking. 1907.03961"},{"key":"11624_CR43","doi-asserted-by":"crossref","unstructured":"Weng X, Kitani K (2019b) Monocular 3d object detection with pseudo-lidar point cloud. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops. 0\u20130","DOI":"10.1109\/ICCVW.2019.00114"},{"key":"11624_CR44","doi-asserted-by":"crossref","unstructured":"Wojke N, Bewley A, Paulus D (2017) Simple online and realtime tracking with a deep association metric. 1703.07402","DOI":"10.1109\/ICIP.2017.8296962"},{"key":"11624_CR45","unstructured":"Xu Y, Ban Y, Alameda-Pineda X, Horaud R (2019) Deepmot: A differentiable framework for training multiple object trackers. arXiv preprint\u00a0arXiv:190606618"},{"issue":"10","key":"11624_CR46","doi-asserted-by":"publisher","first-page":"3337","DOI":"10.3390\/s18103337","volume":"18","author":"Y Yan","year":"2018","unstructured":"Yan Y, Mao Y, Li B (2018) Second: Sparsely embedded convolutional detection. Sensors 18(10):3337","journal-title":"Sensors"},{"key":"11624_CR47","doi-asserted-by":"publisher","first-page":"58443","DOI":"10.1109\/ACCESS.2020.2983149","volume":"8","author":"E Yurtsever","year":"2020","unstructured":"Yurtsever E, Lambert J, Carballo A, Takeda K (2020) A survey of autonomous driving: Common practices and emerging technologies. IEEE access 8:58443\u201358469","journal-title":"IEEE access"},{"key":"11624_CR48","doi-asserted-by":"crossref","unstructured":"Zhang L, Li Y, Nevatia R (2008) Global data association for multi-object tracking using network flows. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition IEEE. 1\u20138","DOI":"10.1109\/CVPR.2008.4587584"},{"key":"11624_CR49","doi-asserted-by":"crossref","unstructured":"Zhang W, Zhou H, Sun S, Wang Z, Shi J, Loy CC (2019) Robust multi-modality multi-object tracking. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision. 2365\u20132374","DOI":"10.1109\/ICCV.2019.00245"},{"key":"11624_CR50","unstructured":"Zhou X, Wang D, Kr\u00e4henb\u00fchl P (2019) Objects as points. In: arXiv preprint\u00a0arXiv:1904.07850"},{"key":"11624_CR51","doi-asserted-by":"publisher","unstructured":"Zhou Y, Tuzel O (2018) Voxelnet: End-to-end learning for point cloud based 3d object detection. 4490\u20134499.\u00a0https:\/\/doi.org\/10.1109\/CVPR.2018.00472","DOI":"10.1109\/CVPR.2018.00472"}],"container-title":["Multimedia Tools and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-021-11624-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11042-021-11624-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-021-11624-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,19]],"date-time":"2022-07-19T07:18:03Z","timestamp":1658215083000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11042-021-11624-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,8]]},"references-count":51,"journal-issue":{"issue":"19","published-print":{"date-parts":[[2022,8]]}},"alternative-id":["11624"],"URL":"https:\/\/doi.org\/10.1007\/s11042-021-11624-2","relation":{},"ISSN":["1380-7501","1573-7721"],"issn-type":[{"type":"print","value":"1380-7501"},{"type":"electronic","value":"1573-7721"}],"subject":[],"published":{"date-parts":[[2022,1,8]]},"assertion":[{"value":"2 February 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 August 2021","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 September 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 January 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}