{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T11:44:29Z","timestamp":1774352669697,"version":"3.50.1"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2023,3,10]],"date-time":"2023-03-10T00:00:00Z","timestamp":1678406400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,10]],"date-time":"2023-03-10T00:00:00Z","timestamp":1678406400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Deutsches Zentrum f\u00fcr Luft- und Raumfahrt e. V. (DLR)"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Vis Comput"],"published-print":{"date-parts":[[2024,2]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Assessing the security status of maritime infrastructures is a key factor for maritime safety and security. Facilities such as ports and harbors are highly active traffic zones with many different agents and infrastructures present, like containers, trucks or vessels. Conveying security-related information in a concise and easily understandable format can support the decision-making process of stakeholders, such as port authorities, law enforcement agencies and emergency services. In this work, we propose a novel real-time 3D reconstruction framework for enhancing maritime situational awareness pictures by joining temporal 2D video data into a single consistent display. We introduce and verify a pipeline prototype for dynamic 3D reconstruction of maritime objects using a static observer and stereoscopic cameras on an GPU-accelerated embedded device. A simulated dataset of a harbor basin was created and used for real-time processing. Usage of a simulated setup allowed verification against synthetic ground-truth data. The presented pipeline runs entirely on a remote, low-power embedded system with <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\sim $$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mo>\u223c<\/mml:mo>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula>6 Hz. A Nvidia Jetson Xavier AGX module was used, featuring 512 CUDA-cores, 16 GB memory and an ARMv8 64-bit octa-core CPU.<\/jats:p>","DOI":"10.1007\/s00371-023-02802-4","type":"journal-article","created":{"date-parts":[[2023,3,10]],"date-time":"2023-03-10T18:03:32Z","timestamp":1678471412000},"page":"571-584","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Embedded 3D reconstruction of dynamic objects in real time for maritime situational awareness pictures"],"prefix":"10.1007","volume":"40","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8869-282X","authenticated-orcid":false,"given":"Felix","family":"Sattler","sequence":"first","affiliation":[]},{"given":"Borja","family":"Carrillo-Perez","sequence":"additional","affiliation":[]},{"given":"Sarah","family":"Barnes","sequence":"additional","affiliation":[]},{"given":"Karsten","family":"Stebner","sequence":"additional","affiliation":[]},{"given":"Maurice","family":"Stephan","sequence":"additional","affiliation":[]},{"given":"Gregor","family":"Lux","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,3,10]]},"reference":[{"key":"2802_CR1","volume-title":"Review of Maritime Transport 2020","author":"SN Sirimanne","year":"2020","unstructured":"Sirimanne, S.N.: Review of Maritime Transport 2020. United Nations Publications, New York (2020)"},{"key":"2802_CR2","doi-asserted-by":"publisher","unstructured":"Whelan, T., Kaess, M., Johannsson, H., Fallon, M.F., Leonard, J.J., McDonald, J.B.: Real-time large scale dense RGB-D SLAM with volumetric fusion. Int. J. Robotics Res. IJRR (2014). https:\/\/doi.org\/10.1177\/0278364914551008","DOI":"10.1177\/0278364914551008"},{"key":"2802_CR3","doi-asserted-by":"publisher","unstructured":"Engelmann, F., Stuckler, J., Leibe, B.: SAMP: Shape and motion priors for 4d vehicle reconstruction. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) (2017). https:\/\/doi.org\/10.1109\/wacv.2017.51","DOI":"10.1109\/wacv.2017.51"},{"key":"2802_CR4","doi-asserted-by":"publisher","unstructured":"Barsan, I.A., Liu, P., Pollefeys, M., Geiger, A.: Robust dense mapping for large-scale dynamic environments. In: 2018 IEEE International Conference on Robotics and Automation (ICRA) (2018). https:\/\/doi.org\/10.1109\/icra.2018.8462974","DOI":"10.1109\/icra.2018.8462974"},{"issue":"11","key":"2802_CR5","doi-asserted-by":"publisher","first-page":"1231","DOI":"10.1177\/0278364913491297","volume":"32","author":"A Geiger","year":"2013","unstructured":"Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the Kitti dataset. Int. J. Robot. Res. 32(11), 1231\u20131237 (2013). https:\/\/doi.org\/10.1177\/0278364913491297","journal-title":"Int. J. Robot. Res."},{"key":"2802_CR6","doi-asserted-by":"publisher","unstructured":"Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohi, P., Shotton, J., Hodges, S., Fitzgibbon, A.: Kinectfusion: Real-time dense surface mapping and tracking. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 127\u2013136 (2011). https:\/\/doi.org\/10.1109\/ISMAR.2011.6092378","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"2802_CR7","doi-asserted-by":"publisher","unstructured":"Grinvald, M., Tombari, F., Siegwart, R., Nieto, J.: TSDF++: a multi-object formulation for dynamic object tracking and reconstruction. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 14192\u201314198 (2021). https:\/\/doi.org\/10.1109\/ICRA48506.2021.9560923","DOI":"10.1109\/ICRA48506.2021.9560923"},{"key":"2802_CR8","doi-asserted-by":"publisher","unstructured":"Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH \u201996, pp. 303\u2013312. Association for Computing Machinery, New York (1996). https:\/\/doi.org\/10.1145\/237170.237269","DOI":"10.1145\/237170.237269"},{"key":"2802_CR9","doi-asserted-by":"publisher","unstructured":"Nie\u00dfner, M., Zollh\u00f6fer, M., Izadi, S., Stamminger, M.: Real-time 3D reconstruction at scale using voxel hashing. ACM Trans. Graph. 32(6) (2013). https:\/\/doi.org\/10.1145\/2508363.2508374","DOI":"10.1145\/2508363.2508374"},{"key":"2802_CR10","doi-asserted-by":"publisher","unstructured":"Bullinger, S.: Image-based 3D reconstruction of dynamic objects using instance-aware multibody structure from motion. Ph.D. thesis, Karlsruher Institut f\u00fcr Technologie (KIT) (2020). https:\/\/doi.org\/10.5445\/KSP\/1000105589","DOI":"10.5445\/KSP\/1000105589"},{"key":"2802_CR11","doi-asserted-by":"publisher","unstructured":"Strecke, M., St\u00fcckler, J.: EM-fusion: Dynamic object-level SLAM with probabilistic data association. In: Proceedings IEEE\/CVF International Conference on Computer Vision 2019 (ICCV), pp. 5864\u20135873. IEEE (2019). https:\/\/doi.org\/10.1109\/ICCV.2019.00596","DOI":"10.1109\/ICCV.2019.00596"},{"key":"2802_CR12","doi-asserted-by":"publisher","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., Girshick, R.: Mask r-cnn. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980\u20132988 (2017). https:\/\/doi.org\/10.1109\/ICCV.2017.322","DOI":"10.1109\/ICCV.2017.322"},{"key":"2802_CR13","doi-asserted-by":"publisher","unstructured":"Zhang, H., Liu, Z.-Q., Wang, Y.-L.: U-loam: A real-time 3D lidar SLAM system for water-surface scene applications. In: 2022 IEEE International Conference on Unmanned Systems (ICUS), pp. 653\u2013657 (2022). https:\/\/doi.org\/10.1109\/ICUS55513.2022.9986766","DOI":"10.1109\/ICUS55513.2022.9986766"},{"key":"2802_CR14","unstructured":"Blender Foundation: Blender: a 3D modelling and rendering package, Stichting Blender Foundation, Amsterdam (2018). http:\/\/www.blender.org"},{"key":"2802_CR15","doi-asserted-by":"publisher","first-page":"8090","DOI":"10.3390\/s22218090","volume":"22","author":"M Ribeiro","year":"2022","unstructured":"Ribeiro, M., Damas, B., Bernardino, A.: Real-time ship segmentation in maritime surveillance videos using automatically annotated synthetic datasets. Sensors 22, 8090 (2022). https:\/\/doi.org\/10.3390\/s22218090","journal-title":"Sensors"},{"key":"2802_CR16","unstructured":"Denninger, M., Sundermeyer, M., Winkelbauer, D., Olefir, D., Hodan, T., Zidan, Y., Elbadrawy, M., Knauer, M., Katam, H., Lodhi, A.: Blenderproc: Reducing the reality gap with photorealistic rendering. In: Robotics: Science and Systems (RSS) (2020). https:\/\/elib.dlr.de\/139317\/"},{"key":"2802_CR17","doi-asserted-by":"publisher","unstructured":"K\u00e1roly, A.I., Galambos, P.: Automated dataset generation with blender for deep learning-based object segmentation. In: 2022 IEEE 20th Jubilee World Symposium on Applied Machine Intelligence and Informatics (SAMI), pp. 000329\u2013000334 (2022). https:\/\/doi.org\/10.1109\/SAMI54271.2022.9780790","DOI":"10.1109\/SAMI54271.2022.9780790"},{"key":"2802_CR18","doi-asserted-by":"publisher","first-page":"909","DOI":"10.1007\/978-3-319-49409-8_75","volume-title":"Computer Vision - ECCV 2016 Workshops","author":"W Qiu","year":"2016","unstructured":"Qiu, W., Yuille, A.: Unrealcv: connecting computer vision to unreal engine. In: Hua, G., J\u00e9gou, H. (eds.) Computer Vision - ECCV 2016 Workshops, pp. 909\u2013916. Springer, Cham (2016)"},{"key":"2802_CR19","doi-asserted-by":"publisher","unstructured":"Yang, Y., Lu, C.: A stereo matching method for 3d image measurement of long-distance sea surface. J. Mar. Sci. Eng. 9(11) (2021). https:\/\/doi.org\/10.3390\/jmse9111281","DOI":"10.3390\/jmse9111281"},{"key":"2802_CR20","doi-asserted-by":"publisher","unstructured":"Jocher, G., Chaurasia, A., Stoken, A., Borovec, J., Kwon, Y.: ultralytics\/yolov5: v6.1 - TensorRT, TensorFlow Edge TPU and OpenVINO Export and Inference. Zenodo (2022). https:\/\/doi.org\/10.5281\/zenodo.6222936","DOI":"10.5281\/zenodo.6222936"},{"key":"2802_CR21","doi-asserted-by":"publisher","unstructured":"Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) Computer Vision \u2013 ECCV 2014, pp. 740\u2013755. Springer, Cham (2014). https:\/\/doi.org\/10.1007\/978-3-319-10602-1_48","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"2802_CR22","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: Pytorch: An imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019 Alch\u00e9-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32, pp. 8024\u20138035. Curran Associates, Inc. (2019). http:\/\/papers.neurips.cc\/paper\/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf"},{"key":"2802_CR23","unstructured":"Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence - Volume 2. IJCAI\u201981, pp. 674\u2013679. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1981)"},{"issue":"12","key":"2802_CR24","doi-asserted-by":"publisher","first-page":"1649","DOI":"10.1109\/TCSVT.2012.2221191","volume":"22","author":"GJ Sullivan","year":"2012","unstructured":"Sullivan, G.J., Ohm, J.-R., Han, W.-J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649\u20131668 (2012). https:\/\/doi.org\/10.1109\/TCSVT.2012.2221191","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"issue":"2","key":"2802_CR25","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1109\/TPAMI.2007.1166","volume":"30","author":"H Hirschm\u00fcller","year":"2008","unstructured":"Hirschm\u00fcller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328\u2013341 (2008). https:\/\/doi.org\/10.1109\/TPAMI.2007.1166","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"2802_CR26","doi-asserted-by":"publisher","unstructured":"Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings Third International Conference on 3-D Digital Imaging and Modeling, pp. 145\u2013152 (2001). https:\/\/doi.org\/10.1109\/IM.2001.924423","DOI":"10.1109\/IM.2001.924423"},{"key":"2802_CR27","doi-asserted-by":"publisher","unstructured":"Chen, Y., Medioni, G.: Object modeling by registration of multiple range images. In: Proceedings. 1991 IEEE International Conference on Robotics and Automation (ICRA), pp. 2724\u201327293 (1991). https:\/\/doi.org\/10.1109\/ROBOT.1991.132043","DOI":"10.1109\/ROBOT.1991.132043"},{"key":"2802_CR28","unstructured":"Diller, C.: KinectFusionLib (2018). https:\/\/github.com\/chrdiller\/KinectFusionLib"},{"key":"2802_CR29","unstructured":"Merrill, D.: CUB v1.16.0. Nvidia Research (2022). https:\/\/nvlabs.github.io\/cub\/index.html"},{"key":"2802_CR30","doi-asserted-by":"publisher","unstructured":"Carrillo-Perez, B., Barnes, S., Stephan, M.: Ship segmentation and georeferencing from static oblique view images. Sensors 22(7) (2022). https:\/\/doi.org\/10.3390\/s22072713","DOI":"10.3390\/s22072713"},{"key":"2802_CR31","doi-asserted-by":"crossref","unstructured":"Qiao, S., Chen, L.-C., Yuille, A.: Detectors: detecting objects with recursive feature pyramid and switchable atrous convolution. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10213\u201310224 (2021)","DOI":"10.1109\/CVPR46437.2021.01008"},{"key":"2802_CR32","doi-asserted-by":"publisher","unstructured":"Solano-Carrillo, E., Carrillo-Perez, B., Flenker, T., Steiniger, Y., Stoppe, J.: Detection and geovisualization of abnormal vessel behavior from video. In: 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pp. 2193\u20132199 (2021). https:\/\/doi.org\/10.1109\/ITSC48978.2021.9564675","DOI":"10.1109\/ITSC48978.2021.9564675"},{"key":"2802_CR33","doi-asserted-by":"publisher","unstructured":"Wang, C.-Y., Bochkovskiy, A., Liao, H.-Y.M.: Scaled-yolov4: Scaling cross stage partial network. In: 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13024\u201313033 (2021). https:\/\/doi.org\/10.1109\/CVPR46437.2021.01283","DOI":"10.1109\/CVPR46437.2021.01283"},{"key":"2802_CR34","doi-asserted-by":"publisher","unstructured":"Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining. KDD\u201996, pp. 226\u2013231 (1996). https:\/\/doi.org\/10.5555\/3001460.3001507","DOI":"10.5555\/3001460.3001507"},{"key":"2802_CR35","doi-asserted-by":"publisher","unstructured":"Peng, S., Liu, Y., Huang, Q., Zhou, X., Bao, H.: Pvnet: pixel-wise voting network for 6dof pose estimation. In: 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4556\u20134565. IEEE Computer Society, Los Alamitos, CA, USA (2019). https:\/\/doi.org\/10.1109\/CVPR.2019.00469","DOI":"10.1109\/CVPR.2019.00469"}],"container-title":["The Visual Computer"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00371-023-02802-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00371-023-02802-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00371-023-02802-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,23]],"date-time":"2024-01-23T19:05:50Z","timestamp":1706036750000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00371-023-02802-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,10]]},"references-count":35,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,2]]}},"alternative-id":["2802"],"URL":"https:\/\/doi.org\/10.1007\/s00371-023-02802-4","relation":{},"ISSN":["0178-2789","1432-2315"],"issn-type":[{"value":"0178-2789","type":"print"},{"value":"1432-2315","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,10]]},"assertion":[{"value":"8 February 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 March 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This work is not attached to any other third-party stakeholders and does not imply any conflict of interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}