{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T10:16:05Z","timestamp":1771236965952,"version":"3.50.1"},"reference-count":54,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2024,11,15]],"date-time":"2024-11-15T00:00:00Z","timestamp":1731628800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012288","name":"Foundation for Research and Technology Hellas","doi-asserted-by":"publisher","award":["2020 FORTH-Synergy Grant"],"award-info":[{"award-number":["2020 FORTH-Synergy Grant"]}],"id":[{"id":"10.13039\/501100012288","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>In this work, we introduce the first pipeline that combines a refraction-aware structure from motion (SfM) method with a deep learning model specifically designed for airborne bathymetry. We accurately estimate the 3D positions of the submerged points by integrating refraction geometry within the SfM optimization problem. This way, no refraction correction as post-processing is required. Experiments with simulated data that approach real-world capturing conditions demonstrate that SfM with refraction correction is extremely accurate, with submillimeter errors. We integrate our refraction-aware SfM within a deep learning framework that also takes into account radiometrical information, developing a combined spectral and geometry-based approach, with further improvements in accuracy and robustness to different seafloor types, both textured and textureless. We conducted experiments with real-world data at two locations in the southern Mediterranean Sea, with varying seafloor types, which demonstrate the benefits of refraction correction for the deep learning framework. We made our refraction-aware SfM open source, providing researchers in airborne bathymetry with a practical tool to apply SfM in shallow water areas.<\/jats:p>","DOI":"10.3390\/rs16224253","type":"journal-article","created":{"date-parts":[[2024,11,15]],"date-time":"2024-11-15T04:47:17Z","timestamp":1731646037000},"page":"4253","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Refraction-Aware Structure from Motion for Airborne Bathymetry"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6453-3082","authenticated-orcid":false,"given":"Alexandros","family":"Makris","sequence":"first","affiliation":[{"name":"Computational Vision and Robotics Laboratory, Institute of Computer Science, Foundation for Research and Technology Hellas (ICS FORTH), 70013 Heraklion, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8523-6749","authenticated-orcid":false,"given":"Vassilis C.","family":"Nicodemou","sequence":"additional","affiliation":[{"name":"Computational Vision and Robotics Laboratory, Institute of Computer Science, Foundation for Research and Technology Hellas (ICS FORTH), 70013 Heraklion, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7276-8666","authenticated-orcid":false,"given":"Evangelos","family":"Alevizos","sequence":"additional","affiliation":[{"name":"Institute for Mediterranean Studies, Foundation for Research and Technology Hellas (IMS FORTH), 74100 Rethymno, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9503-3723","authenticated-orcid":false,"given":"Iason","family":"Oikonomidis","sequence":"additional","affiliation":[{"name":"Computational Vision and Robotics Laboratory, Institute of Computer Science, Foundation for Research and Technology Hellas (ICS FORTH), 70013 Heraklion, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4070-4654","authenticated-orcid":false,"given":"Dimitrios D.","family":"Alexakis","sequence":"additional","affiliation":[{"name":"Institute for Mediterranean Studies, Foundation for Research and Technology Hellas (IMS FORTH), 74100 Rethymno, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anastasios","family":"Roussos","sequence":"additional","affiliation":[{"name":"Computational Vision and Robotics Laboratory, Institute of Computer Science, Foundation for Research and Technology Hellas (ICS FORTH), 70013 Heraklion, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"4431","DOI":"10.1080\/01431161.2017.1421796","article-title":"Shallow Water Bathymetry Mapping Using Support Vector Machine (SVM) Technique and Multispectral Imagery","volume":"39","author":"Misra","year":"2018","journal-title":"Int. J. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sagawa, T., Yamashita, Y., Okumura, T., and Yamanokuchi, T. (2019). Satellite Derived Bathymetry Using Machine Learning and Multi-Temporal Satellite Images. Remote Sens., 11.","DOI":"10.3390\/rs11101155"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1080\/2150704X.2022.2030068","article-title":"Evaluation of Radiometric Calibration of Drone-Based Imagery for Improving Shallow Bathymetry Retrieval","volume":"13","author":"Alevizos","year":"2022","journal-title":"Remote Sens. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zhou, W., Tang, Y., Jing, W., Li, Y., Yang, J., Deng, Y., and Zhang, Y. (2023). A Comparison of Machine Learning and Empirical Approaches for Deriving Bathymetry from Multispectral Imagery. Remote Sens., 15.","DOI":"10.3390\/rs15020393"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"9","DOI":"10.5194\/isprs-archives-XLII-2-W10-9-2019","article-title":"Shallow water bathymetry mapping from uav imagery based on machine learning","volume":"42","author":"Agrafiotis","year":"2019","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1002\/esp.4060","article-title":"Bathymetric Structure-from-Motion: Extracting Shallow Stream Bathymetry from Multi-View Stereo Photogrammetry","volume":"42","author":"Dietrich","year":"2017","journal-title":"Earth Surf. Process. Landf."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Alevizos, E., Nicodemou, V.C., Makris, A., Oikonomidis, I., Roussos, A., and Alexakis, D.D. (2022). Integration of Photogrammetric and Spectral Techniques for Advanced Drone-Based Bathymetry Retrieval Using a Deep Learning Approach. Remote Sens., 14.","DOI":"10.3390\/rs14174160"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1016\/j.isprsjprs.2020.09.002","article-title":"Combined Geometric-Radiometric and Neural Network Approach to Shallow Bathymetric Mapping with UAS Imagery","volume":"169","author":"Slocum","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"109049","DOI":"10.1016\/j.geomorph.2023.109049","article-title":"Pixel-Level Bathymetry Mapping of Optically Shallow Water Areas by Combining Aerial RGB Video and Photogrammetry","volume":"449","author":"Wang","year":"2024","journal-title":"Geomorphology"},{"key":"ref_10","first-page":"71","article-title":"BathyNet: A Deep Neural Network for Water Depth Mapping from Multispectral Aerial Images","volume":"1","author":"Mandlburger","year":"2021","journal-title":"PFG\u2014J. Photogramm. Remote Sens. Geoinf. Sci."},{"key":"ref_11","first-page":"298","article-title":"Bundle Adjustment\u2014A Modern Synthesis","volume":"Volume 1883","author":"Triggs","year":"2000","journal-title":"Vision Algorithms: Theory and Practice"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1486525.1486527","article-title":"SBA: A Software Package for Generic Sparse Bundle Adjustment","volume":"36","author":"Lourakis","year":"2009","journal-title":"ACM Trans. Math. Softw."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Agrafiotis, P., Skarlatos, D., Georgopoulos, A., and Karantzalos, K. (2019). DepthLearn: Learning to Correct the Refraction on Point Clouds Derived from Aerial Imagery for Accurate Dense Shallow Water Bathymetry Based on SVMs-Fusion with LiDAR Point Clouds. Remote Sens., 11.","DOI":"10.3390\/rs11192225"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Agrafiotis, P., Karantzalos, K., Georgopoulos, A., and Skarlatos, D. (2020). Correcting Image Refraction: Towards Accurate Aerial Image-Based Bathymetry Mapping in Shallow Waters. Remote Sens., 12.","DOI":"10.3390\/rs12020322"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Lambert, S.E., and Parrish, C.E. (2023). Refraction Correction for Spectrally Derived Bathymetry Using UAS Imagery. Remote Sens., 15.","DOI":"10.3390\/rs15143635"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1080\/22797254.2019.1658542","article-title":"Shallow Water Bathymetry from WorldView-2 Stereo Imagery Using Two-Media Photogrammetry","volume":"52","author":"Cao","year":"2019","journal-title":"Eur. J. Remote Sens."},{"key":"ref_17","first-page":"91","article-title":"Learning from Synthetic Data: Enhancing Refraction Correction Accuracy for Airborne Image-Based Bathymetric Mapping of Shallow Coastal Waters","volume":"89","author":"Agrafiotis","year":"2021","journal-title":"PFG\u2014J. Photogramm. Remote Sens. Geoinf. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1129","DOI":"10.14358\/PERS.73.9.1129","article-title":"A Photogrammetric Correction Procedure for Light Refraction Effects at a Two-Medium Boundary","volume":"73","author":"Murase","year":"2007","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_19","unstructured":"Wimmer, M. (2016). Comparison of Active and Passive Optical Methods for Mapping River Bathymetry. [Master\u2019s Thesis, Technische Universit\u00e4t Wien]."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"659","DOI":"10.5194\/isprs-archives-XLII-2-659-2018","article-title":"A case study on through-water dense image matching","volume":"42","author":"Mandlburger","year":"2018","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_21","first-page":"102108","article-title":"Universal Algorithm for Water Depth Refraction Correction in Through-Water Stereo Remote Sensing","volume":"91","author":"Cao","year":"2020","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"835","DOI":"10.1007\/s00338-021-02088-9","article-title":"Structure-from-Motion on Shallow Reefs and Beaches: Potential and Limitations of Consumer-Grade Drones to Reconstruct Topography and Bathymetry","volume":"40","author":"David","year":"2021","journal-title":"Coral Reefs"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"249","DOI":"10.5194\/isprs-archives-XLVIII-1-W1-2023-249-2023","article-title":"Iterative refraction-correction method on mvs-sfm for shallow stream bathymetry","volume":"48","author":"Lingua","year":"2023","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"18140","DOI":"10.3390\/s150818140","article-title":"On the Accuracy Potential in Underwater\/Multimedia Photogrammetry","volume":"15","author":"Maas","year":"2015","journal-title":"Sensors"},{"key":"ref_25","first-page":"472","article-title":"A Flexible Multi-Media Bundle Approach","volume":"38","author":"Mulsow","year":"2010","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"739","DOI":"10.5194\/isprs-archives-XLII-2-739-2018","article-title":"Subaquatic digital elevation models from uav-imagery","volume":"42","author":"Mulsow","year":"2018","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"547","DOI":"10.4319\/lo.2003.48.1_part_2.0547","article-title":"Determination of Water Depth with High-Resolution Satellite Imagery over Variable Bottom Types","volume":"48","author":"Stumpf","year":"2003","journal-title":"Limnol. Oceanogr."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2746","DOI":"10.3390\/rs5062746","article-title":"Remotely Sensed Empirical Modeling of Bathymetry in the Southeastern Caspian Sea","volume":"5","author":"Gholamalifard","year":"2013","journal-title":"Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1080\/2150704X.2015.1034885","article-title":"Performance of Two Neural Network Models in Bathymetry","volume":"6","author":"Liu","year":"2015","journal-title":"Remote Sens. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1080\/15481603.2018.1538620","article-title":"Bathymetry Retrieval from Optical Images with Spatially Distributed Support Vector Machines","volume":"56","author":"Wang","year":"2019","journal-title":"GISci. Remote Sens."},{"key":"ref_31","first-page":"201","article-title":"Satellite-derived bathymetry using convolutional neural networks and multispectral sentinel-2 images","volume":"43","author":"Ohori","year":"2021","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Rossi, L., Mammi, I., and Pelliccia, F. (2020). UAV-Derived Multispectral Bathymetry. Remote Sens., 12.","DOI":"10.3390\/rs12233897"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1107","DOI":"10.1007\/s10994-021-05977-w","article-title":"Satellite Derived Bathymetry Using Deep Learning","volume":"112","author":"Thoumyre","year":"2023","journal-title":"Mach. Learn."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.2112\/SI95-197.1","article-title":"A Deep Learning Approach for Estimation of the Nearshore Bathymetry","volume":"95","author":"Benshila","year":"2020","journal-title":"J. Coast. Res."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"104337","DOI":"10.1016\/j.coastaleng.2023.104337","article-title":"Simultaneous Mapping of Nearshore Bathymetry and Waves Based on Physics-Informed Deep Learning","volume":"183","author":"Chen","year":"2023","journal-title":"Coast. Eng."},{"key":"ref_36","unstructured":"Forghani, M., Qian, Y., Lee, J., Farthing, M.W., Hesser, T., Kitanidis, P.K., and Darve, E. (2024, June 05). Deep Learning-Based Estimation of Riverine Bathymetry Using Variational Encoder Geostatistical Approaches (VEGAs); AGU Fall Meeting Abstracts. Available online: https:\/\/ui.adsabs.harvard.edu\/abs\/2021AGUFM.H35S1254F\/abstract."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"103715","DOI":"10.1016\/j.advwatres.2020.103715","article-title":"Deep Learning Technique for Fast Inference of Large-Scale Riverine Bathymetry","volume":"147","author":"Ghorbanidehno","year":"2021","journal-title":"Adv. Water Resour."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.mio.2016.03.001","article-title":"Refractive 3D Reconstruction on Underwater Images","volume":"15\u201316","author":"Jordt","year":"2016","journal-title":"Methods Oceanogr."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Sonogashira, M., Shonai, M., and Iiyama, M. (2020). High-Resolution Bathymetry by Deep-Learning-Based Image Superresolution. PLoS ONE, 15.","DOI":"10.1371\/journal.pone.0235487"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1016\/j.isprsjprs.2020.04.016","article-title":"Efficient Structure from Motion for Large-Scale UAV Images: A Review and a Comparison of SfM Tools","volume":"167","author":"Jiang","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_41","first-page":"1","article-title":"Reflections on Refractions","volume":"4","author":"Glaeser","year":"2000","journal-title":"J. Geom. Graph."},{"key":"ref_42","unstructured":"(2024, June 05). Available online: https:\/\/github.com\/mapillary\/OpenSfM."},{"key":"ref_43","unstructured":"Adorjan, M. (2016). OpenSfM: A Collaborative Structure-from-Motion System. [Diploma Thesis, Technische Universit\u00e4t Wien]."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1007\/s11263-005-3848-x","article-title":"A Comparison of Affine Region Detectors","volume":"65","author":"Mikolajczyk","year":"2005","journal-title":"Int. J. Comput. Vis."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Tareen, S.A.K., and Saleem, Z. (2018, January 3\u20134). A Comparative Analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK. Proceedings of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), Sukkur, Pakistan.","DOI":"10.1109\/ICOMET.2018.8346440"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1007\/978-3-319-46484-8_29","article-title":"Stacked Hourglass Networks for Human Pose Estimation","volume":"Volume 9912","author":"Leibe","year":"2016","journal-title":"Computer Vision\u2014ECCV 2016"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Nicodemou, V.C., Oikonomidis, I., Tzimiropoulos, G., and Argyros, A. (2020, January 19\u201324). Learning to Infer the Depth Map of a Hand from Its Color Image. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.","DOI":"10.1109\/IJCNN48605.2020.9206925"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Jackson, A.S., Bulat, A., Argyriou, V., and Tzimiropoulos, G. (2017, January 22\u201329). Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.117"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Cho, H.-M., Park, J.-W., Lee, J.-S., and Han, S.-K. (2023). Assessment of the GNSS-RTK for Application in Precision Forest Operations. Remote Sens., 16.","DOI":"10.3390\/rs16010148"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.quaint.2015.08.008","article-title":"Vertical Land Movements and Sea Level Changes along the Coast of Crete (Greece) since Late Holocene","volume":"401","author":"Mourtzas","year":"2016","journal-title":"Quat. Int."},{"key":"ref_51","unstructured":"Kingma, D.P., and Ba, J. (2017). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_52","unstructured":"Recht, B., Roelofs, R., Schmidt, L., and Shankar, V. (2019, January 9\u201315). Do Imagenet Classifiers Generalize to Imagenet?. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_53","first-page":"128","article-title":"Deep Learning vs. Traditional Computer Vision","volume":"Volume 943","author":"Arai","year":"2020","journal-title":"Advances in Computer Vision"},{"key":"ref_54","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An Image Is Worth 16\u00d716 Words: Transformers for Image Recognition at Scale. arXiv."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/22\/4253\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:32:44Z","timestamp":1760113964000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/22\/4253"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,15]]},"references-count":54,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["rs16224253"],"URL":"https:\/\/doi.org\/10.3390\/rs16224253","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,15]]}}}