{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T10:48:09Z","timestamp":1776077289374,"version":"3.50.1"},"reference-count":54,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2022,11,29]],"date-time":"2022-11-29T00:00:00Z","timestamp":1669680000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008249","name":"National Institute of Water and Atmospheric Research","doi-asserted-by":"publisher","award":["CARH2302"],"award-info":[{"award-number":["CARH2302"]}],"id":[{"id":"10.13039\/100008249","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>A rip current is a strong, localized current of water which moves along and away from the shore. Recent studies have suggested that drownings due to rip currents are still a major threat to beach safety. Identification of rip currents is important for lifeguards when making decisions on where to designate patrolled areas. The public also require information while deciding where to swim when lifeguards are not on patrol. In the present study we present an artificial intelligence (AI) algorithm that both identifies whether a rip current exists in images\/video, and also localizes where that rip current occurs. While there have been some significant advances in AI for rip current detection and localization, there is a lack of research ensuring that an AI algorithm can generalize well to a diverse range of coastal environments and marine conditions. The present study made use of an interpretable AI method, gradient-weighted class-activation maps (Grad-CAM), which is a novel approach for amorphous rip current detection. The training data\/images were diverse and encompass rip currents in a wide variety of environmental settings, ensuring model generalization. An open-access aerial catalogue of rip currents were used for model training. Here, the aerial imagery was also augmented by applying a wide variety of randomized image transformations (e.g., perspective, rotational transforms, and additive noise), which dramatically improves model performance through generalization. To account for diverse environmental settings, a synthetically generated training set, containing fog, shadows, and rain, was also added to the rip current images, thus increased the training dataset approximately 10-fold. Interpretable AI has dramatically improved the accuracy of unbounded rip current detection, which can correctly classify and localize rip currents about 89% of the time when validated on independent videos from surf-cameras at oblique angles. The novelty also lies in the ability to capture some shape characteristics of the amorphous rip current structure without the need of a predefined bounding box, therefore enabling the use of remote technology like drones. A comparison with well-established coastal image processing techniques is also presented via a short discussion and easy reference table. The strengths and weaknesses of both methods are highlighted and discussed.<\/jats:p>","DOI":"10.3390\/rs14236048","type":"journal-article","created":{"date-parts":[[2022,11,30]],"date-time":"2022-11-30T05:45:22Z","timestamp":1669787122000},"page":"6048","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Interpretable Deep Learning Applied to Rip Current Detection and Localization"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9801-9348","authenticated-orcid":false,"given":"Neelesh","family":"Rampal","sequence":"first","affiliation":[{"name":"Climate and Environmental Applications, National Institute of Water and Atmospheric Research (NIWA), Auckland 1010, New Zealand"}]},{"given":"Tom","family":"Shand","sequence":"additional","affiliation":[{"name":"School of Civil and Environmental Engineering, University of Auckland, Auckland 1010, New Zealand"}]},{"given":"Adam","family":"Wooler","sequence":"additional","affiliation":[{"name":"Surf Life Saving New Zealand, Auckland 1010, New Zealand"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6703-8386","authenticated-orcid":false,"given":"Christo","family":"Rautenbach","sequence":"additional","affiliation":[{"name":"Institute for Coastal and Marine Research, Nelson Mandela University (NMU), Port Elizabeth 6001, South Africa"},{"name":"Coasts and Estuaries, National Institute of Water and Atmospheric Research (NIWA), Hamilton 3216, New Zealand"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.earscirev.2016.09.008","article-title":"Rip current types, circulation and hazard","volume":"163","author":"Castelle","year":"2016","journal-title":"Earth Sci. Rev."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"909","DOI":"10.2112\/JCOASTRES-D-12-00118.1","article-title":"A Probabilistic Rip Current Forecast Model","volume":"29","author":"Dusek","year":"2013","journal-title":"J. Coast. Res."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1007\/s11069-013-0812-x","article-title":"Rip current-related fatalities in India: A new predictive risk scale for forecasting rip currents","volume":"70","author":"Prasad","year":"2014","journal-title":"Nat. Hazards"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1007\/s11069-020-04299-9","article-title":"Rip current hazard assessment on a sandy beach in Liguria, NW Mediterranean","volume":"105","author":"Mucerino","year":"2021","journal-title":"Nat. Hazards"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"105734","DOI":"10.1016\/j.ocecoaman.2021.105734","article-title":"Rip current hazard at coastal recreational beaches in China","volume":"210","author":"Zhang","year":"2021","journal-title":"Ocean. Coast. Manag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"103859","DOI":"10.1016\/j.coastaleng.2021.103859","article-title":"Automated rip current detection with region based convolutional neural networks","volume":"166","author":"Mori","year":"2021","journal-title":"Coast. Eng."},{"key":"ref_7","unstructured":"Voulgaris, G., Kumar, N., and Warner, J.C. (2011). Methodology for Prediction of Rip Currents Using a Three-Dimensional Numerical, Coupled, Wave Current Model, CRC Press."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1687","DOI":"10.5194\/nhess-13-1687-2013","article-title":"Brief Communication: A new perspective on the Australian rip current hazard","volume":"13","author":"Brander","year":"2013","journal-title":"Nat. Hazards Earth Syst. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1659","DOI":"10.1175\/WAF-D-17-0076.1","article-title":"Comparison of Rip Current Hazard Likelihood Forecasts with Observed Rip Current Speeds","volume":"32","author":"Moulton","year":"2017","journal-title":"Weather Forecast."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Carey, W., and Rogers, S. (2005, January 8\u201311). Rip Currents\u2014Coordinating Coastal Research, Outreach and Forecast Methodologies to Improve Public Safety. Proceedings of the Solutions to Coastal Disasters Conference 2005, Charleston, SC, USA.","DOI":"10.1061\/40774(176)29"},{"key":"ref_11","first-page":"11","article-title":"Responses of Swimmers Caught in Rip Currents: Perspectives on Mitigating the Global Rip Current Hazard","volume":"5","author":"Brander","year":"2016","journal-title":"Int. J. Aquat. Res. Educ."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"115","DOI":"10.5194\/nhess-21-115-2021","article-title":"Beachgoers\u2019 ability to identify rip currents at a beach in situ","volume":"21","author":"Pitman","year":"2021","journal-title":"Nat. Hazards Earth Syst. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Tipton, M., and Wooler, A. (2016). Science of the rip current hazard. The Science of Beach Lifeguarding, Taylor & Francis Group.","DOI":"10.1201\/b19650"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"283","DOI":"10.2112\/JCOASTRES-D-12-00093.1","article-title":"Rip Current Prediction: Development, Validation, and Evaluation of an Operational Tool","volume":"29","author":"Austin","year":"2012","journal-title":"J. Coast. Res."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1016\/j.coastaleng.2007.01.009","article-title":"The role of video imagery in predicting daily to monthly coastal evolution","volume":"54","author":"Smit","year":"2007","journal-title":"Coast. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1029\/JC094iC01p00995","article-title":"Quantification of sand bar morphology: A video technique based on wave dissipation","volume":"94","author":"Lippmann","year":"1989","journal-title":"J. Geophys. Res. Ocean."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1016\/0025-3227(91)90028-3","article-title":"Video estimation of subaerial beach profiles","volume":"97","author":"Holman","year":"1991","journal-title":"Mar. Geol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1016\/j.coastaleng.2007.01.003","article-title":"The history and technical capabilities of Argus","volume":"54","author":"Holman","year":"2007","journal-title":"Coast. Eng."},{"key":"ref_19","unstructured":"Bogle, J.A., Bryan, K.R., Black, K.P., Hume, T.M., and Healy, T.R. (2001). Video Observations of Rip Formation and Evolution. J. Coast. Res., 117\u2013127."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1016\/j.cageo.2012.07.013","article-title":"COSMOS: A lightweight coastal video monitoring system","volume":"49","author":"Taborda","year":"2012","journal-title":"Comput. Geosci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1712","DOI":"10.1002\/esp.2025","article-title":"An open source, low cost video-based coastal monitoring system","volume":"35","author":"Nieto","year":"2010","journal-title":"Earth Surf. Process. Landf."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.cageo.2012.06.008","article-title":"A system for beach video-monitoring: Beachkeeper plus","volume":"49","author":"Brignone","year":"2012","journal-title":"Comput. Geosci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1217","DOI":"10.2112\/JCOASTRES-D-16-00022.1","article-title":"ULISES: An Open Source Code for Extrinsic Calibrations and Planview Generations in Coastal Video Monitoring Systems","volume":"33","author":"Simarro","year":"2017","journal-title":"J. Coast. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"103537","DOI":"10.1016\/j.coastaleng.2019.103537","article-title":"Lifeguarding Operational Camera Kiosk System (LOCKS) for flash rip warning: Development and application","volume":"152","author":"Liu","year":"2019","journal-title":"Coast. Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"6483","DOI":"10.1109\/ACCESS.2022.3140340","article-title":"Flow-Based Rip Current Detection and Visualization","volume":"10","author":"Mori","year":"2022","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Rashid, A.H., Razzak, I., Tanveer, M., and Hobbs, M. (2022). Reducing rip current drowning: An improved residual based lightweight deep architecture for rip detection. ISA Trans., in press.","DOI":"10.1016\/j.isatra.2022.05.015"},{"key":"ref_27","unstructured":"Maryan, C.C. (2018). Detecting Rip Currents from Images. [Ph.D. Thesis, University of New Orleans]."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.coastaleng.2019.02.002","article-title":"A novel machine learning algorithm for tracking remotely sensed waves in the surf zone","volume":"147","author":"Stringari","year":"2019","journal-title":"Coast. Eng."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"104021","DOI":"10.1016\/j.coastaleng.2021.104021","article-title":"Wave-by-wave nearshore wave breaking identification using U-Net","volume":"170","author":"Valle","year":"2021","journal-title":"Coast. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Liu, B., Yang, B., Masoud-Ansari, S., Wang, H., and Gahegan, M. (2021). Coastal Image Classification and Pattern Recognition: Tairua Beach, New Zealand. Sensors, 21.","DOI":"10.3390\/s21217352"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5790","DOI":"10.1109\/TGRS.2017.2714202","article-title":"Wavelet-Based Optical Flow Estimation of Instant Surface Currents From Shore-Based and UAV Videos","volume":"55","author":"Almar","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/j.coastaleng.2018.01.007","article-title":"Sensitivity of rip current forecasts to errors in remotely-sensed bathymetry","volume":"135","author":"Radermacher","year":"2018","journal-title":"Coast. Eng."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Anderson, D., Bak, A.S., Brodie, K.L., Cohn, N., Holman, R.A., and Stanley, J. (2021). Quantifying Optically Derived Two-Dimensional Wave-Averaged Currents in the Surf Zone. Remote Sens., 13.","DOI":"10.3390\/rs13040690"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Rodr\u00edguez-Padilla, I., Castelle, B., Marieu, V., Bonneton, P., Mouragues, A., Martins, K., and Morichon, D. (2021). Wave-Filtered Surf Zone Circulation under High-Energy Waves Derived from Video-Based Optical Systems. Remote Sens., 13.","DOI":"10.3390\/rs13101874"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ellenson, A.N., Simmons, J.A., Wilson, G.W., Hesser, T.J., and Splinter, K.D. (2020). Beach state recognition using argus imagery and convolutional neural networks. Remote Sens., 12.","DOI":"10.3390\/rs12233953"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Linardatos, P., Papastefanopoulos, V., and Kotsiantis, S. (2021). Explainable AI: A Review of Machine Learning Interpretability Methods. Entropy, 23.","DOI":"10.3390\/e23010018"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1214\/09-SS054","article-title":"A survey of cross-validation procedures for model selection","volume":"4","author":"Arlot","year":"2010","journal-title":"Stat. Surv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Qin, Z., Zhang, Z., Chen, X., Wang, C., and Peng, Y. (2018, January 7\u201310). Fd-Mobilenet: Improved Mobilenet with a Fast Downsampling Strategy. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.","DOI":"10.1109\/ICIP.2018.8451355"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.patcog.2019.01.006","article-title":"Wider or Deeper: Revisiting the ResNet Model for Visual Recognition","volume":"90","author":"Wu","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"3142","DOI":"10.1109\/TIP.2017.2662206","article-title":"Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising","volume":"26","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_45","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). ImageNet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems\u2014Volume 1, Lake Tahoe, NV, USA."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1186\/s40537-016-0043-6","article-title":"A survey of transfer learning","volume":"3","author":"Weiss","year":"2016","journal-title":"J. Big Data"},{"key":"ref_47","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"100525","DOI":"10.1016\/j.wace.2022.100525","article-title":"High-resolution downscaling with interpretable deep learning: Rainfall extremes over New Zealand","volume":"38","author":"Rampal","year":"2022","journal-title":"Weather. Clim. Extrem."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Montavon, G., Binder, A., Lapuschkin, S., Samek, W., and M\u00fcller, K.-R. (2019). Layer-wise relevance propagation: An overview. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer.","DOI":"10.1007\/978-3-030-28954-6_10"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Xu, F., Uszkoreit, H., Du, Y., Fan, W., Zhao, D., and Zhu, J. (2019, January 9\u201314). Explainable AI: A brief survey on history, research areas, approaches and challenges. Proceedings of the 8th CCF International Conference on Natural Language Processing and Chinese Computing, Dunhuang, China.","DOI":"10.1007\/978-3-030-32236-6_51"},{"key":"ref_51","unstructured":"Meudec, R. (2022, August 25). tf-explain. Available online: https:\/\/github.com\/sicara\/tf-explain."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Buslaev, A., Iglovikov, V.I., Khvedchenya, E., Parinov, A., Druzhinin, M., and Kalinin, A.A. (2020). Albumentations: Fast and Flexible Image Augmentations. Information, 11.","DOI":"10.3390\/info11020125"},{"key":"ref_53","unstructured":"Bailey, D.G., and Shand, R.D. (1996, January 19). Determining large scale sandbar behaviour. Proceedings of the 3rd IEEE International Conference on Image Processing, Lausanne, Switzerland."},{"key":"ref_54","unstructured":"Shand, T., and Quilter, P. (2022, August 25). Surfzone Fun, v1.0 [Source Code]. Available online: https:\/\/doi.org\/10.24433\/CO.5658154.v1."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/23\/6048\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:29:25Z","timestamp":1760146165000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/23\/6048"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,29]]},"references-count":54,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["rs14236048"],"URL":"https:\/\/doi.org\/10.3390\/rs14236048","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,29]]}}}