{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T16:07:36Z","timestamp":1782403656060,"version":"3.54.5"},"reference-count":37,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2020,9,22]],"date-time":"2020-09-22T00:00:00Z","timestamp":1600732800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>The semantic segmentation of underwater imagery is an important step in the ecological analysis of coral habitats. To date, scientists produce fine-scale area annotations manually, an exceptionally time-consuming task that could be efficiently automatized by modern CNNs. This paper extends our previous work presented at the 3DUW\u201919 conference, outlining the workflow for the automated annotation of imagery from the first step of dataset preparation, to the last step of prediction reassembly. In particular, we propose an ecologically inspired strategy for an efficient dataset partition, an over-sampling methodology targeted on ortho-imagery, and a score fusion strategy. We also investigate the use of different loss functions in the optimization of a Deeplab V3+ model, to mitigate the class-imbalance problem and improve prediction accuracy on coral instance boundaries. The experimental results demonstrate the effectiveness of the ecologically inspired split in improving model performance, and quantify the advantages and limitations of the proposed over-sampling strategy. The extensive comparison of the loss functions gives numerous insights on the segmentation task; the Focal Tversky, typically used in the context of medical imaging (but not in remote sensing), results in the most convenient choice. By improving the accuracy of automated ortho image processing, the results presented here promise to meet the fundamental challenge of increasing the spatial and temporal scale of coral reef research, allowing researchers greater predictive ability to better manage coral reef resilience in the context of a changing environment.<\/jats:p>","DOI":"10.3390\/rs12183106","type":"journal-article","created":{"date-parts":[[2020,9,23]],"date-time":"2020-09-23T09:28:08Z","timestamp":1600853288000},"page":"3106","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["On Improving the Training of Models for the Semantic Segmentation of Benthic Communities from Orthographic Imagery"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4083-3835","authenticated-orcid":false,"given":"Gaia","family":"Pavoni","sequence":"first","affiliation":[{"name":"Visual Computing Lab (ISTI-CNR), 56124 Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0543-1638","authenticated-orcid":false,"given":"Massimiliano","family":"Corsini","sequence":"additional","affiliation":[{"name":"Visual Computing Lab (ISTI-CNR), 56124 Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marco","family":"Callieri","sequence":"additional","affiliation":[{"name":"Visual Computing Lab (ISTI-CNR), 56124 Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8687-6609","authenticated-orcid":false,"given":"Giuseppe","family":"Fiameni","sequence":"additional","affiliation":[{"name":"NVIDIA AI Technology Centre (NVAITC), 40134 Bologna, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Clinton","family":"Edwards","sequence":"additional","affiliation":[{"name":"Scripps Institution of Oceanography, UC San Diego, La Jolla, CA 92037, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2686-8567","authenticated-orcid":false,"given":"Paolo","family":"Cignoni","sequence":"additional","affiliation":[{"name":"Visual Computing Lab (ISTI-CNR), 56124 Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1291","DOI":"10.1007\/s00338-017-1624-3","article-title":"Large-area imaging reveals biologically driven non-random spatial patterns of corals at a remote reef","volume":"36","author":"Edwards","year":"2017","journal-title":"Coral Reefs"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"e8957","DOI":"10.7717\/peerj.8957","article-title":"Size-frequency distribution of coral assemblages in insular shallow reefs of the Mexican Caribbean using underwater photogrammetry","volume":"8","year":"2020","journal-title":"PeerJ"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"23166","DOI":"10.1038\/srep23166","article-title":"Improving Automated Annotation of Benthic Survey Images Using Wide-band Fluorescence","volume":"6","author":"Beijbom","year":"2016","journal-title":"Sci. Rep."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1703","DOI":"10.1111\/ecog.04520","article-title":"The influence of habitat and adults on the spatial distribution of juvenile corals","volume":"42","author":"Pedersen","year":"2019","journal-title":"Ecography"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1091","DOI":"10.1007\/s00338-020-01940-8","article-title":"Quantifying life history537demographics of the scleractinian coral genus Pocillopora at Palmyra Atoll","volume":"39","author":"Kodera","year":"2020","journal-title":"Coral Reefs"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Beijbom, O., Edmunds, P.J., Kline, D.I., Mitchell, B.G., and Kriegman, D. (2012, January 16\u201321). Automated annotation of coral reef survey images. Proceedings of the CVPR2012, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6247798"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Beijbom, O., Edmunds, P.J., Roelfsema, C., Smith, J., Kline, D.I., Neal, B.P., Dunlap, M.J., Moriarty, V., Fan, T.Y., and Tan, C.J. (2015). Towards Automated Annotation of Benthic Survey Images: Variability of Human Experts and Operational Modes of Automation. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0130312"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Alonso, I., Cambra, A., Mu\u00f1oz, A., Treibitz, T., and Murillo, A.C. (2017, January 22\u201329). Coral-Segmentation: Training Dense Labeling Models with Sparse Ground Truth. Proceedings of the ICCV 2017 Workshops, Venice, Italy.","DOI":"10.1109\/ICCVW.2017.339"},{"key":"ref_9","unstructured":"Kendall, A., Badrinarayanan, V., and Cipolla, R. (2015). Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1456","DOI":"10.1002\/rob.21915","article-title":"CoralSeg: Learning coral segmentation from sparse annotations","volume":"36","author":"Alonso","year":"2019","journal-title":"J. Field Robot."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"King, A., Bhandarkar, S.M., and Hopkinson, B.M. (2018, January 18\u201322). A Comparison of Deep Learning Methods for Semantic Segmentation of Coral Reef Survey Images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00188"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Pavoni, G., Corsini, M., Pedersen, N., Petrovic, V., and Cignoni, P. (2020). Challenges in the deep learning-based semantic segmentation of benthic communities from Ortho-images. Appl. Geomat., 1\u201316.","DOI":"10.1007\/s12518-020-00331-6"},{"key":"ref_13","unstructured":"Hoeksema, B., and Cairns, S. (2020, July 05). World list of Scleractinia. Scleractinia. Available online: http:\/\/www.marinespecies.org\/aphia.php."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"867","DOI":"10.1038\/nature02685","article-title":"Coral communities are regionally enriched along an oceanic biodiversity gradient","volume":"429","author":"Karlson","year":"2004","journal-title":"Nature"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"107","DOI":"10.3354\/meps265107","article-title":"Estimating cover of benthic organisms from underwater video images: Variability associated with multiple observers","volume":"265","author":"Ninio","year":"2003","journal-title":"Mar. Ecol. Prog. Ser."},{"key":"ref_16","first-page":"151","article-title":"Semantic segmentation of benthic communities from ortho-mosaic maps","volume":"XLII-2\/W10","author":"Pavoni","year":"2019","journal-title":"Isprs Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1367","DOI":"10.1109\/TPAMI.2018.2832629","article-title":"Imbalanced deep learning by minority class incremental rectification","volume":"41","author":"Dong","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_18","unstructured":"Agisoft (2020, September 16). Metashape. Available online: http:\/\/www.agisoft.com\/."},{"key":"ref_19","first-page":"93","article-title":"Visualizing high resolution three-dimensional and two-dimensional data of cultural heritage sites","volume":"14","author":"Petrovic","year":"2014","journal-title":"Mediterr. Archaeol. Archaeom."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1607","DOI":"10.1093\/bioinformatics\/btm158","article-title":"Weighted rank aggregation of cluster validation measures: A monte carlo cross-entropy approach","volume":"23","author":"Pihur","year":"2007","journal-title":"Bioinformatics"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 16\u201320). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Milletari, F., Navab, N., and Ahmadi, S.A. (2016, January 25\u201328). V-net: Fully convolutional neural networks for volumetric medical image segmentation. Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA.","DOI":"10.1109\/3DV.2016.79"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., and Cardoso, M.J. (2017). Generalised Dice Overlap as A Deep Learning Loss Function for Highly Unbalanced Segmentations. Deep Learning in Medical Image Analysis And Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-319-67558-9_28"},{"key":"ref_25","unstructured":"Hashemi, S.R., Salehi, S.S.M., Erdogmus, D., Prabhu, S.P., Warfield, S.K., and Gholipour, A. (2018). Tversky as a loss function for highly unbalanced image segmentation using 3d fully convolutional deep networks. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Abraham, N., and Khan, N.M. (2019, January 8\u201311). A novel focal tversky loss function with improved attention u-net for lesion segmentation. Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy.","DOI":"10.1109\/ISBI.2019.8759329"},{"key":"ref_27","first-page":"285","article-title":"Boundary loss for highly unbalanced segmentation","volume":"Volume 102","author":"Cardoso","year":"2019","journal-title":"Proceedings of the 2nd International Conference on Medical Imaging with Deep Learning"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liu, Y., Minh Nguyen, D., Deligiannis, N., Ding, W., and Munteanu, A. (2017). Hourglass-ShapeNetwork Based Semantic Segmentation for High Resolution Aerial Imagery. Remote Sens., 9.","DOI":"10.3390\/rs9060522"},{"key":"ref_29","unstructured":"Lai, S.H., Lepetit, V., Nishino, K., and Sato, Y. (2016, January 27). Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks. Proceedings of the ACCV 2016, Taipei, Taiwan."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ma, L., Stueckler, J., Kerl, C., and Cremers, D. (2017, January 24\u201328). Multi-View Deep Learning for Consistent Semantic Mapping with RGB-D Cameras. Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8202213"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8\u201314). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_32","unstructured":"Ma, J., and Yarats, D. (2019, January 6\u20139). Quasi-hyperbolic momentum and Adam for deep learning. Proceedings of the International Conference on Learning Representations, New Orleans, LO, USA."},{"key":"ref_33","unstructured":"Tantithamthavorn, C., Hassan, A.E., and Matsumoto, K. (2018). The Impact of Class Rebalancing Techniques on the Performance and Interpretation of Defect Prediction Models. IEEE Trans. Softw. Eng."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Taha, A., Lo, P., Li, J., and Zhao, T. (2018, January 16\u201320). Kid-net: Convolution networks for kidney vessels segmentation from ct-volumes. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain.","DOI":"10.1007\/978-3-030-00937-3_53"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Acuna, D., Kar, A., and Fidler, S. (2019, January 16\u201320). Devil is in the edges: Learning semantic boundaries from noisy annotations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01133"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Takikawa, T., Acuna, D., Jampani, V., and Fidler, S. (2019, January 23\u201325). Gated-scnn: Gated shape cnns for semantic segmentation. Proceedings of the IEEE International Conference on Computer Vision, Thessaloniki, Greece.","DOI":"10.1109\/ICCV.2019.00533"},{"key":"ref_37","unstructured":"Kouw, W.M., and Loog, M. (2019). A review of single-source unsupervised domain adaptation. arXiv."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/18\/3106\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:12:35Z","timestamp":1760177555000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/18\/3106"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,22]]},"references-count":37,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["rs12183106"],"URL":"https:\/\/doi.org\/10.3390\/rs12183106","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,22]]}}}