{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T08:13:55Z","timestamp":1770711235418,"version":"3.49.0"},"reference-count":60,"publisher":"IOP Publishing","issue":"3","license":[{"start":{"date-parts":[[2023,7,14]],"date-time":"2023-07-14T00:00:00Z","timestamp":1689292800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,14]],"date-time":"2023-07-14T00:00:00Z","timestamp":1689292800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/501100022273","name":"Gauss Centre for Supercomputing","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100022273","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100023371","name":"Helmholtz Artificial Intelligence Cooperation Unit","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100023371","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2023,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Advances in <jats:italic>in situ<\/jats:italic> marine life imaging have significantly increased the size and quality of available datasets, but automatic image analysis has not kept pace. Machine learning has shown promise for image processing, but its effectiveness is limited by several open challenges: the requirement for large expert-labeled training datasets, disagreement among experts, under-representation of various species and unreliable or overconfident predictions. To overcome these obstacles for automated underwater imaging, we combine and test recent developments in deep classifier networks and self-supervised feature learning. We use unlabeled images for pretraining deep neural networks to extract task-relevant image features, allowing learning algorithms to cope with scarcity in expert labels, and carefully evaluate performance in subsequent label-based tasks. Performance on rare classes is improved by applying data rebalancing together with a Bayesian correction to avoid biasing inferred <jats:italic>in situ<\/jats:italic> class frequencies. A divergence-based loss allows training on multiple, conflicting labels for the same image, leading to better estimates of uncertainty which we quantify with a novel accuracy measure. Together, these techniques can reduce the required label counts \u223c100-fold while maintaining the accuracy of standard supervised training, shorten training time, cope with expert disagreement and reduce overconfidence.<\/jats:p>","DOI":"10.1088\/2632-2153\/ace417","type":"journal-article","created":{"date-parts":[[2023,7,4]],"date-time":"2023-07-04T22:40:07Z","timestamp":1688510407000},"page":"035007","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Robust detection of marine life with label-free image feature learning and probability calibration"],"prefix":"10.1088","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8455-4179","authenticated-orcid":true,"given":"Tobias","family":"Schanz","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3118-2161","authenticated-orcid":false,"given":"Klas","family":"Ove M\u00f6ller","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4650-6045","authenticated-orcid":false,"given":"Saskia","family":"R\u00fchl","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8515-0459","authenticated-orcid":false,"given":"David S","family":"Greenberg","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2023,7,14]]},"reference":[{"key":"mlstace417bib1","article-title":"TorchVision-maintainers and contributors","year":"2016"},{"key":"mlstace417bib2","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1186\/s40537-021-00444-8","article-title":"Review of deep learning: concepts, CNN architectures, challenges, applications, future directions","volume":"8","author":"Alzubaidi","year":"2021","journal-title":"J. Big Data"},{"key":"mlstace417bib3","article-title":"Vicreg: variance-invariance-covariance regularization for self-supervised learning","author":"Bardes","year":"2022"},{"key":"mlstace417bib4","first-page":"pp 5","article-title":"Deep active learning for in situ plankton classification","author":"Bochinski","year":"2019","edition":"ed"},{"key":"mlstace417bib5","article-title":"A survey of predictive modelling under imbalanced distributions","author":"Branco","year":"2015"},{"key":"mlstace417bib6","first-page":"pp 9650","article-title":"Emerging properties in self-supervised vision transformers","author":"Caron","year":"2021"},{"key":"mlstace417bib7","article-title":"A simple framework for contrastive learning of visual representations","author":"Chen","year":"2020a"},{"key":"mlstace417bib8","article-title":"Big self-supervised models are strong semi-supervised learners","author":"Chen","year":"2020b"},{"key":"mlstace417bib9","article-title":"Improved baselines with momentum contrastive learning","author":"Chen","year":"2020c"},{"key":"mlstace417bib10","article-title":"Keras","author":"Chollet","year":"2015"},{"key":"mlstace417bib11","first-page":"pp 543","article-title":"Information theory and the stock market","author":"Cover","year":"1991"},{"key":"mlstace417bib12","first-page":"pp 1","article-title":"ZooplanktoNet: deep convolutional network for zooplankton classification","author":"Dai","year":"2016"},{"key":"mlstace417bib13","first-page":"pp 3965","article-title":"Coatnet: marrying convolution and attention for all data sizes","author":"Dai","year":"2021","edition":"ed"},{"key":"mlstace417bib14","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1016\/j.seares.2011.04.013","article-title":"Using ZooImage automated system for the estimation of biovolume of copepods from the northern Argentine Sea","volume":"66","author":"Di Mauro","year":"2011","journal-title":"J. Sea Res."},{"key":"mlstace417bib15","article-title":"An image is worth 16 \u00d7 16 words: transformers for image recognition at scale","author":"Dosovitskiy","year":"2020"},{"key":"mlstace417bib16","article-title":"Self-supervised learning for label-efficient sleep stage classification: a comprehensive evaluation","author":"Eldele","year":"2022"},{"key":"mlstace417bib17","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1016\/j.mio.2016.04.003","article-title":"Imperfect automatic image classification successfully describes plankton distribution patterns","volume":"15\u201316","author":"Faillettaz","year":"2016","journal-title":"Methods Oceanogr."},{"key":"mlstace417bib18","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.3828935","article-title":"PyTorch lightning","author":"Falcon","year":"2019"},{"key":"mlstace417bib19","article-title":"On the duality between contrastive and non-contrastive self-supervised learning","author":"Garrido","year":"2022"},{"key":"mlstace417bib20","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1093\/plankt\/fbp124","article-title":"Digital zooplankton image analysis using the ZooScan integrated system","volume":"32","author":"Gorsky","year":"2010","journal-title":"J. Plankton Res."},{"key":"mlstace417bib21","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1016\/j.patcog.2017.10.013","article-title":"Recent advances in convolutional neural networks","volume":"77","author":"Gu","year":"2018","journal-title":"Pattern Recognit."},{"key":"mlstace417bib22","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1002\/lom3.10402","article-title":"Automated plankton classification from holographic imagery with deep convolutional neural networks","volume":"19","author":"Guo","year":"2021","journal-title":"Limnol. Oceanogr.: Methods"},{"key":"mlstace417bib23","doi-asserted-by":"publisher","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","article-title":"Learning from imbalanced data","volume":"21","author":"He","year":"2009","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"mlstace417bib24","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR42600.2020.00975","article-title":"Momentum contrast for unsupervised visual representation learning","author":"He","year":"2020"},{"key":"mlstace417bib25","article-title":"Deep residual learning for image recognition","author":"He","year":"2015"},{"key":"mlstace417bib26","doi-asserted-by":"publisher","first-page":"1286","DOI":"10.1038\/ismej.2009.75","article-title":"Microbial community gene expression within colonies of the diazotroph, trichodesmium, from the southwest pacific ocean","volume":"3","author":"Hewson","year":"2009","journal-title":"ISME J."},{"key":"mlstace417bib27","doi-asserted-by":"crossref","DOI":"10.1109\/ACCESS.2019.2956508","article-title":"A survey on the new generation of deep learning in image processing","author":"Jiao","year":"2019"},{"key":"mlstace417bib28","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014"},{"key":"mlstace417bib29","article-title":"Catalyst - accelerated deep learning r&d","author":"Kolesnikov","year":"2018"},{"key":"mlstace417bib30","article-title":"Going beyond one-hot encoding in classification: can human uncertainty improve model performance?","author":"Koller","year":"2022"},{"key":"mlstace417bib31","doi-asserted-by":"publisher","DOI":"10.3389\/fmars.2022.867695","article-title":"Towards operational phytoplankton recognition with automated high-throughput imaging, near-real-time data processing and convolutional neural networks","volume":"9","author":"Kraft","year":"2022","journal-title":"Front. Marine Sci."},{"key":"mlstace417bib32","doi-asserted-by":"publisher","DOI":"10.3389\/fmicb.2021.746297","article-title":"Deep learning classification of lake zooplankton","volume":"12","author":"Kyathanahally","year":"2021","journal-title":"Front. Microbiol."},{"key":"mlstace417bib33","doi-asserted-by":"publisher","DOI":"10.3389\/fmars.2022.869088","article-title":"Benchmarking and automating the image recognition capability of an in situ plankton imaging system","volume":"9","author":"Le","year":"2022","journal-title":"Front. Marine Sci."},{"key":"mlstace417bib34","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1109\/JOE.2021.3106122","article-title":"Development of a buoy-borne underwater imaging system for in situ mesoplankton monitoring of coastal waters","volume":"47","author":"Li","year":"2021","journal-title":"IEEE J. Ocean. Eng."},{"key":"mlstace417bib35","doi-asserted-by":"publisher","first-page":"695","DOI":"10.1109\/JOE.2013.2280035","article-title":"Pairwise nonparametric discriminant analysis for binary plankton image recognition","volume":"39","author":"Li","year":"2014","journal-title":"IEEE J. Ocean. Eng."},{"key":"mlstace417bib36","first-page":"pp 740","article-title":"Microsoft coco: common objects in context","author":"Lin","year":"2014","edition":"ed"},{"key":"mlstace417bib37","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.ecoinf.2019.02.007","article-title":"Deep learning and transfer learning features for plankton classification","volume":"51","author":"Lumini","year":"2019","journal-title":"Ecol. Inf."},{"key":"mlstace417bib38","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1016\/j.aci.2019.11.004","article-title":"Deep learning for plankton and coral classification","volume":"19","author":"Lumini","year":"2020","journal-title":"Appl. Comput. Inf."},{"key":"mlstace417bib39","doi-asserted-by":"publisher","first-page":"814","DOI":"10.1002\/lom3.10285","article-title":"Automated plankton image analysis using convolutional neural networks","volume":"16","author":"Luo","year":"2018","journal-title":"Limnol. Oceanogr.: Methods"},{"key":"mlstace417bib40","doi-asserted-by":"publisher","first-page":"57","DOI":"10.3354\/meps09984","article-title":"Marine snow, zooplankton and thin layers: indications of a trophic link from small-scale sampling with the video plankton recorder","volume":"468","author":"M\u00f6ller","year":"2012","journal-title":"Marine Ecol. Prog. Ser."},{"key":"mlstace417bib41","doi-asserted-by":"publisher","first-page":"530","DOI":"10.1093\/plankt\/fbv033","article-title":"Effects of climate-induced habitat changes on a key zooplankton species","volume":"37","author":"M\u00f6ller","year":"2015","journal-title":"J. Plankton Res."},{"key":"mlstace417bib42","first-page":"pp 8024","article-title":"Pytorch: an imperative style, high-performance deep learning library","author":"Paszke","year":"2019","edition":"ed"},{"key":"mlstace417bib43","article-title":"Evaluating self and semi-supervised methods for remote sensing segmentation tasks","author":"Patel","year":"2022"},{"key":"mlstace417bib44","doi-asserted-by":"publisher","DOI":"10.3389\/fmars.2022.878803","article-title":"Coupling imaging and omics in plankton surveys: State-of-the-art, challenges and future directions","volume":"9","author":"Pierella Karlusich","year":"2022","journal-title":"Front. Mar. Sci."},{"key":"mlstace417bib45","doi-asserted-by":"crossref","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet large scale visual recognition challenge","author":"Russakovsky","year":"2015"},{"key":"mlstace417bib46","doi-asserted-by":"publisher","first-page":"8972","DOI":"10.3390\/app12188972","article-title":"Deep residual learning for image recognition: a survey","volume":"12","author":"Shafiq","year":"2022","journal-title":"Appl. Sci."},{"key":"mlstace417bib47","article-title":"Improved deep metric learning with multi-class n-pair loss objective","author":"Sohn","year":"2016","edition":"ed"},{"key":"mlstace417bib48","doi-asserted-by":"publisher","first-page":"204","DOI":"10.4319\/lom.2007.5.204","article-title":"Automated taxonomic classification of phytoplankton sampled with imaging-in-flow cytometry","volume":"5","author":"Sosik","year":"2007","journal-title":"Limnol. Oceanogr.: Methods"},{"key":"mlstace417bib49","doi-asserted-by":"crossref","DOI":"10.1109\/ICCV.2017.97","article-title":"Revisiting unreasonable effectiveness of data in deep learning era","author":"Sun","year":"2017"},{"key":"mlstace417bib50","doi-asserted-by":"publisher","first-page":"1155","DOI":"10.1002\/lno.11672","article-title":"Trait-based approach using in situ copepod images reveals contrasting ecological patterns across an arctic ice melt zone","volume":"66","author":"Vilgrain","year":"2021","journal-title":"Limnol. Oceanogr."},{"key":"mlstace417bib51","first-page":"pp 855","article-title":"CGAN-plankton: towards large-scale imbalanced class generation and fine-grained classification","author":"Wang","year":"2017"},{"key":"mlstace417bib52","doi-asserted-by":"publisher","first-page":"3021","DOI":"10.21105\/joss.03021","article-title":"Seaborn: statistical data visualization","volume":"6","author":"Waskom","year":"2021","journal-title":"J. Open Source Softw."},{"key":"mlstace417bib53","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1016\/j.future.2022.05.014","article-title":"A survey of human-in-the-loop for machine learning","volume":"135","author":"Wu","year":"2022","journal-title":"Future Gener. Comput. Syst."},{"key":"mlstace417bib54","article-title":"Hydra - a framework for elegantly configuring complex applications","author":"Yadan","year":"2019"},{"key":"mlstace417bib55","article-title":"Simper: simple self-supervised learning of periodic targets","author":"Yang","year":"2023"},{"key":"mlstace417bib56","first-page":"pp 668","article-title":"Decoupled contrastive learning","author":"Yeh","year":"2022","edition":"ed"},{"key":"mlstace417bib57","article-title":"Large batch training of convolutional networks","author":"You","year":"2017"},{"key":"mlstace417bib58","doi-asserted-by":"crossref","DOI":"10.5244\/C.30.87","article-title":"Wide residual networks","author":"Zagoruyko","year":"2016"},{"key":"mlstace417bib59","doi-asserted-by":"publisher","first-page":"570","DOI":"10.1186\/s12859-017-1954-8","article-title":"Automatic plankton image classification combining multiple view features via multiple kernel learning","volume":"18","author":"Zheng","year":"2017","journal-title":"BMC Bioinform."},{"key":"mlstace417bib60","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1007\/s10462-004-0751-8","article-title":"Class noise vs. attribute noise: a quantitative study","volume":"22","author":"Zhu","year":"2004","journal-title":"Artif. Intell. Rev."}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace417","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace417\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace417","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace417\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace417\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace417\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace417\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace417\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,14]],"date-time":"2023-07-14T12:28:26Z","timestamp":1689337706000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ace417"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,14]]},"references-count":60,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,7,14]]},"published-print":{"date-parts":[[2023,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ace417","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,14]]},"assertion":[{"value":"Robust detection of marine life with label-free image feature learning and probability calibration","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2023 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2022-11-01","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2023-07-04","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2023-07-14","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}