{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T19:07:42Z","timestamp":1772910462983,"version":"3.50.1"},"reference-count":44,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2020,10,20]],"date-time":"2020-10-20T00:00:00Z","timestamp":1603152000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100004326","name":"Bayer AG","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004326","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Life Science Collaboration project"},{"name":"DeepMinDS"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,5,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Image-based profiling combines high-throughput screening with multiparametric feature analysis to capture the effect of perturbations on biological systems. This technology has attracted increasing interest in the field of plant phenotyping, promising to accelerate the discovery of novel herbicides. However, the extraction of meaningful features from unlabeled plant images remains a big challenge.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We describe a novel data-driven approach to find feature representations from plant time-series images in a self-supervised manner by using time as a proxy for image similarity. In the spirit of transfer learning, we first apply an ImageNet-pretrained architecture as a base feature extractor. Then, we extend this architecture with a triplet network to refine and reduce the dimensionality of extracted features by ranking relative similarities between consecutive and non-consecutive time points. Without using any labels, we produce compact, organized representations of plant phenotypes and demonstrate their superior applicability to clustering, image retrieval and classification tasks. Besides time, our approach could be applied using other surrogate measures of phenotype similarity, thus providing a versatile method of general interest to the phenotypic profiling community.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>Source code is provided in https:\/\/github.com\/bayer-science-for-a-better-life\/plant-triplet-net.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btaa905","type":"journal-article","created":{"date-parts":[[2020,10,8]],"date-time":"2020-10-08T19:23:26Z","timestamp":1602185006000},"page":"861-867","source":"Crossref","is-referenced-by-count":17,"title":["Self-supervised feature extraction from image time series in plant phenotyping using triplet networks"],"prefix":"10.1093","volume":"37","author":[{"given":"Paula A","family":"Marin Zapata","sequence":"first","affiliation":[{"name":"Bayer AG, Machine Learning Research, Research and Development , Pharmaceuticals, Berlin, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sina","family":"Roth","sequence":"additional","affiliation":[{"name":"Bayer AG, High Throughput Biology - Weed Control, Research & Development, Crop Science , Frankfurt, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dirk","family":"Schmutzler","sequence":"additional","affiliation":[{"name":"Bayer AG, High Throughput Biology - Weed Control, Research & Development, Crop Science , Frankfurt, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thomas","family":"Wolf","sequence":"additional","affiliation":[{"name":"Bayer AG, Computational Life Sciences - Weed Control, Research & Development , Crop Science, Frankfurt, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Erica","family":"Manesso","sequence":"additional","affiliation":[{"name":"Bayer AG, Computational Life Sciences - Weed Control, Research & Development , Crop Science, Frankfurt, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Djork-Arn\u00e9","family":"Clevert","sequence":"additional","affiliation":[{"name":"Bayer AG, Machine Learning Research, Research and Development , Pharmaceuticals, Berlin, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2020,10,20]]},"reference":[{"key":"2023051705210238500_btaa905-B1","first-page":"79","volume-title":"Gesellschaft f\u00fcr Informatik","author":"Amara","year":"2017"},{"issue":"2","key":"2023051705210238500_btaa905-B2","doi-asserted-by":"crossref","first-page":"256","DOI":"10.3390\/sym11020256","article-title":"Identification and classification of maize drought stress using deep convolutional neural network","volume":"11","author":"An","year":"2019","journal-title":"Symmetry"},{"key":"2023051705210238500_btaa905-B3","article-title":"Improving phenotypic measurements in high-content imaging screens","author":"Ando","year":"2017","journal-title":"bioRxiv"},{"key":"2023051705210238500_btaa905-B4","year":"2005"},{"key":"2023051705210238500_btaa905-B5","doi-asserted-by":"crossref","first-page":"1741","DOI":"10.3389\/fpls.2017.01741","article-title":"X-FIDO: an effective application for detecting olive quick decline syndrome with deep learning and data fusion","volume":"8","author":"Cruz","year":"2017","journal-title":"Front. Plant Sci"},{"key":"2023051705210238500_btaa905-B6","first-page":"41","volume-title":"ICCV Workshop on Computer Vision Problems in Plant Phenotyping","author":"Das Choudhury","year":"2017"},{"key":"2023051705210238500_btaa905-B7","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1186\/s13007-018-0303-x","article-title":"Holistic and component plant phenotyping using temporal image sequence","volume":"14","author":"Das Choudhury","year":"2018","journal-title":"Plant Methods"},{"key":"2023051705210238500_btaa905-B8","volume-title":".","author":"Deng","year":"2009"},{"key":"2023051705210238500_btaa905-B9","author":"Esgario","year":"2019"},{"key":"2023051705210238500_btaa905-B10","first-page":"226","volume-title":"Conference on Knowledge Discovery and Data Mining","author":"Ester","year":"1996"},{"key":"2023051705210238500_btaa905-B11","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1016\/j.compag.2018.01.009","article-title":"Deep learning models for plant disease detection and diagnosis","volume":"145","author":"Ferentinos","year":"2018","journal-title":"Comput. Electron. Agric"},{"key":"2023051705210238500_btaa905-B12","doi-asserted-by":"crossref","first-page":"972","DOI":"10.1126\/science.1136800","article-title":"Clustering by passing messages between data points","volume":"315","author":"Frey","year":"2007","journal-title":"Science"},{"key":"2023051705210238500_btaa905-B13","journal-title":"In:\u00a0"},{"key":"2023051705210238500_btaa905-B14","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1007\/s11263-017-1016-8","article-title":"End-to-end learning of deep visual representations for image retrieval","volume":"124","author":"Gordo","year":"2017","journal-title":"Int. J. Comput. Vis"},{"key":"2023051705210238500_btaa905-B15","author":"He","year":"2015"},{"key":"2023051705210238500_btaa905-B16","author":"Hoffer","year":"2014"},{"key":"2023051705210238500_btaa905-B17","author":"Hughes","year":"2015"},{"key":"2023051705210238500_btaa905-B18","author":"Kingma","year":"2014"},{"key":"2023051705210238500_btaa905-B19","author":"Koch","year":"2015"},{"key":"2023051705210238500_btaa905-B20","author":"Kornblith","year":"2018"},{"key":"2023051705210238500_btaa905-B21","author":"Krizhevsky","year":"2014"},{"key":"2023051705210238500_btaa905-B22","author":"Krizhevsky","year":"2012"},{"key":"2023051705210238500_btaa905-B23","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1162\/neco.1989.1.4.541","article-title":"Backpropagation applied to handwritten zip code recognition","volume":"1","author":"LeCun","year":"1989","journal-title":"Neural Comput"},{"key":"2023051705210238500_btaa905-B24","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"2023051705210238500_btaa905-B25","doi-asserted-by":"crossref","first-page":"e0196615","DOI":"10.1371\/journal.pone.0196615","article-title":"An automated, high-throughput plant phenotyping system using machine learning-based plant segmentation and image analysis","volume":"13","author":"Lee","year":"2018","journal-title":"PLoS One"},{"key":"2023051705210238500_btaa905-B26","doi-asserted-by":"crossref","first-page":"11","DOI":"10.3390\/sym10010011","article-title":"Identification of apple leaf diseases based on deep convolutional neural networks","volume":"10","author":"Liu","year":"2017","journal-title":"Symmetry"},{"key":"2023051705210238500_btaa905-B27","author":"McInnes","year":"2018"},{"key":"2023051705210238500_btaa905-B28","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.patrec.2015.10.013","article-title":"Finely-grained annotated datasets for image-based plant phenotyping","volume":"81","author":"Minervini","year":"2016","journal-title":"Pattern Recognit. Lett"},{"key":"2023051705210238500_btaa905-B29","doi-asserted-by":"crossref","first-page":"1419","DOI":"10.3389\/fpls.2016.01419","article-title":"Using deep learning for image-based plant disease detection","volume":"7","author":"Mohanty","year":"2016","journal-title":"Front. Plant Sci"},{"key":"2023051705210238500_btaa905-B30","doi-asserted-by":"crossref","first-page":"3983","DOI":"10.1093\/jxb\/ert208","article-title":"Chlorophyll fluorescence analysis: a guide to good practice and understanding some new applications","volume":"64","author":"Murchie","year":"2013","journal-title":"J. Exp. Bot"},{"key":"2023051705210238500_btaa905-B31","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A survey on transfer learning","volume":"22","author":"Pan","year":"2010","journal-title":"IEEE Trans. Knowl. Data Eng"},{"key":"2023051705210238500_btaa905-B32","author":"Paszke","year":"2017"},{"key":"2023051705210238500_btaa905-B33","doi-asserted-by":"crossref","first-page":"886","DOI":"10.1038\/nbt.1991","article-title":"Extracting a cellular hierarchy from high-dimensional cytometry data with SPADE","volume":"29","author":"Qiu","year":"2011","journal-title":"Nat. Biotechnol"},{"key":"2023051705210238500_btaa905-B34","doi-asserted-by":"crossref","first-page":"1852","DOI":"10.3389\/fpls.2017.01852","article-title":"Deep learning for image-based cassava disease detection","volume":"8","author":"Ramcharan","year":"2017","journal-title":"Front. Plant Sci"},{"key":"2023051705210238500_btaa905-B35","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis.,"},{"key":"2023051705210238500_btaa905-B36","author":"Schroff","year":"2015"},{"key":"2023051705210238500_btaa905-B37","author":"Simonyan","year":"2014"},{"key":"2023051705210238500_btaa905-B38","doi-asserted-by":"crossref","first-page":"883","DOI":"10.1016\/j.tplants.2018.07.004","article-title":"Deep learning for plant stress phenotyping: trends and future perspectives","volume":"23","author":"Singh","year":"2018","journal-title":"Trends Plant Sci"},{"key":"2023051705210238500_btaa905-B39","author":"Szegedy","year":"2015"},{"key":"2023051705210238500_btaa905-B40","author":"Szegedy","year":"2016"},{"key":"2023051705210238500_btaa905-B41","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1186\/s13007-018-0333-4","article-title":"Deep phenotyping: deep learning for temporal phenotype\/genotype classification","volume":"14","author":"Taghavi","year":"2018","journal-title":"Plant Methods"},{"key":"2023051705210238500_btaa905-B42","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1016\/j.compag.2018.03.032","article-title":"A comparative study of fine-tuning deep learning models for plant disease identification","volume":"161","author":"Too","year":"2019","journal-title":"Comput. Electron. Agric"},{"key":"2023051705210238500_btaa905-B43","author":"Wang","year":"2015"},{"key":"2023051705210238500_btaa905-B44","first-page":"1386","author":"Wang","year":"2014"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaa905\/34542372\/btaa905.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/6\/861\/50357621\/btaa905.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/6\/861\/50357621\/btaa905.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,17]],"date-time":"2023-05-17T05:22:11Z","timestamp":1684300931000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/37\/6\/861\/5932385"}},"subtitle":[],"editor":[{"given":"Zhiyong","family":"Lu","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2020,10,20]]},"references-count":44,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,5,5]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btaa905","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,3,15]]},"published":{"date-parts":[[2020,10,20]]}}}