{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T16:36:52Z","timestamp":1781714212901,"version":"3.54.5"},"reference-count":36,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2016,10,28]],"date-time":"2016-10-28T00:00:00Z","timestamp":1477612800000},"content-version":"vor","delay-in-days":139,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2016,6,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Motivation : High-content screening (HCS) technologies have enabled large scale imaging experiments for studying cell biology and for drug screening. These systems produce hundreds of thousands of microscopy images per day and their utility depends on automated image analysis. Recently, deep learning approaches that learn feature representations directly from pixel intensity values have dominated object recognition challenges. These tasks typically have a single centered object per image and existing models are not directly applicable to microscopy datasets. Here we develop an approach that combines deep convolutional neural networks (CNNs) with multiple instance learning (MIL) in order to classify and segment microscopy images using only whole image level annotations.<\/jats:p>\n               <jats:p>Results : We introduce a new neural network architecture that uses MIL to simultaneously classify and segment microscopy images with populations of cells. We base our approach on the similarity between the aggregation function used in MIL and pooling layers used in CNNs. To facilitate aggregating across large numbers of instances in CNN feature maps we present the Noisy-AND pooling function, a new MIL operator that is robust to outliers. Combining CNNs with MIL enables training CNNs using whole microscopy images with image level labels. We show that training end-to-end MIL CNNs outperforms several previous methods on both mammalian and yeast datasets without requiring any segmentation steps.<\/jats:p>\n               <jats:p>Availability and implementation : Torch7 implementation available upon request.<\/jats:p>\n               <jats:p>Contact : oren.kraus@mail.utoronto.ca<\/jats:p>","DOI":"10.1093\/bioinformatics\/btw252","type":"journal-article","created":{"date-parts":[[2016,6,15]],"date-time":"2016-06-15T15:43:52Z","timestamp":1466005432000},"page":"i52-i59","source":"Crossref","is-referenced-by-count":372,"title":["Classifying and segmenting microscopy images with deep multiple instance learning"],"prefix":"10.1093","volume":"32","author":[{"given":"Oren Z.","family":"Kraus","sequence":"first","affiliation":[{"name":"1 Department of Electrical and Computer Engineering, University of Toronto, Toronto, M5S 2E4, Canada"},{"name":"2 The Donnelly Centre for Cellular and Biomolecular Research, University of Toronto, Toronto, M5S 3E1, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jimmy Lei","family":"Ba","sequence":"additional","affiliation":[{"name":"1 Department of Electrical and Computer Engineering, University of Toronto, Toronto, M5S 2E4, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brendan J.","family":"Frey","sequence":"additional","affiliation":[{"name":"1 Department of Electrical and Computer Engineering, University of Toronto, Toronto, M5S 2E4, Canada"},{"name":"2 The Donnelly Centre for Cellular and Biomolecular Research, University of Toronto, Toronto, M5S 3E1, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2016,6,11]]},"reference":[{"key":"2023020112344731300_btw252-B1","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1016\/j.cell.2010.04.033","article-title":"Cellular heterogeneity: do differences make a difference?","volume":"141","author":"Altschuler","year":"2010","journal-title":"Cell"},{"key":"2023020112344731300_btw252-B2","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1083\/jcb.201301120","article-title":"A novel single-cell screening platform reveals proteome plasticity during yeast stress responses","volume":"200","author":"Breker","year":"2013","journal-title":"J. Cell Biol"},{"key":"2023020112344731300_btw252-B3","doi-asserted-by":"crossref","first-page":"R100","DOI":"10.1186\/gb-2006-7-10-r100","article-title":"Cellprofiler: image analysis software for identifying and quantifying cell phenotypes","volume":"7","author":"Carpenter","year":"2006","journal-title":"Genome Biol"},{"key":"2023020112344731300_btw252-B4","first-page":"7062","article-title":"Semantic image segmentation with deep convolutional nets and fully connected crfs","volume":"1412","author":"Chen","year":"2014","journal-title":"arXiv Preprint arXiv"},{"key":"2023020112344731300_btw252-B5","doi-asserted-by":"crossref","first-page":"1413","DOI":"10.1016\/j.cell.2015.04.051","article-title":"Yeast proteome dynamics from single cell imaging and automated analysis","volume":"161","author":"Chong","year":"2015","journal-title":"Cell"},{"key":"2023020112344731300_btw252-B6","author":"Ciresan","year":"2012"},{"key":"2023020112344731300_btw252-B7","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/S0004-3702(96)00034-3","article-title":"Solving the multiple instance problem with axis-parallel rectangles","volume":"89","author":"Dietterich","year":"1997","journal-title":"Art. Intel"},{"key":"2023020112344731300_btw252-B8","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1038\/nmeth.2084","article-title":"Biological imaging software tools","volume":"9","author":"Eliceiri","year":"2012","journal-title":"Nat. Methods"},{"key":"2023020112344731300_btw252-B9","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/3348.001.0001","volume-title":"Graphical Models for Machine Learning and Digital Communication","author":"Frey","year":"1998"},{"key":"2023020112344731300_btw252-B10","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1038\/nmeth.1486","article-title":"Cellcognition: time-resolved phenotype annotation in high-throughput live cell imaging","volume":"7","author":"Held","year":"2010","journal-title":"Nat. Methods"},{"key":"2023020112344731300_btw252-B11","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1038\/nature02026","article-title":"Global analysis of protein localization in budding yeast","volume":"425","author":"Huh","year":"2003","journal-title":"Nature"},{"key":"2023020112344731300_btw252-B12","author":"Keeler","year":"1991"},{"key":"2023020112344731300_btw252-B13","first-page":"6980","article-title":"Adam: A method for stochastic optimization","volume":"1412","author":"Kingma","year":"2014","journal-title":"arXiv Preprint arXiv"},{"key":"2023020112344731300_btw252-B201","doi-asserted-by":"crossref","first-page":"102","DOI":"10.3109\/10409238.2015.1135868","article-title":"Computer vision for high content screening","volume":"51","author":"Kraus","year":"2016","journal-title":"Critical reviews in biochemistry and molecular biology"},{"key":"2023020112344731300_btw252-B14","author":"Krizhevsky","year":"2012"},{"key":"2023020112344731300_btw252-B15","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"2023020112344731300_btw252-B16","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1038\/nrg3768","article-title":"Single-cell and multivariate approaches in genetic perturbation screens","volume":"16","author":"Liberali","year":"2015","journal-title":"Nat. Rev. Genet"},{"key":"2023020112344731300_btw252-B17","doi-asserted-by":"crossref","first-page":"637hods.","DOI":"10.1038\/nmeth.2083","article-title":"Annotated high-throughput microscopy image sets for validation","volume":"9","author":"Ljosa","year":"2012","journal-title":"Nat. Methods"},{"key":"2023020112344731300_btw252-B18","doi-asserted-by":"crossref","first-page":"1321","DOI":"10.1177\/1087057113503553","article-title":"Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment","volume":"18","author":"Ljosa","year":"2013","journal-title":"J. Biomol. Screen"},{"key":"2023020112344731300_btw252-B19","volume-title":"CVPR","author":"Long","year":"2015"},{"key":"2023020112344731300_btw252-B20","doi-asserted-by":"crossref","first-page":"1360","DOI":"10.1109\/TIP.2005.852470","article-title":"Toward automatic phenotyping of developing embryos from videos","volume":"14","author":"Ning","year":"2005","journal-title":"Image Process. IEEE Trans"},{"key":"2023020112344731300_btw252-B21","author":"Pachitariu","year":"2013"},{"key":"2023020112344731300_btw252-B22","first-page":"7144","article-title":"Fully convolutional multi-class multiple instance learning","volume":"1412","author":"Pathak","year":"2014","journal-title":"arXiv Preprint arXiv"},{"key":"2023020112344731300_btw252-B23","author":"Ramon","year":"2000"},{"key":"2023020112344731300_btw252-B24","first-page":"1556","article-title":"Very deep convolutional networks for large-scale image recognition","volume":"1409","author":"Simonyan","year":"2014","journal-title":"arXiv Preprint arXiv"},{"key":"2023020112344731300_btw252-B25","first-page":"6034","article-title":"Deep inside convolutional networks: visualising image classification models and saliency maps","volume":"1312","author":"Simonyan","year":"2013","journal-title":"arXiv Preprint arXiv"},{"key":"2023020112344731300_btw252-B26","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1177\/1087057114528537","article-title":"Increasing the content of high-content screening an overview","volume":"19","author":"Singh","year":"2014","journal-title":"J. Biomol. Screen"},{"key":"2023020112344731300_btw252-B27","first-page":"6806","article-title":"Striving for simplicity: the all convolutional net","volume":"1412","author":"Springenberg","year":"2014","journal-title":"arXiv Preprint arXiv"},{"key":"2023020112344731300_btw252-B28","first-page":"4842","article-title":"Going deeper with convolutions","volume":"1409","author":"Szegedy","year":"2014","journal-title":"arXiv Preprint arXiv"},{"key":"2023020112344731300_btw252-B29","doi-asserted-by":"crossref","first-page":"966","DOI":"10.1038\/ncb2549","article-title":"Dissecting dna damage response pathways by analysing protein localization and abundance changes during dna replication stress","volume":"14","author":"Tkach","year":"2012","journal-title":"Nat. Cell Biol"},{"key":"2023020112344731300_btw252-B30","first-page":"3371","article-title":"Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion","volume":"11","author":"Vincent","year":"2010","journal-title":"J. Mach. Learn. Res"},{"key":"2023020112344731300_btw252-B31","doi-asserted-by":"crossref","first-page":"714","DOI":"10.1038\/nmeth.1984","article-title":"An image analysis toolbox for high-throughput C. elegans assays","volume":"9","author":"W\u00e4hlby","year":"2012","journal-title":"Nat. Methods"},{"key":"2023020112344731300_btw252-B32","volume-title":"In Advances in Knowledge Discovery and Data Mining","author":"Xu","year":"2004"},{"key":"2023020112344731300_btw252-B33","author":"Xu","year":"2014"},{"key":"2023020112344731300_btw252-B34","author":"Zeiler","year":"2014"},{"key":"2023020112344731300_btw252-B35","author":"Zhang","year":"2005"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/32\/12\/i52\/49021130\/bioinformatics_32_12_i52.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/32\/12\/i52\/49021130\/bioinformatics_32_12_i52.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,1]],"date-time":"2023-02-01T22:46:32Z","timestamp":1675291592000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/32\/12\/i52\/2288769"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,6,11]]},"references-count":36,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2016,6,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btw252","relation":{},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2016,6,15]]},"published":{"date-parts":[[2016,6,11]]}}}