{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T04:26:15Z","timestamp":1774671975114,"version":"3.50.1"},"reference-count":62,"publisher":"Oxford University Press (OUP)","issue":"13","license":[{"start":{"date-parts":[[2017,2,15]],"date-time":"2017-02-15T00:00:00Z","timestamp":1487116800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/about_us\/legal\/notices"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,7,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Identifying phenotypes based on high-content cellular images is challenging. Conventional image analysis pipelines for phenotype identification comprise multiple independent steps, with each step requiring method customization and adjustment of multiple parameters.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Here, we present an approach based on a multi-scale convolutional neural network (M-CNN) that classifies, in a single cohesive step, cellular images into phenotypes by using directly and solely the images\u2019 pixel intensity values. The only parameters in the approach are the weights of the neural network, which are automatically optimized based on training images. The approach requires no a priori knowledge or manual customization, and is applicable to single- or multi-channel images displaying single or multiple cells. We evaluated the classification performance of the approach on eight diverse benchmark datasets. The approach yielded overall a higher classification accuracy compared with state-of-the-art results, including those of other deep CNN architectures. In addition to using the network to simply obtain a yes-or-no prediction for a given phenotype, we use the probability outputs calculated by the network to quantitatively describe the phenotypes. This study shows that these probability values correlate with chemical treatment concentrations. This finding validates further our approach and enables chemical treatment potency estimation via CNNs.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and Implementation<\/jats:title>\n                  <jats:p>The network specifications and solver definitions are provided in Supplementary Software 1.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btx069","type":"journal-article","created":{"date-parts":[[2017,2,15]],"date-time":"2017-02-15T08:58:08Z","timestamp":1487149088000},"page":"2010-2019","source":"Crossref","is-referenced-by-count":173,"title":["A multi-scale convolutional neural network for phenotyping high-content cellular images"],"prefix":"10.1093","volume":"33","author":[{"given":"William J","family":"Godinez","sequence":"first","affiliation":[{"name":"Novartis Institutes for BioMedical Research Inc., Basel, Switzerland"}]},{"given":"Imtiaz","family":"Hossain","sequence":"additional","affiliation":[{"name":"Novartis Institutes for BioMedical Research Inc., Basel, Switzerland"}]},{"given":"Stanley E","family":"Lazic","sequence":"additional","affiliation":[{"name":"Novartis Institutes for BioMedical Research Inc., Basel, Switzerland"}]},{"given":"John W","family":"Davies","sequence":"additional","affiliation":[{"name":"Novartis Institutes for BioMedical Research Inc., Cambridge, MA, USA"}]},{"given":"Xian","family":"Zhang","sequence":"additional","affiliation":[{"name":"Novartis Institutes for BioMedical Research Inc., Basel, Switzerland"}]}],"member":"286","published-online":{"date-parts":[[2017,2,15]]},"reference":[{"key":"2023020206012464100_btx069-B1","doi-asserted-by":"crossref","first-page":"831","DOI":"10.1038\/nbt.3300","article-title":"Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning","volume":"33","author":"Alipanahi","year":"2015","journal-title":"Nat. Biotechnol"},{"key":"2023020206012464100_btx069-B2","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1021\/jm00199a013","article-title":"Synthesis and adrenergic activity of benzimidazole bioisosteres of norepinephrine and isoproterenol","volume":"21","author":"Arnett","year":"1978","journal-title":"J. Med. Chem"},{"key":"2023020206012464100_btx069-B3","doi-asserted-by":"crossref","first-page":"1753","DOI":"10.1126\/science.1140324","article-title":"Quantitative morphological signatures define local signaling networks regulating cell morphology","volume":"316","author":"Bakal","year":"2007","journal-title":"Science"},{"key":"2023020206012464100_btx069-B4","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1093\/toxsci\/kft069","article-title":"Hydroxyurea exposure triggers tissue-specific activation of p38 mitogen-activated protein kinase signaling and the DNA damage response in organogenesis-stage mouse embryos","volume":"133","author":"Banh","year":"2013","journal-title":"Toxicol. Sci"},{"key":"2023020206012464100_btx069-B5","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1074\/mcp.M700325-MCP200","article-title":"Toward a confocal subcellular atlas of the human proteome","volume":"7","author":"Barbe","year":"2008","journal-title":"Mol. Cell. Proteomics"},{"key":"2023020206012464100_btx069-B6","doi-asserted-by":"crossref","first-page":"1213","DOI":"10.1093\/bioinformatics\/17.12.1213","article-title":"A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscope images of HeLa cells","volume":"17","author":"Boland","year":"2001","journal-title":"Bioinformatics"},{"key":"2023020206012464100_btx069-B7","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1002\/(SICI)1097-0320(19981101)33:3<366::AID-CYTO12>3.0.CO;2-R","article-title":"Automated recognition of patterns characteristic of subcellular structures in fluorescence microscopy images","volume":"33","author":"Boland","year":"1998","journal-title":"Cytometry"},{"key":"2023020206012464100_btx069-B8","doi-asserted-by":"crossref","first-page":"1314","DOI":"10.1016\/j.cell.2015.11.007","article-title":"Microscopy-based high-content screening","volume":"163","author":"Boutros","year":"2015","journal-title":"Cell"},{"key":"2023020206012464100_btx069-B9","first-page":"342","article-title":"Multiscale convolutional neural networks for vision-based classification of cells","author":"Buyssens","year":"2012","journal-title":"Proc. Asian Conf. Comput. Vis"},{"key":"2023020206012464100_btx069-B10","doi-asserted-by":"crossref","first-page":"1913","DOI":"10.1158\/1535-7163.MCT-09-1148","article-title":"High-content phenotypic profiling of drug response signatures across distinct cancer cells","volume":"9","author":"Caie","year":"2010","journal-title":"Mol. Cancer. Ther"},{"key":"2023020206012464100_btx069-B11","doi-asserted-by":"crossref","first-page":"R100.","DOI":"10.1186\/gb-2006-7-10-r100","article-title":"CellProfiler: image analysis software for identifying and quantifying cell phenotypes","volume":"7","author":"Carpenter","year":"2006","journal-title":"Genome Biol"},{"key":"2023020206012464100_btx069-B12","doi-asserted-by":"crossref","first-page":"210.","DOI":"10.1186\/1471-2105-8-210","article-title":"A multiresolution approach to automated classification of protein subcellular location images","volume":"8","author":"Chebira","year":"2007","journal-title":"BMC Bioinformatics"},{"key":"2023020206012464100_btx069-B13","doi-asserted-by":"crossref","first-page":"1832","DOI":"10.1093\/bioinformatics\/btw074","article-title":"Gene expression inference with deep learning","volume":"32","author":"Chen","year":"2016","journal-title":"Bioinformatics"},{"key":"2023020206012464100_btx069-B14","first-page":"2843","article-title":"Deep neural networks segment neuronal membranes in electron microscopy images","volume":"25","author":"Ciresan","year":"2012","journal-title":"Proc. Advances in Neural Information Processing Systems"},{"key":"2023020206012464100_btx069-B15","first-page":"3642","article-title":"Multi-column deep neural networks for image classification","author":"Ciresan","year":"2012","journal-title":"Proc. IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"2023020206012464100_btx069-B16","doi-asserted-by":"crossref","first-page":"2343","DOI":"10.1093\/bioinformatics\/btt392","article-title":"Determining the subcellular location of new proteins from microscope images using local features","volume":"29","author":"Coelho","year":"2013","journal-title":"Bioinformatics"},{"key":"2023020206012464100_btx069-B17","first-page":"1223","article-title":"Large scale distributed deep networks","volume":"25","author":"Dean","year":"2012","journal-title":"Proc. Advances in Neural Information Processing Systems"},{"key":"2023020206012464100_btx069-B18","doi-asserted-by":"crossref","first-page":"1915","DOI":"10.1109\/TPAMI.2012.231","article-title":"Learning hierarchical features for scene labeling","volume":"35","author":"Farabet","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"2023020206012464100_btx069-B19","doi-asserted-by":"crossref","first-page":"21718","DOI":"10.18632\/oncotarget.4304","article-title":"Identification of alsterpaullone as a novel small molecule inhibitor to target group 3 medulloblastoma","volume":"6","author":"Faria","year":"2015","journal-title":"Oncotarget"},{"key":"2023020206012464100_btx069-B20","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1016\/j.neuron.2015.02.023","article-title":"Cell-based screening: extracting meaning from complex data","volume":"86","author":"Finkbeiner","year":"2015","journal-title":"Neuron"},{"key":"2023020206012464100_btx069-B21","doi-asserted-by":"crossref","first-page":"370.","DOI":"10.1038\/msb.2010.25","article-title":"Clustering phenotype populations by genome-wide RNAi and multiparametric imaging","volume":"6","author":"Fuchs","year":"2010","journal-title":"Mol. Syst. Biol"},{"key":"2023020206012464100_btx069-B22","article-title":"HEp-2 cell image classification with deep convolutional neural networks","author":"Gao","year":"2016","journal-title":"IEEE J. Biomed. Heal. Informatics"},{"key":"2023020206012464100_btx069-B23","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/j.jneumeth.2010.07.003","article-title":"An imaging assay to analyze primary neurons for cellular neurotoxicity","volume":"192","author":"G\u00f6tte","year":"2010","journal-title":"J. Neurosci. Methods"},{"key":"2023020206012464100_btx069-B24","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1038\/nmeth.3545","article-title":"Trajectories of cell-cycle progression from fixed cell populations","volume":"12","author":"Gut","year":"2015","journal-title":"Nat. Methods"},{"key":"2023020206012464100_btx069-B25","doi-asserted-by":"crossref","first-page":"786","DOI":"10.1109\/PROC.1979.11328","article-title":"Statistical and structural approaches to texture","author":"Haralick","year":"1979","journal-title":"Proc. IEEE"},{"key":"2023020206012464100_btx069-B26","first-page":"770","article-title":"Deep residual learning for image recognition","author":"He","year":"2016","journal-title":"Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR\u201916)"},{"key":"2023020206012464100_btx069-B27","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1038\/nature12346","article-title":"Connectomic reconstruction of the inner plexiform layer in the mouse retina","volume":"500","author":"Helmstaedter","year":"2013","journal-title":"Nature"},{"key":"2023020206012464100_btx069-B28","first-page":"1139","article-title":"Automated classification of subcellular patterns in multicell images without segmentation into single cells","author":"Huang","year":"2004","journal-title":"Proc. IEEE Int. Symp. on Biomedical Imaging: From Nano to Macro (ISBI\u201904)"},{"key":"2023020206012464100_btx069-B29","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.media.2016.10.004","article-title":"Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation","volume":"36","author":"Kamnitsas","year":"2017","journal-title":"Med. Image Anal"},{"key":"2023020206012464100_btx069-B30","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1177\/1087057115623451","article-title":"High-content analysis of breast cancer using single-cell deep transfer learning","volume":"21","author":"Kandaswamy","year":"2016","journal-title":"J. Biomol. Screen"},{"key":"2023020206012464100_btx069-B31","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1021\/jm00348a014","article-title":"Syntheses and adrenergic agonist properties of ring-fluorinated isoproterenols","volume":"25","author":"Kirk","year":"1982","journal-title":"J. Med. Chem"},{"key":"2023020206012464100_btx069-B32","doi-asserted-by":"crossref","first-page":"1683","DOI":"10.1016\/j.ejmech.2009.12.072","article-title":"Novel derivatives of 1,3,4-oxadiazoles are potent mitostatic agents featuring strong microtubule depolymerizing activity in the sea urchin embryo and cell culture assays","volume":"45","author":"Kiselyov","year":"2010","journal-title":"Eur. J. Med. Chem"},{"key":"2023020206012464100_btx069-B33","doi-asserted-by":"crossref","first-page":"i52","DOI":"10.1093\/bioinformatics\/btw252","article-title":"Classifying and segmenting microscopy images with deep multiple instance learning","volume":"32","author":"Kraus","year":"2016","journal-title":"Bioinformatics"},{"key":"2023020206012464100_btx069-B34","first-page":"1097","article-title":"ImageNet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Proc. Advances in Neural Information Processing Systems"},{"key":"2023020206012464100_btx069-B35","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1002\/mc.10114","article-title":"Alsterpaullone, a novel cyclin-dependent kinase inhibitor, induces apoptosis by activation of caspase-9 due to perturbation in mitochondrial membrane potential","volume":"36","author":"Lahusen","year":"2003","journal-title":"Mol. Carcinog"},{"key":"2023020206012464100_btx069-B36","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"2023020206012464100_btx069-B37","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"2023020206012464100_btx069-B38","doi-asserted-by":"crossref","first-page":"e50514.","DOI":"10.1371\/journal.pone.0050514","article-title":"Automated analysis and reannotation of subcellular locations in confocal images from the human protein atlas","volume":"7","author":"Li","year":"2012","journal-title":"PLoS One"},{"key":"2023020206012464100_btx069-B39","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1038\/nrg3768","article-title":"Single-cell and multivariate approaches in genetic perturbation screens","volume":"16","author":"Liberali","year":"2014","journal-title":"Nat. Rev. Genet"},{"key":"2023020206012464100_btx069-B40","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1023\/A:1008045108935","article-title":"Feature detection with automatic scale selection","volume":"30","author":"Lindeberg","year":"1998","journal-title":"Int. J. Comput. Vis"},{"key":"2023020206012464100_btx069-B41","doi-asserted-by":"crossref","first-page":"637\u2013637.","DOI":"10.1038\/nmeth.2083","article-title":"Annotated high-throughput microscopy image sets for validation","volume":"9","author":"Ljosa","year":"2012","journal-title":"Nat. Methods"},{"key":"2023020206012464100_btx069-B42","doi-asserted-by":"crossref","first-page":"1321","DOI":"10.1177\/1087057113503553","article-title":"Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment","volume":"18","author":"Ljosa","year":"2013","journal-title":"J. Biomol. Screen"},{"key":"2023020206012464100_btx069-B43","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1038\/nmeth1032","article-title":"Image-based multivariate profiling of drug responses from single cells","volume":"4","author":"Loo","year":"2007","journal-title":"Nat. Methods"},{"key":"2023020206012464100_btx069-B44","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1002\/cyto.a.20662","article-title":"Single-cell-based image analysis of high-throughput cell array screens for quantification of viral infection","volume":"75A","author":"Matula","year":"2009","journal-title":"Cytometry A"},{"key":"2023020206012464100_btx069-B45","doi-asserted-by":"crossref","first-page":"1360","DOI":"10.1109\/TIP.2005.852470","article-title":"Toward automatic phenotyping of developing embryos from videos","volume":"14","author":"Ning","year":"2005","journal-title":"IEEE Transs. Image Process"},{"key":"2023020206012464100_btx069-B46","doi-asserted-by":"crossref","first-page":"1684","DOI":"10.1016\/j.patrec.2008.04.013","article-title":"WND-CHARM: multi-purpose image classification using compound image transforms","volume":"29","author":"Orlov","year":"2008","journal-title":"Pattern Recognit. Lett"},{"key":"2023020206012464100_btx069-B47","doi-asserted-by":"crossref","first-page":"4806","DOI":"10.1021\/jm0009540","article-title":"Synthesis and antitumor activities of novel pyrimidine derivatives of 2,3-O, O-dibenzyl-6-deoxy-l-ascorbic acid and 4,5-didehydro-5,6-dideoxy-l-ascorbic acid","volume":"43","author":"Rai\u0107-Mali\u0107","year":"2000","journal-title":"J. Med. Chem"},{"key":"2023020206012464100_btx069-B48","first-page":"238","article-title":"U-Net: convolutional networks for biomedical image segmentation","volume":"9350","author":"Ronneberger","year":"2015","journal-title":"Medical Image Computing and Computer-Assisted Intervention"},{"key":"2023020206012464100_btx069-B49","article-title":"OverFeat: integrated recognition, localization and detection using convolutional networks","author":"Sermanet","year":"2014","journal-title":"Proc. International Conference on Learning Representations"},{"key":"2023020206012464100_btx069-B50","doi-asserted-by":"crossref","first-page":"284","DOI":"10.1111\/j.1365-2818.2011.03502.x","article-title":"Assessing the efficacy of low-level image content descriptors for computer-based fluorescence microscopy image analysis","volume":"243","author":"Shamir","year":"2011","journal-title":"J. Microsc"},{"key":"2023020206012464100_btx069-B51","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2015","journal-title":"Proc. International Conference on Learning Representations"},{"key":"2023020206012464100_btx069-B52","article-title":"Deep inside convolutional networks: visualising image classification models and saliency maps","author":"Simonyan","year":"2014","journal-title":"Proc. International Conferences on Learning Representations Workshop"},{"key":"2023020206012464100_btx069-B53","doi-asserted-by":"crossref","first-page":"5605","DOI":"10.1016\/j.bmc.2009.06.030","article-title":"Urea and carbamate derivatives of primaquine: synthesis, cytostatic and antioxidant activities","volume":"17","author":"\u0160imunovi\u0107","year":"2009","journal-title":"Bioorg. Med. Chem"},{"key":"2023020206012464100_btx069-B54","first-page":"5529","article-title":"Machine learning in cell biology\u2014teaching computers to recognize phenotypes","volume":"126","author":"Sommer","year":"2013","journal-title":"J. Cell Sci"},{"key":"2023020206012464100_btx069-B55","first-page":"1","article-title":"Going deeper with convolutions","author":"Szegedy","year":"2015","journal-title":"Proc. IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"2023020206012464100_btx069-B56","doi-asserted-by":"crossref","first-page":", 51.","DOI":"10.1186\/s12859-016-0895-y","article-title":"CP-CHARM: segmentation-free image classification made accessible","volume":"17","author":"Uhlmann","year":"2016","journal-title":"BMC Bioinformatics"},{"key":"2023020206012464100_btx069-B57","doi-asserted-by":"crossref","first-page":"5241","DOI":"10.1016\/S0021-9258(17)37680-9","article-title":"A specific inhibitor of phosphatidylinositol 3-kinase, 2-(4-morpholinyl)-8-phenyl-4H-1-benzopyran-4-one (LY294002)","volume":"269","author":"Vlahos","year":"1994","journal-title":"J. Biol. Chem"},{"key":"2023020206012464100_btx069-B58","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1002\/(SICI)1097-0320(19980101)31:1<20::AID-CYTO3>3.0.CO;2-N","article-title":"Quantification of apoptotic and lytic cell death by video microscopy in combination with artificial neural networks","volume":"31","author":"Weisser","year":"1998","journal-title":"Cytometry"},{"key":"2023020206012464100_btx069-B59","article-title":"Understanding neural networks through deep visualization","author":"Yosinski","year":"2015","journal-title":"Proc. ICML Deep Learning Workshop"},{"key":"2023020206012464100_btx069-B60","first-page":"818","article-title":"Visualizing and understanding convolutional networks","author":"Zeiler","year":"2014","journal-title":"Proc. European Conference on Computer Vision"},{"key":"2023020206012464100_btx069-B61","doi-asserted-by":"crossref","first-page":"931","DOI":"10.1038\/nmeth.3547","article-title":"Predicting effects of noncoding variants with deep learning-based sequence model","volume":"12","author":"Zhou","year":"2015","journal-title":"Nat. Methods"},{"key":"2023020206012464100_btx069-B62","doi-asserted-by":"crossref","first-page":"291.","DOI":"10.1186\/1471-2105-14-291","article-title":"BIOCAT: a pattern recognition platform for customizable biological image classification and annotation","volume":"14","author":"Zhou","year":"2013","journal-title":"BMC Bioinformatics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/33\/13\/2010\/49040404\/bioinformatics_33_13_2010.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/33\/13\/2010\/49040404\/bioinformatics_33_13_2010.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,2]],"date-time":"2023-02-02T06:02:20Z","timestamp":1675317740000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/33\/13\/2010\/2997285"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,2,15]]},"references-count":62,"journal-issue":{"issue":"13","published-print":{"date-parts":[[2017,7,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btx069","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2017,7,1]]},"published":{"date-parts":[[2017,2,15]]}}}