{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T16:14:59Z","timestamp":1786119299131,"version":"build-2736575974"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1014483","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T00:00:00Z","timestamp":1785974400000}}],"reference-count":42,"publisher":"Public Library of Science (PLoS)","issue":"7","license":[{"start":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T00:00:00Z","timestamp":1785283200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Recent advances in self-supervised deep learning have improved our ability to quantify cellular morphological changes in high-throughput microscopy screens, a process known as morphological profiling. However, most current methods only learn from images, despite many screens being inherently multimodal, as they involve both a chemical or genetic perturbation as well as an image-based readout. We hypothesized that incorporating chemical compound structures during self-supervised pre-training could improve learned representations of images from high-throughput microscopy screens. We introduce a representation learning framework, MICON (Molecular-Image Contrastive Learning), that models chemical compounds as treatments that induce transformations of cell phenotypes. MICON significantly outperforms classical hand-crafted features such as CellProfiler and existing deep-learning-based representation learning methods in challenging evaluation settings where models must identify reproducible effects of drugs across independent replicates and data-generating centers. We demonstrate that incorporating chemical compound information into the learning process provides small, but consistent improvements in performance and that modeling compounds specifically as treatments outperforms approaches that directly align images and compounds in a single representation space. Our findings point to a new direction for representation learning in morphological profiling, suggesting that methods should explicitly account for the multimodal nature of microscopy screening data.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1014483","type":"journal-article","created":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T17:45:11Z","timestamp":1785347111000},"page":"e1014483","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Integrating chemical structures as treatments improves representations of microscopy images for morphological profiling"],"prefix":"10.1371","volume":"22","author":[{"given":"Yemin","family":"Yu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emre","family":"Hayir","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Neil","family":"Tenenholtz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1102-0387","authenticated-orcid":true,"given":"Lester","family":"Mackey","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ying","family":"Wei","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Alvarez-Melis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ava P.","family":"Amini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9568-3155","authenticated-orcid":true,"given":"Alex X.","family":"Lu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,7,29]]},"reference":[{"issue":"8","key":"pcbi.1014483.ref001","doi-asserted-by":"crossref","first-page":"1348","DOI":"10.1016\/j.drudis.2020.06.001","article-title":"Image-based high-content screening in drug discovery","volume":"25","author":"S Lin","year":"2020","journal-title":"Drug Discov Today"},{"issue":"3","key":"pcbi.1014483.ref002","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1016\/j.chembiol.2021.02.012","article-title":"Morphological profiling of small molecules","volume":"28","author":"S Ziegler","year":"2021","journal-title":"Cell Chem Biol"},{"issue":"9","key":"pcbi.1014483.ref003","doi-asserted-by":"crossref","first-page":"1757","DOI":"10.1038\/nprot.2016.105","article-title":"Cell Painting, a high-content image-based assay for morphological profiling using multiplexed fluorescent dyes","volume":"11","author":"M-A Bray","year":"2016","journal-title":"Nat Protoc"},{"issue":"9","key":"pcbi.1014483.ref004","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1038\/nmeth.4397","article-title":"Data-analysis strategies for image-based cell profiling","volume":"14","author":"JC Caicedo","year":"2017","journal-title":"Nat Methods"},{"key":"pcbi.1014483.ref005","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.cbpa.2021.04.001","article-title":"Image-based cell phenotyping with deep learning","volume":"65","author":"A Pratapa","year":"2021","journal-title":"Curr Opin Chem Biol"},{"issue":"10","key":"pcbi.1014483.ref006","doi-asserted-by":"crossref","DOI":"10.1186\/gb-2006-7-10-r100","article-title":"CellProfiler: image analysis software for identifying and quantifying cell phenotypes","volume":"7","author":"AE Carpenter","year":"2006","journal-title":"Genome Biol"},{"issue":"4","key":"pcbi.1014483.ref007","doi-asserted-by":"crossref","DOI":"10.1093\/bib\/bbae284","article-title":"Morphological profiling for drug discovery in the era of deep learning","volume":"25","author":"Q Tang","year":"2024","journal-title":"Brief Bioinform"},{"issue":"4","key":"pcbi.1014483.ref008","doi-asserted-by":"crossref","first-page":"2761","DOI":"10.1007\/s11831-023-09884-2","article-title":"Self-supervised Learning: A Succinct Review","volume":"30","author":"V Rani","year":"2023","journal-title":"Arch Comput Methods Eng"},{"key":"pcbi.1014483.ref009","doi-asserted-by":"crossref","unstructured":"Kim V, Adaloglou N, Osterland M, Morelli FM, Halawa M, K\u00f6nig T, et al. Self-supervision advances morphological profiling by unlocking powerful image representations. BioRxiv. 2023;:2023\u201304.","DOI":"10.1101\/2023.04.28.538691"},{"key":"pcbi.1014483.ref010","unstructured":"Chen T, Kornblith S, Norouzi M, Hinton G. A simple framework for contrastive learning of visual representations. In: International conference on machine learning, 2020. 1597\u2013607."},{"key":"pcbi.1014483.ref011","doi-asserted-by":"crossref","unstructured":"Perakis A, Gorji A, Jain S, Chaitanya K, Rizza S, Konukoglu E. Contrastive learning of single-cell phenotypic representations for treatment classification. In: Machine Learning in Medical Imaging: 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings 12, 2021. 565\u201375.","DOI":"10.1007\/978-3-030-87589-3_58"},{"key":"pcbi.1014483.ref012","first-page":"74","article-title":"Incorporating knowledge of plates in batch normalization improves generalization of deep learning for microscopy images","volume-title":"Machine Learning in Computational Biology","author":"A Lin","year":"2022"},{"key":"pcbi.1014483.ref013","doi-asserted-by":"crossref","unstructured":"He K, Chen X, Xie S, Li Y, Dollar P, Girshick R. Masked Autoencoders Are Scalable Vision Learners. In: 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 15979\u201388. https:\/\/doi.org\/10.1109\/cvpr52688.2022.01553","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"pcbi.1014483.ref014","doi-asserted-by":"crossref","unstructured":"Kraus O, Kenyon-Dean K, Saberian S, Fallah M, McLean P, Leung J, et al. Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology. In: 2024 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 11757\u201368. https:\/\/doi.org\/10.1109\/cvpr52733.2024.01117","DOI":"10.1109\/CVPR52733.2024.01117"},{"key":"pcbi.1014483.ref015","doi-asserted-by":"crossref","unstructured":"Caron M, Touvron H, Misra I, Jegou H, Mairal J, Bojanowski P, et al. Emerging Properties in Self-Supervised Vision Transformers. In: 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), 2021. 9630\u201340. https:\/\/doi.org\/10.1109\/iccv48922.2021.00951","DOI":"10.1109\/ICCV48922.2021.00951"},{"key":"pcbi.1014483.ref016","doi-asserted-by":"crossref","unstructured":"Doron M, Moutakanni T, Chen ZS, Moshkov N, Caron M, Touvron H, et al. Unbiased single-cell morphology with self-supervised vision transformers. bioRxiv. 2023.","DOI":"10.1101\/2023.06.16.545359"},{"key":"pcbi.1014483.ref017","first-page":"9309","article-title":"Weakly Supervised Learning of Single-Cell Feature Embeddings","volume":"2018","author":"JC Caicedo","year":"2018","journal-title":"Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit"},{"issue":"1","key":"pcbi.1014483.ref018","doi-asserted-by":"crossref","first-page":"7339","DOI":"10.1038\/s41467-023-42328-w","article-title":"CLOOME: contrastive learning unlocks bioimaging databases for queries with chemical structures","volume":"14","author":"A Sanchez-Fernandez","year":"2023","journal-title":"Nat Commun"},{"key":"pcbi.1014483.ref019","doi-asserted-by":"crossref","unstructured":"Fradkin P, Azadi P, Suri K, Wenkel F, Bashashati A, Sypetkowski M. How Molecules Impact Cells: Unlocking Contrastive PhenoMolecular Retrieval. In: 2024. https:\/\/doi.org\/arXiv:240908302","DOI":"10.52202\/079017-3513"},{"key":"pcbi.1014483.ref020","unstructured":"Radford A, Kim JW, Hallacy C, Ramesh A, Goh G, Agarwal S. Learning transferable visual models from natural language supervision. In: International conference on machine learning, 2021. 8748\u201363."},{"issue":"1","key":"pcbi.1014483.ref021","doi-asserted-by":"crossref","first-page":"6516","DOI":"10.1038\/s41467-024-50613-5","article-title":"Evaluating batch correction methods for image-based cell profiling","volume":"15","author":"J Arevalo","year":"2024","journal-title":"Nat Commun"},{"key":"pcbi.1014483.ref022","doi-asserted-by":"crossref","unstructured":"Yao H, Hanslovsky P, Huetter J-C, Hoeckendorf B, Richmond D. Weakly Supervised Set-Consistency Learning Improves Morphological Profiling of Single-Cell Images. In: 2024 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2024. 6978\u201387. https:\/\/doi.org\/10.1109\/cvprw63382.2024.00691","DOI":"10.1109\/CVPRW63382.2024.00691"},{"key":"pcbi.1014483.ref023","unstructured":"Cross-Zamirski JO, Williams G, Mouchet E, Sch\u00f6nlieb CB, Turkki R, Wang Y. Self-supervised learning of phenotypic representations from cell images with weak labels. 2022. https:\/\/arxiv.org\/abs\/2209.07819"},{"issue":"1","key":"pcbi.1014483.ref024","doi-asserted-by":"crossref","first-page":"1594","DOI":"10.1038\/s41467-024-45999-1","article-title":"Learning representations for image-based profiling of perturbations","volume":"15","author":"N Moshkov","year":"2024","journal-title":"Nat Commun"},{"key":"pcbi.1014483.ref025","doi-asserted-by":"crossref","unstructured":"Chandrasekaran SN, Ackerman J, Alix E, Ando DM, Arevalo J, Bennion M. JUMP Cell Painting dataset: morphological impact of 136,000 chemical and genetic perturbations. bioRxiv. 2023;:2023\u201303.","DOI":"10.1101\/2023.03.23.534023"},{"key":"pcbi.1014483.ref026","doi-asserted-by":"crossref","unstructured":"Chen ZS, Pham C, Wang S, Doron M, Moshkov N, Plummer B, et al. CHAMMI: A benchmark for channel-adaptive models in microscopy imaging. In: Advances in Neural Information Processing Systems 36, 2023. 19700\u201313. https:\/\/doi.org\/10.52202\/075280-0865","DOI":"10.52202\/075280-0865"},{"issue":"10","key":"pcbi.1014483.ref027","doi-asserted-by":"crossref","first-page":"1321","DOI":"10.1177\/1087057113503553","article-title":"Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment","volume":"18","author":"V Ljosa","year":"2013","journal-title":"J Biomol Screen"},{"key":"pcbi.1014483.ref028","doi-asserted-by":"crossref","unstructured":"Ando DM, McLean CY, Berndl M. Improving phenotypic measurements in high-content imaging screens. BioRxiv. 2017;:161422.","DOI":"10.1101\/161422"},{"issue":"23","key":"pcbi.1014483.ref029","doi-asserted-by":"crossref","first-page":"4548","DOI":"10.1093\/bioinformatics\/btab497","article-title":"Fully unsupervised deep mode of action learning for phenotyping high-content cellular images","volume":"37","author":"R Janssens","year":"2021","journal-title":"Bioinformatics"},{"issue":"7","key":"pcbi.1014483.ref030","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pbio.2005970","article-title":"CellProfiler 3.0: Next-generation image processing for biology","volume":"16","author":"C McQuin","year":"2018","journal-title":"PLoS Biology"},{"issue":"4","key":"pcbi.1014483.ref031","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1038\/nm.4306","article-title":"The Drug Repurposing Hub: a next-generation drug library and information resource","volume":"23","author":"SM Corsello","year":"2017","journal-title":"Nat Med"},{"issue":"3","key":"pcbi.1014483.ref032","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1111\/jmi.12178","article-title":"Pipeline for illumination correction of images for high-throughput microscopy","volume":"256","author":"S Singh","year":"2014","journal-title":"J Microsc"},{"issue":"11","key":"pcbi.1014483.ref033","article-title":"Morphology and gene expression profiling provide complementary information for mapping cell state","volume":"13","author":"GP Way","year":"2022","journal-title":"Cell Syst"},{"key":"pcbi.1014483.ref034","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 770\u20138. https:\/\/doi.org\/10.1109\/cvpr.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"issue":"12","key":"pcbi.1014483.ref035","doi-asserted-by":"crossref","DOI":"10.1093\/bioinformatics\/btw252","article-title":"Classifying and segmenting microscopy images with deep multiple instance learning","volume":"32","author":"OZ Kraus","year":"2016","journal-title":"Bioinformatics"},{"key":"pcbi.1014483.ref036","doi-asserted-by":"crossref","unstructured":"Pawlowski N, Caicedo JC, Singh S, Carpenter AE, Storkey A. Automating morphological profiling with generic deep convolutional networks. BioRxiv. 2016;:085118.","DOI":"10.1101\/085118"},{"issue":"13","key":"pcbi.1014483.ref037","doi-asserted-by":"crossref","first-page":"2010","DOI":"10.1093\/bioinformatics\/btx069","article-title":"A multi-scale convolutional neural network for phenotyping high-content cellular images","volume":"33","author":"WJ Godinez","year":"2017","journal-title":"Bioinformatics"},{"issue":"5","key":"pcbi.1014483.ref038","doi-asserted-by":"crossref","first-page":"742","DOI":"10.1021\/ci100050t","article-title":"Extended-connectivity fingerprints","volume":"50","author":"D Rogers","year":"2010","journal-title":"J Chem Inf Model"},{"key":"pcbi.1014483.ref039","doi-asserted-by":"crossref","unstructured":"Zhou G, Gao Z, Ding Q, Zheng H, Xu H, Wei Z. Uni-mol: A universal 3d molecular representation learning framework. ChemRxiv. 2023.","DOI":"10.26434\/chemrxiv-2022-jjm0j-v4"},{"key":"pcbi.1014483.ref040","unstructured":"Kingma DP. Adam: A method for stochastic optimization. 2014. https:\/\/arxiv.org\/abs\/1412.6980"},{"key":"pcbi.1014483.ref041","unstructured":"Sim\u00e9oni O, Vo HV, Seitzer M, Baldassarre F, Oquab M, Jose C. Dinov3. In: 2025. https:\/\/doi.org\/arXiv:250810104"},{"key":"pcbi.1014483.ref042","doi-asserted-by":"crossref","unstructured":"Palma A, Theis FJ, Lotfollahi M. Predicting cell morphological responses to perturbations using generative modeling. bioRxiv. 2023;:2023\u201307.","DOI":"10.1101\/2023.07.17.549216"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1014483","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T00:00:00Z","timestamp":1785974400000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1014483","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T17:51:16Z","timestamp":1786038676000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1014483"}},"subtitle":[],"editor":[{"given":"Virginie","family":"Uhlmann","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,7,29]]},"references-count":42,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,7,29]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1014483","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,29]]}}}