{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,5]],"date-time":"2026-09-05T15:35:53Z","timestamp":1788622553021,"version":"build-2803163510"},"reference-count":49,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2023,9,22]],"date-time":"2023-09-22T00:00:00Z","timestamp":1695340800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62122025"],"award-info":[{"award-number":["62122025"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U22A2037"],"award-info":[{"award-number":["U22A2037"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62250028"],"award-info":[{"award-number":["62250028"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62202153"],"award-info":[{"award-number":["62202153"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Hunan Provincial Natural Science Outstanding Youth Fund","award":["2021JJ10020"],"award-info":[{"award-number":["2021JJ10020"]}]},{"name":"Postgraduate Scientific Research Innovation Project of Hunan Province","award":["CX20220380"],"award-info":[{"award-number":["CX20220380"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,9,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Current methods of molecular image-based drug discovery face two major challenges: (1) work effectively in absence of labels, and (2) capture chemical structure from implicitly encoded images. Given that chemical structures are explicitly encoded by molecular graphs (such as nitrogen, benzene rings and double bonds), we leverage self-supervised contrastive learning to transfer chemical knowledge from graphs to images. Specifically, we propose a novel Contrastive Graph-Image Pre-training (CGIP) framework for molecular representation learning, which learns explicit information in graphs and implicit information in images from large-scale unlabeled molecules via carefully designed intra- and inter-modal contrastive learning. We evaluate the performance of CGIP on multiple experimental settings (molecular property prediction, cross-modal retrieval and distribution similarity), and the results show that CGIP can achieve state-of-the-art performance on all 12 benchmark datasets and demonstrate that CGIP transfers chemical knowledge in graphs to molecular images, enabling image encoder to perceive chemical structures in images. We hope this simple and effective framework will inspire people to think about the value of image for molecular representation learning.<\/jats:p>","DOI":"10.1093\/bib\/bbad404","type":"journal-article","created":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T13:24:29Z","timestamp":1700227469000},"source":"Crossref","is-referenced-by-count":33,"title":["Chemical structure-aware molecular image representation learning"],"prefix":"10.1093","volume":"24","author":[{"given":"Hongxin","family":"Xiang","sequence":"first","affiliation":[{"name":"School of Information Science and Engineering, Hunan University , Hunan, 410082 , China"},{"name":"Department of AIDD, Shanghai Yuyao Biotechnology Co., Ltd. , Shanghai, 201109 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuting","family":"Jin","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Wuhan University of Science and Technology , Wuhan, 430065 , China"},{"name":"School of Informatics, Xiamen University , Xiamen, 361005 , China"},{"name":"Department of AIDD, Shanghai Yuyao Biotechnology Co., Ltd. , Shanghai, 201109 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiangrong","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Informatics, Xiamen University , Xiamen, 361005 , China"},{"name":"Zhejiang Lab , Hangzhou, 310013 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiangxiang","family":"Zeng","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Hunan University , Hunan, 410082 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Li","family":"Zeng","sequence":"additional","affiliation":[{"name":"Department of AIDD, Shanghai Yuyao Biotechnology Co., Ltd. , Shanghai, 201109 , China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2023,11,16]]},"reference":[{"issue":"9","key":"2023111707033659900_ref1","doi-asserted-by":"crossref","first-page":"899","DOI":"10.1016\/j.scib.2022.01.029","article-title":"X-Mol: large-scale pre-training for molecular understanding and diverse molecular analysis","volume":"67","author":"Xue","year":"2022","journal-title":"Science Bulletin"},{"issue":"2","key":"2023111707033659900_ref2","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1038\/s42256-021-00438-4","article-title":"Geometry-enhanced molecular representation learning for property prediction","volume":"4","author":"Fang","year":"2022","journal-title":"Nat Mach Intell"},{"issue":"5","key":"2023111707033659900_ref3","doi-asserted-by":"crossref","first-page":"742","DOI":"10.1021\/ci100050t","article-title":"Extended-connectivity fingerprints","volume":"50","author":"Rogers","year":"2010","journal-title":"J Chem Inf Model"},{"issue":"5","key":"2023111707033659900_ref4","doi-asserted-by":"crossref","first-page":"996","DOI":"10.1021\/ci200028n","article-title":"Classification of cytochrome p450 inhibitors and noninhibitors using combined classifiers","volume":"51","author":"Cheng","year":"2011","journal-title":"J Chem Inf Model"},{"key":"2023111707033659900_ref5","first-page":"4116","volume-title":"International Conference on Machine Learning","author":"Hassani","year":"2020"},{"key":"2023111707033659900_ref6","article-title":"Deep molecular representation learning via fusing physical and chemical information","volume":"34","author":"Yang","year":"2021","journal-title":"Adv Neural Inf Process Syst"},{"key":"2023111707033659900_ref7","article-title":"Smiles transformer: pre-trained molecular fingerprint for low data drug discovery","author":"Honda","year":"2019","journal-title":"arXiv preprint arXiv:191104738"},{"key":"2023111707033659900_ref8","first-page":"28,877","article-title":"Do transformers really perform badly for graph representation?","volume":"34","author":"Ying","year":"2021","journal-title":"Adv Neural Inf Process Syst"},{"issue":"3","key":"2023111707033659900_ref9","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1038\/s42256-022-00447-x","article-title":"Molecular contrastive learning of representations via graph neural networks","volume":"4","author":"Wang","year":"2022","journal-title":"Nat Mach Intell"},{"key":"2023111707033659900_ref10","first-page":"1263","volume-title":"International conference on machine learning","author":"Gilmer","year":"2017"},{"key":"2023111707033659900_ref11","first-page":"1052","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Lu","year":"2019"},{"issue":"3","key":"2023111707033659900_ref12","doi-asserted-by":"crossref","first-page":"816","DOI":"10.1039\/D1SC05180F","article-title":"Mgraphdta: deep multiscale graph neural network for explainable drug\u2013target binding affinity prediction","volume":"13","author":"Yang","year":"2022","journal-title":"Chem Sci"},{"key":"2023111707033659900_ref13","first-page":"3438","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Chen","year":"2020"},{"key":"2023111707033659900_ref14","article-title":"Dual-view molecule pre-training","author":"Zhu","year":"2021","journal-title":"arXiv preprint arXiv:210610234"},{"issue":"11","key":"2023111707033659900_ref15","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc IEEE"},{"key":"2023111707033659900_ref16","doi-asserted-by":"crossref","first-page":"1200","DOI":"10.1109\/TMM.2020.2993952","article-title":"Attribute-guided feature learning for few-shot image recognition","volume":"23","author":"Zhu","year":"2020","journal-title":"IEEE Trans Multimed"},{"key":"2023111707033659900_ref17","first-page":"8748","volume-title":"International Conference on Machine Learning","author":"Radford","year":"2021"},{"key":"2023111707033659900_ref18","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1145\/3307339.3342186","volume-title":"Proceedings of the 10th ACM international conference on bioinformatics, computational biology and health informatics","author":"Wang","year":"2019"},{"key":"2023111707033659900_ref19","article-title":"Strategies for pre-training graph neural networks","author":"Hu","year":"2019","journal-title":"arXiv preprint arXiv:190512265"},{"key":"2023111707033659900_ref20","volume-title":"NeurIPS","author":"Rong","year":"2020"},{"key":"2023111707033659900_ref21","doi-asserted-by":"crossref","DOI":"10.1093\/bib\/bbab109","article-title":"An effective self-supervised framework for learning expressive molecular global representations to drug discovery","volume":"22","author":"Li","year":"2021","journal-title":"Brief Bioinform"},{"key":"2023111707033659900_ref22","doi-asserted-by":"crossref","first-page":"857","DOI":"10.1145\/3534678.3539426","volume-title":"Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","author":"Li","year":"2022"},{"issue":"7","key":"2023111707033659900_ref23","doi-asserted-by":"crossref","first-page":"1757","DOI":"10.1021\/ci3001277","article-title":"Zinc: a free tool to discover chemistry for biology","volume":"52","author":"Irwin","year":"2012","journal-title":"J Chem Inf Model"},{"issue":"12","key":"2023111707033659900_ref24","doi-asserted-by":"crossref","first-page":"1256","DOI":"10.1038\/s42256-022-00580-7","article-title":"Large-scale chemical language representations capture molecular structure and properties","volume":"4","author":"Ross","year":"2022","journal-title":"Nat Mach Intell"},{"key":"2023111707033659900_ref25","volume-title":"International Conference on Learning Representations","author":"Sun","year":"2020"},{"key":"2023111707033659900_ref26","first-page":"5812","article-title":"Graph contrastive learning with augmentations","volume":"33","author":"You","year":"2020","journal-title":"Adv Neural Inf Process Syst"},{"key":"2023111707033659900_ref27","doi-asserted-by":"crossref","first-page":"3968","DOI":"10.1609\/aaai.v36i4.20313","article-title":"Molecular contrastive learning with chemical element knowledge graph","volume":"36","author":"Fang","year":"2022","journal-title":"AAAI"},{"issue":"17","key":"2023111707033659900_ref28","doi-asserted-by":"crossref","first-page":"3948","DOI":"10.1021\/acs.jcim.2c00521","article-title":"Smiclr: contrastive learning on multiple molecular representations for semisupervised and unsupervised representation learning","volume":"62","author":"Pinheiro","year":"2022","journal-title":"J Chem Inf Model"},{"key":"2023111707033659900_ref29","article-title":"Motif-based graph self-supervised learning for molecular property prediction","volume":"34","author":"Zhang","year":"2021","journal-title":"Adv Neural Inf Process Syst"},{"key":"2023111707033659900_ref30","first-page":"15,980","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Subramonian","year":"2021"},{"key":"2023111707033659900_ref31","first-page":"3441","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics","author":"Guo","year":"2022"},{"key":"2023111707033659900_ref32","first-page":"20,479","volume-title":"International Conference on Machine Learning","author":"St\u00e4rk","year":"2022"},{"key":"2023111707033659900_ref33","volume-title":"ICLR 2022 Workshop on Geometrical and Topological Representation Learning","author":"Liu"},{"key":"2023111707033659900_ref34","first-page":"3968","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Fang","year":"2022"},{"key":"2023111707033659900_ref35","article-title":"Chemception: a deep neural network with minimal chemistry knowledge matches the performance of expert-developed qsar\/qspr models","author":"Goh","year":"2017"},{"issue":"8","key":"2023111707033659900_ref36","doi-asserted-by":"crossref","first-page":"1533","DOI":"10.1021\/acs.jcim.8b00338","article-title":"Toxic colors: the use of deep learning for predicting toxicity of compounds merely from their graphic images","volume":"58","author":"Fernandez","year":"2018","journal-title":"J Chem Inf Model"},{"key":"2023111707033659900_ref37","volume-title":"Densely connected convolutional networks","author":"Huang","year":"2016"},{"key":"2023111707033659900_ref38","doi-asserted-by":"crossref","first-page":"127,998","DOI":"10.1016\/j.cej.2020.127998","article-title":"Molecular image-convolutional neural network (cnn) assisted qsar models for predicting contaminant reactivity toward oh radicals: transfer learning, data augmentation and model interpretation","volume":"408","author":"Zhong","year":"2021","journal-title":"Chem Eng J"},{"key":"2023111707033659900_ref39","first-page":"1","article-title":"Accurate prediction of molecular properties and drug targets using a self-supervised image representation learning framework","author":"Zeng","year":"2022","journal-title":"Nat Mach Intell"},{"issue":"D1","key":"2023111707033659900_ref40","doi-asserted-by":"crossref","first-page":"D1102","DOI":"10.1093\/nar\/gky1033","article-title":"Pubchem 2019 update: improved access to chemical data","volume":"47","author":"Kim","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023111707033659900_ref41","volume-title":"Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling","author":"Landrum","year":"2013"},{"key":"2023111707033659900_ref42","article-title":"Open graph benchmark: datasets for machine learning on graphs","author":"Hu","year":"2020","journal-title":"arXiv preprint arXiv:200500687"},{"key":"2023111707033659900_ref43","first-page":"770","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"He","year":"2016"},{"key":"2023111707033659900_ref44","article-title":"Deepgcns: making gcns go as deep as cnns","author":"Li","year":"2021","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2023111707033659900_ref45","article-title":"Inductive representation learning on large graphs","volume":"30","author":"Hamilton","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"issue":"3","key":"2023111707033659900_ref46","first-page":"4","article-title":"Deep graph infomax","volume":"2","author":"Velickovic","year":"2019","journal-title":"ICLR (Poster)"},{"key":"2023111707033659900_ref47","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1109\/TPAMI.1979.4766909","article-title":"A cluster separation measure","volume":"2","author":"Davies","year":"1979","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2023111707033659900_ref48","first-page":"618","volume-title":"Proceedings of the IEEE international conference on computer vision","author":"Selvaraju","year":"2017"},{"key":"2023111707033659900_ref49","first-page":"1597","volume-title":"International conference on machine learning","author":"Chen","year":"2020"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/24\/6\/bbad404\/53471469\/bbad404.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/24\/6\/bbad404\/53471469\/bbad404.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T13:24:59Z","timestamp":1700227499000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbad404\/7424448"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,22]]},"references-count":49,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,9,22]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbad404","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,11,1]]},"published":{"date-parts":[[2023,9,22]]},"article-number":"bbad404"}}