{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,19]],"date-time":"2026-07-19T04:10:01Z","timestamp":1784434201063,"version":"3.55.0"},"reference-count":69,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2021,5,3]],"date-time":"2021-05-03T00:00:00Z","timestamp":1620000000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"Department of Education Key Innovation Research"},{"name":"Institute Guoqiang at Tsinghua University"},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61836004"],"award-info":[{"award-number":["61836004"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Beijing Brain Science Special Project","award":["Z181100001518006"],"award-info":[{"award-number":["Z181100001518006"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,11,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>How to produce expressive molecular representations is a fundamental challenge in artificial intelligence-driven drug discovery. Graph neural network (GNN) has emerged as a powerful technique for modeling molecular data. However, previous supervised approaches usually suffer from the scarcity of labeled data and poor generalization capability. Here, we propose a novel molecular pre-training graph-based deep learning framework, named MPG, that learns molecular representations from large-scale unlabeled molecules. In MPG, we proposed a powerful GNN for modelling molecular graph named MolGNet, and designed an effective self-supervised strategy for pre-training the model at both the node and graph-level. After pre-training on 11 million unlabeled molecules, we revealed that MolGNet can capture valuable chemical insights to produce interpretable representation. The pre-trained MolGNet can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of drug discovery tasks, including molecular properties prediction, drug-drug interaction and drug-target interaction, on 14 benchmark datasets. The pre-trained MolGNet in MPG has the potential to become an advanced molecular encoder in the drug discovery pipeline.<\/jats:p>","DOI":"10.1093\/bib\/bbab109","type":"journal-article","created":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T12:09:59Z","timestamp":1615550999000},"source":"Crossref","is-referenced-by-count":134,"title":["An effective self-supervised framework for learning expressive molecular global representations to drug discovery"],"prefix":"10.1093","volume":"22","author":[{"given":"Pengyong","family":"Li","sequence":"first","affiliation":[{"name":"Department of Biomedical Engineering at Tsinghua University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jun","family":"Wang","sequence":"additional","affiliation":[{"name":"Ping An Healthcare Technology, Chaoyang, 100027 Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yixuan","family":"Qiao","sequence":"additional","affiliation":[{"name":"Operations Research and Cybernetics at Beijing University of Technology, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Chen","sequence":"additional","affiliation":[{"name":"Cybernetics at Beijing University of Technology, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yihuan","family":"Yu","sequence":"additional","affiliation":[{"name":"Beijing University of Biomedical Engineering, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaojun","family":"Yao","sequence":"additional","affiliation":[{"name":"Analytical Chemistry and Chemoinformatics at Lanzhou University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peng","family":"Gao","sequence":"additional","affiliation":[{"name":"Ping An Healthcare Technology, Chaoyang, 100027 Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guotong","family":"Xie","sequence":"additional","affiliation":[{"name":"Ping An Healthcare Technology, Chaoyang, 100027 Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sen","family":"Song","sequence":"additional","affiliation":[{"name":"Tsinghua Laboratory of Brain and Intelligence and Department of Biomedical Engineering, Tsinghua University, Haidian, 100084 Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,5,3]]},"reference":[{"key":"2021110814191458600_ref1","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2018"},{"key":"2021110814191458600_ref2","article-title":"Deep learning in drug target interaction prediction: Current and future perspective","author":"Abbasi","year":"2020","journal-title":"Curr Med Chem"},{"issue":"15","key":"2021110814191458600_ref3","first-page":"2887","article-title":"The properties of known drugs","volume":"39","author":"Bemis","year":"1996","journal-title":"1. molecular frameworks. J Med Chem"},{"key":"2021110814191458600_ref4","article-title":"Language models are few-shot learners","author":"Brown","year":"2020"},{"key":"2021110814191458600_ref5","first-page":"592","article-title":"Advancing drug discovery via artificial intelligence","volume-title":"Trends in pharmacological sciences","author":"Chan","year":"2019"},{"issue":"9","key":"2021110814191458600_ref6","doi-asserted-by":"crossref","first-page":"2208","DOI":"10.3390\/molecules23092208","article-title":"Machine learning for drug-target interaction prediction","volume":"23","author":"Chen","year":"2018","journal-title":"Molecules"},{"issue":"12","key":"2021110814191458600_ref7","doi-asserted-by":"crossref","first-page":"4977","DOI":"10.1021\/jm4004285","article-title":"Qsar modeling: where have you been? where are you going to?","volume":"57","author":"Artem Cherkasov","year":"2014","journal-title":"J Med Chem"},{"key":"2021110814191458600_ref8","article-title":"Chemberta: Large-scale self-supervised pretraining for molecular property prediction","author":"Chithrananda","year":"2020"},{"key":"2021110814191458600_ref9","author":"Cho","year":"2014"},{"key":"2021110814191458600_ref10","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1109\/TPAMI.1979.4766909","article-title":"A cluster separation measure","author":"Davies","year":"1979","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"4","key":"2021110814191458600_ref11","doi-asserted-by":"crossref","first-page":"748","DOI":"10.1016\/j.drudis.2020.03.003","article-title":"Machine learning models for drug\u2013target interactions: current knowledge and future directions","volume":"25","author":"Sofia","year":"2020","journal-title":"Drug Discov Today"},{"issue":"D1","key":"2021110814191458600_ref12","doi-asserted-by":"crossref","first-page":"D1100","DOI":"10.1093\/nar\/gkr777","article-title":"ChEMBL: a large-scale bioactivity database for drug discovery","volume":"40","author":"Gaulton","year":"2011","journal-title":"Nucleic Acids Res"},{"issue":"10","key":"2021110814191458600_ref13","doi-asserted-by":"crossref","first-page":"1784","DOI":"10.1016\/j.drudis.2018.06.016","article-title":"Neural network and deep-learning algorithms used in qsar studies: merits and drawbacks","volume":"23","author":"Ghasemi","year":"2018","journal-title":"Drug Discov Today"},{"key":"2021110814191458600_ref14","first-page":"1263","article-title":"Neural message passing for quantum chemistry","author":"Gilmer","year":"2017","journal-title":"International Conference on Machine Learning"},{"issue":"2","key":"2021110814191458600_ref15","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1021\/acscentsci.7b00572","article-title":"Automatic chemical design using a data-driven continuous representation of molecules","volume":"4","author":"G\u00f3mez-Bombarelli","year":"2018","journal-title":"ACS central science"},{"key":"2021110814191458600_ref16","article-title":"Inductive representation learning on large graphs","author":"William","year":"2017"},{"key":"2021110814191458600_ref17","first-page":"9729","article-title":"Momentum contrast for unsupervised visual representation learning","author":"He","year":"2020","journal-title":"In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"issue":"18","key":"2021110814191458600_ref18","doi-asserted-by":"crossref","first-page":"2333","DOI":"10.1093\/bioinformatics\/bts437","article-title":"Metabolite identification and molecular fingerprint prediction through machine learning","volume":"28","author":"Heinonen","year":"2012","journal-title":"Bioinformatics"},{"key":"2021110814191458600_ref19","article-title":"Gaussian error linear units (gelus)","author":"Hendrycks","year":"2016"},{"key":"2021110814191458600_ref20","article-title":"Drug discovery and development-E-book: technology in transition","author":"Hill","year":"2012","journal-title":"Elsevier Health Sciences"},{"key":"2021110814191458600_ref21","article-title":"Smiles transformer: Pre-trained molecular fingerprint for low data drug discovery","author":"Honda","year":"2019"},{"key":"2021110814191458600_ref22","article-title":"Strategies for pre-training graph neural networks","author":"Hu","year":"2019","journal-title":"In International Conference on Learning Representations"},{"issue":"9","key":"2021110814191458600_ref23","doi-asserted-by":"crossref","first-page":"4062","DOI":"10.1021\/acs.jmedchem.5b01746","article-title":"Computational exploration of molecular scaffolds in medicinal chemistry: Miniperspective","volume":"59","author":"Hu","year":"2016","journal-title":"J Med Chem"},{"key":"2021110814191458600_ref24","doi-asserted-by":"crossref","first-page":"702","DOI":"10.1609\/aaai.v34i01.5412","article-title":"Caster: Predicting drug interactions with chemical substructure representation","volume":"34","author":"Huang","year":"2020","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"issue":"1","key":"2021110814191458600_ref25","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1021\/acs.jcim.7b00616","article-title":"Mol2vec: unsupervised machine learning approach with chemical intuition","volume":"58","author":"Jaeger","year":"2018","journal-title":"J Chem Inf Model"},{"issue":"2","key":"2021110814191458600_ref26","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.cbi.2006.12.006","article-title":"Computer-aided drug discovery and development (caddd): in silico-chemico-biological approach","volume":"171","author":"Kapetanovic","year":"2008","journal-title":"Chem Biol Interact"},{"issue":"8","key":"2021110814191458600_ref27","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1007\/s10822-016-9938-8","article-title":"Molecular graph convolutions: moving beyond fingerprints","volume":"30","author":"Kearnes","year":"2016","journal-title":"J Comput Aided Mol Des"},{"key":"2021110814191458600_ref28","author":"Kipf","year":"2016"},{"key":"2021110814191458600_ref29","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2021110814191458600_ref30","first-page":"9267","article-title":"Deepgcns: Can gcns go as deep as cnns?","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Li","year":"2019"},{"key":"2021110814191458600_ref31","article-title":"Trimnet: learning molecular representation from triplet messages for biomedicine","author":"Li","year":"2020","journal-title":"Brief Bioinform"},{"key":"2021110814191458600_ref32","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1145\/3394486.3403076","article-title":"Towards deeper graph neural networks","volume-title":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Liu","year":"2020"},{"key":"2021110814191458600_ref33","first-page":"8466","article-title":"N-gram graph: Simple unsupervised representation for graphs, with applications to molecules","volume-title":"Advances in Neural Information Processing Systems","author":"Liu","year":"2019"},{"key":"2021110814191458600_ref34","article-title":"N-gram graph: Simple unsupervised representation for graphs, with applications to molecules","author":"Liu","year":"2018"},{"key":"2021110814191458600_ref35","article-title":"Self-supervised learning","volume-title":"Generative or contrastive","author":"Liu","year":"2020"},{"key":"2021110814191458600_ref36","first-page":"1051","article-title":"Molecular property prediction: A multilevel quantum interactions modeling perspective","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Lu","year":"2019"},{"key":"2021110814191458600_ref37","article-title":"BioSNAP Datasets: Stanford biomedical network dataset collection","author":"Zitnik","year":"2018"},{"key":"2021110814191458600_ref38","article-title":"Umap: Uniform manifold approximation and projection for dimension reduction","author":"McInnes","year":"2018"},{"key":"2021110814191458600_ref39","first-page":"2265","article-title":"Learning word embeddings efficiently with noise-contrastive estimation","author":"Mnih","year":"2013","journal-title":"In Advances in Neural Information Processing Systems"},{"issue":"9","key":"2021110814191458600_ref40","doi-asserted-by":"crossref","first-page":"1273","DOI":"10.1517\/17425255.2014.950222","article-title":"Drug\u2013target interaction prediction via chemogenomic space: learning-based methods","volume":"10","author":"Mousavian","year":"2014","journal-title":"Expert Opin Drug Metab Toxicol"},{"issue":"1","key":"2021110814191458600_ref41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-020-18671-7","article-title":"Transfer learning enables the molecular transformer to predict regio-and stereoselective reactions on carbohydrates","volume":"11","author":"Pesciullesi","year":"2020","journal-title":"Nat Commun"},{"key":"2021110814191458600_ref42","doi-asserted-by":"crossref","first-page":"1150","DOI":"10.1145\/3394486.3403168","article-title":"Graph contrastive coding for graph neural network pre-training","volume-title":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Qiu","year":"2020"},{"key":"2021110814191458600_ref43","author":"Ramsundar","year":"2015"},{"key":"2021110814191458600_ref44","doi-asserted-by":"crossref","DOI":"10.1201\/9780429131967","volume-title":"Drug-drug interactions","author":"David Rodrigues","year":"2019"},{"issue":"5","key":"2021110814191458600_ref45","doi-asserted-by":"crossref","first-page":"742","DOI":"10.1021\/ci100050t","article-title":"Extended-connectivity fingerprints","volume":"50","author":"Rogers","year":"2010","journal-title":"J Chem Inf Model"},{"key":"2021110814191458600_ref46","first-page":"33","article-title":"Self-supervised graph transformer on large-scale molecular data","author":"Yu","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"18","key":"2021110814191458600_ref47","doi-asserted-by":"crossref","first-page":"E4304","DOI":"10.1073\/pnas.1803294115","article-title":"Deep learning improves prediction of drug\u2013drug and drug\u2013food interactions","volume":"115","author":"Ryu","year":"2018","journal-title":"Proc Natl Acad Sci"},{"key":"2021110814191458600_ref48","first-page":"991","article-title":"Schnet: A continuous-filter convolutional neural network for modeling quantum interactions","volume-title":"Advances in Neural Information Processing Systems","author":"Sch\u00fctt","year":"2017"},{"issue":"25","key":"2021110814191458600_ref49","doi-asserted-by":"crossref","first-page":"6118","DOI":"10.1002\/chem.201604556","article-title":"Modelling chemical reasoning to predict and invent reactions","volume":"23","author":"Segler","year":"2017","journal-title":"Chem"},{"issue":"1","key":"2021110814191458600_ref50","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1124\/pr.112.007336","article-title":"Computational methods in drug discovery","volume":"66","author":"Sliwoski","year":"2014","journal-title":"Pharmacol Rev"},{"issue":"11","key":"2021110814191458600_ref51","doi-asserted-by":"crossref","first-page":"2324","DOI":"10.1021\/acs.jcim.5b00559","article-title":"Zinc 15-ligand discovery for everyone","volume":"55","author":"Sterling","year":"2015","journal-title":"J Chem Inf Model"},{"key":"2021110814191458600_ref52","article-title":"Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization","author":"Sun","year":"2019"},{"issue":"125","key":"2021110814191458600_ref53","first-page":"125ra31","article-title":"Data-driven prediction of drug effects and interactions","volume":"4","author":"Nicholas","year":"2012","journal-title":"Science Translational Medicine"},{"issue":"2","key":"2021110814191458600_ref54","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1093\/bioinformatics\/bty535","article-title":"Compound-protein interaction prediction with end-to-end learning of neural networks for graphs and sequences","volume":"35","author":"Tsubaki","year":"2019","journal-title":"Bioinformatics"},{"key":"2021110814191458600_ref55","first-page":"5998","article-title":"Attention is all you need","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani","year":"2017"},{"key":"2021110814191458600_ref56","article-title":"Graph Attention Networks","volume-title":"International Conference on Learning Representations","author":"Veli\u010dkovi\u0107","year":"2018"},{"key":"2021110814191458600_ref57","author":"Veli\u010dkovi\u0107","year":"2018"},{"issue":"9","key":"2021110814191458600_ref58","doi-asserted-by":"crossref","first-page":"2147","DOI":"10.1038\/nprot.2014.151","article-title":"Similarity-based modeling in large-scale prediction of drug-drug interactions","volume":"9","author":"Vilar","year":"2014","journal-title":"Nat Protoc"},{"key":"2021110814191458600_ref59","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1145\/3307339.3342186","article-title":"Smiles-bert: large scale unsupervised pre-training for molecular property prediction","volume-title":"Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics","author":"Wang","year":"2019"},{"issue":"1","key":"2021110814191458600_ref60","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1021\/ci00057a005","article-title":"Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules","volume":"28","author":"Weininger","year":"1988","journal-title":"J Chem Inf Comput Sci"},{"issue":"4","key":"2021110814191458600_ref61","doi-asserted-by":"crossref","first-page":"1401","DOI":"10.1021\/acs.jproteome.6b00618","article-title":"Deep-learning-based drug\u2013target interaction prediction","volume":"16","author":"Wen","year":"2017","journal-title":"J Proteome Res"},{"issue":"6","key":"2021110814191458600_ref62","doi-asserted-by":"crossref","first-page":"1692","DOI":"10.1039\/C8SC04175J","article-title":"Learning continuous and data-driven molecular descriptors by translating equivalent chemical representations","volume":"10","author":"Winter","year":"2019","journal-title":"Chem Sci"},{"issue":"2","key":"2021110814191458600_ref63","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1039\/C7SC02664A","article-title":"MoleculeNet: A benchmark for molecular machine learning","volume":"9","author":"Wu","year":"2018","journal-title":"Chem Sci"},{"issue":"2","key":"2021110814191458600_ref64","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1039\/C7SC02664A","article-title":"Moleculenet: a benchmark for molecular machine learning","volume":"9","author":"Wu","year":"2018","journal-title":"Chem Sci"},{"issue":"16","key":"2021110814191458600_ref65","doi-asserted-by":"crossref","first-page":"8749","DOI":"10.1021\/acs.jmedchem.9b00959","article-title":"Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism","volume":"63","author":"Xiong","year":"2019","journal-title":"J Med Chem"},{"key":"2021110814191458600_ref66","first-page":"285","article-title":"An unsupervised deep molecular embedding for drug discovery","volume-title":"Proceedings of the 8th ACM international conference on bioinformatics, computational biology, and health informatics","author":"Zheng","year":"2017"},{"issue":"5","key":"2021110814191458600_ref67","doi-asserted-by":"crossref","first-page":"363","DOI":"10.2174\/1386207003331454","article-title":"Molecular descriptors in chemoinformatics, computational combinatorial chemistry, and virtual screening","volume":"3","author":"Xue","year":"2000","journal-title":"Comb Chem High Throughput Screen"},{"issue":"8","key":"2021110814191458600_ref68","doi-asserted-by":"crossref","first-page":"3370","DOI":"10.1021\/acs.jcim.9b00237","article-title":"Analyzing learned molecular representations for property prediction","volume":"59","author":"Yang","year":"2019","journal-title":"J Chem Inf Model"},{"issue":"19","key":"2021110814191458600_ref69","first-page":"1","article-title":"Ddi-pulearn: a positive-unlabeled learning method for large-scale prediction of drug-drug interactions","volume":"20","author":"Zheng","year":"2019","journal-title":"BMC bioinformatics"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/22\/6\/bbab109\/41087438\/bbab109.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/22\/6\/bbab109\/41087438\/bbab109.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,11,8]],"date-time":"2021-11-08T14:21:11Z","timestamp":1636381271000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbab109\/6262238"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,3]]},"references-count":69,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,11,5]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbab109","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,11]]},"published":{"date-parts":[[2021,5,3]]},"article-number":"bbab109"}}