{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,26]],"date-time":"2026-10-26T23:58:54Z","timestamp":1793059134590,"version":"4.5.0"},"reference-count":122,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2025,11,11]],"date-time":"2025-11-11T00:00:00Z","timestamp":1762819200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62402390"],"award-info":[{"award-number":["62402390"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62433016"],"award-info":[{"award-number":["62433016"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006469","name":"FDCT","doi-asserted-by":"publisher","award":["62361166662"],"award-info":[{"award-number":["62361166662"]}],"id":[{"id":"10.13039\/501100006469","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2023YFC3503400"],"award-info":[{"award-number":["2023YFC3503400"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2022YFC3400400"],"award-info":[{"award-number":["2022YFC3400400"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","award":["D5000240148"],"award-info":[{"award-number":["D5000240148"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014718","name":"Innovative Research Group Project","doi-asserted-by":"publisher","award":["2024JJ1002"],"award-info":[{"award-number":["2024JJ1002"]}],"id":[{"id":"10.13039\/100014718","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Hunan Science and Technology Innovation Plan","award":["2025ZYJ003"],"award-info":[{"award-number":["2025ZYJ003"]}]},{"DOI":"10.13039\/501100019091","name":"Key Research and Development Program of Hunan Province of China","doi-asserted-by":"publisher","award":["2023GK2004"],"award-info":[{"award-number":["2023GK2004"]}],"id":[{"id":"10.13039\/501100019091","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100019091","name":"Key Research and Development Program of Hunan Province of China","doi-asserted-by":"publisher","award":["2023SK2059"],"award-info":[{"award-number":["2023SK2059"]}],"id":[{"id":"10.13039\/501100019091","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100019091","name":"Key Research and Development Program of Hunan Province of China","doi-asserted-by":"publisher","award":["2023SK2060"],"award-info":[{"award-number":["2023SK2060"]}],"id":[{"id":"10.13039\/501100019091","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012165","name":"Key Technologies Research and Development Program","doi-asserted-by":"publisher","award":["2023B1111030004"],"award-info":[{"award-number":["2023B1111030004"]}],"id":[{"id":"10.13039\/501100012165","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,1,10]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:p>With the great success of large language models in natural language processing, self-supervised pre-training models have emerged as an important technique in drug discovery. In particular, multimodal pre-training models have opened a new avenue for drug discovery. The experience and ideas from previous works can provide important reference points for further research in drug discovery. Therefore, this review summarizes the foundation of multimodal pre-training models and their progress in the field of drug discovery. We emphasize the adaptability between various modalities and network frameworks or pre-training tasks. At the same time, we summarize the difference and relevance between various modalities or pre-training models. Importantly, we identify two increasing trends that may serve as reference points for future research. Specifically, Transformers and graph neural networks are often integrated as encoders and then combined with multiple pre-training tasks to learn cross-scale molecular representation, thereby promoting the accuracy of drug discovery. In addition, molecular captions as brief biomedical text provide a bridge for collaboration between drug discovery and large language models. Finally, we discuss the challenges of multimodal pre-training models in drug discovery, and explore future opportunities.<\/jats:p>","DOI":"10.1093\/nsr\/nwaf495","type":"journal-article","created":{"date-parts":[[2025,11,10]],"date-time":"2025-11-10T12:29:32Z","timestamp":1762777772000},"source":"Crossref","is-referenced-by-count":24,"title":["Multimodal pre-training models of molecular representation for drug discovery"],"prefix":"10.1093","volume":"13","author":[{"given":"Xiaoqi","family":"Wang","sequence":"first","affiliation":[{"name":"School of Computer Science, Northwestern Polytechnical University , Xi\u2019an, 710129 ,","place":["China"]},{"name":"Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology , Northwestern Polytechnical University, Xi\u2019an, 710129 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chuanshi","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Northwestern Polytechnical University , Xi\u2019an, 710129 ,","place":["China"]},{"name":"Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology , Northwestern Polytechnical University, Xi\u2019an, 710129 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4647-2615","authenticated-orcid":false,"given":"Boya","family":"Ji","sequence":"additional","affiliation":[{"name":"College of Computer Science and Electronic Engineering, Hunan University , Changsha, 410082 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4432-4707","authenticated-orcid":false,"given":"Junwen","family":"Wang","sequence":"additional","affiliation":[{"name":"Division of Applied Oral Sciences & Community Dental Care, Faculty of Dentistry, the University of Hong Kong , Hong Kong 999077 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3323-3092","authenticated-orcid":false,"given":"Mingyue","family":"Zheng","sequence":"additional","affiliation":[{"name":"Drug Discovery and Design Center, State Key Laboratory of Drug Research, Shanghai Institute of Materia Medica, Chinese Academy of Sciences , Shanghai 201203 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lingyun","family":"Song","sequence":"additional","affiliation":[{"name":"School of Computer Science, Northwestern Polytechnical University , Xi\u2019an, 710129 ,","place":["China"]},{"name":"Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology , Northwestern Polytechnical University, Xi\u2019an, 710129 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shaoliang","family":"Peng","sequence":"additional","affiliation":[{"name":"College of Computer Science and Electronic Engineering, Hunan University , Changsha, 410082 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuequn","family":"Shang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Northwestern Polytechnical University , Xi\u2019an, 710129 ,","place":["China"]},{"name":"Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology , Northwestern Polytechnical University, Xi\u2019an, 710129 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,11,11]]},"reference":[{"key":"2026011311372154500_bib1","doi-asserted-by":"publisher","first-page":"nwae451","DOI":"10.1093\/nsr\/nwae451","article-title":"A deep learning framework for in silico screening of anticancer drugs at the single-cell level","volume":"12","author":"Zhang","year":"2025","journal-title":"Natl Sci Rev"},{"key":"2026011311372154500_bib2","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1038\/s41591-024-03434-4","article-title":"Artificial intelligence in drug development","volume":"31","author":"Zhang","year":"2025","journal-title":"Nat Med"},{"key":"2026011311372154500_bib3","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1016\/j.tips.2019.05.005","article-title":"Artificial intelligence for clinical trial design","volume":"40","author":"Harrer","year":"2019","journal-title":"Trends Pharmacol Sci"},{"key":"2026011311372154500_bib4","doi-asserted-by":"publisher","first-page":"314","DOI":"10.1038\/534314a","article-title":"New tricks for old drugs: faced with skyrocketing costs for developing new drugs, researchers are looking at ways to repurpose older ones\u2013and even some that failed in initial trials","volume":"534","author":"Nosengo","year":"2016","journal-title":"Nature"},{"key":"2026011311372154500_bib5","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1093\/nsr\/nww003","article-title":"Applying high-performance computing in drug discovery and molecular simulation","volume":"3","author":"Liu","year":"2016","journal-title":"Natl Sci Rev"},{"key":"2026011311372154500_bib6","doi-asserted-by":"publisher","first-page":"822","DOI":"10.1007\/s12539-025-00713-7","article-title":"A multi-modal drug target affinity prediction based on graph features and pre-trained sequence embeddings","volume":"17","author":"Tang","year":"2025","journal-title":"Interdiscip Sci Comput Life Sci"},{"key":"2026011311372154500_bib7","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","article-title":"Highly accurate protein structure prediction with alphafold","volume":"596","author":"Jumper","year":"2021","journal-title":"Nature"},{"key":"2026011311372154500_bib8","doi-asserted-by":"publisher","first-page":"15930","DOI":"10.1073\/pnas.1905068116","article-title":"Assembling multidomain protein structures through analogous global structural alignments","volume":"116","author":"Zhou","year":"2019","journal-title":"Proc Natl Acad Sci USA"},{"key":"2026011311372154500_bib9","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1038\/s42003-023-04605-8","article-title":"Protein structure and folding pathway prediction based on remote homologs recognition using pathreader","volume":"6","author":"Zhao","year":"2023","journal-title":"Commun Biol"},{"key":"2026011311372154500_bib10","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1186\/s13059-024-03291-x","article-title":"FoldPAthreader: predicting protein folding pathway using a novel folding force field model derived from known protein universe","volume":"25","author":"Zhao","year":"2024","journal-title":"Genome Biol"},{"key":"2026011311372154500_bib11","doi-asserted-by":"publisher","first-page":"1904","DOI":"10.1109\/TPAMI.2015.2389824","article-title":"Spatial pyramid pooling in deep convolutional networks for visual recognition","volume":"37","author":"He","year":"2015","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2026011311372154500_bib12","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun ACM"},{"key":"2026011311372154500_bib13","article-title":"Empirical evaluation of gated recurrent neural networks on sequence modeling","author":"Chung"},{"key":"2026011311372154500_bib14","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput"},{"key":"2026011311372154500_bib15","article-title":"Semi-supervised classification with graph convolutional networks","author":"Kipf"},{"key":"2026011311372154500_bib16","first-page":"1024","article-title":"Inductive representation learning on large graphs","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems","author":"Hamilton","year":"2017"},{"key":"2026011311372154500_bib17","article-title":"Graph attention networks","author":"Veli\u010dkovi\u0107"},{"key":"2026011311372154500_bib18","doi-asserted-by":"publisher","first-page":"998","DOI":"10.1007\/s12539-025-00714-6","article-title":"WDGBANDTI: a deep graph convolutional network-based bilinear attention network for drug-target interaction prediction with domain adaptation","volume":"17","author":"Wang","year":"2025","journal-title":"Interdiscip Sci Comput Life Sci"},{"key":"2026011311372154500_bib19","first-page":"6000","article-title":"Attention is all you need","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems","author":"Vaswani","year":"2017"},{"key":"2026011311372154500_bib20","doi-asserted-by":"publisher","first-page":"445","DOI":"10.1038\/s42256-023-00640-6","article-title":"Multitask joint strategies of self-supervised representation learning on biomedical networks for drug discovery","volume":"5","author":"Wang","year":"2023","journal-title":"Nat Mach Intell"},{"key":"2026011311372154500_bib21","first-page":"1877","article-title":"Language models are few-shot learners","volume-title":"Proceedings of the 34th International Conference on Neural Information Processing Systems","author":"Brown","year":"2020"},{"key":"2026011311372154500_bib22","first-page":"11324","article-title":"PaLM: scaling language modeling with pathways","volume":"24","author":"Chowdhery","year":"2023","journal-title":"J Mach Learn Res"},{"key":"2026011311372154500_bib23","doi-asserted-by":"publisher","first-page":"nwaf028","DOI":"10.1093\/nsr\/nwaf028","article-title":"Foundation models in bioinformatics","volume":"12","author":"Guo","year":"2025","journal-title":"Natl Sci Rev"},{"key":"2026011311372154500_bib24","first-page":"5879","article-title":"Graph self-supervised learning: A survey","volume":"35","author":"Liu","year":"2022","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"2026011311372154500_bib25","article-title":"KGARevion: an AI agent for knowledge-intensive biomedical QA","author":"Su","year":"24\u201328  2025"},{"key":"2026011311372154500_bib26","doi-asserted-by":"crossref","article-title":"UniCure: a foundation model for predicting personalized cancer therapy response","author":"Chen","DOI":"10.1101\/2025.06.14.658531"},{"key":"2026011311372154500_bib27","doi-asserted-by":"crossref","first-page":"6787","DOI":"10.24963\/ijcai.2023\/760","article-title":"A systematic survey of chemical pre-trained models","volume-title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","author":"Xia","year":"2023"},{"key":"2026011311372154500_bib28","first-page":"4171","article-title":"BERT: pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin","year":"2019"},{"key":"2026011311372154500_bib29","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1021\/ci00057a005","article-title":"SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules","volume":"28","author":"Weininger","year":"1988","journal-title":"J Chem Inf Comput Sci"},{"key":"2026011311372154500_bib30","doi-asserted-by":"crossref","first-page":"1256","DOI":"10.1038\/s42256-022-00580-7","article-title":"Large-scale chemical language representations capture molecular structure and properties","volume":"4","author":"Ross","year":"2022","journal-title":"Nat Mach Intell"},{"key":"2026011311372154500_bib31","article-title":"ChemBERTa: large-scale self-supervised pretraining for molecular property prediction","author":"Chithrananda"},{"key":"2026011311372154500_bib32","article-title":"Pre-training molecular graph representation with 3D geometry","author":"Liu"},{"key":"2026011311372154500_bib33","doi-asserted-by":"publisher","first-page":"1112","DOI":"10.1021\/acs.jcim.3c01468","article-title":"Molecular contrastive pretraining with collaborative featurizations","volume":"64","author":"Zhu","year":"2024","journal-title":"J Chem Inf Model"},{"key":"2026011311372154500_bib34","doi-asserted-by":"publisher","first-page":"542","DOI":"10.1038\/s42256-023-00654-0","article-title":"Knowledge graph-enhanced molecular contrastive learning with functional prompt","volume":"5","author":"Fang","year":"2023","journal-title":"Nat Mach Intell"},{"key":"2026011311372154500_bib35","doi-asserted-by":"publisher","first-page":"375","DOI":"10.18653\/v1\/2022.emnlp-main.26","article-title":"Translation between molecules and natural language","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Edwards","year":"2022"},{"key":"2026011311372154500_bib36","doi-asserted-by":"publisher","first-page":"1102","DOI":"10.18653\/v1\/2023.emnlp-main.70","article-title":"BioT5: enriching cross-modal integration in biology with chemical knowledge and natural language associations","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Pei","year":"2023"},{"key":"2026011311372154500_bib37","doi-asserted-by":"publisher","first-page":"862","DOI":"10.1038\/s41467-022-28494-3","article-title":"A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals","volume":"13","author":"Zeng","year":"2022","journal-title":"Nat Commun"},{"key":"2026011311372154500_bib38","doi-asserted-by":"crossref","first-page":"2921","DOI":"10.1145\/3442381.3449786","article-title":"Multi-view graph contrastive representation learning for drug-drug interaction prediction","volume-title":"Proceedings of the Web Conference 2021","author":"Wang","year":"2021"},{"key":"2026011311372154500_bib39","doi-asserted-by":"publisher","first-page":"5299","DOI":"10.1109\/TPAMI.2024.3429301","article-title":"Self-supervised multimodal learning: a survey","volume":"47","author":"Zong","year":"2024","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2026011311372154500_bib40","first-page":"1606","article-title":"MolXPT: wrapping molecules with text for generative pre-training","author":"Liu","year":"2023","journal-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics"},{"key":"2026011311372154500_bib41","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1186\/s13321-020-00456-1","article-title":"An open source chemical structure curation pipeline using RDKit","volume":"12","author":"Bento","year":"2020","journal-title":"J Cheminf"},{"key":"2026011311372154500_bib42","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1186\/1758-2946-3-33","article-title":"Open babel: an open chemical toolbox","volume":"3","author":"O\u2019Boyle","year":"2011","journal-title":"J Cheminf"},{"key":"2026011311372154500_bib43","doi-asserted-by":"publisher","first-page":"340","DOI":"10.1038\/s42256-023-00624-6","article-title":"Multimodal learning with graphs","volume":"5","author":"Ektefaie","year":"2023","journal-title":"Nat Mach Intell"},{"key":"2026011311372154500_bib44","doi-asserted-by":"publisher","first-page":"7363","DOI":"10.1021\/acs.jcim.3c01527","article-title":"A multimodal deep learning framework for predicting PPI-modulator interactions","volume":"63","author":"Sun","year":"2023","journal-title":"J Chem Inf Model"},{"key":"2026011311372154500_bib45","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1007\/s12539-025-00697-4","article-title":"MultiKD-DTA: enhancing drug-target affinity prediction through multiscale feature extraction","volume":"17","author":"Hu","year":"2025","journal-title":"Interdiscip Sci Comput Life Sci"},{"key":"2026011311372154500_bib46","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"2026011311372154500_bib47","doi-asserted-by":"publisher","first-page":"2531","DOI":"10.1039\/C9SC03414E","article-title":"Deepscreen: high performance drug\u2013target interaction prediction with convolutional neural networks using 2-D structural compound representations","volume":"11","author":"Rifaioglu","year":"2020","journal-title":"Chem Sci"},{"key":"2026011311372154500_bib48","doi-asserted-by":"publisher","first-page":"1533","DOI":"10.1021\/acs.jcim.8b00338","article-title":"Toxic colors: the use of deep learning for predicting toxicity of compounds merely from their graphic images","volume":"58","author":"Fernandez","year":"2018","journal-title":"J Chem Inf Model"},{"key":"2026011311372154500_bib49","doi-asserted-by":"publisher","first-page":"1004","DOI":"10.1038\/s42256-022-00557-6","article-title":"Accurate prediction of molecular properties and drug targets using a self-supervised image representation learning framework","volume":"4","author":"Zeng","year":"2022","journal-title":"Nat Mach Intell"},{"key":"2026011311372154500_bib50","doi-asserted-by":"publisher","first-page":"107398","DOI":"10.1016\/j.ymssp.2020.107398","article-title":"1d convolutional neural networks and applications: a survey","volume":"151","author":"Kiranyaz","year":"2021","journal-title":"Mech Syst Sig Process"},{"key":"2026011311372154500_bib51","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1016\/j.aiopen.2021.01.001","article-title":"Graph neural networks: a review of methods and applications","volume":"1","author":"Zhou","year":"2020","journal-title":"AI Open"},{"key":"2026011311372154500_bib52","doi-asserted-by":"publisher","DOI":"10.1063\/1.5019779","article-title":"Schnet\u2013a deep learning architecture for molecules and materials","volume":"148","author":"Sch\u00fctt","year":"2018","journal-title":"J Chem Phys"},{"key":"2026011311372154500_bib53","doi-asserted-by":"publisher","first-page":"13890","DOI":"10.1038\/ncomms13890","article-title":"Quantum-chemical insights from deep tensor neural networks","volume":"8","author":"Sch\u00fctt","year":"2017","journal-title":"Nat Commun"},{"key":"2026011311372154500_bib54","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1145\/3292500.3330961","article-title":"Heterogeneous graph neural network","volume-title":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Zhang","year":"2019"},{"key":"2026011311372154500_bib55","doi-asserted-by":"publisher","first-page":"2022","DOI":"10.1145\/3308558.3313562","article-title":"Heterogeneous graph attention network","volume-title":"The World Wide Web Conference","author":"Wang","year":"2019"},{"key":"2026011311372154500_bib56","article-title":"A survey on oversmoothing in graph neural networks","author":"Rusch"},{"key":"2026011311372154500_bib57","doi-asserted-by":"publisher","first-page":"102485","DOI":"10.1016\/j.inffus.2024.102485","article-title":"A hierarchical attention network integrating multi-scale relationship for drug response prediction","volume":"110","author":"Wang","year":"2024","journal-title":"Inform Fusion"},{"key":"2026011311372154500_bib58","first-page":"770","article-title":"Deep residual learning for image recognition","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"He","year":"2016"},{"key":"2026011311372154500_bib59","article-title":"Layer normalization","author":"Ba"},{"key":"2026011311372154500_bib60","first-page":"3441","article-title":"Multilingual molecular representation learning via contrastive pre-training","author":"Guo","year":"2022","journal-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics"},{"key":"2026011311372154500_bib61","first-page":"3615","article-title":"Dual-view molecular pre-training","author":"Zhu","year":"2023","journal-title":"Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining"},{"key":"2026011311372154500_bib62","doi-asserted-by":"publisher","first-page":"4217","DOI":"10.1038\/s41467-023-39856-w","article-title":"Sequence-based drug design as a concept in computational drug design","volume":"14","author":"Chen","year":"2023","journal-title":"Nat Commun"},{"key":"2026011311372154500_bib63","doi-asserted-by":"publisher","first-page":"9360","DOI":"10.1038\/s41467-024-53632-4","article-title":"TamGen: drug design with target-aware molecule generation through a chemical language model","volume":"15","author":"Wu","year":"2024","journal-title":"Nat Commun"},{"key":"2026011311372154500_bib64","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1038\/s41551-024-01312-5","article-title":"Interpretable identification of cancer genes across biological networks via transformer-powered graph representation learning","volume":"9","author":"Su","year":"2025","journal-title":"Nat Biomed Eng"},{"key":"2026011311372154500_bib65","first-page":"28877","article-title":"Do transformers really perform badly for graph representation?","volume-title":"34th Annual Conference on Advances in Neural Information Processing Systems","author":"Ying","year":"2021"},{"key":"2026011311372154500_bib66","article-title":"Molecule attention transformer","author":"Maziarka"},{"key":"2026011311372154500_bib67","article-title":"One transformer can understand both 2D & 3D molecular data","author":"Luo"},{"key":"2026011311372154500_bib68","article-title":"Multimodal molecular pretraining via modality blending","author":"Yu"},{"key":"2026011311372154500_bib69","doi-asserted-by":"publisher","first-page":"10223","DOI":"10.1038\/s41467-024-54440-6","article-title":"Interformer: an interaction-aware model for protein-ligand docking and affinity prediction","volume":"15","author":"Lai","year":"2024","journal-title":"Nat Commun"},{"key":"2026011311372154500_bib70","doi-asserted-by":"publisher","first-page":"1564","DOI":"10.1109\/JBHI.2023.3347794","article-title":"Dual-view learning based on images and sequences for molecular property prediction","volume":"28","author":"Zhang","year":"2023","journal-title":"IEEE J Biomed Health Inform"},{"key":"2026011311372154500_bib71","first-page":"8748","article-title":"Learning transferable visual models from natural language supervision","volume-title":"Proceedings of the 38th International Conference on Machine Learning (Proc Mach Learn Res 139)","author":"Radford","year":"2021"},{"key":"2026011311372154500_bib72","doi-asserted-by":"publisher","first-page":"3948","DOI":"10.1021\/acs.jcim.2c00521","article-title":"Smiclr: contrastive learning on multiple molecular representations for semisupervised and unsupervised representation learning","volume":"62","author":"Pinheiro","year":"2022","journal-title":"J Chem Inf Model"},{"key":"2026011311372154500_bib73","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1007\/978-3-030-58577-8_7","article-title":"UNITER: universal Image-TExt Representation Learning","volume-title":"Computer Vision\u2013ECCV 2020","author":"Chen","year":"2020"},{"key":"2026011311372154500_bib74","doi-asserted-by":"publisher","first-page":"2978","DOI":"10.18653\/v1\/P19-1285","article-title":"Transformer-XL: attentive language models beyond a fixed-length context","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Dai","year":"2019"},{"key":"2026011311372154500_bib75","first-page":"1597","article-title":"A simple framework for contrastive learning of visual representations","volume-title":"Proceedings of the 37th International Conference on Machine Learning (Proc Mach Learn Res\n119)","author":"Chen","year":"2020"},{"key":"2026011311372154500_bib76","doi-asserted-by":"publisher","first-page":"4216","DOI":"10.1109\/TKDE.2021.3131584","article-title":"Self-supervised learning on graphs: contrastive, generative, or predictive","volume":"35","author":"Wu","year":"2021","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"2026011311372154500_bib77","doi-asserted-by":"publisher","first-page":"15623","DOI":"10.18653\/v1\/2023.emnlp-main.966","article-title":"MolCA: molecular graph-language modeling with cross-modal projector and uni-modal adapter","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Liu","year":"2023"},{"key":"2026011311372154500_bib78","doi-asserted-by":"publisher","first-page":"2064","DOI":"10.1021\/acs.jcim.1c00600","article-title":"MolGPT: molecular generation using a transformer-decoder model","volume":"62","author":"Bagal","year":"2021","journal-title":"J Chem Inf Model"},{"key":"2026011311372154500_bib79","doi-asserted-by":"publisher","first-page":"5339","DOI":"10.1609\/aaai.v37i4.25665","article-title":"Multi-relational contrastive learning graph neural network for drug-drug interaction event prediction","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Xiong","year":"2023"},{"key":"2026011311372154500_bib80","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1609\/aaai.v38i1.27777","article-title":"Dual-channel learning framework for drug-drug interaction prediction via relation-aware heterogeneous graph transformer","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Su","year":"2024"},{"key":"2026011311372154500_bib81","doi-asserted-by":"publisher","first-page":"bbab226","DOI":"10.1093\/bib\/bbab226","article-title":"DeepR2cov: deep representation learning on heterogeneous drug networks to discover anti-inflammatory agents for covid-19","volume":"22","author":"Wang","year":"2021","journal-title":"Brief Bioinform"},{"key":"2026011311372154500_bib82","doi-asserted-by":"publisher","first-page":"4793","DOI":"10.1093\/bioinformatics\/btab565","article-title":"BioERP: biomedical heterogeneous network-based self-supervised representation learning approach for entity relationship predictions","volume":"37","author":"Wang","year":"2021","journal-title":"Bioinformatics"},{"key":"2026011311372154500_bib83","article-title":"Parameter-efficient fine-tuning for large models: a comprehensive survey","author":"Han"},{"key":"2026011311372154500_bib84","article-title":"Strategies for pre-training graph neural networks","author":"Hu"},{"key":"2026011311372154500_bib85","first-page":"36","article-title":"Hierarchical structure-aware graph prompting for drug-drug interaction prediction","volume-title":"Machine Learning and Knowledge Discovery in Databases","author":"Ye","year":"2024"},{"key":"2026011311372154500_bib86","doi-asserted-by":"publisher","first-page":"637","DOI":"10.1039\/D4SC06864E","article-title":"3DSMILES-GPT: 3D molecular pocket-based generation with token-only large language model","volume":"16","author":"Wang","year":"2025","journal-title":"Chem Sci"},{"key":"2026011311372154500_bib87","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1038\/s42256-023-00775-6","article-title":"Generation of 3D molecules in pockets via a language model","volume":"6","author":"Feng","year":"2024","journal-title":"Nat Mach Intell"},{"key":"2026011311372154500_bib88","first-page":"5485","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"J Mach Learn Res"},{"key":"2026011311372154500_bib89","doi-asserted-by":"publisher","first-page":"21958","DOI":"10.1609\/aaai.v38i20.30198","article-title":"From artificially real to real: Leveraging pseudo data from large language models for low-resource molecule discovery","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Chen","year":"2024"},{"key":"2026011311372154500_bib90","doi-asserted-by":"publisher","first-page":"btae534","DOI":"10.1093\/bioinformatics\/btae534","article-title":"ChatMol: interactive molecular discovery with natural language","volume":"40","author":"Zeng","year":"2024","journal-title":"Bioinformatics"},{"key":"2026011311372154500_bib91","first-page":"6140","article-title":"Unifying molecular and textual representations via multi-task language modelling","volume-title":"Proceedings of the 40th International Conference on Machine Learning (Proc Mach Learn Res 202)","author":"Christofidellis","year":"2023"},{"key":"2026011311372154500_bib92","doi-asserted-by":"publisher","first-page":"8380","DOI":"10.1039\/D4SC00966E","article-title":"nach0: multimodal natural and chemical languages foundation model","volume":"15","author":"Livne","year":"2024","journal-title":"Chem Sci"},{"key":"2026011311372154500_bib93","doi-asserted-by":"publisher","first-page":"108073","DOI":"10.1016\/j.compbiomed.2024.108073","article-title":"GIT-Mol: a multi-modal large language model for molecular science with graph, image, and text","volume":"171","author":"Liu","year":"2024","journal-title":"Comput Biol Med"},{"key":"2026011311372154500_bib94","first-page":"20479","article-title":"3d infomax improves gnns for molecular property prediction","volume-title":"Proceedings of the 39th International Conference on Machine Learning (Proc Mach Learn Res 162)","author":"St\u00e4rk","year":"2022"},{"key":"2026011311372154500_bib95","doi-asserted-by":"publisher","first-page":"7659","DOI":"10.1109\/JBHI.2024.3422488","article-title":"Self-supervised pre-training via multi-view graph information bottleneck for molecular property prediction","volume":"28","author":"Zang","year":"2024","journal-title":"IEEE J Biomed Health Inform"},{"key":"2026011311372154500_bib96","doi-asserted-by":"publisher","first-page":"i357","DOI":"10.1093\/bioinformatics\/btae260","article-title":"MolLM: a unified language model for integrating biomedical text with 2D and 3D molecular representations","volume":"40","author":"Tang","year":"2024","journal-title":"Bioinformatics"},{"key":"2026011311372154500_bib97","doi-asserted-by":"publisher","first-page":"1447","DOI":"10.1038\/s42256-023-00759-6","article-title":"Multi-modal molecule structure\u2013text model for text-based retrieval and editing","volume":"5","author":"Liu","year":"2023","journal-title":"Nat Mach Intell"},{"key":"2026011311372154500_bib98","article-title":"A molecular multimodal foundation model associating molecule graphs with natural language","author":"Su"},{"key":"2026011311372154500_bib99","doi-asserted-by":"crossref","first-page":"4283","DOI":"10.1145\/3627673.3679936","article-title":"H2D: hierarchical heterogeneous graph learning framework for drug-drug interaction prediction","volume-title":"Proceedings of the 33rd ACM International Conference on Information and Knowledge Management","author":"Zhang","year":"2024"},{"key":"2026011311372154500_bib100","doi-asserted-by":"publisher","first-page":"bbad385","DOI":"10.1093\/bib\/bbad385","article-title":"HetDDI: a pre-trained heterogeneous graph neural network model for drug\u2013drug interaction prediction","volume":"24","author":"Li","year":"2023","journal-title":"Brief Bioinform"},{"key":"2026011311372154500_bib101","doi-asserted-by":"publisher","first-page":"673","DOI":"10.1038\/nrd1468","article-title":"Drug repositioning: identifying and developing new uses for existing drugs","volume":"3","author":"Ashburn","year":"2004","journal-title":"Nat Rev Drug Discov"},{"key":"2026011311372154500_bib102","doi-asserted-by":"publisher","first-page":"btae693","DOI":"10.1093\/bioinformatics\/btae693","article-title":"Accurate and transferable drug\u2013target interaction prediction with druglamp","volume":"40","author":"Luo","year":"2024","journal-title":"Bioinformatics"},{"key":"2026011311372154500_bib103","doi-asserted-by":"publisher","first-page":"bbad516","DOI":"10.1093\/bib\/bbae516","article-title":"Predicting drug\u2013target binding affinity with cross-scale graph contrastive learning","volume":"25","author":"Wang","year":"2024","journal-title":"Brief Bioinform"},{"key":"2026011311372154500_bib104","doi-asserted-by":"publisher","first-page":"045024","DOI":"10.1088\/2632-2153\/aba947","article-title":"Self-referencing embedded strings (SELFIES): a 100% robust molecular string representation","volume":"1","author":"Krenn","year":"2020","journal-title":"Mach Learn Sci Technol"},{"key":"2026011311372154500_bib105","doi-asserted-by":"publisher","first-page":"6071","DOI":"10.1109\/TKDE.2024.3393356","article-title":"Empowering molecule discovery for molecule-caption translation with large language models: A chatgpt perspective","volume":"36","author":"Li","year":"2024","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"2026011311372154500_bib106","first-page":"19730","article-title":"BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models","volume-title":"Proceedings of the 40th International Conference on Machine Learning (Proc Mach Learn Res 202)","author":"Li","year":"2023"},{"key":"2026011311372154500_bib107","doi-asserted-by":"publisher","first-page":"15623","DOI":"10.18653\/v1\/2023.emnlp-main.966","article-title":"MolCA: molecular graph-language modeling with cross-modal projector and uni-modal adapter","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Liu","year":"2023"},{"key":"2026011311372154500_bib108","article-title":"MolFM: a multimodal molecular foundation model","author":"Luo"},{"key":"2026011311372154500_bib109","article-title":"Towards 3D molecule-text interpretation in language models","author":"Li"},{"key":"2026011311372154500_bib110","article-title":"UniMoT: unified molecule-text language model with discrete token representation","author":"Zhang"},{"key":"2026011311372154500_bib111","article-title":"3D-MolT5: leveraging discrete structural information for molecule-text modeling","author":"Pei"},{"key":"2026011311372154500_bib112","first-page":"32897","article-title":"VLMo: unified vision-language pre-training with mixture-of-modality-experts","volume-title":"35th Annual Conference on Advances in Neural Information Processing Systems","author":"Bao","year":"2022"},{"key":"2026011311372154500_bib113","doi-asserted-by":"crossref","article-title":"SaProt: protein language modeling with structure-aware vocabulary","author":"Su","DOI":"10.1101\/2023.10.01.560349"},{"key":"2026011311372154500_bib114","first-page":"19175","article-title":"Image as a foreign language: BEIT pretraining for vision and vision-language tasks","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Wang","year":"2023"},{"key":"2026011311372154500_bib115","doi-asserted-by":"publisher","first-page":"9696","DOI":"10.1038\/s41467-024-53742-z","article-title":"A molecular video-derived foundation model for scientific drug discovery","volume":"15","author":"Xiang","year":"2024","journal-title":"Nat Commun"},{"key":"2026011311372154500_bib116","doi-asserted-by":"publisher","first-page":"6494","DOI":"10.1038\/s41467-022-34277-7","article-title":"Deep transfer learning of cancer drug responses by integrating bulk and single-cell RNA-seq data","volume":"13","author":"Chen","year":"2022","journal-title":"Nat Commun"},{"key":"2026011311372154500_bib117","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1038\/s41592-024-02185-x","article-title":"VIBRANT: spectral profiling for single-cell drug responses","volume":"21","author":"Liu","year":"2024","journal-title":"Nat Methods"},{"key":"2026011311372154500_bib118","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1038\/s41573-020-00117-w","article-title":"Image-based profiling for drug discovery: due for a machine-learning upgrade?","volume":"20","author":"Chandrasekaran","year":"2021","journal-title":"Nat Rev Drug Discov"},{"key":"2026011311372154500_bib119","doi-asserted-by":"publisher","first-page":"bbad398","DOI":"10.1093\/bib\/bbad398","article-title":"FG-BERT: a generalized and self-supervised functional group-based molecular representation learning framework for properties prediction","volume":"24","author":"Li","year":"2023","journal-title":"Brief Bioinform"},{"key":"2026011311372154500_bib120","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1021\/acscentsci.2c01200","article-title":"Slicing and dicing: optimal coarse-grained representation to preserve molecular kinetics","volume":"9","author":"Yang","year":"2023","journal-title":"ACS Cent Sci"},{"key":"2026011311372154500_bib121","first-page":"125581","article-title":"Molecular representation learning via heterogeneous motif graph neural networks","volume-title":"Proceedings of the 39th International Conference on Machine Learning (Proc Mach Learn Res 162)","author":"Yu","year":"2022"},{"key":"2026011311372154500_bib122","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1186\/1758-2946-1-8","article-title":"Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions","volume":"1","author":"Ertl","year":"2009","journal-title":"J Cheminf"}],"container-title":["National Science Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/nsr\/advance-article-pdf\/doi\/10.1093\/nsr\/nwaf495\/65269418\/nwaf495.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/nsr\/article-pdf\/13\/1\/nwaf495\/65269418\/nwaf495.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/nsr\/article-pdf\/13\/1\/nwaf495\/65269418\/nwaf495.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T16:37:35Z","timestamp":1768322255000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/nsr\/article\/doi\/10.1093\/nsr\/nwaf495\/8320534"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,11]]},"references-count":122,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1,10]]}},"URL":"https:\/\/doi.org\/10.1093\/nsr\/nwaf495","relation":{},"ISSN":["2095-5138","2053-714X"],"issn-type":[{"value":"2095-5138","type":"print"},{"value":"2053-714X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,1]]},"published":{"date-parts":[[2025,11,11]]},"article-number":"nwaf495"}}