{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T19:37:57Z","timestamp":1782848277926,"version":"3.54.5"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,10,21]],"date-time":"2022-10-21T00:00:00Z","timestamp":1666310400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,21]],"date-time":"2022-10-21T00:00:00Z","timestamp":1666310400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005047","name":"Natural Science Foundation of Liaoning Province","doi-asserted-by":"publisher","award":["2020-MS-135"],"award-info":[{"award-number":["2020-MS-135"]}],"id":[{"id":"10.13039\/501100005047","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61673084"],"award-info":[{"award-number":["61673084"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100019010","name":"Key Laboratory of Intelligent Perception and Advanced Control of State Ethnic Affairs Commission","doi-asserted-by":"publisher","award":["MD-IPAC-2019103"],"award-info":[{"award-number":["MD-IPAC-2019103"]}],"id":[{"id":"10.13039\/501100019010","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Dalian High Level Talent Innovation Support Program","award":["2020RQ060"],"award-info":[{"award-number":["2020RQ060"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cheminform"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Molecular property prediction (MPP) is vital in drug discovery and drug reposition. Deep learning-based MPP models capture molecular property-related features from various molecule representations. In this paper, we propose a molecule sequence embedding and prediction model facing with MPP task. We pre-trained a bi-directional encoder representations from Transformers (BERT) encoder to obtain the semantic representation of compound fingerprints, called Fingerprints-BERT (FP-BERT), in a self-supervised learning manner. Then, the encoded molecular representation by the FP-BERT is input to the convolutional neural network (CNN) to extract higher-level abstract features, and the predicted properties of the molecule are finally obtained through fully connected layer for distinct classification or regression MPP tasks. Comparison with the baselines shows that the proposed model achieves high prediction performance on all of the classification tasks and regression tasks.<\/jats:p>","DOI":"10.1186\/s13321-022-00650-3","type":"journal-article","created":{"date-parts":[[2022,10,21]],"date-time":"2022-10-21T08:07:37Z","timestamp":1666339657000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":71,"title":["A fingerprints based molecular property prediction method using the BERT model"],"prefix":"10.1186","volume":"14","author":[{"given":"Naifeng","family":"Wen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guanqun","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jie","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rubo","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yating","family":"Fu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xu","family":"Han","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,10,21]]},"reference":[{"key":"650_CR1","unstructured":"Dahl GE, Jaitly N, Salakhutdinov R (2014) Multi-task neural networks for qsar predictions. arXiv preprint . arXiv:1406.1231"},{"issue":"2","key":"650_CR2","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1021\/ci500747n","volume":"55","author":"J Ma","year":"2015","unstructured":"Ma J, Sheridan RP, Liaw A, Dahl GE, Svetnik V (2015) Deep neural nets as a method for quantitative structure-activity relationships. J Chem Inf Model 55(2):263\u2013274","journal-title":"J Chem Inf Model"},{"issue":"10","key":"650_CR3","doi-asserted-by":"publisher","first-page":"2490","DOI":"10.1021\/acs.jcim.7b00087","volume":"57","author":"Y Xu","year":"2017","unstructured":"Xu Y, Ma J, Liaw A, Sheridan RP, Svetnik V (2017) Demystifying multitask deep neural networks for quantitative structure-activity relationships. J Chem Inf Model 57(10):2490\u20132504","journal-title":"J Chem Inf Model"},{"issue":"8","key":"650_CR4","doi-asserted-by":"publisher","first-page":"3370","DOI":"10.1021\/acs.jcim.9b00237","volume":"59","author":"K Yang","year":"2019","unstructured":"Yang K, Swanson K, Jin W, Coley C, Eiden P, Gao H et al (2019) Analyzing learned molecular representations for property prediction. J Chem Inf Model 59(8):3370\u20133388","journal-title":"J Chem Inf Model"},{"key":"650_CR5","doi-asserted-by":"crossref","unstructured":"Lu C, Liu Q, Wang C, Huang Z, Lin P, He L (2019) Molecular property prediction: a multilevel quantum interactions modeling perspective. In: Proceedings of the AAAI Conference on Artificial Intelligence: 27 January-01 February 2019; Hawaii, vol. 33, pp. 1052\u2013 1060","DOI":"10.1609\/aaai.v33i01.33011052"},{"issue":"9","key":"650_CR6","doi-asserted-by":"publisher","first-page":"3817","DOI":"10.1021\/acs.jcim.9b00410","volume":"59","author":"X Wang","year":"2019","unstructured":"Wang X, Li Z, Jiang M, Wang S, Zhang S, Wei Z (2019) Molecule property prediction based on spatial graph embedding. J Chem Inf Model 59(9):3817\u20133828","journal-title":"J Chem Inf Model"},{"issue":"23","key":"650_CR7","doi-asserted-by":"publisher","first-page":"4979","DOI":"10.1093\/bioinformatics\/btz307","volume":"35","author":"W Jeon","year":"2019","unstructured":"Jeon W, Kim DZ (2019) Fp2vec: a new molecular featurizer for learning molecular properties. Bioinformatics 35(23):4979\u20134985","journal-title":"Bioinformatics"},{"issue":"1","key":"650_CR8","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1021\/acs.jcim.7b00616","volume":"58","author":"S Jaeger","year":"2018","unstructured":"Jaeger S, Fulle S, Turk S (2018) Mol2vec: unsupervised machine learning approach with chemical intuition. J Chem Inf Model 58(1):27\u201335","journal-title":"J Chem Inf Model"},{"key":"650_CR9","doi-asserted-by":"publisher","unstructured":"Fabian B, Edlich T, Gaspar H, Segler M, Ahmed M (2020) Molecular representation learning with language models and domain-relevant auxiliary tasks. arXiv preprint. https:\/\/doi.org\/10.48550\/arXiv.2011.13230","DOI":"10.48550\/arXiv.2011.13230"},{"key":"650_CR10","doi-asserted-by":"crossref","unstructured":"Wang S, Guo Y, Wang Y, Sun H, Huang J (2019) Smiles-bert: large scale unsupervised pre-training for molecular property prediction. In: Proceedings of The10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics: 7-10 September 2019; Niagara Falls, pp. 429\u2013436","DOI":"10.1145\/3307339.3342186"},{"key":"650_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.scib.2022.01.029","author":"X Dongyu","year":"2022","unstructured":"Dongyu X, Han Z, Dongling X, Yukang G, Guohui C, Yu S, Hao T, Hua W, Yukun L, Qi L (2022) X-mol: large-scale pre-training for molecular understanding and diverse molecular analysis. Sci Bull. https:\/\/doi.org\/10.1016\/j.scib.2022.01.029","journal-title":"Sci Bull"},{"key":"650_CR12","doi-asserted-by":"publisher","unstructured":"Ross I, Spyridon D, Jiazhen H, Esben B (2022) Jannik: Chemformer: a pre-trained transformer for computational chemistry. Mach Learn Sci Technol.  https:\/\/doi.org\/10.1088\/2632-2153\/ac3ffb","DOI":"10.1088\/2632-2153\/ac3ffb"},{"issue":"3","key":"650_CR13","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1093\/bib\/bbac131","volume":"23","author":"W Zhenxing","year":"2022","unstructured":"Zhenxing W, Dejun J, Jike W, Xujun Z, Hongyan D, Lurong P, Chang-Yu H, Dongsheng C, Tingjun H (2022) Knowledge-based bert: a method to extract molecular features like computational chemists. Brief Bioinform 23(3):131","journal-title":"Brief Bioinform"},{"key":"650_CR14","doi-asserted-by":"publisher","unstructured":"Jannis B, Matteo M (2022) Regression transformer: concurrent conditional generation and regression by blending numerical and textual tokens. arXiv preprint. https:\/\/doi.org\/10.48550\/arXiv.2202.01338","DOI":"10.48550\/arXiv.2202.01338"},{"key":"650_CR15","doi-asserted-by":"publisher","unstructured":"Shion H, Shoi S, Hiroki U R (2019) Smiles transformer: pre-trained molecular fingerprint for low data drug discovery. arXiv preprint. https:\/\/doi.org\/10.48550\/arXiv.1911.04738","DOI":"10.48550\/arXiv.1911.04738"},{"key":"650_CR16","unstructured":"Jacob D, Chang M, Lee K, K T(2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint ( 2018). arXiv:1810.04805"},{"key":"650_CR17","doi-asserted-by":"crossref","unstructured":"Vig J, Madani A, Varshney LR, Xiong C, Socher R, Rajani NF (2020) Bertology meets biology: interpreting attention in protein language models. bioRxiv preprint (2020). 10.1101\/2020.06.26.174417","DOI":"10.1101\/2020.06.26.174417"},{"key":"650_CR18","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btac550","author":"J Jing","year":"2022","unstructured":"Jing J, Ruisheng Z, Zhili Z, Jun M, Yunwu L, Yongna Y, Bojuan N (2022) Multigran-smiles: multi-granularity smiles learning for molecular property prediction. Bioinformatics. https:\/\/doi.org\/10.1093\/bioinformatics\/btac550","journal-title":"Bioinformatics"},{"issue":"5","key":"650_CR19","doi-asserted-by":"publisher","first-page":"742","DOI":"10.1021\/ci100050t","volume":"50","author":"D Rogers","year":"2010","unstructured":"Rogers D, Hahn M (2010) Extended-connectivity fingerprints. J Chem Inf Model 50(5):742\u2013754","journal-title":"J Chem Inf Model"},{"key":"650_CR20","unstructured":"Chithrananda S, Grand G, Ramsundar B (2020) Chemberta: Large-scale self-supervised pretraining for molecular property prediction. arXiv preprint ( 2020). arXiv:2010.09885"},{"key":"650_CR21","unstructured":"RDKit: Open-Source Cheminformatics Software. http:\/\/www.rdkit.org\/AccessedAccessed10March2021"},{"key":"650_CR22","first-page":"107","volume":"5","author":"HL Morgan","year":"1965","unstructured":"Morgan HL (1965) The generation of a unique machine description for chemical structures-a technique developed at chemical abstracts service. J Chem 5:107\u2013113","journal-title":"J Chem"},{"key":"650_CR23","unstructured":"Babuji Y, Blaiszik B, Brettin T, Chard K, Chard R, Clyde A, et al (2020) Targeting sars-cov-2 with ai- and hpc-enabled lead generation: a first data release. arXiv preprint . arXiv:2006.02431"},{"issue":"5","key":"650_CR24","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1093\/bfgp\/elab036","volume":"20","author":"L Yu","year":"2021","unstructured":"Yu L, Su Y, Liu Y, Zeng X (2021) Review of unsupervised pretraining strategies for molecules representation. Brief Funct Genom 20(5):323\u2013332","journal-title":"Brief Funct Genom"},{"issue":"2","key":"650_CR25","doi-asserted-by":"publisher","first-page":"513","DOI":"10.1039\/C7SC02664A","volume":"9","author":"Z Wu","year":"2018","unstructured":"Wu Z, Bharath R, Feinberg EN, Joseph G, Caleb G, Pappu AS et al (2018) Moleculenet: a benchmark for molecular machine learning. Chem Sci 9(2):513\u2013530","journal-title":"Chem Sci"},{"key":"650_CR26","unstructured":"Duvenaud D, Maclaurin D, Aguilera-Iparraguirre J, G\u00f3mez-Bombarelli R, Hirzel T, Aspuru-Guzik A, et al (2015) Convolutional networks on graphs for learning molecular fingerprints. arXiv preprint (2015). arXiv:1509.09292"},{"key":"650_CR27","unstructured":"AIDS Antiviral Screen Data. http:\/\/wiki.nci.nih.gov\/display\/NCIDTPdata\/AIDS+Antiviral+Screen+DataAccessedAccessed10March2021"},{"key":"650_CR28","doi-asserted-by":"publisher","first-page":"1686","DOI":"10.1021\/ci300124c","volume":"52","author":"IF Martins","year":"2012","unstructured":"Martins IF, Teixeira AL, Pinheiro L, Falcao AO (2012) A bayesian approach to in silico blood-brain barrier penetration modeling. J Chem Inf Model 52:1686\u20131697","journal-title":"J Chem Inf Model"},{"issue":"3","key":"650_CR29","doi-asserted-by":"publisher","first-page":"1000","DOI":"10.1021\/ci034243x","volume":"44","author":"JS Delaney","year":"2004","unstructured":"Delaney JS (2004) Esol: estimating aqueous solubility directly from molecular structure. J Chem Inf Comput Sci 44(3):1000\u20131005","journal-title":"J Chem Inf Comput Sci"},{"issue":"7","key":"650_CR30","doi-asserted-by":"publisher","first-page":"711","DOI":"10.1007\/s10822-014-9747-x","volume":"28","author":"DL Mobley","year":"2014","unstructured":"Mobley DL, Guthrie JP (2014) Freesolv: a database of experimental and calculated hydration free energies, with input files. J Comput-aided Mol Design 28(7):711\u2013720","journal-title":"J Comput-aided Mol Design"},{"issue":"7296","key":"650_CR31","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1038\/nature09107","volume":"465","author":"G Francisco-Javier","year":"2010","unstructured":"Francisco-Javier G, Sanz LM, Vidal J, Cozar CD, Alvarez E, Lavandera JL et al (2010) Thousands of chemical starting points for antimalarial lead identification. Nature 465(7296):305\u2013310","journal-title":"Nature"},{"issue":"17","key":"650_CR32","doi-asserted-by":"publisher","first-page":"2241","DOI":"10.1021\/jz200866s","volume":"2","author":"J Hachmann","year":"2011","unstructured":"Hachmann J, Olivares-Amaya R, Atahan-Evrenk S, Amador-Bedolla C, SaNchez-Carrera RS, Gold-Parker A et al (2011) The harvard clean energy project: large-scale computational screening and design of organic photovoltaics on the world community grid. J Phys Chem Lett 2(17):2241\u20132251","journal-title":"J Phys Chem Lett"},{"issue":"15","key":"650_CR33","doi-asserted-by":"publisher","first-page":"2887","DOI":"10.1021\/jm9602928","volume":"39","author":"GW Bemis","year":"1996","unstructured":"Bemis GW, Murcko MA (1996) The properties of known drugs. 1. Molecular frameworks. J Med Chem 39(15): 2887\u20132893","journal-title":"J Med Chem"},{"key":"650_CR34","unstructured":"Ramsundar B, Leswing K, Eastman P, Pande V (2019) Deep Learning for the Life Sciences: Applying Deep Learning to Genomics, Microscopy, Drug Discovery, and More. O\u2019Reilly, US"},{"key":"650_CR35","unstructured":"DeepChem: Deep-learning models for Drug Discovery and Quantum Chemistry. http:\/\/github.com\/deepchem\/deepchem. Accessed 10 March 2021"}],"container-title":["Journal of Cheminformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-022-00650-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13321-022-00650-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-022-00650-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,21]],"date-time":"2022-10-21T08:18:04Z","timestamp":1666340284000},"score":1,"resource":{"primary":{"URL":"https:\/\/jcheminf.biomedcentral.com\/articles\/10.1186\/s13321-022-00650-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,21]]},"references-count":35,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["650"],"URL":"https:\/\/doi.org\/10.1186\/s13321-022-00650-3","relation":{},"ISSN":["1758-2946"],"issn-type":[{"value":"1758-2946","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,21]]},"assertion":[{"value":"16 March 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 October 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 October 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All authors have approved the manuscript and agree with its submission to the \u201cJournal of Cheminformatics\u201d.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"71"}}