{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,30]],"date-time":"2026-08-30T20:29:12Z","timestamp":1788121752058,"version":"build-2803163510"},"reference-count":38,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2021,10,13]],"date-time":"2021-10-13T00:00:00Z","timestamp":1634083200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["21403002"],"award-info":[{"award-number":["21403002"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,1,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Motivation<\/jats:title><jats:p>As one of the most important post-translational modifications (PTMs), protein lysine crotonylation (Kcr) has attracted wide attention, which involves in important physiological activities, such as cell differentiation and metabolism. However, experimental methods are expensive and time-consuming for Kcr identification. Instead, computational methods can predict Kcr sites in silico with high efficiency and low cost.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>In this study, we proposed a novel predictor, BERT-Kcr, for protein Kcr sites prediction, which was developed by using a transfer learning method with pre-trained bidirectional encoder representations from transformers (BERT) models. These models were originally used for natural language processing (NLP) tasks, such as sentence classification. Here, we transferred each amino acid into a word as the input information to the pre-trained BERT model. The features encoded by BERT were extracted and then fed to a BiLSTM network to build our final model. Compared with the models built by other machine learning and deep learning classifiers, BERT-Kcr achieved the best performance with AUROC of 0.983 for 10-fold cross validation. Further evaluation on the independent test set indicates that BERT-Kcr outperforms the state-of-the-art model Deep-Kcr with an improvement of about 5% for AUROC. The results of our experiment indicate that the direct use of sequence information and advanced pre-trained models of NLP could be an effective way for identifying PTM sites of proteins.<\/jats:p><\/jats:sec><jats:sec><jats:title>Availability and implementation<\/jats:title><jats:p>The BERT-Kcr model is publicly available on http:\/\/zhulab.org.cn\/BERT-Kcr_models\/.<\/jats:p><\/jats:sec><jats:sec><jats:title>Supplementary information<\/jats:title><jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p><\/jats:sec>","DOI":"10.1093\/bioinformatics\/btab712","type":"journal-article","created":{"date-parts":[[2021,10,11]],"date-time":"2021-10-11T14:36:22Z","timestamp":1633962982000},"page":"648-654","source":"Crossref","is-referenced-by-count":83,"title":["BERT-Kcr: prediction of lysine crotonylation sites by a transfer learning method with pre-trained BERT models"],"prefix":"10.1093","volume":"38","author":[{"given":"Yanhua","family":"Qiao","sequence":"first","affiliation":[{"name":"School of Life Sciences, Tsinghua University , Beijing 100084, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1967-2806","authenticated-orcid":false,"given":"Xiaolei","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Sciences, Anhui Agricultural University , Hefei, Anhui 230036, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5532-1640","authenticated-orcid":false,"given":"Haipeng","family":"Gong","sequence":"additional","affiliation":[{"name":"School of Life Sciences, Tsinghua University , Beijing 100084, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,10,13]]},"reference":[{"key":"2023020108484786700_btab712-B1","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/45.329294","article-title":"Feed-forward neural networks","volume":"13","author":"Bebis","year":"1990","journal-title":"IEEE Potentials"},{"key":"2023020108484786700_btab712-B2","first-page":"105","article-title":"Random Forests","volume":"36","author":"Breiman","year":"2001","journal-title":"Machine Learning"},{"key":"2023020108484786700_btab712-B3","doi-asserted-by":"crossref","first-page":"2556","DOI":"10.1093\/bioinformatics\/btab133","article-title":"BERT4Bitter: a bidirectional encoder representations from transformers (BERT)-based model for improving the prediction of bitter peptides","volume":"37","author":"Charoenkwan","year":"2021","journal-title":"Bioinformatics"},{"key":"2023020108484786700_btab712-B4","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1613\/jair.953","article-title":"SMOTE: Synthetic Minority Over-sampling Technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J. Artif. Intell. Res"},{"key":"2023020108484786700_btab712-B5","first-page":"785","article-title":"XGBoost: a scalable tree boosting system","author":"Chen","year":"2016","journal-title":"ACM"},{"key":"2023020108484786700_btab712-B6","first-page":"4171","article-title":"BERT: pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2019"},{"key":"2023020108484786700_btab712-B7","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1038\/s41467-017-02651-5","article-title":"Microbiota derived short chain fatty acids promote histone crotonylation in the colon through histone deacetylases","volume":"9","author":"Fellows","year":"2018","journal-title":"Nat. Commun"},{"key":"2023020108484786700_btab712-B8","doi-asserted-by":"crossref","first-page":"4051","DOI":"10.1007\/s00216-018-1021-y","article-title":"Proteomic approaches beyond expression profiling and PTM analysis","volume":"410","author":"Fu","year":"2018","journal-title":"Anal. Bioanal. Chem"},{"key":"2023020108484786700_btab712-B9","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput"},{"key":"2023020108484786700_btab712-B10","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1093\/bioinformatics\/btq003","article-title":"CD-HIT Suite: a web server for clustering and comparing biological sequences","volume":"26","author":"Huang","year":"2010","journal-title":"Bioinformatics"},{"key":"2023020108484786700_btab712-B11","doi-asserted-by":"crossref","first-page":"458","DOI":"10.1016\/j.cell.2014.09.037","article-title":"SnapShot: histone modifications","volume":"159","author":"Huang","year":"2014","journal-title":"Cell"},{"key":"2023020108484786700_btab712-B12","doi-asserted-by":"crossref","first-page":"e1700230","DOI":"10.1002\/pmic.201700230","article-title":"Quantitative crotonylome analysis expands the roles of p300 in the regulation of lysine crotonylation pathway","volume":"18","author":"Huang","year":"2018","journal-title":"Proteomics"},{"key":"2023020108484786700_btab712-B13","doi-asserted-by":"crossref","first-page":"3374","DOI":"10.1038\/s41467-018-05567-w","article-title":"Lysine benzoylation is a histone mark regulated by SIRT2","volume":"9","author":"Huang","year":"2018","journal-title":"Nat. Commun"},{"key":"2023020108484786700_btab712-B14","doi-asserted-by":"crossref","first-page":"1190","DOI":"10.1172\/JCI98071","article-title":"HIV latency is reversed by ACSS2-driven histone crotonylation","volume":"128","author":"Jiang","year":"2018","journal-title":"J. Clin. Investig"},{"key":"2023020108484786700_btab712-B15","first-page":"427","article-title":"Bag of tricks for efficient text classification","author":"Joulin","year":"2017"},{"key":"2023020108484786700_btab712-B16","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/j.jmgm.2017.08.020","article-title":"Prediction of lysine crotonylation sites by incorporating the composition of k-spaced amino acid pairs into Chou's general PseAAC","volume":"77","author":"Ju","year":"2017","journal-title":"J. Mol. Graph. Model"},{"key":"2023020108484786700_btab712-B17","article-title":"ImageNet classification with deep convolutional neural networks","author":"Krizhevsky","year":"2012","journal-title":"In Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS 2012), Vol. 1, pp. 1097-1105."},{"key":"2023020108484786700_btab712-B18","doi-asserted-by":"crossref","first-page":"bbab005","DOI":"10.1093\/bib\/bbab005","article-title":"A transformer architecture based on BERT and 2D convolutional neural network to identify DNA enhancers from sequence information","volume":"22","author":"Le","year":"2021","journal-title":"Brief. Bioinf"},{"key":"2023020108484786700_btab712-B19","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1016\/j.molcel.2017.07.011","article-title":"Chromodomain protein CDYL acts as a crotonyl-coa hydratase to regulate histone crotonylation and spermatogenesis","volume":"67","author":"Liu","year":"2017","journal-title":"Mol. Cell"},{"key":"2023020108484786700_btab712-B20","doi-asserted-by":"crossref","first-page":"113903","DOI":"10.1016\/j.ab.2020.113903","article-title":"Prediction of protein crotonylation sites through LightGBM classifier based on SMOTE and elastic net","volume":"609","author":"Liu","year":"2020","journal-title":"Anal. Biochem"},{"key":"2023020108484786700_btab712-B21","doi-asserted-by":"crossref","first-page":"bbaa255","DOI":"10.1093\/bib\/bbaa255","article-title":"Deep-Kcr: accurate detection of lysine crotonylation sites using deep learning method","volume":"22","author":"Lv","year":"2020","journal-title":"Brief. Bioinf"},{"key":"2023020108484786700_btab712-B22","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/N18-1202","article-title":"Deep contextualized word representations","author":"Peters","year":"2018"},{"key":"2023020108484786700_btab712-B23","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.artmed.2017.02.007","article-title":"Identify and analysis crotonylation sites in histone by using support vector machines","volume":"83","author":"Qiu","year":"2017","journal-title":"Artif. Intell. Med"},{"key":"2023020108484786700_btab712-B24","doi-asserted-by":"crossref","first-page":"e2016239118","DOI":"10.1073\/pnas.2016239118","article-title":"Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences","volume":"118","author":"Rives","year":"2021","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023020108484786700_btab712-B25","doi-asserted-by":"crossref","first-page":"633","DOI":"10.1242\/dmm.024455","article-title":"Histone lysine crotonylation during acute kidney injury in mice","volume":"9","author":"Ruizandres","year":"2016","journal-title":"Dis. Models Mech"},{"key":"2023020108484786700_btab712-B26","doi-asserted-by":"crossref","first-page":"2673","DOI":"10.1109\/78.650093","article-title":"Bidirectional recurrent neural networks","volume":"45","author":"Schuster","year":"1997","journal-title":"IEEE Trans. Signal Process"},{"key":"2023020108484786700_btab712-B28","doi-asserted-by":"crossref","first-page":"1016","DOI":"10.1016\/j.cell.2011.08.008","article-title":"Identification of 67 histone marks and histone lysine crotonylation as a new type of histone modification","volume":"146","author":"Tan","year":"2011","journal-title":"Cell"},{"key":"2023020108484786700_btab712-B29","doi-asserted-by":"crossref","first-page":"D214","DOI":"10.1093\/nar\/gkq1020","article-title":"Ongoing and future developments at the Universal Protein Resource","volume":"39","author":"UniProt","year":"2011","journal-title":"Nucleic Acids Res"},{"key":"2023020108484786700_btab712-B30","doi-asserted-by":"crossref","first-page":"1536","DOI":"10.1093\/bioinformatics\/btl151","article-title":"Two Sample Logo: a graphical representation of the differences between two sets of sequence alignments","volume":"22","author":"Vacic","year":"2006","journal-title":"Bioinformatics"},{"key":"2023020108484786700_btab712-B31","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-2440-0","volume-title":"The Nature of Statistical Learning Theory","author":"Vapnik","year":"1995"},{"key":"2023020108484786700_btab712-B32","first-page":"6000","article-title":"Attention is all you need","author":"Vaswani","year":"2017"},{"key":"2023020108484786700_btab712-B33","doi-asserted-by":"crossref","first-page":"2365","DOI":"10.1073\/pnas.1717664115","article-title":"Identification of the YEATS domain of GAS41 as a pH-dependent reader of histone succinylation","volume":"115","author":"Wang","year":"2018","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023020108484786700_btab712-B34","doi-asserted-by":"crossref","first-page":"898","DOI":"10.1038\/cr.2017.68","article-title":"Class I histone deacetylases are major histone decrotonylases: evidence for critical and broad function of histone crotonylation in transcription","volume":"27","author":"Wei","year":"2017","journal-title":"Cell Res"},{"key":"2023020108484786700_btab712-B35","doi-asserted-by":"crossref","first-page":"1743","DOI":"10.1021\/acs.jproteome.7b00012","article-title":"Large-scale identification of protein crotonylation reveals its role in multiple cellular functions","volume":"16","author":"Wei","year":"2017","journal-title":"J. Proteome Res"},{"key":"2023020108484786700_btab712-B36","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1162\/neco.1996.8.7.1341","article-title":"The lack of a priori distinctions between learning algorithms","volume":"8","author":"Wolpert","year":"1996","journal-title":"Neural Comput"},{"key":"2023020108484786700_btab712-B37","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1016\/j.jgg.2017.03.007","article-title":"PLMD: an updated data resource of protein lysine modifications","volume":"44","author":"Xu","year":"2017","journal-title":"J. Genet. Genomics"},{"key":"2023020108484786700_btab712-B38","doi-asserted-by":"crossref","first-page":"eaay4697","DOI":"10.1126\/sciadv.aay4697","article-title":"Global crotonylome reveals CDYL-regulated RPA1 crotonylation in homologous recombination-mediated DNA repair","volume":"6","author":"Yu","year":"2020","journal-title":"Sci. Adv"},{"key":"2023020108484786700_btab712-B27","first-page":"73","author":"Zhang","year":"2015"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btab712\/41135124\/btab712.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/38\/3\/648\/49008320\/btab712.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/38\/3\/648\/49008320\/btab712.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,9]],"date-time":"2024-09-09T20:17:15Z","timestamp":1725913035000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/38\/3\/648\/6395351"}},"subtitle":[],"editor":[{"given":"Jinbo","family":"Xu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2021,10,13]]},"references-count":38,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,1,12]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btab712","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,2,1]]},"published":{"date-parts":[[2021,10,13]]}}}