{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T04:11:02Z","timestamp":1772165462583,"version":"3.50.1"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,1,4]],"date-time":"2022-01-04T00:00:00Z","timestamp":1641254400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,4]],"date-time":"2022-01-04T00:00:00Z","timestamp":1641254400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Protein-protein interactions (PPIs) are critical to normal cellular function and are related to many disease pathways. A range of protein functions are mediated and regulated by protein interactions through post-translational modifications (PTM). However, only 4% of PPIs are annotated with PTMs in biological knowledge databases such as IntAct, mainly performed through manual curation, which is neither time- nor cost-effective. Here we aim to facilitate annotation by extracting PPIs along with their pairwise PTM from the literature by using distantly supervised training data using deep learning to aid human curation.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method<\/jats:title>\n                    <jats:p>We use the IntAct PPI database to create a distant supervised dataset annotated with interacting protein pairs, their corresponding PTM type, and associated abstracts from the PubMed database. We train an ensemble of BioBERT models\u2014dubbed PPI-BioBERT-x10\u2014to improve confidence calibration. We extend the use of ensemble average confidence approach with confidence variation to counteract the effects of class imbalance to extract high confidence predictions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results and conclusion<\/jats:title>\n                    <jats:p>\n                      The PPI-BioBERT-x10 model evaluated on the test set resulted in a modest F1-micro 41.3 (P\u00a0=5\u00a08.1, R\u00a0=\u00a032.1). However, by combining high confidence and low variation to identify high quality predictions, tuning the predictions for precision, we retained 19% of the test predictions with 100% precision. We evaluated PPI-BioBERT-x10 on 18 million PubMed abstracts and extracted 1.6 million (546507 unique PTM-PPI triplets) PTM-PPI predictions, and filter\n                      <jats:inline-formula>\n                        <jats:alternatives>\n                          <jats:tex-math>$$\\approx 5700$$<\/jats:tex-math>\n                          <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                            <mml:mrow>\n                              <mml:mo>\u2248<\/mml:mo>\n                              <mml:mn>5700<\/mml:mn>\n                            <\/mml:mrow>\n                          <\/mml:math>\n                        <\/jats:alternatives>\n                      <\/jats:inline-formula>\n                      (4584 unique) high confidence predictions. Of the 5700, human evaluation on a small randomly sampled subset shows that the precision drops to 33.7% despite confidence calibration and highlights the challenges of generalisability beyond the test set even with confidence calibration. We circumvent the problem by only including predictions associated with multiple papers, improving the precision to 58.8%. In this work, we highlight the benefits and challenges of deep learning-based text mining in practice, and the need for increased emphasis on confidence calibration to facilitate human curation efforts.\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12859-021-04504-x","type":"journal-article","created":{"date-parts":[[2022,1,4]],"date-time":"2022-01-04T12:26:46Z","timestamp":1641299206000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Large-scale protein-protein post-translational modification extraction with distant supervision and confidence calibrated BioBERT"],"prefix":"10.1186","volume":"23","author":[{"given":"Aparna","family":"Elangovan","sequence":"first","affiliation":[]},{"given":"Yuan","family":"Li","sequence":"additional","affiliation":[]},{"given":"Douglas E. V.","family":"Pires","sequence":"additional","affiliation":[]},{"given":"Melissa J.","family":"Davis","sequence":"additional","affiliation":[]},{"given":"Karin","family":"Verspoor","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,1,4]]},"reference":[{"issue":"D1","key":"4504_CR1","doi-asserted-by":"publisher","first-page":"358","DOI":"10.1093\/nar\/gkt1115","volume":"42","author":"S Orchard","year":"2013","unstructured":"Orchard S, Ammari M, Aranda B, Breuza L, Briganti L, Broackes-Carter F, Campbell NH, Chavali G, Chen C, Del-Toro N. The mintact project-intact as a common curation platform for 11 molecular interaction databases. Nucleic Acids Res. 2013;42(D1):358\u201363.","journal-title":"Nucleic Acids Res."},{"key":"4504_CR2","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1093\/nar\/gkj141","volume":"34","author":"GR Mishra","year":"2006","unstructured":"Mishra GR, Suresh M, Kumaran K, Kannabiran N, Suresh S, Bala P, Shivakumar K, Anuradha N, Reddy R, Raghavan TM. Human protein reference database-2006 update. Nucleic Acids Res. 2006;34:411\u20134.","journal-title":"Nucleic Acids Res."},{"issue":"(D1)","key":"4504_CR3","doi-asserted-by":"publisher","first-page":"358","DOI":"10.1093\/nar\/gkt1115","volume":"42","author":"S Orchard","year":"2013","unstructured":"Orchard S, Ammari M, Aranda B, Breuza L, Briganti L, BroackesCarter F, Campbell NH, Chavali G, Chen C, delToro N, Duesbury M, Dumousseau M, Galeota E, Hinz U, Iannuccelli M, Jagannathan S, Jimenez R, Khadake J, Lagreid A, Licata L, Lovering RC, Meldal B, Melidoni AN, Milagros M, Peluso D, Perfetto L, Porras P, Raghunath A, RicardBlum S, Roechert B, Stutz A, Tognolli M, van Roey K, Cesareni G, Hermjakob H. The MIntAct projectIntAct as a common curation platform for 11 molecular interaction databases. Nucleic Acids Res. 2013;42((D1)):358\u201363. https:\/\/doi.org\/10.1093\/nar\/gkt1115.","journal-title":"Nucleic Acids Res."},{"key":"4504_CR4","doi-asserted-by":"publisher","DOI":"10.1093\/database\/bav009","author":"S Orchard","year":"2015","unstructured":"Orchard S, Hermjakob H. Shared resources, shared costs-leveraging biocuration resources. Database. 2015. https:\/\/doi.org\/10.1093\/database\/bav009.","journal-title":"Database"},{"issue":"D1","key":"4504_CR5","doi-asserted-by":"publisher","first-page":"506","DOI":"10.1093\/nar\/gky1049","volume":"47","author":"TU Consortium","year":"2018","unstructured":"Consortium TU. UniProt: a worldwide hub of protein knowledge. Nucleic Acids Res. 2018;47(D1):506\u201315. https:\/\/doi.org\/10.1093\/nar\/gky1049.","journal-title":"Nucleic Acids Res."},{"key":"4504_CR6","unstructured":"Lakshminarayanan B, Pritzel A, Blundell C. Simple and scalable predictive uncertainty estimation using deep ensembles. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in Neural Information Processing Systems, vol 30. Curran Associates, Inc.; 2017, pp 6402\u20136413. https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/9ef2ed4b7fd2c810847ffa5fa85bce38-Paper.pdf"},{"key":"4504_CR7","unstructured":"Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: Proceedings of the 34th International Conference on Machine Learning, vol 70. ICML\u201917, pp 1321\u20131330. JMLR.org; 2017"},{"key":"4504_CR8","unstructured":"Craven, M., Kumlien, J.: Constructing biological knowledge bases by extracting information from text sources. In: Proceedings of the 7th International Conference on Intelligent Systems for Molecular Biology. AAAI Press; 1999. p. 77\u201386"},{"key":"4504_CR9","doi-asserted-by":"crossref","unstructured":"Mintz M, Bills S, Snow R, Jurafsky D. Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. Association for Computational Linguistics, Suntec, Singapore; 2009. p. 1003\u20131011https:\/\/www.aclweb.org\/anthology\/P09-1113","DOI":"10.3115\/1690219.1690287"},{"issue":"4","key":"4504_CR10","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","volume":"36","author":"J Lee","year":"2019","unstructured":"Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, Kang J. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2019;36(4):1234\u201340. https:\/\/doi.org\/10.1093\/bioinformatics\/btz682.","journal-title":"Bioinformatics"},{"issue":"2","key":"4504_CR11","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1016\/j.artmed.2004.07.016","volume":"33","author":"R Bunescu","year":"2005","unstructured":"Bunescu R, Ge R, Kate RJ, Marcotte EM, Mooney RJ, Ramani AK, Wong YW. Comparative experiments on learning information extractors for proteins and their interactions. Artif Intell Med. 2005;33(2):139\u201355.","journal-title":"Artif Intell Med"},{"issue":"1","key":"4504_CR12","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1186\/1471-2105-8-50","volume":"8","author":"S Pyysalo","year":"2007","unstructured":"Pyysalo S, Ginter F, Heimonen J, Bj\u00f6rne J, Boberg J, J\u00e4rvinen J, Salakoski T. Bioinfer: a corpus for information extraction in the biomedical domain. BMC Bioinf. 2007;8(1):50.","journal-title":"BMC Bioinf"},{"key":"4504_CR13","unstructured":"Hsieh Y-L, Chang Y-C, Chang N-W, Hsu W-L. Identifying protein-protein interactions in biomedical literature using recurrent neural networks with long short-term memory. In: Proceedings of the 8th International Joint Conference on Natural Language Processing (volume 2: Short Papers), p. 240\u2013245"},{"key":"4504_CR14","doi-asserted-by":"publisher","unstructured":"Peng Y, Lu Z. Deep learning for extracting protein-protein interactions from biomedical literature. In: BioNLP 2017. Association for Computational Linguistics. p. 29\u201338 https:\/\/www.aclweb.org\/anthology\/W17-2304https:\/\/doi.org\/10.18653\/v1\/W17-2304","DOI":"10.18653\/v1\/W17-2304"},{"issue":"D1","key":"4504_CR15","doi-asserted-by":"publisher","first-page":"808","DOI":"10.1093\/nar\/gks1094","volume":"41","author":"A Franceschini","year":"2012","unstructured":"Franceschini A, Szklarczyk D, Frankild S, Kuhn M, Simonovic M, Roth A, Lin J, Minguez P, Bork P, von Mering C, Jensen LJ. STRING v9.1: protein-protein interaction networks, with increased coverage and integration. Nucleic Acids Res. 2012;41(D1):808\u201315. https:\/\/doi.org\/10.1093\/nar\/gks1094.","journal-title":"Nucleic Acids Res"},{"key":"4504_CR16","doi-asserted-by":"publisher","DOI":"10.1093\/database\/bav020","author":"CO Tudor","year":"2015","unstructured":"Tudor CO, Ross KE, Li G, Vijay-Shanker K, Wu CH, Arighi CN. Construction of phosphorylation interaction networks by text mining of full-length articles using the eFIP system. Database. 2015. https:\/\/doi.org\/10.1093\/database\/bav020.","journal-title":"Database"},{"issue":"1","key":"4504_CR17","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1109\/TCBB.2014.2372765","volume":"12","author":"M Torii","year":"2015","unstructured":"Torii M, Arighi CN, Li G, Wang Q, Wu CH, VijayShanker K. Rlims-p 2.0: A generalizable rule-based information extraction system for literature mining of protein phosphorylation information. IEEE\/ACM Trans Comput Biol Bioinf. 2015;12(1):17\u201329. https:\/\/doi.org\/10.1109\/TCBB.2014.2372765.","journal-title":"IEEE\/ACM Trans Comput Biol Bioinf"},{"issue":"D1","key":"4504_CR18","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1093\/nar\/gky1131","volume":"47","author":"D Szklarczyk","year":"2018","unstructured":"Szklarczyk D, Gable AL, Lyon D, Junge A, Wyder S, Huerta-Cepas J, Simonovic M, Doncheva NT, Morris JH, Bork P, Jensen LJ, Mering C. STRING v11: protein-protein association networks with increased coverage, supporting functional discovery in genome-wide experimental datasets. Nucleic Acids Res. 2018;47(D1):607\u201313. https:\/\/doi.org\/10.1093\/nar\/gky1131.","journal-title":"Nucleic Acids Res."},{"key":"4504_CR19","doi-asserted-by":"publisher","DOI":"10.1093\/database\/bay122","author":"Q Chen","year":"2018","unstructured":"Chen Q, Panyam NC, Elangovan A, Verspoor K. BioCreative VI precision medicine track system performance is constrained by entity recognition and variations in corpus characteristics. Database. 2018. https:\/\/doi.org\/10.1093\/database\/bay122.","journal-title":"Database"},{"issue":"D1","key":"4504_CR20","doi-asserted-by":"publisher","first-page":"542","DOI":"10.1093\/nar\/gkx1104","volume":"46","author":"H Huang","year":"2017","unstructured":"Huang H, Arighi CN, Ross KE, Ren J, Li G, Chen S-C, Wang Q, Cowart J, Vijay-Shanker K, Wu CH. iPTMnet: an integrated resource for protein post-translational modification network discovery. Nucleic Acids Res. 2017;46(D1):542\u201350. https:\/\/doi.org\/10.1093\/nar\/gkx1104.","journal-title":"Nucleic Acids Res"},{"issue":"D1","key":"4504_CR21","doi-asserted-by":"publisher","first-page":"512","DOI":"10.1093\/nar\/gku1267","volume":"43","author":"PV Hornbeck","year":"2014","unstructured":"Hornbeck PV, Zhang B, Murray B, Kornhauser JM, Latham V, Skrzypek E. PhosphoSitePlus, 2014: mutations, PTMs and recalibrations. Nucleic Acids Res. 2014;43(D1):512\u201320. https:\/\/doi.org\/10.1093\/nar\/gku1267.","journal-title":"Nucleic Acids Res"},{"key":"4504_CR22","unstructured":"Bj\u00f6rne J, Van\u00a0Landeghem S, Pyysalo S, Ohta T, Ginter F, Van\u00a0de Peer Y, Ananiadou S, Salakoski T. PubMed-scale event extraction for post-translational modifications, epigenetics and protein structural relations. In: BioNLP: Proceedings of the 2012 Workshop on Biomedical Natural Language Processing. Association for Computational Linguistics, Montr\u00e9al, Canada; 2012. p. 82\u201390. https:\/\/aclanthology.org\/W12-2410"},{"key":"4504_CR23","doi-asserted-by":"crossref","unstructured":"Elangovan A, He J, Verspoor K. Memorization vs. generalization: quantifying data leakage in NLP performance evaluation. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Association for Computational Linguistics, Online; 2021. p. 1325\u20131335. https:\/\/www.aclweb.org\/anthology\/2021.eacl-main.113","DOI":"10.18653\/v1\/2021.eacl-main.113"},{"issue":"9","key":"4504_CR24","doi-asserted-by":"publisher","first-page":"489","DOI":"10.1016\/S2589-7500(20)30186-2","volume":"2","author":"J Futoma","year":"2020","unstructured":"Futoma J, Simons M, Panch T, Doshi-Velez F, Celi LA. The myth of generalisability in clinical research and machine learning in health care. Lancet Digital Health. 2020;2(9):489\u201392. https:\/\/doi.org\/10.1016\/S2589-7500(20)30186-2.","journal-title":"Lancet Digital Health"},{"key":"4504_CR25","doi-asserted-by":"publisher","unstructured":"Li G, Wu C, Vijay-Shanker K. Noise reduction methods for distantly supervised biomedical relation extraction. In: BioNLP 2017. Association for Computational Linguistics, Vancouver, Canada; 2017. p. 184\u2013193. https:\/\/doi.org\/10.18653\/v1\/W17-2323. https:\/\/www.aclweb.org\/anthology\/W17-2323","DOI":"10.18653\/v1\/W17-2323"},{"key":"4504_CR26","doi-asserted-by":"publisher","first-page":"89354","DOI":"10.1109\/ACCESS.2019.2927253","volume":"7","author":"H Zhang","year":"2019","unstructured":"Zhang H, Guan R, Zhou F, Liang Y, Zhan Z-H, Huang L, Feng X. Deep residual convolutional neural network for protein-protein interaction extraction. IEEE Access. 2019;7:89354\u201365. https:\/\/doi.org\/10.1109\/ACCESS.2019.2927253.","journal-title":"IEEE Access"},{"issue":"4","key":"4504_CR27","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1038\/nmeth.1931","volume":"9","author":"S Orchard","year":"2012","unstructured":"Orchard S, Kerrien S, Abbani S, Aranda B, Bhate J, Bidwell S, Bridge A, Briganti L, Brinkman FS, Cesareni G. Protein interaction data curation: the international molecular exchange (imex) consortium. Nature Methods. 2012;9(4):345.","journal-title":"Nature Methods"},{"key":"4504_CR28","doi-asserted-by":"crossref","unstructured":"Wei C-H, Kao H-Y, Lu Z. Gnormplus: an integrative approach for tagging genes, gene families, and protein domains. BioMed Res Int 2015 (2015)","DOI":"10.1155\/2015\/918710"},{"issue":"D1","key":"4504_CR29","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1093\/nar\/gku1055","volume":"43","author":"GR Brown","year":"2014","unstructured":"Brown GR, Hem V, Katz KS, Ovetsky M, Wallin C, Ermolaeva O, Tolstoy I, Tatusova T, Pruitt KD, Maglott DR, Murphy TD. Gene: a gene-centered information resource at NCBI. Nucleic Acids Res. 2014;43(D1):36\u201342. https:\/\/doi.org\/10.1093\/nar\/gku1055.","journal-title":"Nucleic Acids Res"},{"key":"4504_CR30","doi-asserted-by":"publisher","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K. Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics. p. 4171\u20134186. https:\/\/www.aclweb.org\/anthology\/N19-1423https:\/\/doi.org\/10.18653\/v1\/N19-1423","DOI":"10.18653\/v1\/N19-1423"},{"issue":"56","key":"4504_CR31","first-page":"1929","volume":"15","author":"N Srivastava","year":"2014","unstructured":"Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15(56):1929\u201358.","journal-title":"J Mach Learn Res"},{"key":"4504_CR32","doi-asserted-by":"crossref","unstructured":"Yadav S, Ekbal A, Saha S, Kumar A, Bhattacharyya P. Feature assisted stacked attentive shortest dependency path based bi-lstm model for protein-protein interaction. Knowl Based Syst (2018)","DOI":"10.1016\/j.knosys.2018.11.020"},{"key":"4504_CR33","doi-asserted-by":"publisher","unstructured":"McCoy RT, Min J, Linzen T. BERTs of a feather do not generalize together: large variability in generalization across models with similar test set performance. In: Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP. Association for Computational Linguistics, Online; 2020. p. 217\u2013227. https:\/\/doi.org\/10.18653\/v1\/2020.blackboxnlp-1.21. https:\/\/aclanthology.org\/2020.blackboxnlp-1.21","DOI":"10.18653\/v1\/2020.blackboxnlp-1.21"},{"issue":"3","key":"4504_CR34","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1007\/s10994-021-05946-3","volume":"110","author":"E H\u00fcllermeier","year":"2021","unstructured":"H\u00fcllermeier E, Waegeman W. Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Mach Learn. 2021;110(3):457\u2013506.","journal-title":"Mach Learn"},{"key":"4504_CR35","unstructured":"Kendall A, Gal Y. What uncertainties do we need in bayesian deep learning for computer vision? In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc.; 2017. https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/2650d6089a6d640c5e85b2b88265dc2b-Paper.pdf"},{"key":"4504_CR36","doi-asserted-by":"publisher","DOI":"10.1093\/database\/bax040","author":"L Mottin","year":"2017","unstructured":"Mottin L, Pasche E, Gobeill J, Rech\u00a0de Laval V, Gleizes A, Michel P-A, Bairoch A, Gaudet P, Ruch P. Triage by ranking to support the curation of protein interactions. Database. 2017. https:\/\/doi.org\/10.1093\/database\/bax040.","journal-title":"Database"},{"key":"4504_CR37","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1016\/j.ijhcs.2019.05.008","volume":"131","author":"R Raisamo","year":"2019","unstructured":"Raisamo R, Rakkolainen I, Majaranta P, Salminen K, Rantala J, Farooq A. Human augmentation: past, present and future (50 years of the International Journal of Human-Computer Studies. Reflections on the past, present and future of human-centred technologies). Int J Human-Comput Stud. 2019;131:131\u201343. https:\/\/doi.org\/10.1016\/j.ijhcs.2019.05.008.","journal-title":"Int J Human-Comput Stud."}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-021-04504-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-021-04504-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-021-04504-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,1,4]],"date-time":"2022-01-04T12:26:51Z","timestamp":1641299211000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-021-04504-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,4]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["4504"],"URL":"https:\/\/doi.org\/10.1186\/s12859-021-04504-x","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-898489\/v1","asserted-by":"object"}]},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,4]]},"assertion":[{"value":"12 September 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 November 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 January 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"We certify that there are no financial and non-financial competing interests regarding the materials discussed in the manuscript.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"4"}}