{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T14:56:50Z","timestamp":1776092210696,"version":"3.50.1"},"reference-count":55,"publisher":"Oxford University Press (OUP)","license":[{"start":{"date-parts":[[2024,8,30]],"date-time":"2024-08-30T00:00:00Z","timestamp":1724976000000},"content-version":"vor","delay-in-days":242,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006087","name":"Direcci\u00f3n General de Asuntos del Personal Acad\u00e9mico, Universidad Nacional Aut\u00f3noma de M\u00e9xico","doi-asserted-by":"publisher","award":["IN219523"],"award-info":[{"award-number":["IN219523"]}],"id":[{"id":"10.13039\/501100006087","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006087","name":"Direcci\u00f3n General de Asuntos del Personal Acad\u00e9mico, Universidad Nacional Aut\u00f3noma de M\u00e9xico","doi-asserted-by":"publisher","award":["IN219523"],"award-info":[{"award-number":["IN219523"]}],"id":[{"id":"10.13039\/501100006087","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,8,30]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Transcriptional regulatory networks (TRNs) give a global view of the regulatory mechanisms of bacteria to respond to environmental signals. These networks are published in biological databases as a valuable resource for experimental and bioinformatics researchers. Despite the efforts to publish TRNs of diverse bacteria, many of them still lack one and many of the existing TRNs are incomplete. In addition, the manual extraction of information from biomedical literature (\u201cliterature curation\u201d) has been the traditional way to extract these networks, despite this being demanding and time-consuming. Recently, language models based on pretrained transformers have been used to extract relevant knowledge from biomedical literature. Moreover, the benefit of fine-tuning a large pretrained model with new limited data for a specific task (\u201ctransfer learning\u201d) opens roads to address new problems of biomedical information extraction. Here, to alleviate this lack of knowledge and assist literature curation, we present a new approach based on the Bidirectional Transformer for Language Understanding (BERT) architecture to classify transcriptional regulatory interactions of bacteria as a first step to extract TRNs from literature. The approach achieved a significant performance in a test dataset of sentences of Escherichia coli (F1-Score: 0.8685, Matthew\u2019s correlation coefficient: 0.8163). The examination of model predictions revealed that the model learned different ways to express the regulatory interaction. The approach was evaluated to extract a TRN of Salmonella using 264 complete articles. The evaluation showed that the approach was able to accurately extract 82% of the network and that it was able to extract interactions absent in curation data. To the best of our knowledge, the present study is the first effort to obtain a BERT-based approach to extract this specific kind of interaction. This approach is a starting point to address the limitations of reconstructing TRNs of bacteria and diseases of biological interest.<\/jats:p>\n               <jats:p>Database URL: https:\/\/github.com\/laigen-unam\/BERT-trn-extraction.<\/jats:p>","DOI":"10.1093\/database\/baae094","type":"journal-article","created":{"date-parts":[[2024,8,30]],"date-time":"2024-08-30T20:04:40Z","timestamp":1725048280000},"source":"Crossref","is-referenced-by-count":5,"title":["Automatic extraction of transcriptional regulatory interactions of bacteria from biomedical literature using a BERT-based approach"],"prefix":"10.1093","volume":"2024","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4857-4997","authenticated-orcid":false,"given":"Alfredo","family":"Varela-Vega","sequence":"first","affiliation":[{"name":"Programa de Gen\u00f3mica Computacional, Centro de Ciencias Gen\u00f3micas, UNAM , Av. Universidad S\/N Col. Chamilpa, Cuernavaca, Morelos 62210, M\u00e9xico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3165-6099","authenticated-orcid":false,"given":"Ali-Berenice","family":"Posada-Reyes","sequence":"additional","affiliation":[{"name":"Laboratorio de Microbiolog\u00eda, Inmunolog\u00eda y Salud P\u00fablica, Facultad de Estudios Superiores Cuautitl\u00e1n, UNAM , Carretera Cuautitl\u00e1n-Teoloyucan Km. 2.5, Xhala, Cuautitl\u00e1n Izcalli, Estado de M\u00e9xico 54714, M\u00e9xico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2549-1614","authenticated-orcid":false,"given":"Carlos-Francisco","family":"M\u00e9ndez-Cruz","sequence":"additional","affiliation":[{"name":"Programa de Gen\u00f3mica Computacional, Centro de Ciencias Gen\u00f3micas, UNAM , Av. Universidad S\/N Col. Chamilpa, Cuernavaca, Morelos 62210, M\u00e9xico"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2024,8,30]]},"reference":[{"key":"2024083016301690100_R1","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.gpb.2015.01.006","article-title":"Biological databases for human research","volume":"13","author":"Zou","year":"2015","journal-title":"Genomic, Proteomics Bioinf"},{"key":"2024083016301690100_R2","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1093\/bfgp\/elu015","article-title":"Event-based text mining for biology and functional genomics","volume":"14","author":"Ananiadou","year":"2014","journal-title":"Briefings Funct Genomics"},{"key":"2024083016301690100_R3","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1093\/bib\/bbv024","article-title":"Community challenges in biomedical text mining over 10 years: success, failure and the future","volume":"17","author":"Huang","year":"2015","journal-title":"Briefings Bioinf"},{"key":"2024083016301690100_R4","doi-asserted-by":"crossref","DOI":"10.1016\/j.jbi.2021.103982","article-title":"Ammu: A survey of transformer-based biomedical pretrained language models","volume":"126","author":"Kalyan","year":"2022","journal-title":"J Biomed Inf"},{"key":"2024083016301690100_R5","article-title":"Recent advances in biomedical literature mining","volume":"22","author":"Zhao","year":"2020","journal-title":"Briefings Bioinf"},{"key":"2024083016301690100_R6","first-page":"4171","article-title":"BERT: pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2019"},{"key":"2024083016301690100_R7","doi-asserted-by":"crossref","first-page":"856","DOI":"10.1093\/bib\/bbt006","article-title":"Biological network extraction from scientific literature: state of the art and challenges","volume":"15","author":"Li","year":"2013","journal-title":"Briefings Bioinf"},{"key":"2024083016301690100_R8","volume-title":"The Cell: A Molecular Approach","author":"Cooper","year":"2000"},{"key":"2024083016301690100_R9","doi-asserted-by":"crossref","DOI":"10.1099\/mgen.0.000833","article-title":"Regulondb 11.0: Comprehensive high-throughput datasets on transcriptional regulation in Escherichia coli k-12","volume":"8","author":"Tierrafr\u00eda","year":"2022","journal-title":"Microb Genomics"},{"key":"2024083016301690100_R10","doi-asserted-by":"crossref","DOI":"10.1038\/s41540-017-0034-z","article-title":"Salmonet, an integrated network of ten salmonella enterica strains reveals common and distinct pathways to host adaptation","volume":"3","author":"M\u00e9tris","year":"2017","journal-title":"NPJ Syst Biol Appl"},{"key":"2024083016301690100_R11","doi-asserted-by":"crossref","first-page":"1228","DOI":"10.1016\/j.csbj.2020.05.015","article-title":"Abasy atlas v2.2: The most comprehensive and up-to-date inventory of meta-curated, historical, bacterial regulatory networks, their completeness and system-level characterization","volume":"18","author":"Escorcia-Rodr\u00edguez","year":"2020","journal-title":"Comput Struct Biotechnol J"},{"key":"2024083016301690100_R12","doi-asserted-by":"crossref","DOI":"10.3390\/ijms23042179","article-title":"Bacterial transcriptional regulators: a road map for functional, structural, and biophysical characterization","volume":"23","author":"Pis Diez","year":"2022","journal-title":"Int J Mol Sci"},{"key":"2024083016301690100_R13","doi-asserted-by":"crossref","first-page":"6857","DOI":"10.1128\/jb.178.23.6857-6864.1996","article-title":"Phop-phoq activates transcription of pmrab, encoding a two-component regulatory system involved in Salmonella typhimurium antimicrobial peptide resistance","volume":"178","author":"Gunn","year":"1996","journal-title":"J Bacteriol"},{"key":"2024083016301690100_R14","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1006\/jmbi.1994.0138","article-title":"The complex formed between tet repressor and tetracyclinemg2\u2014 ihsbop\u2014+reveals mechanism of antibiotic resistance","volume":"247","author":"Kisker","year":"1995","journal-title":"J Mol Biol"},{"key":"2024083016301690100_R15","doi-asserted-by":"crossref","DOI":"10.1038\/s42003-022-03336-6","article-title":"Nutrient and salt depletion synergistically boosts glucose metabolism in individual Escherichia coli cells","volume":"5","author":"Glover","year":"2022","journal-title":"Commun Biol"},{"key":"2024083016301690100_R16","article-title":"Editorial: transcription and its regulation in bacteria","volume":"14","author":"Wang","year":"2023","journal-title":"Front Microbiol"},{"key":"2024083016301690100_R17","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1038\/nrmicro787","article-title":"The regulation of bacterial transcription initiation","volume":"2","author":"Browning","year":"2004","journal-title":"Nat Rev Microbiol"},{"key":"2024083016301690100_R18","first-page":"58","article-title":"Transfer learning in biomedical natural language processing: an evaluation of BERT and ELMo on ten benchmarking datasets","author":"Peng","year":"2019"},{"key":"2024083016301690100_R19","doi-asserted-by":"crossref","DOI":"10.1186\/s12859-023-05520-9","article-title":"Deep learning-enabled natural language processing to identify directional pharmacokinetic drug\u2013drug interactions","volume":"24","author":"Zirkle","year":"2023","journal-title":"BMC Bioinf"},{"key":"2024083016301690100_R20","doi-asserted-by":"crossref","DOI":"10.1186\/s12859-022-04642-w","article-title":"Investigation of improving the pretraining and fine-tuning of bert model for biomedical relation extraction","volume":"23","author":"Su","year":"2022","journal-title":"BMC Bioinf"},{"key":"2024083016301690100_R21","doi-asserted-by":"crossref","DOI":"10.1186\/s12859-023-05236-w","article-title":"Automatic extraction of ranked snp-phenotype associations from text using a BERT-LSTM-based method","volume":"24","author":"Bokharaeian","year":"2023","journal-title":"BMC Bioinf"},{"key":"2024083016301690100_R22","doi-asserted-by":"crossref","DOI":"10.1093\/database\/baab077","article-title":"Automated extraction of genes associated with antibiotic resistance from the biomedical literature","volume":"2022","author":"Brincat","year":"2022","journal-title":"Database"},{"key":"2024083016301690100_R23","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/978-1-0716-2305-3_12","volume-title":"Biomedical Text Mining","author":"Bhasuran","year":"2022"},{"key":"2024083016301690100_R24","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"2024083016301690100_R25","first-page":"8003","article-title":"LinkBERT: pretraining language models with document links","author":"Yasunaga","year":"2022"},{"key":"2024083016301690100_R26","first-page":"4700","article-title":"BioMegatron: larger biomedical domain language model","author":"Shin","year":"2020"},{"key":"2024083016301690100_R27","first-page":"8342","article-title":"Don\u2019t stop pretraining: adapt language models to domains and tasks","author":"Gururangan","year":"2020"},{"key":"2024083016301690100_R28","first-page":"6442","article-title":"LUKE: Deep contextualized entity representations with entity aware self-attention","author":"Yamada","year":"2020"},{"key":"2024083016301690100_R29","doi-asserted-by":"crossref","DOI":"10.1093\/database\/bau049","article-title":"Assisted curation of regulatory interactions and growth conditions of OxyR in E. coli K-12","volume":"2014","author":"Gama-Castro","year":"2014","journal-title":"Database"},{"key":"2024083016301690100_R30","first-page":"35","article-title":"Position-aware attention and supervised data improve slot filling","author":"Zhang","year":"2017"},{"key":"2024083016301690100_R31","first-page":"6000","article-title":"Attention is all you need","author":"Vaswani","year":"2017"},{"key":"2024083016301690100_R32","first-page":"5484","article-title":"Transformer feed-forward layers are key-value memories","author":"Geva","year":"2021"},{"key":"2024083016301690100_R33","doi-asserted-by":"crossref","DOI":"10.1186\/s40537-022-00652-w","article-title":"Transfer learning: a friendly introduction","volume":"9","author":"Hosna","year":"2022","journal-title":"J Big Data"},{"key":"2024083016301690100_R34","article-title":"Roberta: a robustly optimized BERT pretraining approach","author":"Liu","year":"2019","journal-title":"arXiv"},{"key":"2024083016301690100_R35","first-page":"157","article-title":"Drug protein interaction extraction using Scibert based deep learning model","author":"GabAllah","year":"2022"},{"key":"2024083016301690100_R36","article-title":"On optimal early stopping: over-informative versus under-informative parametrization","author":"Shen","year":"2022","journal-title":"arXiv"},{"key":"2024083016301690100_R37","first-page":"1","article-title":"Cross-entropy loss functions: theoretical analysis and applications","author":"Mao","year":"2023"},{"key":"2024083016301690100_R38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-021-00444-8","article-title":"Review of deep learning: concepts, cnn architectures, challenges, applications, future directions","volume":"8","author":"Alzubaidi","year":"2021","journal-title":"J Big Data"},{"key":"2024083016301690100_R39","volume-title":"Introduction to Information Retrieval","author":"Christopher","year":"2008"},{"key":"2024083016301690100_R40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13040-021-00244-z","article-title":"The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation","volume":"14","author":"Chicco","year":"2021","journal-title":"BioData Min"},{"key":"2024083016301690100_R41","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1128\/CMR.00066-12","article-title":"Salmonella enterica serovar typhimurium skills to succeed in the host: virulence and regulation","volume":"26","author":"F\u00e0brega","year":"2013","journal-title":"Clin Microbiol Rev"},{"key":"2024083016301690100_R42","first-page":"55","article-title":"The Stanford CoreNLP natural language processing toolkit","author":"Manning","year":"2014"},{"key":"2024083016301690100_R43","first-page":"41","article-title":"Dual control by regulators, GntH and GntR, of the GntII genes for gluconate metabolism in Escherichia coli","volume":"6","author":"Tsunedomi","year":"2003","journal-title":"J Mol Microbiol Biotechnol"},{"key":"2024083016301690100_R44","doi-asserted-by":"crossref","first-page":"752","DOI":"10.1073\/pnas.78.2.752","article-title":"Mechanism of Arac autoregulation and the domains of two overlapping promoters, pc and pbad, in the l-arabinose regulatory region of Escherichia coli","volume":"78","author":"Lee","year":"1981","journal-title":"Proc Natl Acad Sci"},{"key":"2024083016301690100_R45","first-page":"782","article-title":"Transformer interpretability beyond attention visualization","author":"Chefer","year":"2021"},{"key":"2024083016301690100_R46","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1099\/mic.0.2007\/012146-0","article-title":"Characterization of the Escherichia coli k-12 ydhyvwxut operon: regulation by fnr, narl and narp","volume":"154","author":"Partridge","year":"2008","journal-title":"Microbiol"},{"key":"2024083016301690100_R47","article-title":"Bioformer: an efficient transformer language model for biomedical text mining","author":"Fang","year":"2023","journal-title":"arXiv"},{"key":"2024083016301690100_R48","doi-asserted-by":"crossref","first-page":"20742","DOI":"10.1073\/pnas.1108963108","article-title":"Regulation of phenotypic heterogeneity permits Salmonella evasion of the host caspase-1 inflammatory response","volume":"108","author":"Stewart","year":"2011","journal-title":"Proc Natl Acad Sci"},{"key":"2024083016301690100_R49","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1186\/1471-2105-12-393","article-title":"Enriching a biomedical event corpus with meta-knowledge annotation","volume":"12","author":"Thompson","year":"2011","journal-title":"BMC Bioinf"},{"key":"2024083016301690100_R50","first-page":"79","article-title":"Multitask learning of negation and speculation using transformers","author":"Khandelwal","year":"2020"},{"key":"2024083016301690100_R51","doi-asserted-by":"crossref","DOI":"10.3390\/app12105209","article-title":"Negation and speculation in nlp: a survey, corpora, methods, and applications","volume":"12","author":"Mahany","year":"2022","journal-title":"Appl Sci"},{"key":"2024083016301690100_R52","first-page":"26","article-title":"CadC-mediated activation of the cadBA promoter in Escherichia coli","volume":"10","author":"K\u00fcper","year":"2005","journal-title":"J Mol Microbiol Biotechnol"},{"key":"2024083016301690100_R53","doi-asserted-by":"crossref","first-page":"9037","DOI":"10.1074\/jbc.M313602200","article-title":"The Escherichia coli transcriptional regulator mara directly represses transcription of pura and hdea","volume":"279","author":"Schneiders","year":"2004","journal-title":"J Biol Chem"},{"key":"2024083016301690100_R54","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1471-2180-2-13","article-title":"Oxygen and nitrate-dependent regulation of dmsabc operon expression in Escherichia coli: sites for fnr and narl protein interactions","volume":"2","author":"Bearson","year":"2002","journal-title":"BMC Microbiol"},{"key":"2024083016301690100_R55","doi-asserted-by":"crossref","first-page":"6774","DOI":"10.1128\/JB.182.23.6774-6782.2000","article-title":"Interdependence of activation at rhaSR by cyclic amp receptor protein, the RNA polymerase alpha subunit c-terminal domain, and rhar","volume":"182","author":"Holcroft","year":"2000","journal-title":"J Bacteriol"}],"container-title":["Database"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/database\/article-pdf\/doi\/10.1093\/database\/baae094\/58978508\/baae094.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/database\/article-pdf\/doi\/10.1093\/database\/baae094\/58978508\/baae094.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,30]],"date-time":"2024-08-30T20:05:10Z","timestamp":1725048310000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/database\/article\/doi\/10.1093\/database\/baae094\/7745981"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":55,"URL":"https:\/\/doi.org\/10.1093\/database\/baae094","relation":{},"ISSN":["1758-0463"],"issn-type":[{"value":"1758-0463","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]},"article-number":"baae094"}}