{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T18:04:32Z","timestamp":1787508272559,"version":"build-2736575974"},"reference-count":28,"publisher":"Public Library of Science (PLoS)","issue":"11","license":[{"start":{"date-parts":[[2024,11,26]],"date-time":"2024-11-26T00:00:00Z","timestamp":1732579200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001711","name":"Schweizerischer Nationalfonds zur F\u00f6rderung der Wissenschaftlichen Forschung","doi-asserted-by":"publisher","award":["407940_206504"],"award-info":[{"award-number":["407940_206504"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000065","name":"NINDS","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100006447","name":"Universit\u00e4t Z\u00fcrich","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100006447","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:sec id=\"sec001\">\n                    <jats:title>Background and methods<\/jats:title>\n                    <jats:p>\n                      Systematic reviews, i.e., research summaries that address focused questions in a structured and reproducible manner, are a cornerstone of evidence-based medicine and research. However, certain steps in systematic reviews, such as data extraction, are labour-intensive, which hampers their feasibility, especially with the rapidly expanding body of biomedical literature. To bridge this gap, we aimed to develop a data mining tool in the R programming environment to automate data extraction from neuroscience\n                      <jats:italic>in vivo<\/jats:italic>\n                      publications. The function was trained on a literature corpus (n = 45 publications) of animal motor neuron disease studies and tested in two validation corpora (motor neuron diseases, n = 31 publications; multiple sclerosis, n = 244 publications).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec002\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Our data mining tool, STEED (STructured Extraction of Experimental Data), successfully extracted key experimental parameters such as animal models and species, as well as risk of bias items like randomization or blinding, from\n                      <jats:italic>in vivo<\/jats:italic>\n                      studies. Sensitivity and specificity were over 85% and 80%, respectively, for most items in both validation corpora. Accuracy and F1-score were above 90% and 0.9 for most items in the validation corpora, respectively. Time savings were above 99%.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec003\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>\n                      Our text mining tool, STEED, can extract key experimental parameters and risk of bias items from the neuroscience\n                      <jats:italic>in vivo<\/jats:italic>\n                      literature. This enables the tool\u2019s deployment for probing a field in a research improvement context or replacing one human reader during data extraction, resulting in substantial time savings and contributing towards the automation of systematic reviews.\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.1371\/journal.pone.0311358","type":"journal-article","created":{"date-parts":[[2024,11,26]],"date-time":"2024-11-26T14:00:45Z","timestamp":1732629645000},"page":"e0311358","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["STEED: A data mining tool for automated extraction of experimental parameters and risk of bias items from in vivo publications"],"prefix":"10.1371","volume":"19","author":[{"given":"Wolfgang Emanuel","family":"Zurrer","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amelia Elaine","family":"Cannon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ewoud","family":"Ewing","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1476-3619","authenticated-orcid":true,"given":"David","family":"Br\u00fcschweiler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julia","family":"Bugajska","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bernard Friedrich","family":"Hild","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marianna","family":"Rosso","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2628-4334","authenticated-orcid":true,"given":"Daniel Salo","family":"Reich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1362-4819","authenticated-orcid":true,"given":"Benjamin Victor","family":"Ineichen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2024,11,26]]},"reference":[{"issue":"4","key":"pone.0311358.ref001","doi-asserted-by":"crossref","first-page":"498","DOI":"10.1038\/s41559-020-1153-2","article-title":"A new ecosystem for evidence synthesis.","volume":"4","author":"S Nakagawa","year":"2020","journal-title":"Nature Ecology & Evolution"},{"key":"pone.0311358.ref002","doi-asserted-by":"crossref","DOI":"10.1002\/9781119099369","volume-title":"Systematic reviews in health research: Meta-analysis in context","author":"M Egger","year":"2022"},{"key":"pone.0311358.ref003","doi-asserted-by":"crossref","DOI":"10.1002\/9781119536604","volume-title":"Cochrane handbook for systematic reviews of interventions","author":"JP Higgins","year":"2019"},{"issue":"9","key":"pone.0311358.ref004","doi-asserted-by":"crossref","first-page":"1949","DOI":"10.1097\/j.pain.0000000000001974","article-title":"A practical guide to preclinical systematic review and meta-analysis","volume":"161","author":"N Soliman","year":"2020","journal-title":"Pain"},{"issue":"5","key":"pone.0311358.ref005","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1177\/01410768221093551","article-title":"The role of systematic reviews in identifying the limitations of preclinical animal research, 2000\u20132022: part 1.","volume":"115","author":"M Ritskes-Hoitinga","year":"2022","journal-title":"Journal of the Royal Society of Medicine"},{"issue":"1","key":"pone.0311358.ref006","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1152\/physrev.00028.2022","article-title":"Systematic reviews for basic scientists: a different beast","volume":"103","author":"JP Ioannidis","year":"2022","journal-title":"Physiological reviews"},{"issue":"1","key":"pone.0311358.ref007","doi-asserted-by":"crossref","first-page":"e100103","DOI":"10.1136\/bmjos-2020-100103","article-title":"Development and uptake of an online systematic review platform: the early years of the CAMARADES Systematic Review Facility (SyRF).","volume":"5","author":"Z Bahor","year":"2021","journal-title":"BMJ Open Science"},{"issue":"12","key":"pone.0311358.ref008","doi-asserted-by":"crossref","first-page":"1163","DOI":"10.3390\/ani9121163","article-title":"How can systematic reviews teach us more about the implementation of the 3Rs and animal welfare?","volume":"9","author":"M Ritskes-Hoitinga","year":"2019","journal-title":"Animals"},{"issue":"1","key":"pone.0311358.ref009","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1093\/ilar\/ilz015","article-title":"Reproducibility and rigor in animal-based research","volume":"60","author":"M Macleod","year":"2019","journal-title":"ILAR journal"},{"issue":"2","key":"pone.0311358.ref010","doi-asserted-by":"crossref","first-page":"e012545","DOI":"10.1136\/bmjopen-2016-012545","article-title":"Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry","volume":"7","author":"R Borah","year":"2017","journal-title":"BMJ open"},{"issue":"10","key":"pone.0311358.ref011","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1038\/s41684-023-01256-4","article-title":"From data deluge to publomics: How AI can transform animal research.","volume":"52","author":"BV Ineichen","year":"2023","journal-title":"Lab Anim (NY)."},{"issue":"11","key":"pone.0311358.ref012","doi-asserted-by":"crossref","first-page":"2215","DOI":"10.1002\/asi.23329","article-title":"Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references","volume":"66","author":"L Bornmann","year":"2015","journal-title":"Journal of the Association for Information Science and Technology"},{"issue":"151","key":"pone.0311358.ref013","doi-asserted-by":"crossref","first-page":"151ps15","DOI":"10.1126\/scitranslmed.3004631","article-title":"Extrapolating from animals to humans","volume":"4","author":"JP Ioannidis","year":"2012","journal-title":"Science translational medicine"},{"issue":"1","key":"pone.0311358.ref014","doi-asserted-by":"crossref","first-page":"e100131","DOI":"10.1136\/bmjos-2020-100131","article-title":"Technological advances in preclinical meta-research","volume":"5","author":"A Bannach-Brown","year":"2021","journal-title":"BMJ Open Science"},{"key":"pone.0311358.ref015","first-page":"1","article-title":"Semi-Automated Evidence Synthesis in Health Psychology: Current Methods and Future Prospects.","author":"IJ Marshall","year":"2020","journal-title":"Health psychology review"},{"key":"pone.0311358.ref016","doi-asserted-by":"crossref","first-page":"1135282","DOI":"10.3389\/fvets.2023.1135282","article-title":"Neuroimaging findings in preclinical amyotrophic lateral sclerosis models\u2014How well do they mimic the clinical phenotype? A systematic review","volume":"10","author":"AE Cannon","year":"2023","journal-title":"Frontiers in Veterinary Science"},{"key":"pone.0311358.ref017","doi-asserted-by":"crossref","first-page":"102371","DOI":"10.1016\/j.nicl.2020.102371","article-title":"Magnetic resonance imaging in multiple sclerosis animal models: A systematic review, meta-analysis, and white paper","author":"BV Ineichen","year":"2020","journal-title":"NeuroImage: Clinical"},{"issue":"9","key":"pone.0311358.ref018","doi-asserted-by":"crossref","first-page":"1769","DOI":"10.1177\/0271678X20943823","article-title":"The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research.","volume":"40","author":"N Percie du Sert","year":"2020","journal-title":"Journal of Cerebral Blood Flow & Metabolism"},{"issue":"1","key":"pone.0311358.ref019","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1177\/0165551520932855","article-title":"Exploring research trends in big data across disciplines: A text mining analysis","volume":"48","author":"E Mohammadi","year":"2022","journal-title":"Journal of Information Science"},{"key":"pone.0311358.ref020","article-title":"Risk of bias assessment in preclinical literature using natural language processing.","author":"Q Wang","year":"2021","journal-title":"Res Synth Methods"},{"issue":"1","key":"pone.0311358.ref021","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1186\/s13643-019-1074-9","article-title":"Toward systematic review automation: a practical guide to using machine learning tools in research synthesis.","volume":"8","author":"IJ Marshall","year":"2019","journal-title":"Syst Rev"},{"issue":"12","key":"pone.0311358.ref022","doi-asserted-by":"crossref","first-page":"e0226176","DOI":"10.1371\/journal.pone.0226176","article-title":"Menagerie: A text-mining tool to support animal-human translation in neurodegeneration research","volume":"14","author":"CJ Zeiss","year":"2019","journal-title":"PloS one"},{"issue":"11","key":"pone.0311358.ref023","doi-asserted-by":"crossref","first-page":"101698","DOI":"10.1016\/j.isci.2020.101698","article-title":"The Rigor and Transparency Index quality metric for assessing biological and medical science methods","volume":"23","author":"J Menke","year":"2020","journal-title":"Iscience"},{"issue":"20","key":"pone.0311358.ref024","doi-asserted-by":"crossref","first-page":"2525","DOI":"10.1042\/CS20160722","article-title":"Risk of bias reporting in the recent animal focal cerebral ischaemia literature","volume":"131","author":"Z Bahor","year":"2017","journal-title":"Clinical Science"},{"key":"pone.0311358.ref025","article-title":"Protocol for an analysis of in vivo reporting standards by journal, institution and funder","author":"Q Wang","year":"2021","journal-title":"OSF (https:\/\/osfio\/preprints\/metaarxiv\/cjxtf\/)."},{"key":"pone.0311358.ref026","article-title":"Can large language models replace humans in the systematic review process? Evaluating GPT-4\u2019s efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages","author":"Q Khraisha","year":"2023","journal-title":"arXiv preprint arXiv:231017526"},{"issue":"1","key":"pone.0311358.ref027","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1038\/s41746-023-00896-7","article-title":"Evaluating large language models on medical evidence summarization","volume":"6","author":"L Tang","year":"2023","journal-title":"npj Digital Medicine"},{"key":"pone.0311358.ref028","article-title":"Space-junk spear, depression drug and the EU\u2019s digital copyright","author":"brief NSD-Tni","year":"2019"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0311358","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,26]],"date-time":"2024-11-26T14:00:55Z","timestamp":1732629655000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0311358"}},"subtitle":[],"editor":[{"given":"John","family":"Blake","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2024,11,26]]},"references-count":28,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11,26]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0311358","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,26]]}}}