{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T03:52:00Z","timestamp":1790999520703,"version":"4.1.0"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,1,3]],"date-time":"2023-01-03T00:00:00Z","timestamp":1672704000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,1,3]],"date-time":"2023-01-03T00:00:00Z","timestamp":1672704000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Syst Rev"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>The importance of systematic reviews in collating and summarising available research output on a particular topic cannot be over-emphasized. However, initial screening of retrieved literature is significantly time and labour intensive. Attempts at automating parts of the systematic review process have been made with varying degree of success partly due to being domain-specific, requiring vendor-specific software or manually labelled training data.<\/jats:p>\n                    <jats:p>Our primary objective was to develop statistical methodology for performing automated title and abstract screening for systematic reviews. Secondary objectives included (1) to retrospectively apply the automated screening methodology to previously manually screened systematic reviews and (2) to characterize the performance of the automated screening methodology scoring algorithm in a simulation study.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We implemented a Latent Dirichlet Allocation-based topic model to derive representative topics from the retrieved documents\u2019 title and abstract. The second step involves defining a score threshold for classifying the documents as relevant for full-text review or not. The score is derived based on a set of search keywords (often the database retrieval search terms). Two systematic review studies were retrospectively used to illustrate the methodology.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      In one case study (helminth dataset),\n                      <jats:inline-formula>\n                        <jats:alternatives>\n                          <jats:tex-math>$$69.83\\%$$<\/jats:tex-math>\n                          <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                            <mml:mrow>\n                              <mml:mn>69.83<\/mml:mn>\n                              <mml:mo>%<\/mml:mo>\n                            <\/mml:mrow>\n                          <\/mml:math>\n                        <\/jats:alternatives>\n                      <\/jats:inline-formula>\n                      sensitivity compared to manual title and abstract screening was achieved. This is against a false positive rate of\n                      <jats:inline-formula>\n                        <jats:alternatives>\n                          <jats:tex-math>$$22.63\\%$$<\/jats:tex-math>\n                          <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                            <mml:mrow>\n                              <mml:mn>22.63<\/mml:mn>\n                              <mml:mo>%<\/mml:mo>\n                            <\/mml:mrow>\n                          <\/mml:math>\n                        <\/jats:alternatives>\n                      <\/jats:inline-formula>\n                      . For the second case study (Wilson disease dataset), a sensitivity of\n                      <jats:inline-formula>\n                        <jats:alternatives>\n                          <jats:tex-math>$$54.02\\%$$<\/jats:tex-math>\n                          <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                            <mml:mrow>\n                              <mml:mn>54.02<\/mml:mn>\n                              <mml:mo>%<\/mml:mo>\n                            <\/mml:mrow>\n                          <\/mml:math>\n                        <\/jats:alternatives>\n                      <\/jats:inline-formula>\n                      and specificity of\n                      <jats:inline-formula>\n                        <jats:alternatives>\n                          <jats:tex-math>$$67.03\\%$$<\/jats:tex-math>\n                          <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                            <mml:mrow>\n                              <mml:mn>67.03<\/mml:mn>\n                              <mml:mo>%<\/mml:mo>\n                            <\/mml:mrow>\n                          <\/mml:math>\n                        <\/jats:alternatives>\n                      <\/jats:inline-formula>\n                      were achieved.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>\n                      Unsupervised title and abstract screening has the potential to reduce the workload involved in conducting systematic review. While sensitivity of the methodology on the tested data is low, approximately\n                      <jats:inline-formula>\n                        <jats:alternatives>\n                          <jats:tex-math>$$70\\%$$<\/jats:tex-math>\n                          <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                            <mml:mrow>\n                              <mml:mn>70<\/mml:mn>\n                              <mml:mo>%<\/mml:mo>\n                            <\/mml:mrow>\n                          <\/mml:math>\n                        <\/jats:alternatives>\n                      <\/jats:inline-formula>\n                      specificity was achieved. Users ought to keep in mind that potentially low sensitivity might occur. One approach to mitigate this might be to incorporate additional targeted search keywords such as the indexing databases terms into the search term copora. Moreover, automated screening can be used as an additional screener to the manual screeners.\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s13643-022-02163-4","type":"journal-article","created":{"date-parts":[[2023,1,3]],"date-time":"2023-01-03T09:04:07Z","timestamp":1672736647000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["Unsupervised title and abstract screening for systematic review: a retrospective case-study using topic modelling methodology"],"prefix":"10.1186","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5639-4972","authenticated-orcid":false,"given":"Agnes","family":"Natukunda","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leacky K.","family":"Muchene","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,1,3]]},"reference":[{"key":"2163_CR1","unstructured":"JPT H, J T, J C, M C, T L, MJ P, et\u00a0al., editors. Cochrane Handbook for Systematic Reviews of Interventions. 2nd ed. Chichester: Wiley; 2019."},{"issue":"3","key":"2163_CR2","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1136\/ebn.2011.0049","volume":"14","author":"J Clarke","year":"2011","unstructured":"Clarke J. What is a systematic review? Evid-Based Nurs. 2011;14(3):64. https:\/\/doi.org\/10.1136\/ebn.2011.0049.","journal-title":"Evid-Based Nurs."},{"issue":"2","key":"2163_CR3","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1177\/0885412219881135","volume":"35","author":"HR Kwon","year":"2019","unstructured":"Kwon HR, Silva EA. Mapping the Landscape of Behavioral Theories: Systematic Literature Review. J Plan Lit. 2019;35(2):161\u201379. https:\/\/doi.org\/10.1177\/0885412219881135.","journal-title":"J Plan Lit."},{"key":"2163_CR4","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1016\/j.envsci.2014.05.010","volume":"42","author":"GS Bilotta","year":"2014","unstructured":"Bilotta GS, Milner AM, Boyd I. On the use of systematic reviews to inform environmental policies. Environ Sci Policy. 2014;42:67\u201377. https:\/\/doi.org\/10.1016\/j.envsci.2014.05.010.","journal-title":"Environ Sci Policy."},{"key":"2163_CR5","doi-asserted-by":"publisher","unstructured":"Zawacki-Richter O, Kerres M, Bedenlier S, Bond M, Buntins K, editors. Systematic Reviews in Educational Research. USA: Springer Fachmedien Wiesbaden; 2020. https:\/\/doi.org\/10.1007\/978-3-658-27602-7.","DOI":"10.1007\/978-3-658-27602-7"},{"issue":"1","key":"2163_CR6","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1002\/jrsm.1071","volume":"4","author":"BT Johnson","year":"2013","unstructured":"Johnson BT, Low RE, LaCroix JM. Systematic Reviews to Support Evidence-based Medicine (2nd edition) by Khalid Khan, Regina Kunz, Jos Kleijnen and Gerd Antes: A Review. Res Synth Methods. 2013;4(1):102\u20138. https:\/\/doi.org\/10.1002\/jrsm.1071.","journal-title":"Res Synth Methods"},{"key":"2163_CR7","doi-asserted-by":"publisher","unstructured":"Konstantinidis I, Siaminos G, Timplalexis C, Zervas P, Peristeras V, Decker S. Blockchain for Business Applications: A Systematic Literature Review. In: Business Information Systems. Cham: Springer International Publishing; 2018. p. 384\u2013399. https:\/\/doi.org\/10.1007\/978-3-319-93931-5_28.","DOI":"10.1007\/978-3-319-93931-5_28"},{"key":"2163_CR8","doi-asserted-by":"publisher","unstructured":"Edwards P, Clarke M, DiGuiseppi C, Pratap S, Roberts I, Wentz R. Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Stat Med. 2002;21(11):1635\u201340. https:\/\/doi.org\/10.1002\/sim.1190. https:\/\/onlinelibrary.wiley.com\/doi\/abs\/10.1002\/sim.1190","DOI":"10.1002\/sim.1190"},{"key":"2163_CR9","doi-asserted-by":"publisher","unstructured":"Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7(2). https:\/\/doi.org\/10.1136\/bmjopen-2016-012545. https:\/\/bmjopen.bmj.com\/content\/7\/2\/e012545.","DOI":"10.1136\/bmjopen-2016-012545"},{"key":"2163_CR10","doi-asserted-by":"publisher","unstructured":"Haddaway NR, Westgate MJ. Predicting the time needed for environmental systematic reviews and systematic maps. Conserv Biol. 2019;33(2):434\u201343. https:\/\/doi.org\/10.1111\/cobi.13231. https:\/\/conbio.onlinelibrary.wiley.com\/doi\/abs\/10.1111\/cobi.13231.","DOI":"10.1111\/cobi.13231"},{"key":"2163_CR11","doi-asserted-by":"publisher","unstructured":"Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Syst Rev. 2014;3(1). https:\/\/doi.org\/10.1186\/2046-4053-3-74.","DOI":"10.1186\/2046-4053-3-74"},{"key":"2163_CR12","doi-asserted-by":"publisher","unstructured":"Olofsson H, Brolund A, Hellberg C, Silverstein R, Stenstr\u00f6m K, \u00d6sterberg M, et al. Can abstract screening workload be reduced using text mining? User experiences of the tool Rayyan. Res Synth Methods. 2017;8(3):275\u201380. https:\/\/doi.org\/10.1002\/jrsm.1237. https:\/\/onlinelibrary.wiley.com\/doi\/abs\/10.1002\/jrsm.1237.","DOI":"10.1002\/jrsm.1237"},{"key":"2163_CR13","doi-asserted-by":"publisher","unstructured":"Beller E, , Clark J, Tsafnat G, Adams C, Diehl H, et\u00a0al. Making progress with the automation of systematic reviews: principles of the International Collaboration for the Automation of Systematic Reviews (ICASR). Syst Rev. 2018;7(1). https:\/\/doi.org\/10.1186\/s13643-018-0740-7.","DOI":"10.1186\/s13643-018-0740-7"},{"key":"2163_CR14","doi-asserted-by":"publisher","unstructured":"Marshall IJ, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev. 2019;8(1). https:\/\/doi.org\/10.1186\/s13643-019-1074-9.","DOI":"10.1186\/s13643-019-1074-9"},{"key":"2163_CR15","doi-asserted-by":"publisher","unstructured":"Feng L, Chiam YK, Lo SK. Text-Mining Techniques and Tools for Systematic Literature Reviews: A Systematic Literature Review. In: 2017 24th Asia-Pacific Software Engineering Conference (APSEC). 2017. p. 41\u201350. https:\/\/doi.org\/10.1109\/APSEC.2017.10.","DOI":"10.1109\/APSEC.2017.10"},{"key":"2163_CR16","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1016\/j.jbi.2014.06.005","volume":"51","author":"M Miwa","year":"2014","unstructured":"Miwa M, Thomas J, O\u2019Mara-Eves A, Ananiadou S. Reducing systematic review workload through certainty-based screening. J Biomed Inform. 2014;51:242\u201353. https:\/\/doi.org\/10.1016\/j.jbi.2014.06.005.","journal-title":"J Biomed Inform."},{"key":"2163_CR17","doi-asserted-by":"publisher","unstructured":"Wallace BC, Trikalinos TA, Lau J, Brodley C, Schmid CH. Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics. 2010;11(1). https:\/\/doi.org\/10.1186\/1471-2105-11-55.","DOI":"10.1186\/1471-2105-11-55"},{"key":"2163_CR18","doi-asserted-by":"publisher","unstructured":"O\u2019Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4(1). https:\/\/doi.org\/10.1186\/2046-4053-4-5.","DOI":"10.1186\/2046-4053-4-5"},{"key":"2163_CR19","doi-asserted-by":"publisher","unstructured":"Berry MW, Mohamed A, Yap BW, editors. Supervised and Unsupervised Learning for Data Science. Springer International Publishing; 2020. https:\/\/doi.org\/10.1007\/978-3-030-22475-2.","DOI":"10.1007\/978-3-030-22475-2"},{"issue":"2","key":"2163_CR20","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1197\/jamia.m1929","volume":"13","author":"AM Cohen","year":"2006","unstructured":"Cohen AM, Hersh WR, Peterson K, Yen PY. Reducing Workload in Systematic Review Preparation Using Automated Citation Classification. J Am Med Inform Assoc. 2006;13(2):206\u201319. https:\/\/doi.org\/10.1197\/jamia.m1929.","journal-title":"J Am Med Inform Assoc."},{"key":"2163_CR21","unstructured":"Blei DM, Ng AY, Jordan MI. Latent Dirichlet Allocation. J Mach Learn Res. 2003;3(null):993\u20131022."},{"key":"2163_CR22","doi-asserted-by":"publisher","unstructured":"Mo Y, Kontonatsios G, Ananiadou S. Supporting systematic reviews using LDA-based document representations. Syst Rev. 2015;4(1). https:\/\/doi.org\/10.1186\/s13643-015-0117-0.","DOI":"10.1186\/s13643-015-0117-0"},{"key":"2163_CR23","unstructured":"Li D, Wang Z, Wang L, Sohn S, Shen F, Murad MH, et\u00a0al. A Text-Mining Framework for Supporting Systematic Reviews. Am J Inf Manag. 2016;1(1):1\u20139. https:\/\/pubmed.ncbi.nlm.nih.gov\/29071308. Acessed\u00a017 Feb 2021."},{"key":"2163_CR24","unstructured":"Hatcher E, Gospodnetic O, McCandless M. Lucene in Action. 2nd ed. Manning; 2010. http:\/\/amazon.de\/o\/ASIN\/1933988177\/.\u00a0Acessed\u00a017 Feb 2021."},{"key":"2163_CR25","doi-asserted-by":"publisher","unstructured":"Natukunda A, Zirimenya L, Nassuuna J, Nkurunungi G, Cose S, Elliott AM, Webb EL.\u00a0 The effects of helminth infection on vaccine responses in humans and animal models: a systematic\u00a0review and meta-analysis.\u00a0In Parasite Immunology (Vol. 44, Issue 9). John Wiley and Sons Inc. https:\/\/doi.org\/10.1111\/pim.12939.","DOI":"10.1111\/pim.12939"},{"key":"2163_CR26","doi-asserted-by":"publisher","unstructured":"Appenzeller-Herzog C, Mathes T, Heeres MLS, Weiss KH, Houwen RHJ, Ewald H. Comparative effectiveness of common therapies for Wilson disease: A systematic review and meta-analysis of controlled studies. Liver International. 2019;39(11):2136\u201352. https:\/\/doi.org\/10.1111\/liv.14179. https:\/\/onlinelibrary.wiley.com\/doi\/abs\/10.1111\/liv.14179.","DOI":"10.1111\/liv.14179"},{"key":"2163_CR27","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.3625931","author":"C Appenzeller-Herzog","year":"2020","unstructured":"Appenzeller-Herzog C. Data from Comparative effectiveness of common therapies for Wilson disease: A systematic review and meta-analysis of controlled studies. 2020. https:\/\/doi.org\/10.5281\/zenodo.3625931.","journal-title":"Data from Comparative effectiveness of common therapies for Wilson disease: A systematic review and meta-analysis of controlled studies."},{"key":"2163_CR28","doi-asserted-by":"publisher","unstructured":"Feinerer I, Hornik K, Meyer D. Text Mining Infrastructure in R. J Stat Softw. 2008;25(5). https:\/\/doi.org\/10.18637\/jss.v025.i05.","DOI":"10.18637\/jss.v025.i05"},{"key":"2163_CR29","doi-asserted-by":"publisher","unstructured":"Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. Bayesian Data Analysis. Chapman and Hall\/CRC; 2013. https:\/\/doi.org\/10.1201\/b16018.","DOI":"10.1201\/b16018"},{"key":"2163_CR30","doi-asserted-by":"publisher","unstructured":"Gr\u00fcn B, Hornik K. topicmodels: An R Package for Fitting Topic Models. J Stat Softw. 2011;40(13). https:\/\/doi.org\/10.18637\/jss.v040.i13.","DOI":"10.18637\/jss.v040.i13"},{"key":"2163_CR31","unstructured":"Nikita M. ldatuning: Tuning of the Latent Dirichlet Allocation Models Parameters. 2020. R package version 1.0.2. https:\/\/CRAN.R-project.org\/package=ldatuning.\u00a0Acessed\u00a017 Feb 2021."},{"key":"2163_CR32","doi-asserted-by":"publisher","unstructured":"Griffiths TL, Steyvers M. Finding scientific topics. Proc Natl Acad Sci. 2004;101(suppl 1):5228\u201335. https:\/\/doi.org\/10.1073\/pnas.0307752101. https:\/\/www.pnas.org\/content\/101\/suppl_1\/5228.","DOI":"10.1073\/pnas.0307752101"},{"key":"2163_CR33","doi-asserted-by":"publisher","unstructured":"Cao J, Xia T, Li J, Zhang Y, Tang S. A density-based method for adaptive LDA model selection. Neurocomputing. 2009;72(7):1775\u20131781. Advances in Machine Learning and Computational Intelligence. https:\/\/doi.org\/10.1016\/j.neucom.2008.06.011.","DOI":"10.1016\/j.neucom.2008.06.011"},{"key":"2163_CR34","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1007\/978-3-642-13657-3_43","volume-title":"Advances in Knowledge Discovery and Data Mining","author":"R Arun","year":"2010","unstructured":"Arun R, Suresh V, Veni Madhavan CE, Narasimha Murthy MN. On Finding the Natural Number of Topics with Latent Dirichlet Allocation: Some Observations. In: Zaki MJ, Yu JX, Ravindran B, Pudi V, editors. Advances in Knowledge Discovery and Data Mining. Springer, Berlin Heidelberg: Berlin, Heidelberg; 2010. p. 391\u2013402."}],"container-title":["Systematic Reviews"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13643-022-02163-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13643-022-02163-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13643-022-02163-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,3]],"date-time":"2023-01-03T09:05:16Z","timestamp":1672736716000},"score":1,"resource":{"primary":{"URL":"https:\/\/systematicreviewsjournal.biomedcentral.com\/articles\/10.1186\/s13643-022-02163-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,3]]},"references-count":34,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["2163"],"URL":"https:\/\/doi.org\/10.1186\/s13643-022-02163-4","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-1302520\/v1","asserted-by":"object"}]},"ISSN":["2046-4053"],"issn-type":[{"value":"2046-4053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,3]]},"assertion":[{"value":"27 January 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 December 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 January 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"1"}}