{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T13:11:03Z","timestamp":1787317863614,"version":"3.56.0"},"reference-count":68,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2025,5,2]],"date-time":"2025-05-02T00:00:00Z","timestamp":1746144000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,5,2]],"date-time":"2025-05-02T00:00:00Z","timestamp":1746144000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"OCRE"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Data Sci Anal"],"published-print":{"date-parts":[[2025,11]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Despite progress in active learning, evaluation remains limited by constraints in simulation size, infrastructure, and dataset availability. This study advocates for large-scale simulations as the gold standard for evaluating active learning models in systematic review screening. Two large-scale simulations, totaling over 29 thousand runs, assessed active learning solutions. The first study evaluated 13 combinations of classification models and feature extraction techniques using high-quality datasets from the SYNERGY dataset. The second expanded this to 92 model combinations with additional classifiers and feature extractors. In every scenario tested, active learning outperformed random screening. The performance gained varied across datasets, models, and screening progression, ranging from considerable to near-flawless results. The findings demonstrate that active learning consistently outperforms random screening in systematic review tasks, offering significant efficiency gains. While the extent of improvement varies depending on the dataset, model choice, and screening stage, the overall advantage is clear. Since model performance differs, active learning systems should remain adaptable to accommodate new classifiers and feature extraction techniques. The publicly available results underscore the importance of open benchmarking to ensure reproducibility and the development of robust, generalizable active learning strategies.<\/jats:p>","DOI":"10.1007\/s41060-025-00777-0","type":"journal-article","created":{"date-parts":[[2025,5,2]],"date-time":"2025-05-02T11:16:17Z","timestamp":1746184577000},"page":"5435-5456","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Large-scale simulation study of active learning models for systematic reviews"],"prefix":"10.1007","volume":"20","author":[{"given":"Jelle Jasper","family":"Teijema","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonathan","family":"de Bruin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ayoub","family":"Bagheri","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rens","family":"van de Schoot","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,5,2]]},"reference":[{"key":"777_CR1","doi-asserted-by":"publisher","unstructured":"Adam, G.P., Wallace, B.C., Trikalinos, T.A.: Semi-automated tools for systematic searches. In: Methods in Molecular Biology. Springer US, pp. 17\u201340 (2021). https:\/\/doi.org\/10.1007\/978-1-0716-1566-9_2","DOI":"10.1007\/978-1-0716-1566-9_2"},{"issue":"DEC","key":"777_CR2","doi-asserted-by":"publisher","first-page":"38","DOI":"10.3389\/fninf.2013.00038","volume":"7","author":"KH Ambert","year":"2013","unstructured":"Ambert, K.H., Cohen, A.M., Burns, G.A.P.C., et al.: Virk: an active learning-based system for bootstrapping knowledge base development in the neurosciences. Front. Neuroinform. 7(DEC), 38 (2013). https:\/\/doi.org\/10.3389\/fninf.2013.00038","journal-title":"Front. Neuroinform."},{"issue":"1","key":"777_CR3","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1186\/s13643-018-0740-7","volume":"7","author":"E Beller","year":"2018","unstructured":"Beller, E., Clark, J., Tsafnat, G., et al.: Making progress with the automation of systematic reviews: principles of the international collaboration for the automation of systematic reviews (ICASR). Syst. Rev. 7(1), 77 (2018). https:\/\/doi.org\/10.1186\/s13643-018-0740-7","journal-title":"Syst. Rev."},{"issue":"1","key":"777_CR4","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1186\/s13643-024-02502-7","volume":"13","author":"J Boetje","year":"2024","unstructured":"Boetje, J., van de Schoot, R.: The safe procedure: a practical stopping heuristic for active learning-based screening in systematic reviews and meta-analyses. Syst. Rev. 13(1), 81 (2024)","journal-title":"Syst. Rev."},{"key":"777_CR5","doi-asserted-by":"crossref","unstructured":"Bojanowski, P., Grave, E., Joulin, A., et\u00a0al.: Enriching word vectors with subword information. (2016). arXiv preprint arXiv:1607.04606","DOI":"10.1162\/tacl_a_00051"},{"key":"777_CR6","unstructured":"Bron, M.P., van\u00a0der Heijden, P.G., Feelders, A.J., et\u00a0al.: Using chao\u2019s estimator as a stopping criterion for technology-assisted review (2024). arXiv preprint arXiv:2404.01176"},{"issue":"1","key":"777_CR7","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1186\/s13643-024-02587-0","volume":"13","author":"F Byrne","year":"2024","unstructured":"Byrne, F., Hofstee, L., Teijema, J., et al.: Impact of active learning model and prior knowledge on discovery time of elusive relevant papers: a simulation study. Syst. Rev. 13(1), 175 (2024)","journal-title":"Syst. Rev."},{"issue":"S1","key":"777_CR8","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1017\/S0266462321000696","volume":"37","author":"N Carey","year":"2021","unstructured":"Carey, N., Harte, M., McCullagh, L.: The use of a text-mining screening tool for systematic review of treatments for relapsed\/refractory diffuse large b-cell lymphoma. Int. J. Technol. Assess. Health Care 37(S1), 2 (2021)","journal-title":"Int. J. Technol. Assess. Health Care"},{"issue":"3","key":"777_CR9","doi-asserted-by":"publisher","first-page":"3047","DOI":"10.1007\/s11192-020-03648-6","volume":"125","author":"A Carvallo","year":"2020","unstructured":"Carvallo, A., Parra, D., Lobel, H., et al.: Automatic document screening of medical literature using word and text embeddings in an active learning setting. SCIENTOMETRICS 125(3), 3047\u20133084 (2020). https:\/\/doi.org\/10.1007\/s11192-020-03648-6","journal-title":"SCIENTOMETRICS"},{"key":"777_CR10","doi-asserted-by":"crossref","unstructured":"Chen, T., Guestrin, C.: Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining, pp. 785\u2013794 (2016)","DOI":"10.1145\/2939672.2939785"},{"issue":"2","key":"777_CR11","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1197\/jamia.M1929","volume":"13","author":"AM Cohen","year":"2006","unstructured":"Cohen, A.M., Hersh, W.R., Peterson, K., et al.: Reducing workload in systematic review preparation using automated citation classification. J. Am. Med. Inform. Assoc. 13(2), 206\u2013219 (2006)","journal-title":"J. Am. Med. Inform. Assoc."},{"issue":"5","key":"777_CR12","doi-asserted-by":"publisher","DOI":"10.2196\/33219","volume":"10","author":"K Cowie","year":"2022","unstructured":"Cowie, K., Rahmatullah, A., Hardy, N., et al.: Web-based software tools for systematic literature review in medicine: systematic search and feature analysis. JMIR Med. Inform. 10(5), e33219 (2022). https:\/\/doi.org\/10.2196\/33219","journal-title":"JMIR Med. Inform."},{"key":"777_CR13","doi-asserted-by":"crossref","unstructured":"Cumpston, M., Li, T., Page, M.J., et\u00a0al.: Updated guidance for trusted systematic reviews: a new edition of the cochrane handbook for systematic reviews of interventions. Cochrane Database Syst. Rev. 2019(10) (2019)","DOI":"10.1002\/14651858.ED000142"},{"key":"777_CR14","doi-asserted-by":"publisher","unstructured":"De\u00a0Bruin, J., Ma, Y., Ferdinands, G., et\u00a0al.: SYNERGY - Open machine learning dataset on study selection in systematic reviews (2023). https:\/\/doi.org\/10.34894\/HE6NAQ","DOI":"10.34894\/HE6NAQ"},{"key":"777_CR15","doi-asserted-by":"publisher","unstructured":"developers, A.L.: Asreview lab v1.2\u2014a tool for AI-assisted systematic reviews (2023). https:\/\/doi.org\/10.5281\/zenodo.7821585","DOI":"10.5281\/zenodo.7821585"},{"issue":"11","key":"777_CR16","doi-asserted-by":"publisher","first-page":"1395","DOI":"10.1007\/s40262-021-01042-w","volume":"60","author":"AA Donners","year":"2021","unstructured":"Donners, A.A., Rademaker, C.M., Bevers, L.A., et al.: Pharmacokinetics and associated efficacy of emicizumab in humans: a systematic review. Clin. Pharmacokinet. 60(11), 1395\u20131406 (2021)","journal-title":"Clin. Pharmacokinet."},{"key":"777_CR17","unstructured":"Feng, F., Yang, Y., Cer, D., et\u00a0al.: Language-agnostic BERT sentence embedding (2020). CoRR arxiv:2007.01852"},{"issue":"1","key":"777_CR18","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1080\/00273171.2020.1853501","volume":"56","author":"G Ferdinands","year":"2021","unstructured":"Ferdinands, G.: Ai-assisted systematic reviewing: selecting studies to compare Bayesian versus frequentist SEM for small sample sizes. Multivar. Behav. Res. 56(1), 153\u2013154 (2021)","journal-title":"Multivar. Behav. Res."},{"issue":"1","key":"777_CR19","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1186\/s13643-023-02257-7","volume":"12","author":"G Ferdinands","year":"2023","unstructured":"Ferdinands, G., Schram, R., de Bruin, J., et al.: Performance of active learning models for screening prioritization in systematic reviews: a simulation study into the average time to discover relevant records. Syst. Rev. 12(1), 100 (2023)","journal-title":"Syst. Rev."},{"issue":"6","key":"777_CR20","doi-asserted-by":"publisher","first-page":"1276","DOI":"10.1109\/TSE.2011.103","volume":"38","author":"T Hall","year":"2011","unstructured":"Hall, T., Beecham, S., Bowes, D., et al.: A systematic literature review on fault prediction performance in software engineering. IEEE Trans. Softw. Eng. 38(6), 1276\u20131304 (2011)","journal-title":"IEEE Trans. Softw. Eng."},{"key":"777_CR21","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-020-01129-1","author":"C Hamel","year":"2020","unstructured":"Hamel, C., Kelly, S.E., Thavorn, K., et al.: An evaluation of Distillersr\u2019s machine learning-based prioritization tool for title\/abstract screening\u2014impact on reviewer-relevant outcomes. BMC Med. Res. Methodol. (2020). https:\/\/doi.org\/10.1186\/s12874-020-01129-1","journal-title":"BMC Med. Res. Methodol."},{"issue":"1","key":"777_CR22","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1186\/s13643-024-02590-5","volume":"13","author":"W Harmsen","year":"2024","unstructured":"Harmsen, W., de Groot, J., Harkema, A., et al.: Machine learning to optimize literature screening in medical guideline development. Syst. Rev. 13(1), 177 (2024)","journal-title":"Syst. Rev."},{"key":"777_CR23","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1016\/j.jbi.2016.06.001","volume":"62","author":"K Hashimoto","year":"2016","unstructured":"Hashimoto, K., Kontonatsios, G., Miwa, M., et al.: Topic detection using paragraph vectors to support active learning in systematic reviews. J. Biomed. Inform. 62, 59\u201365 (2016). https:\/\/doi.org\/10.1016\/j.jbi.2016.06.001","journal-title":"J. Biomed. Inform."},{"key":"777_CR24","doi-asserted-by":"publisher","DOI":"10.1016\/j.envint.2020.105623","volume":"138","author":"BE Howard","year":"2020","unstructured":"Howard, B.E., Phillips, J., Tandon, A., et al.: Swift-active screener: accelerated document screening through active learning and integrated recall estimation. Environ. Int. 138, 105623 (2020)","journal-title":"Environ. Int."},{"issue":"1","key":"777_CR25","doi-asserted-by":"publisher","first-page":"322","DOI":"10.1186\/s12874-022-01805-4","volume":"22","author":"RC Jimenez","year":"2022","unstructured":"Jimenez, R.C., Lee, T., Rosillo, N., et al.: Machine learning computational tools to assist the performance of systematic reviews: a mapping review. BMC Med. Res. Methodol. 22(1), 322 (2022). https:\/\/doi.org\/10.1186\/s12874-022-01805-4","journal-title":"BMC Med. Res. Methodol."},{"key":"777_CR26","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1016\/j.jclinepi.2021.12.005","volume":"144","author":"H Khalil","year":"2022","unstructured":"Khalil, H., Ameen, D., Zarnegar, A.: Tools to support the automation of systematic reviews: a scoping review. J. Clin. Epidemiol. 144, 22\u201342 (2022). https:\/\/doi.org\/10.1016\/j.jclinepi.2021.12.005","journal-title":"J. Clin. Epidemiol."},{"key":"777_CR27","unstructured":"Lee, S., Shakir, A., Koenig, D., et\u00a0al.: Open source strikes bread\u2014new fluffy embeddings model (2024). https:\/\/www.mixedbread.ai\/blog\/mxbai-embed-large-v1"},{"key":"777_CR28","doi-asserted-by":"publisher","first-page":"7","DOI":"10.5334\/jcr.183","volume":"17","author":"CH Leenaars","year":"2019","unstructured":"Leenaars, C.H., Drinkenburg, W.P., Nolten, C., et al.: Sleep and microdialysis: an experiment and a systematic review of histamine and several amino acids. J. Circadian Rhythms 17, 7 (2019)","journal-title":"J. Circadian Rhythms"},{"issue":"9","key":"777_CR29","doi-asserted-by":"publisher","first-page":"3842","DOI":"10.3390\/app14093842","volume":"14","author":"P Lombaers","year":"2024","unstructured":"Lombaers, P., de Bruin, J., van de Schoot, R.: Reproducibility and data storage for active learning-aided systematic reviews. Appl. Sci. 14(9), 3842 (2024). https:\/\/doi.org\/10.3390\/app14093842","journal-title":"Appl. Sci."},{"issue":"1","key":"777_CR30","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1186\/s13643-019-1074-9","volume":"8","author":"IJ Marshall","year":"2019","unstructured":"Marshall, I.J., Wallace, B.C.: Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst. Rev. 8(1), 163 (2019). https:\/\/doi.org\/10.1186\/s13643-019-1074-9","journal-title":"Syst. Rev."},{"key":"777_CR31","doi-asserted-by":"publisher","unstructured":"Mauricio, D., Gonzalez, N.: Optimizacion de estrategias de busquedas cientificas medicas utilizando tecnicas de inteligencia artificial. Ph.D. thesis (2021). https:\/\/doi.org\/10.11144\/javeriana.10554.58492","DOI":"10.11144\/javeriana.10554.58492"},{"key":"777_CR32","volume":"16","author":"A Molinari","year":"2022","unstructured":"Molinari, A., Kanoulas, E.: Transferring knowledge between topics in systematic reviews. Intell. Syst. Appl. 16, 200150 (2022)","journal-title":"Intell. Syst. Appl."},{"issue":"3","key":"777_CR33","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1097\/BRS.0000000000003645","volume":"46","author":"S Muthu","year":"2021","unstructured":"Muthu, S., Ramakrishnan, E.: Fragility analysis of statistically significant outcomes of randomized control trials in spine surgery: a systematic review. Spine 46(3), 198\u2013208 (2021)","journal-title":"Spine"},{"issue":"1","key":"777_CR34","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1186\/s13643-019-1162-x","volume":"8","author":"CR Norman","year":"2019","unstructured":"Norman, C.R., Leeflang, M.M.G., Porcher, R., et al.: Measuring the impact of screening automation on meta-analyses of diagnostic test accuracy. Syst. Rev. 8(1), 243 (2019). https:\/\/doi.org\/10.1186\/s13643-019-1162-x","journal-title":"Syst. Rev."},{"issue":"1","key":"777_CR35","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1186\/s13643-019-0975-y","volume":"8","author":"AM O\u2019Connor","year":"2019","unstructured":"O\u2019Connor, A.M., Tsafnat, G., Gilbert, S.B., et al.: Still moving toward automation of the systematic review process: a summary of discussions at the third meeting of the international collaboration for automation of systematic reviews (ICASR). Syst. Rev. 8(1), 57 (2019). https:\/\/doi.org\/10.1186\/s13643-019-0975-y","journal-title":"Syst. Rev."},{"key":"777_CR36","doi-asserted-by":"publisher","unstructured":"Olorisade, B.K., De Quincey, E., Andras, P., et al.: A critical analysis of studies that address the use of text mining for citation screening in systematic reviews (2016). https:\/\/doi.org\/10.1145\/2915970.2915982","DOI":"10.1145\/2915970.2915982"},{"key":"777_CR37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jbi.2017.07.010","volume":"73","author":"BK Olorisade","year":"2017","unstructured":"Olorisade, B.K., Brereton, P., Andras, P.: Reproducibility of studies on text mining for citation screening in systematic reviews: evaluation and checklist. J. Biomed. Inform. 73, 1\u201313 (2017). https:\/\/doi.org\/10.1016\/j.jbi.2017.07.010","journal-title":"J. Biomed. Inform."},{"issue":"10","key":"777_CR38","doi-asserted-by":"publisher","first-page":"949","DOI":"10.1177\/0004867418791257","volume":"52","author":"M Oud","year":"2018","unstructured":"Oud, M., Arntz, A., Hermens, M.L., et al.: Specialized psychotherapies for adults with borderline personality disorder: a systematic review and meta-analysis. Aust. N. Z. J. Psychiatry 52(10), 949\u2013961 (2018)","journal-title":"Aust. N. Z. J. Psychiatry"},{"issue":"1","key":"777_CR39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/2046-4053-4-1","volume":"4","author":"A O\u2019Mara-Eves","year":"2015","unstructured":"O\u2019Mara-Eves, A., Thomas, J., McNaught, J., et al.: Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst. Rev. 4(1), 1\u201322 (2015)","journal-title":"Syst. Rev."},{"key":"777_CR40","doi-asserted-by":"publisher","first-page":"n160","DOI":"10.1136\/bmj.n160","volume":"372","author":"MJ Page","year":"2021","unstructured":"Page, M.J., Moher, D., Bossuyt, P.M., et al.: Prisma 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 372, n160 (2021)","journal-title":"BMJ"},{"key":"777_CR41","doi-asserted-by":"publisher","unstructured":"Pellegrini, M., Marsili, F.: Evaluating software tools to conduct systematic reviews: a feature analysis and user survey. Form@re Open Journal per la formazione in rete 21(2), 124\u2013140 (2021). https:\/\/doi.org\/10.36253\/form-11343","DOI":"10.36253\/form-11343"},{"issue":"3","key":"777_CR42","doi-asserted-by":"publisher","first-page":"470","DOI":"10.1002\/jrsm.1311","volume":"9","author":"P Przyby\u0142a","year":"2018","unstructured":"Przyby\u0142a, P., Brockmeier, A.J., Kontonatsios, G., et al.: Prioritising references for systematic reviews with robotanalyst: a user study. Res. Synthesis Methods 9(3), 470\u2013488 (2018)","journal-title":"Res. Synthesis Methods"},{"key":"777_CR43","doi-asserted-by":"crossref","unstructured":"Reimers, N., Gurevych, I.: Sentence-bert: Sentence embeddings using siamese bert-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (2019). http:\/\/arxiv.org\/abs\/1908.10084","DOI":"10.18653\/v1\/D19-1410"},{"key":"777_CR44","doi-asserted-by":"publisher","unstructured":"Robledo, S., Aguirre, A.M.G., Hughes, M., et\u00a0al.: \u201cHasta la vista, baby\u201d \u2013 will machine learning terminate human literature reviews in entrepreneurship? J. Small Bus. Manag. 1\u201330 (2021). https:\/\/doi.org\/10.1080\/00472778.2021.1955125","DOI":"10.1080\/00472778.2021.1955125"},{"key":"777_CR45","doi-asserted-by":"publisher","unstructured":"Romanov, S., Siqueira, A.S., de\u00a0Bruin, J., et\u00a0al.: Optimizing ASReview simulations: a generic multiprocessing solution for \u2018light-data\u2019 and \u2018heavy-data\u2019 users. Data Intell. 1\u201319 (2024). https:\/\/doi.org\/10.1162\/dint_a_00244","DOI":"10.1162\/dint_a_00244"},{"key":"777_CR46","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1016\/j.jclinepi.2021.06.030","volume":"138","author":"AM Scott","year":"2021","unstructured":"Scott, A.M., Forbes, C., Clark, J., et al.: Systematic review automation tools improve efficiency but lack of knowledge impedes their adoption: a survey. J. Clin. Epidemiol. 138, 80\u201394 (2021). https:\/\/doi.org\/10.1016\/j.jclinepi.2021.06.030","journal-title":"J. Clin. Epidemiol."},{"issue":"7","key":"777_CR47","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0249102","volume":"16","author":"MS Sep","year":"2021","unstructured":"Sep, M.S., Vellinga, M., Sarabdjitsingh, R.A., et al.: The rodent object-in-context task: a systematic review and meta-analysis of important variables. PLoS ONE 16(7), e0249102 (2021)","journal-title":"PLoS ONE"},{"key":"777_CR48","unstructured":"Settles, B.: Active learning literature survey. Tech. rep (2009)"},{"key":"777_CR49","doi-asserted-by":"crossref","unstructured":"Sun, C., Qiu, X., Xu, Y., et\u00a0al.: How to fine-tune bert for text classification? (2020). arXiv:1905.05583","DOI":"10.1007\/978-3-030-32381-3_16"},{"key":"777_CR50","doi-asserted-by":"publisher","unstructured":"Teijema, J.J.: jteijema\/asreview-simulation-project: v1.1.4 (2023). https:\/\/doi.org\/10.5281\/zenodo.7993561","DOI":"10.5281\/zenodo.7993561"},{"key":"777_CR51","doi-asserted-by":"publisher","unstructured":"Teijema, J.J.: Simulation data for: large-scale simulation study of active learning models for systematic reviews (2023). https:\/\/doi.org\/10.34894\/NYFSJY","DOI":"10.34894\/NYFSJY"},{"key":"777_CR52","doi-asserted-by":"publisher","unstructured":"Teijema, J.J.: jteijema\/synergy-simulations-website: release on Zenodo (2024). https:\/\/doi.org\/10.5281\/zenodo.13169790","DOI":"10.5281\/zenodo.13169790"},{"key":"777_CR53","doi-asserted-by":"publisher","unstructured":"Teijema, J.J., van\u00a0den Brand SAGE, Bagheri, A., et\u00a0al.: Simulation-based active learning for systematic reviews: a systematic review of the literature-repository (2023). https:\/\/doi.org\/10.17605\/OSF.IO\/T9HGM","DOI":"10.17605\/OSF.IO\/T9HGM"},{"key":"777_CR54","doi-asserted-by":"publisher","first-page":"1178181","DOI":"10.3389\/frma.2023.1178181","volume":"8","author":"JJ Teijema","year":"2023","unstructured":"Teijema, J.J., Hofstee, L., Brouwer, M., et al.: Active learning-based systematic reviewing using switching classification models: the case of the onset, maintenance, and relapse of depressive disorders. Front. Res. Metrics Anal. 8, 1178181 (2023)","journal-title":"Front. Res. Metrics Anal."},{"key":"777_CR55","doi-asserted-by":"crossref","unstructured":"Teijema, J.J., Seuren, S., Anadria, D., et\u00a0al.: Simulation-based active learning for systematic reviews: a scoping review of the literature. Preprint (2023)","DOI":"10.31234\/osf.io\/67zmt"},{"key":"777_CR56","doi-asserted-by":"crossref","unstructured":"Teijema, J.J., van de Schoot, R., Ferdinands, G., et al.: Makita-a workflow generator for large-scale and reproducible simulation studies mimicking text labeling. Softw. Impacts 21, 100663 (2024)","DOI":"10.1016\/j.simpa.2024.100663"},{"issue":"1","key":"777_CR57","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/jrsm.27","volume":"2","author":"J Thomas","year":"2011","unstructured":"Thomas, J., McNaught, J., Ananiadou, S.: Applications of text mining within systematic reviews. Res. Synth. Methods 2(1), 1\u201314 (2011). https:\/\/doi.org\/10.1002\/jrsm.27","journal-title":"Res. Synth. Methods"},{"key":"777_CR58","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1016\/j.jclinepi.2017.08.011","volume":"91","author":"J Thomas","year":"2017","unstructured":"Thomas, J., Noel-Storr, A., Marshall, I., et al.: Living systematic reviews: 2. Combining human and machine effort. J. Clin. Epidemiol. 91, 31\u201337 (2017)","journal-title":"J. Clin. Epidemiol."},{"key":"777_CR59","doi-asserted-by":"publisher","unstructured":"van de Schoot, R., de Bruin, J., Schram, R., et al.: An open source machine learning framework for efficient and transparent systematic reviews. Nat. Mach. Intell. 3(2), 125\u2013133 (2021). https:\/\/doi.org\/10.1038\/s42256-020-00287-7","DOI":"10.1038\/s42256-020-00287-7"},{"key":"777_CR60","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2021.106589","volume":"136","author":"R Van Dinter","year":"2021","unstructured":"Van Dinter, R., Tekinerdogan, B., Catal, C.: Automation of systematic literature reviews: a systematic literature review. Inf. Softw. Technol. 136, 106589 (2021). https:\/\/doi.org\/10.1016\/j.infsof.2021.106589","journal-title":"Inf. Softw. Technol."},{"issue":"2","key":"777_CR61","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1177\/02683962211048201","volume":"37","author":"G Wagner","year":"2022","unstructured":"Wagner, G., Lukyanenko, R., Par\u00e9, G.: Artificial intelligence and the conduct of literature reviews. J. Inf. Technol. 37(2), 209\u2013226 (2022)","journal-title":"J. Inf. Technol."},{"key":"777_CR62","unstructured":"Wallace, B.: Abstrackr: Software for semi-automatic citation screening. https:\/\/effectivehealthcare.ahrq.gov\/products\/abstractr\/abstract (2012)"},{"issue":"1","key":"777_CR63","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1186\/1471-2105-11-55","volume":"11","author":"BC Wallace","year":"2010","unstructured":"Wallace, B.C., Trikalinos, T.A., Lau, J., et al.: Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinform. 11(1), 55 (2010). https:\/\/doi.org\/10.1186\/1471-2105-11-55","journal-title":"BMC Bioinform."},{"issue":"2","key":"777_CR64","doi-asserted-by":"publisher","first-page":"781","DOI":"10.1093\/bib\/bbaa296","volume":"22","author":"LL Wang","year":"2021","unstructured":"Wang, L.L., Lo, K.: Text mining approaches for dealing with the rapidly expanding literature on covid-19. Brief. Bioinform. 22(2), 781\u2013799 (2021)","journal-title":"Brief. Bioinform."},{"key":"777_CR65","doi-asserted-by":"crossref","unstructured":"Wang, W., Wei, F., Dong, L., et\u00a0al.: Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers (2020). arXiv:2002.10957","DOI":"10.18653\/v1\/2021.findings-acl.188"},{"issue":"6","key":"777_CR66","doi-asserted-by":"publisher","first-page":"715","DOI":"10.7507\/1672-2531.202012150","volume":"21","author":"Q Xuan","year":"2021","unstructured":"Xuan, Q., Jiali, L., Yuning, W., et al.: Application of natural language processing in systematic reviews. Chin. J. Evid.-Based Med. 21(6), 715\u2013720 (2021). https:\/\/doi.org\/10.7507\/1672-2531.202012150","journal-title":"Chin. J. Evid.-Based Med."},{"key":"777_CR67","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.116958","volume":"200","author":"Z Yu","year":"2022","unstructured":"Yu, Z., Carver, J.C., Rothermel, G., et al.: Assessing expert system-assisted literature reviews with a case study. Expert Syst. Appl. 200, 116958 (2022)","journal-title":"Expert Syst. Appl."},{"issue":"3","key":"777_CR68","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3388640","volume":"38","author":"J Zou","year":"2020","unstructured":"Zou, J., Kanoulas, E.: Towards question-based high-recall information retrieval. ACM Trans. Inf. Syst. 38(3), 1\u201335 (2020). https:\/\/doi.org\/10.1145\/3388640","journal-title":"ACM Trans. Inf. Syst."}],"container-title":["International Journal of Data Science and Analytics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-025-00777-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41060-025-00777-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-025-00777-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,27]],"date-time":"2025-09-27T12:16:57Z","timestamp":1758975417000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41060-025-00777-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,2]]},"references-count":68,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,11]]}},"alternative-id":["777"],"URL":"https:\/\/doi.org\/10.1007\/s41060-025-00777-0","relation":{},"ISSN":["2364-415X","2364-4168"],"issn-type":[{"value":"2364-415X","type":"print"},{"value":"2364-4168","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,2]]},"assertion":[{"value":"20 September 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 March 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 May 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"During the preparation of this work, the authors used\n                      Open Source\n                      Generative AI to increase language readability. After the use of this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Usage of generative AI and AI-assisted technologies in the writing process"}}]}}