{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T05:04:50Z","timestamp":1778216690156,"version":"3.51.4"},"reference-count":41,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,5,16]],"date-time":"2023-05-16T00:00:00Z","timestamp":1684195200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Res. Metr. Anal."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>This study examines the performance of active learning-aided systematic reviews using a deep learning-based model compared to traditional machine learning approaches, and explores the potential benefits of model-switching strategies.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>Comprising four parts, the study: 1) analyzes the performance and stability of active learning-aided systematic review; 2) implements a convolutional neural network classifier; 3) compares classifier and feature extractor performance; and 4) investigates the impact of model-switching strategies on review performance.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Lighter models perform well in early simulation stages, while other models show increased performance in later stages. Model-switching strategies generally improve performance compared to using the default classification model alone.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>The study's findings support the use of model-switching strategies in active learning-based systematic review workflows. It is advised to begin the review with a light model, such as Na\u00efve Bayes or logistic regression, and switch to a heavier classification model based on a heuristic rule when needed.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frma.2023.1178181","type":"journal-article","created":{"date-parts":[[2023,5,16]],"date-time":"2023-05-16T05:41:29Z","timestamp":1684215689000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":26,"title":["Active learning-based systematic reviewing using switching classification models: the case of the onset, maintenance, and relapse of depressive disorders"],"prefix":"10.3389","volume":"8","author":[{"given":"Jelle Jasper","family":"Teijema","sequence":"first","affiliation":[]},{"given":"Laura","family":"Hofstee","sequence":"additional","affiliation":[]},{"given":"Marlies","family":"Brouwer","sequence":"additional","affiliation":[]},{"given":"Jonathan","family":"de Bruin","sequence":"additional","affiliation":[]},{"given":"Gerbrich","family":"Ferdinands","sequence":"additional","affiliation":[]},{"given":"Jan","family":"de Boer","sequence":"additional","affiliation":[]},{"given":"Pablo","family":"Vizan","sequence":"additional","affiliation":[]},{"given":"Sofie","family":"van den Brand","sequence":"additional","affiliation":[]},{"given":"Claudi","family":"Bockting","sequence":"additional","affiliation":[]},{"given":"Rens","family":"van de Schoot","sequence":"additional","affiliation":[]},{"given":"Ayoub","family":"Bagheri","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2023,5,16]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1016\/j.jocm.2018.07.002","article-title":"Is your dataset big enough? Sample size requirements when using artificial neural networks for discrete choice analysis","volume":"28","author":"Alwosheel","year":"2018","journal-title":"J. Choice Model"},{"key":"B2","doi-asserted-by":"publisher","first-page":"627","DOI":"10.1108\/14777260911001644","article-title":"Building knowledge integration systems for evidence-informed decisions","volume":"23","author":"Best","year":"2009","journal-title":"J. Health Organiz. Manage"},{"key":"B3","doi-asserted-by":"crossref","DOI":"10.1002\/9781119558378","volume-title":"Introduction to Meta-Analysis.","author":"Borenstein","year":"2021"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.17605\/OSF.IO\/R45YZ","author":"Brouwer","year":"2021","journal-title":"Systematic review data from \u201cPsychological theories of depressive relapse and recurrence\u201d"},{"key":"B5","unstructured":"BrouwerM.\n            van de SchootR.\n          Results Reanalyzing Meta-Analysis Depression Data without Hard-to-Find papers2021"},{"key":"B6","doi-asserted-by":"publisher","first-page":"101773","DOI":"10.1016\/j.cpr.2019.101773","article-title":"Psychological theories of depressive relapse and recurrence: A systematic review and meta-analysis of prospective studies","volume":"74","author":"Brouwer","year":"2019","journal-title":"Clin. Psychol. Rev"},{"key":"B7","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1016\/j.jbi.2011.11.003","article-title":"Applying active learning to assertion classification of concepts in clinical text","volume":"45","author":"Chen","year":"2012","journal-title":"J. Biomed. Inform"},{"key":"B8","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1197\/jamia.M1929","article-title":"Reducing workload in systematic review preparation using automated citation classification","volume":"13","author":"Cohen","year":"2006","journal-title":"J. Am. Med. Inform. Assoc"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390177","article-title":"\u201cA unified architecture for natural language processing: Deep neural networks with multitask learning,\u201d","author":"Collobert","year":"2008","journal-title":"Proceedings of the 25th International Conference on Machine Learning."},{"key":"B10","author":"De Boer","year":"2021","journal-title":"Systematic Reviews at Utrecht University and UMC Utrecht 2020"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.31219\/osf.io\/w6qbg","author":"Ferdinands","year":"2020","journal-title":"Active learning for screening prioritization in systematic reviews - A simulation study"},{"key":"B12","author":"Ferdinands","year":"2021","journal-title":"Scripts and Output for the Simulation Study Determining the Time to Discovery for the Depression Data (1, 0.)."},{"key":"B13","doi-asserted-by":"publisher","first-page":"e050129","DOI":"10.1136\/bmjopen-2021-050129","article-title":"Psychological factors for the onset of depression: a meta-analysis of prospective studies","volume":"11","author":"Fu","year":"2021","journal-title":"BMJ Open"},{"key":"B14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2523813","article-title":"survey on concept drift adaptation","volume":"46","author":"Gama","year":"2014","journal-title":"ACM Comput. Surv"},{"key":"B15","article-title":"\u201cDeep sparse rectifier neural networks,\u201d","author":"Glorot","year":"2011","journal-title":"Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings."},{"key":"B16","author":"Goodfellow","year":"2016","journal-title":"Deep learning: MIT press;"},{"key":"B17","first-page":"246","article-title":"\u201cMedical text classification using convolutional neural networks,\u201d","volume-title":"Informatics for Health: Connected Citizen-Led Wellness and Population Health","author":"Hughes","year":"2017"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11263-015-0823-z","article-title":"Reading text in the wild with convolutional neural networks","volume":"116","author":"Jaderberg","year":"2016","journal-title":"Int. J. Comput. Vision"},{"key":"B19","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1021\/acschemneuro.5b00302","article-title":"Failure of standard training sets in the analysis of fast-scan cyclic voltammetry data","volume":"7","author":"Johnson","year":"2016","journal-title":"ACS Chem. Neurosci"},{"key":"B20","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1038\/s41380-019-0585-z","article-title":"Prospective biomarkers of major depressive disorder: a systematic review and meta-analysis","volume":"25","author":"Kennis","year":"2020","journal-title":"Mol Psychiat"},{"key":"B21","doi-asserted-by":"publisher","first-page":"1249","DOI":"10.1038\/s41598-023-28328-2","article-title":"A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge","volume":"13","author":"Keren","year":"2023","journal-title":"Scient. Rep"},{"key":"B22","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1080\/10627190801968224","article-title":"Assessing knowledge integration in science: Construct, measures, and evidence","volume":"13","author":"Liu","year":"2008","journal-title":"Educ. Assess"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/2046-4053-4-1","article-title":"Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement","volume":"4","author":"Moher","year":"2015","journal-title":"Syst. Rev"},{"key":"B24","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-35289-8","volume-title":"Neural Networks: Tricks of the Trade.","author":"Montavon","year":"2012"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3434237","article-title":"A comprehensive survey on word representation models: From classical to state-of-the-art word representation language models","volume":"20","author":"Naseem","year":"2021","journal-title":"Trans. Asian Low-Resour. Language Inform. Proc"},{"key":"B26","author":"Parker","year":"2011","journal-title":"English Gigaword"},{"key":"B27","author":"Rolnick","year":"2017","journal-title":"The power of deeper networks for expressing natural functions"},{"key":"B28","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1007\/s42488-021-00062-2","article-title":"Computer aided functional style identification and correction in modern russian texts","volume":"4","author":"Savchenko","year":"2022","journal-title":"J. Data Inf. Manage"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2200\/S00429ED1V01Y201207AIM018","article-title":"Active learning","volume":"6","author":"Settles","year":"2012","journal-title":"Synth. Lectur. Artif. Intell. Mach. Learn"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1929","DOI":"10.5555\/2627435.2670313","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res"},{"key":"B31","author":"Teijema","year":"","journal-title":"ASReview CNN 17 layer model plugin. v1, 0.2 ed."},{"key":"B32","author":"Teijema","year":"","journal-title":"ASReview wide doc2vec plugin. v0, 1.2 ed."},{"key":"B33","author":"Teijema","year":"","journal-title":"ASReview model switcher plugin. v1, 0.2 ed."},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.31234\/osf.io\/t7bpd","author":"Teijema","year":"2022","journal-title":"A code repository for: Active learning-based Systematic reviewing using switching classification models: the case of the onset, maintenance, and relapse of depressive disorders (0, 1.)."},{"key":"B35","doi-asserted-by":"publisher","first-page":"899","DOI":"10.1016\/j.infsof.2004.03.006","article-title":"An empirical study of the effect of knowledge integration on software development performance","volume":"46","author":"Tiwana","year":"2004","journal-title":"Inf. Softw. Technol"},{"key":"B36","author":"van de Brand","year":"2021","journal-title":"A Systematic Review on Studies Evaluating the Performance of Active Learning Compared to Human Reading for Systematic Review Data"},{"key":"B37","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1038\/s42256-020-00287-7","article-title":"An open source machine learning framework for efficient and transparent systematic reviews","volume":"3","author":"van de Schoot","year":"","journal-title":"Nat. Mach. Intell"},{"key":"B38","author":"van de Schoot","year":"","journal-title":"ASReview: Active learning for systematic reviews [Software]"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.5565336","author":"van de Schoot","year":"","journal-title":"ASReview: Active learning for systematic reviews"},{"key":"B40","author":"van den Brand","year":"2021","journal-title":"Scripts for Post-Processing Mega-Meta Screening Results. v1, 0.1 ed."},{"key":"B41","doi-asserted-by":"publisher","first-page":"e0227742","DOI":"10.1371\/journal.pone.0227742","article-title":"Error rates of human reviewers during abstract screening in systematic reviews","volume":"15","author":"Wang","year":"2020","journal-title":"PLoS ONE"}],"container-title":["Frontiers in Research Metrics and Analytics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frma.2023.1178181\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,16]],"date-time":"2023-05-16T05:41:56Z","timestamp":1684215716000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frma.2023.1178181\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,16]]},"references-count":41,"alternative-id":["10.3389\/frma.2023.1178181"],"URL":"https:\/\/doi.org\/10.3389\/frma.2023.1178181","relation":{},"ISSN":["2504-0537"],"issn-type":[{"value":"2504-0537","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,16]]},"article-number":"1178181"}}