{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T22:28:29Z","timestamp":1790288909916,"version":"4.1.0"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T00:00:00Z","timestamp":1753315200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T00:00:00Z","timestamp":1753315200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000925","name":"National Health and Medical Research Council","doi-asserted-by":"publisher","award":["2024622"],"award-info":[{"award-number":["2024622"]}],"id":[{"id":"10.13039\/501100000925","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000925","name":"National Health and Medical Research Council","doi-asserted-by":"publisher","award":["2024622"],"award-info":[{"award-number":["2024622"]}],"id":[{"id":"10.13039\/501100000925","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000925","name":"National Health and Medical Research Council","doi-asserted-by":"publisher","award":["2024622"],"award-info":[{"award-number":["2024622"]}],"id":[{"id":"10.13039\/501100000925","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Malar J"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Background<\/jats:title>\n            <jats:p>Machine learning algorithms have been used to predict malaria risk and severity, identify immunity biomarkers for malaria vaccine candidates, and determine molecular biomarkers of antimalarial drug resistance. Developing these prediction models requires large training datasets to ensure prediction accuracy when applied to new individuals in the target population. Learning curves can be used to assess the sample size required for the training dataset by evaluating the predictive performance of a model trained using different dataset sizes. These curves are agnostic to the specific prediction model, but their construction does require existing data. This tutorial demonstrates how to generate and interpret learning curves for malaria prediction models developed using machine learning algorithms.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>To illustrate the approach, training dataset sizes were evaluated to inform the design of a \u201cmock\u201d prediction modelling study aimed to predict the artemisinin resistance status of <jats:italic>Plasmodium falciparum<\/jats:italic> malaria isolates from gene expression data. Data were simulated based on a previously published in vivo parasite gene expression dataset, which contained transcriptomes of 1043\u00a0<jats:italic>P. falciparum<\/jats:italic> isolates from patients with acute malaria, of which 29% (299\/1043) were from slow clearing infections (parasite clearance half-life\u2009&gt;\u20095\u00a0h). Learning curves were produced for two machine learning algorithms, sparse Partial Least Squares-Discriminant Analysis plus Support Vector Machines (sPLSDA\u2009+\u2009SVMs) and random forests. Prediction error was measured using the balanced error rate (average of percentage of slow clearing infections incorrectly predicted as fast and percentage of fast clearing infections predicted as slow).<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>For this mock malaria prediction study, the balanced error rate on a test dataset not used for model training (208 samples) was 50% for sPLSDA\u2009+\u2009SVMs and 50% for random forests on the smallest training dataset evaluated (20 samples) and 14% for sPLSDA\u2009+\u2009SVMs and 22% for random forests on the largest training dataset evaluated (835 samples). The shape of the learning curves indicates that increasing the training dataset size beyond 835 samples is unlikely to significantly reduce the balanced error rates further.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusions<\/jats:title>\n            <jats:p>Learning curves are a simple tool that can be used to determine the minimum sample size required for future prediction modelling studies of different malaria outcomes that use machine learning algorithms for prediction. These curves need to be generated for each specific prediction modelling application.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1186\/s12936-025-05479-3","type":"journal-article","created":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T11:42:39Z","timestamp":1753357359000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["How to use learning curves to evaluate the sample size for malaria prediction models developed using machine learning algorithms"],"prefix":"10.1186","volume":"24","author":[{"given":"Sophie G.","family":"Zaloumis","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Megha","family":"Rajasekhar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julie A.","family":"Simpson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,7,24]]},"reference":[{"key":"5479_CR1","doi-asserted-by":"publisher","first-page":"m441","DOI":"10.1136\/bmj.m441","volume":"368","author":"RD Riley","year":"2020","unstructured":"Riley RD, Ensor J, Snell KIE, Harrell FE, Martin GP, Reitsma JB, et al. Calculating the sample size required for developing a clinical prediction model. BMJ. 2020;368:m441.","journal-title":"BMJ"},{"key":"5479_CR2","doi-asserted-by":"publisher","first-page":"502","DOI":"10.1186\/s12916-023-03212-y","volume":"21","author":"RD Riley","year":"2023","unstructured":"Riley RD, Pate A, Dhiman P, Archer L, Martin GP, Collins GS. Clinical prediction models and the multiverse of madness. BMC Med. 2023;21:502.","journal-title":"BMC Med"},{"key":"5479_CR3","doi-asserted-by":"publisher","first-page":"2455","DOI":"10.1177\/0962280218784726","volume":"28","author":"M van Smeden","year":"2019","unstructured":"van Smeden M, Moons KG, de Groot JA, Collins GS, Altman DG, Eijkemans MJ, et al. Sample size for binary logistic prediction models: beyond events per variable criteria. Stat Methods Med Res. 2019;28:2455\u201374.","journal-title":"Stat Methods Med Res"},{"key":"5479_CR4","doi-asserted-by":"publisher","first-page":"1262","DOI":"10.1002\/sim.7993","volume":"38","author":"RD Riley","year":"2019","unstructured":"Riley RD, Snell KIE, Ensor J, Burke DL, Harrell FE, Moons KGM, et al. Minimum sample size for developing a multivariable prediction model: part I\u2014continuous outcomes. Stat Med. 2019;38:1262\u201375.","journal-title":"Stat Med"},{"key":"5479_CR5","doi-asserted-by":"publisher","first-page":"1276","DOI":"10.1002\/sim.7992","volume":"38","author":"RD Riley","year":"2019","unstructured":"Riley RD, Snell KI, Ensor J, Burke DL, Harrell FE Jr, Moons KG, et al. Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes. Stat Med. 2019;38:1276\u201396.","journal-title":"Stat Med"},{"key":"5479_CR6","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1186\/s12911-021-01403-2","volume":"21","author":"A Brnabic","year":"2021","unstructured":"Brnabic A, Hess LM. Systematic literature review of machine learning methods used in the analysis of real-world data for patient-provider decision making. BMC Med Inform Decis Mak. 2021;21:54.","journal-title":"BMC Med Inform Decis Mak"},{"key":"5479_CR7","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1186\/s13063-021-05489-x","volume":"22","author":"EH Weissler","year":"2021","unstructured":"Weissler EH, Naumann T, Andersson T, Ranganath R, Elemento O, Luo Y, et al. The role of machine learning in clinical research: transforming the future of evidence generation. Trials. 2021;22:537.","journal-title":"Trials"},{"key":"5479_CR8","doi-asserted-by":"publisher","first-page":"e65776","DOI":"10.7554\/eLife.65776","volume":"10","author":"EH Aitken","year":"2021","unstructured":"Aitken EH, Damelang T, Ortega-Pajares A, Alemu A, Hasang W, Dini S, et al. Developing a multivariate prediction model of antibody features associated with protection of malaria-infected pregnant women from placental malaria. Elife. 2021;10:e65776.","journal-title":"Elife"},{"key":"5479_CR9","doi-asserted-by":"publisher","first-page":"e474","DOI":"10.1093\/infdis\/jiae041","volume":"230","author":"NI Silva","year":"2024","unstructured":"Silva NI, Souza PFL, Silva BF, Fonseca SG, Gardinassi LG. Host transcriptional meta-signatures reveal diagnostic biomarkers for Plasmodium falciparum malaria. J Infect Dis. 2024;230:e474\u201385.","journal-title":"J Infect Dis"},{"key":"5479_CR10","doi-asserted-by":"publisher","first-page":"685","DOI":"10.3390\/biomedicines11030685","volume":"11","author":"DM Goldenholz","year":"2023","unstructured":"Goldenholz DM, Sun H, Ganglberger W, Westover MB. Sample size analysis for machine learning clinical validation studies. Biomedicines. 2023;11:685.","journal-title":"Biomedicines"},{"key":"5479_CR11","doi-asserted-by":"publisher","first-page":"7799","DOI":"10.1109\/TPAMI.2022.3220744","volume":"45","author":"T Viering","year":"2023","unstructured":"Viering T, Loog M. The shape of learning curves: a review. IEEE Trans Pattern Anal Mach Intell. 2023;45:7799\u2013819.","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"5479_CR12","doi-asserted-by":"publisher","first-page":"3062","DOI":"10.1002\/sim.10121","volume":"43","author":"A Dayimu","year":"2024","unstructured":"Dayimu A, Simidjievski N, Demiris N, Abraham J. Sample size determination for prediction models via learning-type curves. Stat Med. 2024;43:3062\u201372.","journal-title":"Stat Med"},{"key":"5479_CR13","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1186\/1472-6947-12-8","volume":"12","author":"RL Figueroa","year":"2012","unstructured":"Figueroa RL, Zeng-Treitler Q, Kandula S, Ngo LH. Predicting sample size required for classification performance. BMC Med Inform Decis Mak. 2012;12:8.","journal-title":"BMC Med Inform Decis Mak"},{"key":"5479_CR14","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-024-06619-7","author":"F Mohr","year":"2022","unstructured":"Mohr F, van Rijn JN. Learning curves for decision making in supervised machine learning\u2014a survey. arXiv. 2022. https:\/\/doi.org\/10.1007\/s10994-024-06619-7.","journal-title":"arXiv"},{"key":"5479_CR15","doi-asserted-by":"publisher","first-page":"e60231","DOI":"10.2196\/60231","volume":"26","author":"S Silvey","year":"2024","unstructured":"Silvey S, Liu J. Sample size requirements for popular classification algorithms in tabular clinical data: empirical study. J Med Internet Res. 2024;26:e60231.","journal-title":"J Med Internet Res"},{"key":"5479_CR16","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-0716-1418-1","volume-title":"An introduction to statistical learning: with applications in R","author":"G James","year":"2021","unstructured":"James G, Witten D, Hastie T, Tibshirani R. An introduction to statistical learning: with applications in R. New York: Springer; 2021."},{"key":"5479_CR17","unstructured":"Olteanu A. Tutorial: Learning Curves for Machine Learning in Python. 2018. Available from: https:\/\/www.dataquest.io\/blog\/learning-curves-machine-learning\/"},{"key":"5479_CR18","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-84858-7","volume-title":"The elements of statistical learning","author":"T Hastie","year":"2009","unstructured":"Hastie T, Tibshirani R, Friedman J. The elements of statistical learning. New York: Springer; 2009."},{"key":"5479_CR19","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1126\/science.1260403","volume":"347","author":"S Mok","year":"2015","unstructured":"Mok S, Ashley EA, Ferreira PE, Zhu L, Lin Z, Yeo T, et al. Population transcriptomics of human malaria parasites reveals the mechanism of artemisinin resistance. Science. 2015;347:431\u20135.","journal-title":"Science"},{"key":"5479_CR20","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1056\/NEJMoa1314981","volume":"371","author":"EA Ashley","year":"2014","unstructured":"Ashley EA, Dhorda M, Fairhurst RM, Amaratunga C, Lim P, Suon S, et al. Spread of artemisinin resistance in Plasmodium falciparum malaria. N Engl J Med. 2014;371:411\u201323.","journal-title":"N Engl J Med"},{"key":"5479_CR21","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L. Random forests. Mach Learn. 2001;45:5\u201332.","journal-title":"Mach Learn"},{"key":"5479_CR22","first-page":"18","volume":"2","author":"A Liaw","year":"2002","unstructured":"Liaw A, Wiener M. Classification and regression by randomForest. R News. 2002;2:18\u201322.","journal-title":"R News"},{"key":"5479_CR23","doi-asserted-by":"publisher","first-page":"e1005752","DOI":"10.1371\/journal.pcbi.1005752","volume":"13","author":"F Rohart","year":"2017","unstructured":"Rohart F, Gautier B, Singh A, L\u00ea Cao K-A. mixOmics: an R package for \u2018omics feature selection and multiple data integration. PLoS Comput Biol. 2017;13:e1005752.","journal-title":"PLoS Comput Biol"},{"key":"5479_CR24","unstructured":"Meyer D, Dimitriadou E, Hornik K, Weingessel A, Leisch F. _e1071: Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien. R package version 1.7\u201313, 2023"},{"key":"5479_CR25","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1023\/A:1022627411411","volume":"20","author":"C Cortes","year":"1995","unstructured":"Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20:273\u201397.","journal-title":"Mach Learn"},{"key":"5479_CR26","unstructured":"R Core Team. _R: A Language and Environment for Statistical Computing. 2023; Available from: https:\/\/www.R-project.org\/"},{"key":"5479_CR27","doi-asserted-by":"publisher","first-page":"giz045","DOI":"10.1093\/gigascience\/giz045","volume":"8","author":"R-H Chung","year":"2019","unstructured":"Chung R-H, Kang C-Y. A multi-omics data simulator for complex disease studies and its application to evaluate multi-omics data analysis methods for disease classification. GigaScience. 2019;8:giz045.","journal-title":"GigaScience"},{"key":"5479_CR28","doi-asserted-by":"publisher","DOI":"10.1101\/2021.12.19.473339v1.abstract","author":"H Syed","year":"2021","unstructured":"Syed H, Otto GW, Kelberman D, Bacchelli C, Beales PL. MOPower: an R-shiny application for the simulation and power calculation of multi-omics studies. bioRxiv. 2021. https:\/\/doi.org\/10.1101\/2021.12.19.473339v1.abstract.","journal-title":"bioRxiv"},{"key":"5479_CR29","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbaf110","author":"C Monz\u00f3","year":"2024","unstructured":"Monz\u00f3 C, Mart\u00ednez-Mira C, Arzalluz-Luque \u00c1, Conesa A, Tarazona S. MOSim: bulk and single-cell multi-layer regulatory network simulator. bioRxiv. 2024. https:\/\/doi.org\/10.1093\/bib\/bbaf110.","journal-title":"bioRxiv"},{"key":"5479_CR30","doi-asserted-by":"publisher","first-page":"406","DOI":"10.1038\/nbt.3790","volume":"35","author":"Y Perez-Riverol","year":"2017","unstructured":"Perez-Riverol Y, Bai M, da Veiga LF, Squizzato S, Park YM, Haug K, et al. Discovering and linking public omics data sets using the omics discovery index. Nat Biotechnol. 2017;35:406\u20139.","journal-title":"Nat Biotechnol"},{"key":"5479_CR31","unstructured":"Hoiem D, Gupta T, Li Z, Shlapentokh-Rothman M. Learning Curves for Analysis of Deep Networks. Proceedings of the 38th International Conference on Machine Learning. Proc Machine Learning Res. 2021;139:4287\u201396."},{"key":"5479_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v028.i05","volume":"28","author":"M Kuhn","year":"2008","unstructured":"Kuhn M. Building predictive models in R using the caret package. J Stat Softw. 2008;28:1\u201326.","journal-title":"J Stat Softw"},{"key":"5479_CR33","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011;12:2825\u201330.","journal-title":"J Mach Learn Res"}],"container-title":["Malaria Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12936-025-05479-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12936-025-05479-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12936-025-05479-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,7]],"date-time":"2025-09-07T21:37:07Z","timestamp":1757281027000},"score":1,"resource":{"primary":{"URL":"https:\/\/malariajournal.biomedcentral.com\/articles\/10.1186\/s12936-025-05479-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,24]]},"references-count":33,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["5479"],"URL":"https:\/\/doi.org\/10.1186\/s12936-025-05479-3","relation":{},"ISSN":["1475-2875"],"issn-type":[{"value":"1475-2875","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,24]]},"assertion":[{"value":"5 February 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 July 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 July 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"242"}}