{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,12]],"date-time":"2026-01-12T22:57:13Z","timestamp":1768258633999,"version":"3.49.0"},"reference-count":61,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,1,30]],"date-time":"2020-01-30T00:00:00Z","timestamp":1580342400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,1,30]],"date-time":"2020-01-30T00:00:00Z","timestamp":1580342400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000156","name":"Fonds de Recherche du Qu\u00e9bec - Sant\u00e9","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000156","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100023334","name":"Foundation of stars","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100023334","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000002","name":"U.S. Department of Health & Human Services | National Institutes of Health","doi-asserted-by":"publisher","award":["1R01EB026299-01"],"award-info":[{"award-number":["1R01EB026299-01"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"Gouvernement du Canada | Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["436355-13"],"award-info":[{"award-number":["436355-13"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001804","name":"Canada Research Chairs","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001804","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Meningiomas are known to have relatively lower aggressiveness and better outcomes than other central nervous system (CNS) tumors. However, there is considerable overlap between clinical and radiological features characterizing benign, atypical, and malignant tumors. In this study, we developed methods and a practical app designed to assist with the diagnosis and prognosis of meningiomas. Statistical learning models were trained and validated on 62,844 patients from the Surveillance, Epidemiology, and End Results database. We used balanced logistic regression-random forest ensemble classifiers and proportional hazards models to learn multivariate patterns of association between malignancy, survival, and a series of basic clinical variables\u2014such as tumor size, location, and surgical procedure. We demonstrate that our models are capable of predicting meaningful individual-specific clinical outcome variables and show good generalizability across 16 SEER registries. A free smartphone and web application is provided for readers to access and test the predictive models (<jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"http:\/\/www.meningioma.app\">www.meningioma.app<\/jats:ext-link>). Future model improvements and prospective replication will be necessary to demonstrate true clinical utility. Rather than being used in isolation, we expect that the proposed models will be integrated into larger and more comprehensive models that integrate imaging and molecular biomarkers. Whether for meningiomas or other tumors of the CNS, the power of these methods to make individual-patient predictions could lead to improved diagnosis, patient counseling, and outcomes.<\/jats:p>","DOI":"10.1038\/s41746-020-0219-5","type":"journal-article","created":{"date-parts":[[2020,1,30]],"date-time":"2020-01-30T11:02:57Z","timestamp":1580382177000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["Individual-patient prediction of meningioma malignancy and survival using the Surveillance, Epidemiology, and End Results database"],"prefix":"10.1038","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6545-7525","authenticated-orcid":false,"given":"Jeremy T.","family":"Moreau","sequence":"first","affiliation":[]},{"given":"Todd C.","family":"Hankinson","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6762-5713","authenticated-orcid":false,"given":"Sylvain","family":"Baillet","sequence":"additional","affiliation":[]},{"given":"Roy W. R.","family":"Dudley","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,1,30]]},"reference":[{"key":"219_CR1","doi-asserted-by":"publisher","first-page":"v1","DOI":"10.1093\/neuonc\/nox158","volume":"19","author":"QT Ostrom","year":"2017","unstructured":"Ostrom, Q. T. et al. CBTRUS statistical report: primary brain and other central nervous system tumors diagnosed in the United States in 2010-2014. Neuro. Oncol. 19, v1\u2013v88 (2017).","journal-title":"Neuro. Oncol."},{"key":"219_CR2","doi-asserted-by":"publisher","first-page":"520","DOI":"10.1093\/neuonc\/nop066","volume":"12","author":"KR Porter","year":"2010","unstructured":"Porter, K. R., McCarthy, B. J., Freels, S., Kim, Y. & Davis, F. G. Prevalence estimates for primary brain tumors in the United States by age, gender, behavior, and histology. Neuro. Oncol. 12, 520\u2013527 (2010).","journal-title":"Neuro. Oncol."},{"key":"219_CR3","doi-asserted-by":"publisher","first-page":"4376","DOI":"10.1002\/cncr.29639","volume":"121","author":"AA Aizer","year":"2015","unstructured":"Aizer, A. A. et al. Extent of resection and overall survival for patients with atypical and malignant meningioma. Cancer 121, 4376\u20134381 (2015).","journal-title":"Cancer"},{"key":"219_CR4","doi-asserted-by":"publisher","first-page":"621","DOI":"10.1007\/s11060-018-2756-1","volume":"137","author":"RWR Dudley","year":"2018","unstructured":"Dudley, R. W. R. et al. Pediatric versus adult meningioma: comparison of epidemiology, treatments, and outcomes using the Surveillance, Epidemiology, and End Results database. J. Neurooncol. 137, 621\u2013629 (2018).","journal-title":"J. Neurooncol."},{"key":"219_CR5","unstructured":"National Cancer Institute. Overview of the SEER Program. Available at: https:\/\/seer.cancer.gov\/about\/overview.html. Accessed in 2019."},{"key":"219_CR6","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.wneu.2016.04.098","volume":"92","author":"S Ohba","year":"2016","unstructured":"Ohba, S., Abe, M., Hasegawa, M. & Hirose, Y. Intraparenchymal meningioma: clinical, radiologic, and histologic review. World Neurosurg. 92, 23\u201330 (2016).","journal-title":"World Neurosurg."},{"key":"219_CR7","doi-asserted-by":"publisher","first-page":"57","DOI":"10.3389\/fpubh.2015.00057","volume":"3","author":"Y Yuan","year":"2015","unstructured":"Yuan, Y., Su, W. & Zhu, M. Threshold-free measures for assessing the performance of medical screening tests. Front. Public Health 3, 57 (2015).","journal-title":"Front. Public Health"},{"key":"219_CR8","unstructured":"Cai, H., Yuan, Y., Zhou, Q. M. & Li, B. APtools: Average Positive Predictive Values (AP) for Binary Outcomes and Censored Event Times. R package version 6.8.8. https:\/\/CRAN.R-project.org\/package=APtools (2018)."},{"key":"219_CR9","doi-asserted-by":"publisher","first-page":"1105","DOI":"10.1002\/sim.4154","volume":"30","author":"H Uno","year":"2011","unstructured":"Uno, H., Cai, T., Pencina, M. J., D\u2019Agostino, R. B. & Wei, L. J. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Stat. Med. 30, 1105\u20131117 (2011).","journal-title":"Stat. Med."},{"key":"219_CR10","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4","volume":"15","author":"FE Harrell Jr.","year":"1996","unstructured":"Harrell, F. E. Jr., Lee, K. L. & Mark, D. B. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat. Med. 15, 361\u2013387 (1996).","journal-title":"Stat. Med."},{"key":"219_CR11","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1016\/j.cmpb.2017.12.011","volume":"155","author":"N Shukla","year":"2018","unstructured":"Shukla, N., Hagenbuchner, M., Win, K. T. & Yang, J. Breast cancer data analysis for survivability studies and prediction. Comput. Methods Prog. Biomed. 155, 199\u2013208 (2018).","journal-title":"Comput. Methods Prog. Biomed."},{"key":"219_CR12","doi-asserted-by":"publisher","first-page":"304","DOI":"10.1016\/j.ijmedinf.2016.11.001","volume":"97","author":"RJ Kate","year":"2017","unstructured":"Kate, R. J. & Nadig, R. Stage-specific predictive models for breast cancer survivability. Int. J. Med. Inform. 97, 304\u2013311 (2017).","journal-title":"Int. J. Med. Inform."},{"key":"219_CR13","doi-asserted-by":"publisher","first-page":"392","DOI":"10.5539\/gjhs.v7n4p392","volume":"7","author":"H Lotfnezhad Afshar","year":"2015","unstructured":"Lotfnezhad Afshar, H., Ahmadi, M., Roudbari, M. & Sadoughi, F. Prediction of breast cancer survival through knowledge discovery in databases. Glob. J. Health Sci. 7, 392\u2013398 (2015).","journal-title":"Glob. J. Health Sci."},{"key":"219_CR14","unstructured":"Wang, X., Hershman, D. L. & Neugut, A. I. Using machine learning, general regression, and Cox proportional hazards regression to predict the effectiveness of treatment in patients with breast cancer. AMIA Annu. Symp. Proc. 1133. https:\/\/www.ncbi.nlm.nih.gov\/pubmed\/17238752 (2006)."},{"issue":"Suppl 1","key":"219_CR15","doi-asserted-by":"publisher","DOI":"10.1186\/1755-8794-7-S1-S4","volume":"7","author":"H Shin","year":"2014","unstructured":"Shin, H. & Nam, Y. A coupling approach of a predictor and a descriptor for breast cancer prognosis. BMC Med. Genomics 7(Suppl 1), S4 (2014).","journal-title":"BMC Med. Genomics"},{"key":"219_CR16","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0184370","volume":"12","author":"CM Lynch","year":"2017","unstructured":"Lynch, C. M., van Berkel, V. H. & Frieboes, H. B. Application of unsupervised analysis techniques to lung cancer patient data. PLoS ONE 12, e0184370 (2017).","journal-title":"PLoS ONE"},{"key":"219_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ijmedinf.2017.09.013","volume":"108","author":"CM Lynch","year":"2017","unstructured":"Lynch, C. M. et al. Prediction of lung cancer patient survival via supervised machine learning classification techniques. Int. J. Med. Inform. 108, 1\u20138 (2017).","journal-title":"Int. J. Med. Inform."},{"key":"219_CR18","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1038\/nn.4478","volume":"20","author":"C-W Woo","year":"2017","unstructured":"Woo, C. -W., Chang, L. J., Lindquist, M. A. & Wager, T. D. Building better biomarkers: brain models in translational neuroimaging. Nat. Neurosci. 20, 365\u2013377 (2017).","journal-title":"Nat. Neurosci."},{"key":"219_CR19","doi-asserted-by":"publisher","first-page":"1166","DOI":"10.1093\/neuonc\/nov069","volume":"17","author":"VR Kshettry","year":"2015","unstructured":"Kshettry, V. R. et al. Descriptive epidemiology of World Health Organization grades II and III intracranial meningiomas in the United States. Neuro. Oncol. 17, 1166\u20131173 (2015).","journal-title":"Neuro. Oncol."},{"key":"219_CR20","doi-asserted-by":"publisher","first-page":"368","DOI":"10.3171\/2014.12.SPINE14974","volume":"23","author":"HJ Westwick","year":"2015","unstructured":"Westwick, H. J. & Shamji, M. F. Effects of sex on the incidence and prognosis of spinal meningiomas: a Surveillance, Epidemiology, and End Results study. J. Neurosurg. Spine 23, 368\u2013373 (2015).","journal-title":"J. Neurosurg. Spine"},{"key":"219_CR21","volume":"8","author":"M Amsbaugh","year":"2016","unstructured":"Amsbaugh, M. et al. Patterns of care and outcomes of adjuvant radiotherapy for meningiomas: a surveillance, epidemiology, and end results and medicare linked analysis. Cureus 8, e567 (2016).","journal-title":"Cureus"},{"key":"219_CR22","doi-asserted-by":"publisher","first-page":"2400","DOI":"10.1002\/cncr.29379","volume":"121","author":"TA Dolecek","year":"2015","unstructured":"Dolecek, T. A. et al. Epidemiology of meningiomas post-Public Law 107-206: The Benign Brain Tumor Cancer Registries Amendment Act. Cancer 121, 2400\u20132410 (2015).","journal-title":"Cancer"},{"key":"219_CR23","doi-asserted-by":"publisher","unstructured":"Garzon-Muvdi, T., Yang, W., Lim, M., Brem, H. & Huang, J. Atypical and anaplastic meningioma: outcomes in a population based study. J. Neurooncol. (2017) https:\/\/doi.org\/10.1007\/s11060-017-2436-6.","DOI":"10.1007\/s11060-017-2436-6"},{"key":"219_CR24","unstructured":"Chen, C, Liaw, A. & Breiman, L. Using Random Forest to Learn Imbalanced Data. Vol. 110 1\u201312 (University of California, Berkeley, 2004)."},{"key":"219_CR25","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1111\/j.2517-6161.1972.tb00899.x","volume":"34","author":"DR Cox","year":"1972","unstructured":"Cox, D. R. Regression models and life-tables. J. R. Stat. Soc. Ser. B Stat. Methodol. 34, 187\u2013220 (1972).","journal-title":"J. R. Stat. Soc. Ser. B Stat. Methodol."},{"key":"219_CR26","doi-asserted-by":"publisher","unstructured":"Davidson-Pilon, C. et al. CamDavidsonPilon\/lifelines: v0.20.3. (2019). https:\/\/doi.org\/10.5281\/zenodo.2604107.","DOI":"10.5281\/zenodo.2604107"},{"key":"219_CR27","doi-asserted-by":"crossref","unstructured":"Dietterich, T. G. Ensemble Methods in Machine Learning. in Multiple Classifier Systems 1\u201315 (Springer, Berlin, Heidelberg, 2000).","DOI":"10.1007\/3-540-45014-9_1"},{"key":"219_CR28","doi-asserted-by":"publisher","first-page":"570","DOI":"10.1016\/j.tranon.2017.04.006","volume":"10","author":"P-F Yan","year":"2017","unstructured":"Yan, P. -F. et al. The potential value of preoperative MRI texture and shape analysis in grading meningiomas: a preliminary investigation. Transl. Oncol. 10, 570\u2013577 (2017).","journal-title":"Transl. Oncol."},{"key":"219_CR29","doi-asserted-by":"crossref","unstructured":"Sachdeva, J., Kumar, V., Gupta, I., Khandelwal, N. & Ahuja, C. K. Multiclass brain tumor classification using GA-SVM. in 2011 Developments in E-systems Engineering 182\u2013187 (Institute of Electrical and Electronics Engineers, 2011).","DOI":"10.1109\/DeSE.2011.31"},{"key":"219_CR30","doi-asserted-by":"crossref","unstructured":"Qureshi, H., Sertel, O., Rajpoot, N., Wilson, R. & Gurcan, M. Adaptive Discriminant Wavelet Packet Transform and Local Binary Patterns for Meningioma Subtype Classification. in Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2008 196\u2013204 (Springer, Berlin, Heidelberg, 2008).","DOI":"10.1007\/978-3-540-85990-1_24"},{"key":"219_CR31","doi-asserted-by":"publisher","first-page":"1588","DOI":"10.1093\/neuonc\/nox101","volume":"19","author":"M Nowosielski","year":"2017","unstructured":"Nowosielski, M. et al. Diagnostic challenges in meningioma. Neuro. Oncol. 19, 1588\u20131598 (2017).","journal-title":"Neuro. Oncol."},{"key":"219_CR32","doi-asserted-by":"publisher","first-page":"401","DOI":"10.1158\/2159-8290.CD-12-0095","volume":"2","author":"E Cerami","year":"2012","unstructured":"Cerami, E. et al. The cBio cancer genomics portal: an open platform for exploring multidimensional cancer genomics data. Cancer Disco. 2, 401\u2013404 (2012).","journal-title":"Cancer Disco."},{"key":"219_CR33","doi-asserted-by":"publisher","first-page":"1045","DOI":"10.1007\/s10278-013-9622-7","volume":"26","author":"K Clark","year":"2013","unstructured":"Clark, K. et al. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26, 1045\u20131057 (2013).","journal-title":"J. Digit. Imaging"},{"key":"219_CR34","doi-asserted-by":"publisher","first-page":"1182","DOI":"10.1016\/j.neuroimage.2015.04.028","volume":"124","author":"G Niso","year":"2016","unstructured":"Niso, G. et al. OMEGA: the open MEG archive. Neuroimage 124, 1182\u20131187 (2016).","journal-title":"Neuroimage"},{"key":"219_CR35","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1038\/s41586-018-0579-z","volume":"562","author":"C Bycroft","year":"2018","unstructured":"Bycroft, C. et al. The UK Biobank resource with deep phenotyping and genomic data. Nature 562, 203\u2013209 (2018).","journal-title":"Nature"},{"key":"219_CR36","doi-asserted-by":"publisher","first-page":"1523","DOI":"10.1038\/nn.4393","volume":"19","author":"KL Miller","year":"2016","unstructured":"Miller, K. L. et al. Multimodal population brain imaging in the UK Biobank prospective epidemiological study. Nat. Neurosci. 19, 1523\u20131536 (2016).","journal-title":"Nat. Neurosci."},{"key":"219_CR37","first-page":"90","volume":"60","author":"J Gagnon","year":"2014","unstructured":"Gagnon, J., Leggett, J. A., Richard, C. & Lussier, M. -T. Facilitating informed consent for EMR research in Quebec. Can. Fam. Physician 60, 90 (2014).","journal-title":"Can. Fam. Physician"},{"key":"219_CR38","doi-asserted-by":"publisher","first-page":"1188","DOI":"10.1016\/j.neuroimage.2015.08.076","volume":"124","author":"S Das","year":"2016","unstructured":"Das, S. et al. The MNI data-sharing and processing ecosystem. Neuroimage 124, 1188\u20131195 (2016).","journal-title":"Neuroimage"},{"key":"219_CR39","first-page":"53","volume":"10","author":"S Das","year":"2016","unstructured":"Das, S. et al. Cyberinfrastructure for open science at the Montreal Neurological Institute. Front. Neuroinform 10, 53 (2016).","journal-title":"Front. Neuroinform"},{"key":"219_CR40","first-page":"392","volume":"60","author":"K Keshavjee","year":"2014","unstructured":"Keshavjee, K. et al. Getting to usable EMR data. Can. Fam. Physician 60, 392 (2014).","journal-title":"Can. Fam. Physician"},{"key":"219_CR41","doi-asserted-by":"crossref","unstructured":"Owens, B. Family doctors call for guaranteed access to EMR data for research and quality improvement. CMAJ 190, E60\u2013E61 (2018).","DOI":"10.1503\/cmaj.109-5543"},{"key":"219_CR42","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1038\/nrg3208","volume":"13","author":"PB Jensen","year":"2012","unstructured":"Jensen, P. B., Jensen, L. J. & Brunak, S. Mining electronic health records: towards better research applications and clinical care. Nat. Rev. Genet. 13, 395\u2013405 (2012).","journal-title":"Nat. Rev. Genet."},{"key":"219_CR43","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","volume":"1","author":"A Rajkomar","year":"2018","unstructured":"Rajkomar, A. et al. Scalable and accurate deep learning with electronic health records. NPJ Digital Med. 1, 18 (2018).","journal-title":"NPJ Digital Med."},{"key":"219_CR44","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1227\/NEU.0000000000000619","volume":"76","author":"RWR Dudley","year":"2015","unstructured":"Dudley, R. W. R. et al. Pediatric low-grade ganglioglioma: epidemiology, treatments, and outcome analysis on 348 children from the surveillance, epidemiology, and end results database. Neurosurgery 76, 313\u2013319 (2015).","journal-title":"Neurosurgery"},{"key":"219_CR45","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1007\/s11060-014-1628-6","volume":"121","author":"RWR Dudley","year":"2015","unstructured":"Dudley, R. W. R. et al. Pediatric choroid plexus tumors: epidemiology, treatments, and outcome analysis on 202 children from the SEER database. J. Neurooncol. 121, 201\u2013207 (2015).","journal-title":"J. Neurooncol."},{"key":"219_CR46","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1007\/s11060-012-0966-5","volume":"110","author":"TC Hankinson","year":"2012","unstructured":"Hankinson, T. C. et al. Limited utility despite accuracy of the national SEER dataset for the study of craniopharyngioma. J. Neurooncol. 110, 271\u2013278 (2012).","journal-title":"J. Neurooncol."},{"key":"219_CR47","doi-asserted-by":"publisher","first-page":"289","DOI":"10.3171\/2015.7.PEDS15224","volume":"17","author":"TC Hankinson","year":"2016","unstructured":"Hankinson, T. C. et al. Short-term mortality following surgical procedures for the diagnosis of pediatric brain tumors: outcome analysis in 5533 children from SEER, 2004-2011. J. Neurosurg. Pediatr. 17, 289\u2013297 (2016).","journal-title":"J. Neurosurg. Pediatr."},{"key":"219_CR48","doi-asserted-by":"publisher","first-page":"e55","DOI":"10.1097\/MLR.0000000000000073","volume":"54","author":"A-M Noone","year":"2016","unstructured":"Noone, A. -M. et al. Comparison of SEER treatment data with medicare claims. Med. Care 54, e55\u2013e64 (2016).","journal-title":"Med. Care"},{"key":"219_CR49","doi-asserted-by":"publisher","first-page":"669","DOI":"10.3171\/2012.7.JNS111439","volume":"117","author":"AM Stessin","year":"2012","unstructured":"Stessin, A. M. et al. Does adjuvant external-beam radiotherapy improve outcomes for nonbenign meningiomas? A Surveillance, Epidemiology, and End Results (SEER)-based analysis. J. Neurosurg. 117, 669\u2013675 (2012).","journal-title":"J. Neurosurg."},{"key":"219_CR50","doi-asserted-by":"publisher","first-page":"55","DOI":"10.7326\/M14-0697","volume":"162","author":"GS Collins","year":"2015","unstructured":"Collins, G. S., Reitsma, J. B., Altman, D. G. & Moons, K. G. M. Transparent reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann. Intern. Med. 162, 55\u201363 (2015).","journal-title":"Ann. Intern. Med."},{"key":"219_CR51","unstructured":"Ripley, B. D. Pattern Recognition and Neural Networks. (Cambridge University Press, 2007)."},{"key":"219_CR52","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825\u20132830 (2011).","journal-title":"J. Mach. Learn. Res."},{"key":"219_CR53","first-page":"1","volume":"18","author":"G Lema\u00eetre","year":"2017","unstructured":"Lema\u00eetre, G., Nogueira, F. & Aridas, C. K. Imbalanced-learn: a Python toolbox to tackle the curse of imbalanced datasets in machine learning. J. Mach. Learn. Res. 18, 1\u20135 (2017).","journal-title":"J. Mach. Learn. Res."},{"key":"219_CR54","doi-asserted-by":"publisher","first-page":"638","DOI":"10.21105\/joss.00638","volume":"3","author":"S Raschka","year":"2018","unstructured":"Raschka, S. MLxtend: providing machine learning and data science utilities and extensions to Python\u2019s scientific computing stack. J. Open Sour. Softw. 3, 638 (2018).","journal-title":"J. Open Sour. Softw."},{"key":"219_CR55","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman, L. Random Forests. Mach. Learn. 45, 5\u201332 (2001).","journal-title":"Mach. Learn."},{"key":"219_CR56","doi-asserted-by":"crossref","unstructured":"Platt, J. C. Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods. in Advances in Large Margin Classifiers 61\u201374 (MIT Press, 1999).","DOI":"10.7551\/mitpress\/1113.003.0008"},{"key":"219_CR57","first-page":"281","volume":"13","author":"J Bergstra","year":"2012","unstructured":"Bergstra, J. & Bengio, Y. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, 281\u2013305 (2012).","journal-title":"J. Mach. Learn. Res."},{"key":"219_CR58","doi-asserted-by":"publisher","first-page":"1323","DOI":"10.1109\/TIP.2017.2781298","volume":"27","author":"M Liu","year":"2018","unstructured":"Liu, M. et al. Cost-sensitive feature selection by optimizing F-measures. IEEE Trans. Image Process 27, 1323\u20131335 (2018).","journal-title":"IEEE Trans. Image Process"},{"key":"219_CR59","doi-asserted-by":"publisher","first-page":"855","DOI":"10.1016\/j.jclinepi.2015.02.010","volume":"68","author":"B Ozenne","year":"2015","unstructured":"Ozenne, B., Subtil, F. & Maucort-Boulch, D. The precision\u2013recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases. J. Clin. Epidemiol. 68, 855\u2013859 (2015).","journal-title":"J. Clin. Epidemiol."},{"key":"219_CR60","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1111\/j.0006-341X.2000.00337.x","volume":"56","author":"PJ Heagerty","year":"2000","unstructured":"Heagerty, P. J., Lumley, T. & Pepe, M. S. Time-dependent ROC curves for censored survival data and a diagnostic marker. Biometrics 56, 337\u2013344 (2000).","journal-title":"Biometrics"},{"key":"219_CR61","doi-asserted-by":"crossref","unstructured":"P\u00f6lsterl, S., Navab, N. & Katouzian, A. in Machine Learning and Knowledge Discovery in Databases. 243\u2013259 (Springer International Publishing, 2015).","DOI":"10.1007\/978-3-319-23525-7_15"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0219-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0219-5","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0219-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,30]],"date-time":"2024-07-30T21:27:54Z","timestamp":1722374874000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0219-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,30]]},"references-count":61,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["219"],"URL":"https:\/\/doi.org\/10.1038\/s41746-020-0219-5","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1,30]]},"assertion":[{"value":"7 November 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 January 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 January 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"12"}}