{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:35:24Z","timestamp":1785537324161,"version":"3.56.0"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,10,16]],"date-time":"2020-10-16T00:00:00Z","timestamp":1602806400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,10,16]],"date-time":"2020-10-16T00:00:00Z","timestamp":1602806400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The rate of disability accumulation varies across multiple sclerosis (MS) patients. Machine learning techniques may offer more powerful means to predict disease course in MS patients. In our study, 724 patients from the Comprehensive Longitudinal Investigation in MS at Brigham and Women\u2019s Hospital (CLIMB study) and 400 patients from the EPIC dataset, University of California, San Francisco, were included in the analysis. The primary outcome was an increase in <jats:italic>Expanded Disability Status Scale<\/jats:italic> (<jats:italic>EDSS<\/jats:italic>)\u2009\u2265\u20091.5 (worsening) or not (non-worsening) at up to 5 years after the baseline visit. Classification models were built using the CLIMB dataset with patients\u2019 clinical and MRI longitudinal observations in first 2 years, and further validated using the EPIC dataset. We compared the performance of three popular machine learning algorithms (<jats:italic>SVM, Logistic Regression,<\/jats:italic> and <jats:italic>Random Forest<\/jats:italic>) and three ensemble learning approaches (<jats:italic>XGBoost, LightGBM<\/jats:italic>, and a Meta-learner <jats:italic>L<\/jats:italic>). A \u201cthreshold\u201d was established to trade-off the performance between the two classes. Predictive features were identified and compared among different models. Machine learning models achieved 0.79 and 0.83 AUC scores for the CLIMB and EPIC datasets, respectively, shortly after disease onset. Ensemble learning methods were more effective and robust compared to standalone algorithms. Two ensemble models, XGBoost and LightGBM were superior to the other four models evaluated in our study. Of variables evaluated, EDSS, <jats:italic>Pyramidal Function<\/jats:italic>, and <jats:italic>Ambulatory Index<\/jats:italic> were the top common predictors in forecasting the MS disease course. Machine learning techniques, in particular ensemble methods offer increased accuracy for the prediction of MS disease course.<\/jats:p>","DOI":"10.1038\/s41746-020-00338-8","type":"journal-article","created":{"date-parts":[[2020,10,16]],"date-time":"2020-10-16T10:02:55Z","timestamp":1602842575000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":48,"title":["Ensemble learning predicts multiple sclerosis disease course in the SUMMIT study"],"prefix":"10.1038","volume":"3","author":[{"given":"Yijun","family":"Zhao","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tong","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Riley","family":"Bove","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bruce","family":"Cree","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roland","family":"Henry","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hrishikesh","family":"Lokhande","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1244-1802","authenticated-orcid":false,"given":"Mariann","family":"Polgar-Turcsanyi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mark","family":"Anderson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8601-5534","authenticated-orcid":false,"given":"Rohit","family":"Bakshi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Howard L.","family":"Weiner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9897-4422","authenticated-orcid":false,"given":"Tanuja","family":"Chitnis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"name":"SUMMIT Investigators","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,10,16]]},"reference":[{"key":"338_CR1","doi-asserted-by":"crossref","unstructured":"Mowry, E. M. Natural history of multiple sclerosis: early prognostic factors. Neurol. Clin. 29, 279\u2013292 (2011).","DOI":"10.1016\/j.ncl.2011.01.001"},{"key":"338_CR2","doi-asserted-by":"publisher","first-page":"595","DOI":"10.1093\/brain\/awh714","volume":"129","author":"C Confavreux","year":"2006","unstructured":"Confavreux, C. & Vukusic, S. Age at disability milestones in multiple sclerosis. Brain 129, 595\u2013605 (2006).","journal-title":"Brain"},{"key":"338_CR3","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1093\/brain\/awg081","volume":"126","author":"C Confavreux","year":"2003","unstructured":"Confavreux, C., Vukusic, S. & Adeleine, P. Early clinical predictors and progression of irreversible disability in multiple sclerosis: an amnesic process. Brain 126, 770\u2013782 (2003).","journal-title":"Brain"},{"key":"338_CR4","doi-asserted-by":"publisher","first-page":"2603","DOI":"10.1056\/NEJMoa067597","volume":"356","author":"C Renoux","year":"2007","unstructured":"Renoux, C. et al. Natural history of multiple sclerosis with childhood onset. N. Engl. J. Med. 356, 2603\u20132613 (2007).","journal-title":"N. Engl. J. Med."},{"key":"338_CR5","doi-asserted-by":"crossref","unstructured":"Amato, M. & Ponziani, G. A prospective study on the prognosis of multiple sclerosis. Neurol. Sci. 21, S831\u2013S838 (2000).","DOI":"10.1007\/s100720070021"},{"key":"338_CR6","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1093\/brain\/112.1.133","volume":"112","author":"BG Weinshenker","year":"1989","unstructured":"Weinshenker, B. G. et al. The natural history of multiple sclerosis: a geographically based study: I. Clinical course and disability. Brain 112, 133\u2013146 (1989).","journal-title":"Brain"},{"key":"338_CR7","doi-asserted-by":"publisher","first-page":"1419","DOI":"10.1093\/brain\/112.6.1419","volume":"112","author":"BG Weinshenker","year":"1989","unstructured":"Weinshenker, B. G. et al. The natural history of multiple sclerosis: a geographically based study: 2 predictive value of the early clinical course. Brain 112, 1419\u20131428 (1989).","journal-title":"Brain"},{"key":"338_CR8","doi-asserted-by":"publisher","first-page":"1044","DOI":"10.1001\/archneurol.65.8.noc80042","volume":"65","author":"C Krishnan","year":"2008","unstructured":"Krishnan, C. et al. Reduction of disease activity and disability with high-dose cyclophosphamide in patients with aggressive multiple sclerosis. Arch. Neurol. 65, 1044\u20131051 (2008).","journal-title":"Arch. Neurol."},{"key":"338_CR9","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1016\/S0140-6736(07)61194-5","volume":"370","author":"L Kappos","year":"2007","unstructured":"Kappos, L. et al. Effect of early versus delayed interferon beta-1b treatment on disability after a first clinical event suggestive of multiple sclerosis: a 3-year follow-up analysis of the BENEFIT study. Lancet 370, 389\u2013397 (2007).","journal-title":"Lancet"},{"key":"338_CR10","doi-asserted-by":"publisher","first-page":"1444","DOI":"10.1212\/WNL.33.11.1444","volume":"33","author":"JF Kurtzke","year":"1983","unstructured":"Kurtzke, J. F. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology 33, 1444\u20131444 (1983).","journal-title":"Neurology"},{"key":"338_CR11","unstructured":"Nielsen, D. Tree Boosting with Xgboost-why Does Xgboost Win Every Machine Learning Competition? Master\u2019s thesis, NTNU (2016)."},{"key":"338_CR12","unstructured":"Kasturi, S. N. XGBOOST vs LightGBM: Which Algorithm Wins the Race!!! https:\/\/towardsdatascience.com\/lightgbm-vs-xgboost-which-algorithm-win-the-race-1ff7dd4917d (2019)."},{"key":"338_CR13","doi-asserted-by":"publisher","first-page":"370","DOI":"10.2307\/2344614","volume":"135","author":"JA Nelder","year":"1972","unstructured":"Nelder, J. A. & Wedderburn, R. W. Generalized linear models. J. R. Stat. Soc. Ser. A (Gen.) 135, 370\u2013384 (1972).","journal-title":"J. R. Stat. Soc. Ser. A (Gen.)"},{"key":"338_CR14","first-page":"81","volume":"1","author":"JR Quinlan","year":"1986","unstructured":"Quinlan, J. R. Induction of decision trees. Mach. Learn. 1, 81\u2013106 (1986).","journal-title":"Mach. Learn."},{"key":"338_CR15","doi-asserted-by":"publisher","first-page":"1485","DOI":"10.1177\/1352458517726657","volume":"24","author":"R Bove","year":"2018","unstructured":"Bove, R. et al. SUMMIT (Serially Unified Multicenter Multiple Sclerosis Investigation): creating a repository of deeply phenotyped contemporary multiple sclerosis cohorts. Mult. Scler. 24, 1485\u20131498 (2018).","journal-title":"Mult. Scler."},{"key":"338_CR16","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1016\/j.autrev.2006.02.012","volume":"5","author":"SA Gauthier","year":"2006","unstructured":"Gauthier, S. A., Glanz, B. I., Mandel, M. & Weiner, H. L. A model for the comprehensive investigation of a chronic autoimmune disease: the multiple sclerosis CLIMB study. Autoimmun. Rev. 5, 532\u2013536 (2006).","journal-title":"Autoimmun. Rev."},{"key":"338_CR17","unstructured":"Longadge, R., Dongre, S. & Malik, L. Class imbalance problem in data mining: review. Int. J. Comp. Sci. Netw. 2, 83 (2013)."},{"key":"338_CR18","first-page":"273","volume":"20","author":"C Cortes","year":"1995","unstructured":"Cortes, C. & Vapnik, V. Support-vector networks. Mach. Learn. 20, 273\u2013297 (1995).","journal-title":"Mach. Learn."},{"key":"338_CR19","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman, L. Random forests. Mach. Learn. 45, 5\u201332 (2001).","journal-title":"Mach. Learn."},{"key":"338_CR20","doi-asserted-by":"crossref","unstructured":"Hosmer, D. W. Jr, Lemeshow, S. & Sturdivant, R. X. Applied Logistic Regression, Vol. 398 (Wiley, Hoboken, 2013).","DOI":"10.1002\/9781118548387"},{"key":"338_CR21","doi-asserted-by":"crossref","unstructured":"Dietterich, T. G. Ensemble methods in machine learning. In Proc. of the International Workshop on Multiple Classifier Systems, Vol. 1857 (MCS, Cagliari, Italy, 2000).","DOI":"10.1007\/3-540-45014-9_1"},{"key":"338_CR22","doi-asserted-by":"crossref","unstructured":"Chen, T. & Guestrin, C. Xgboost: A scalable tree boosting system. In Proc. of the 22nd acm sigkdd International Conference on Knowledge Discovery and Data Mining, 785\u2013794 (KDD, New York, NY, 2016).","DOI":"10.1145\/2939672.2939785"},{"key":"338_CR23","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","volume":"5","author":"DH Wolpert","year":"1992","unstructured":"Wolpert, D. H. Stacked generalization. Neural Netw. 5, 241\u2013259 (1992).","journal-title":"Neural Netw."},{"key":"338_CR24","first-page":"771","volume":"14","author":"Y Freund","year":"1999","unstructured":"Freund, Y., Schapire, R. & Abe, N. A short introduction to boosting. J.-Jpn. Soc. Artif. Intell. 14, 771\u2013780 (1999).","journal-title":"J.-Jpn. Soc. Artif. Intell."},{"key":"338_CR25","unstructured":"Ke, G. et al. Lightgbm: A highly efficient gradient boosting decision tree. In Proc. of the 31st Conference on Neural Information Processing Systems (NIPS, Long Beach, CA, 2017)."}],"updated-by":[{"DOI":"10.1038\/s41746-020-00361-9","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2020,11,20]],"date-time":"2020-11-20T00:00:00Z","timestamp":1605830400000}}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00338-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00338-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00338-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T02:39:48Z","timestamp":1670380788000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00338-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,16]]},"references-count":25,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["338"],"URL":"https:\/\/doi.org\/10.1038\/s41746-020-00338-8","relation":{"correction":[{"id-type":"doi","id":"10.1038\/s41746-020-00361-9","asserted-by":"object"}]},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,16]]},"assertion":[{"value":"18 February 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 September 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 October 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 November 2020","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Update","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A Correction to this paper has been published:","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"https:\/\/doi.org\/10.1038\/s41746-020-00361-9","URL":"https:\/\/doi.org\/10.1038\/s41746-020-00361-9","order":7,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 November 2020","order":8,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Correction","order":9,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A Correction to this paper has been published:","order":10,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"https:\/\/doi.org\/10.1038\/s41746-020-00361-9","URL":"https:\/\/doi.org\/10.1038\/s41746-020-00361-9","order":11,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests. Complete disclosures are listed on ICJME forms.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"135"}}