{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,3]],"date-time":"2026-11-03T16:45:33Z","timestamp":1793724333856,"version":"4.6.1"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,2,24]],"date-time":"2021-02-24T00:00:00Z","timestamp":1614124800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,2,24]],"date-time":"2021-02-24T00:00:00Z","timestamp":1614124800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    As the burden of neurodegenerative diseases increases, time-limited clinic encounters do not allow quantification of complex neurological functions. Patient-collected digital biomarkers may remedy this, if they provide reliable information. However, psychometric properties of digital tools remain largely un-assessed. We developed a smartphone adaptation of the cognitive test, the Symbol-Digit Modalities Test (SDMT) by randomizing the test\u2019s symbol-number codes and testing sequences. The smartphone SDMT showed comparable psychometric properties in 154 multiple sclerosis (MS) patients and 39 healthy volunteers (HV). E.g., smartphone SDMT achieved slightly higher correlations with cognitive subscores of neurological examinations and with brain injury measured by MRI (\n                    <jats:italic>R<\/jats:italic>\n                    <jats:sup>2<\/jats:sup>\n                    \u2009=\u20090.75,\u2009Rho\u2009=\u20090.83,\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.0001) than traditional SDMT. Mathematical adjustment for motoric disability of the dominant hand, measured by another smartphone test, compensates for the disadvantage of touch-based test. Averaging granular home measurements of the digital biomarker also increases accuracy of identifying true neurological decline.\n                  <\/jats:p>","DOI":"10.1038\/s41746-021-00401-y","type":"journal-article","created":{"date-parts":[[2021,2,24]],"date-time":"2021-02-24T06:03:35Z","timestamp":1614146615000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":64,"title":["Smartphone-based symbol-digit modalities test reliably captures brain damage in multiple sclerosis"],"prefix":"10.1038","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6664-726X","authenticated-orcid":false,"given":"Linh","family":"Pham","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Harris","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mihael","family":"Varosanec","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5854-8277","authenticated-orcid":false,"given":"Vanessa","family":"Morgan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter","family":"Kosa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0959-9430","authenticated-orcid":false,"given":"Bibiana","family":"Bielekova","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,2,24]]},"reference":[{"key":"401_CR1","doi-asserted-by":"publisher","first-page":"2254","DOI":"10.1212\/WNL.0000000000000509","volume":"82","author":"BA Racette","year":"2014","unstructured":"Racette, B. A., Holtzman, D. M., Dall, T. M. & Drogan, O. Supply and demand analysis of the current and future US neurology workforce. Neurology 82, 2254\u20132255 (2014).","journal-title":"Neurology"},{"key":"401_CR2","doi-asserted-by":"publisher","first-page":"358","DOI":"10.3389\/fneur.2019.00358","volume":"10","author":"AK Boukhvalova","year":"2019","unstructured":"Boukhvalova, A. K. et al. Smartphone level test measures disability in several neurological domains for patients with multiple sclerosis. Front Neurol. 10, 358 (2019).","journal-title":"Front Neurol."},{"key":"401_CR3","doi-asserted-by":"publisher","unstructured":"Boukhvalova, A. K. et al. Identifying and quantifying neurological disability via smartphone. Front. Neurol. 9, https:\/\/doi.org\/10.3389\/fneur.2018.00740 (2018).","DOI":"10.3389\/fneur.2018.00740"},{"key":"401_CR4","unstructured":"Smith, A. Symbol digit modalities test: Manual. (Western Psychological Corporation, Los Angeles, California, 2002)."},{"key":"401_CR5","doi-asserted-by":"publisher","first-page":"721","DOI":"10.1177\/1352458517690821","volume":"23","author":"RH Benedict","year":"2017","unstructured":"Benedict, R. H. et al. Validity of the Symbol Digit Modalities Test as a cognition performance outcome measure for multiple sclerosis. Mult. Scler. 23, 721\u2013733 (2017).","journal-title":"Mult. Scler."},{"key":"401_CR6","doi-asserted-by":"publisher","first-page":"940","DOI":"10.1177\/1352458508090923","volume":"14","author":"RHB Benedict","year":"2008","unstructured":"Benedict, R. H. B. et al. Repeated assessment of neuropsychological deficits in multiple sclerosis using the Symbol Digit Modalities Test and the MS Neuropsychological Screening Questionnaire. Mult. Scler. 14, 940\u2013946 (2008).","journal-title":"Mult. Scler."},{"key":"401_CR7","doi-asserted-by":"publisher","unstructured":"Pham, L., Harris, T., Varosanec, M., Kosa, P. & Bielekova, B. Smartphone-based symbol-digit modalities test reliably measures cognitive function in multiple sclerosis patients. medRxiv (2020) https:\/\/doi.org\/10.1101\/2020.03.09.20033316.","DOI":"10.1101\/2020.03.09.20033316"},{"key":"401_CR8","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1177\/135245850100700409","volume":"7","author":"J Boringa","year":"2001","unstructured":"Boringa, J. et al. The brief repeatable battery of neuropsychological tests: normative values allow application in multiple sclerosis clinical practice. Mult. Scler. J. 7, 263\u2013267 (2001).","journal-title":"Mult. Scler. J."},{"key":"401_CR9","doi-asserted-by":"publisher","first-page":"1320","DOI":"10.1177\/1352458511435717","volume":"18","author":"RH Benedict","year":"2012","unstructured":"Benedict, R. H. et al. Reliability and equivalence of alternate forms for the Symbol Digit Modalities Test: implications for multiple sclerosis clinical trials. Mult. Scler. 18, 1320\u20131325 (2012).","journal-title":"Mult. Scler."},{"key":"401_CR10","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1016\/S0140-6736(86)90837-8","volume":"1","author":"JM Bland","year":"1986","unstructured":"Bland, J. M. & Altman, D. G. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1, 307\u2013310 (1986).","journal-title":"Lancet"},{"key":"401_CR11","doi-asserted-by":"publisher","first-page":"1241","DOI":"10.1002\/acn3.640","volume":"5","author":"P Kosa","year":"2018","unstructured":"Kosa, P. et al. NeurEx: digitalized neurological examination offers a novel high-resolution disability scale. Ann. Clin. Transl. Neurol. 5, 1241\u20131249 (2018).","journal-title":"Ann. Clin. Transl. Neurol."},{"key":"401_CR12","first-page":"975803","volume":"2014","author":"SM Rao","year":"2014","unstructured":"Rao, S. M. et al. Correlations between MRI and information processing speed in MS: a meta-analysis. Mult. Scler. Int. 2014, 975803 (2014).","journal-title":"Mult. Scler. Int."},{"key":"401_CR13","doi-asserted-by":"publisher","first-page":"1745","DOI":"10.1177\/1352458514533229","volume":"20","author":"RH Benedict","year":"2014","unstructured":"Benedict, R. H. et al. Characterizing cognitive function during relapse in multiple sclerosis. Mult. Scler. 20, 1745\u20131752 (2014).","journal-title":"Mult. Scler."},{"key":"401_CR14","doi-asserted-by":"publisher","first-page":"1131","DOI":"10.1080\/13854046.2010.511272","volume":"24","author":"SA Morrow","year":"2010","unstructured":"Morrow, S. A. et al. Predicting loss of employment over three years in multiple sclerosis: clinically meaningful cognitive decline. Clin. Neuropsychol. 24, 1131\u20131145 (2010).","journal-title":"Clin. Neuropsychol."},{"key":"401_CR15","doi-asserted-by":"publisher","first-page":"1035","DOI":"10.1136\/jnnp-2013-307275","volume":"85","author":"M Pardini","year":"2014","unstructured":"Pardini, M. et al. Isolated cognitive relapses in multiple sclerosis. J. Neurol. Neurosurg. Psychiatry 85, 1035\u20131037 (2014).","journal-title":"J. Neurol. Neurosurg. Psychiatry"},{"key":"401_CR16","doi-asserted-by":"publisher","first-page":"131","DOI":"10.3389\/fneur.2016.00131","volume":"7","author":"P Kosa","year":"2016","unstructured":"Kosa, P. et al. Development of a sensitive outcome for economical drug screening for progressive multiple sclerosis treatment. Front. Neurol. 7, 131 (2016).","journal-title":"Front. Neurol."},{"key":"401_CR17","unstructured":"Food & Administration, D. General principles of software validation; Final guidance for industry and FDA staff. Food Drug Admin. 11 (2002). https:\/\/www.fda.gov\/regulatory-information\/search-fda-guidance-documents\/general-principles-software-validation."},{"key":"401_CR18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-020-0286-7","volume":"3","author":"CM Kanzler","year":"2020","unstructured":"Kanzler, C. M. et al. A data-driven framework for selecting and validating digital health metrics: use-case in neurological sensorimotor impairments. NPJ Digital Med. 3, 1\u201317 (2020).","journal-title":"NPJ Digital Med."},{"key":"401_CR19","doi-asserted-by":"publisher","DOI":"10.7224\/1537-2073.2018-108","author":"M Beier","year":"2019","unstructured":"Beier, M. et al. iCAMS: assessing the reliability of a BICAMS tablet application. Int. J. MS Care Prepr. https:\/\/doi.org\/10.7224\/1537-2073.2018-108 (2019).","journal-title":"Int. J. MS Care Prepr."},{"key":"401_CR20","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1016\/j.msard.2016.09.009","volume":"10","author":"M Roar","year":"2016","unstructured":"Roar, M., Illes, Z. & Sejbaek, T. Practice effect in symbol digit modalities test in multiple sclerosis patients treated with natalizumab. Mult. Scler. Relat. Disord. 10, 116\u2013122 (2016).","journal-title":"Mult. Scler. Relat. Disord."},{"key":"401_CR21","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1002\/ana.22366","volume":"69","author":"CH Polman","year":"2011","unstructured":"Polman, C. H. et al. Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann. Neurol. 69, 292\u2013302 (2011).","journal-title":"Ann. Neurol."},{"key":"401_CR22","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1016\/S1474-4422(17)30470-2","volume":"17","author":"AJ Thompson","year":"2018","unstructured":"Thompson, A. J. et al. Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. Lancet Neurol. 17, 162\u2013173 (2018).","journal-title":"Lancet Neurol."},{"key":"401_CR23","doi-asserted-by":"publisher","first-page":"526","DOI":"10.1016\/j.msard.2015.08.009","volume":"4","author":"P Kosa","year":"2015","unstructured":"Kosa, P. et al. Novel composite MRI scale correlates highly with disability in multiple sclerosis patients. Mult. Scler. Relat. Disord. 4, 526\u2013535 (2015).","journal-title":"Mult. Scler. Relat. Disord."},{"key":"401_CR24","doi-asserted-by":"publisher","first-page":"402","DOI":"10.1016\/j.nicl.2013.03.002","volume":"2","author":"EM Sweeney","year":"2013","unstructured":"Sweeney, E. M. et al. OASIS is automated statistical inference for segmentation, with applications to multiple sclerosis lesion segmentation in MRI. Neuroimage Clin. 2, 402\u2013413 (2013).","journal-title":"Neuroimage Clin."},{"key":"401_CR25","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0168321","volume":"11","author":"RA Parker","year":"2016","unstructured":"Parker, R. A. et al. Application of mixed effects limits of agreement in the presence of multiple sources of variability: exemplar from the comparison of several devices to measure respiratory rate in COPD patients. PLoS ONE 11, e0168321 (2016).","journal-title":"PLoS ONE"},{"key":"401_CR26","unstructured":"R Core Team. R: A Language and Environment for Statistical Computing (2018). RFoundation for Statistical Computing, Vienna, Austria. https:\/\/www.R-project.org\/."},{"key":"401_CR27","doi-asserted-by":"crossref","unstructured":"Sarkar, D. Lattice: Multivariate Data Visualization with R. (Springer, 2008).","DOI":"10.1007\/978-0-387-75969-2"},{"key":"401_CR28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v067.i01","volume":"67","author":"D Bates","year":"2015","unstructured":"Bates, D., Martin, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using {lme4}. J. Stat. Softw. 67, 1\u201348 (2015).","journal-title":"J. Stat. Softw."},{"key":"401_CR29","unstructured":"Ludecke, D. sjPlot: Data Visualization for Statistics in Social Science v. R package version 2.8.2 (2020)."},{"key":"401_CR30","unstructured":"Kassambara, A. ggcorrplot: Visualization of a Correlation Matrix using \u2018ggplot2\u2019 v. R package version 0.1.3 (2019)."},{"key":"401_CR31","unstructured":"Signorell, A. et al. DescTools: Tools for Descriptive Statistics v. R package version 0.99.28 (2019)."},{"key":"401_CR32","doi-asserted-by":"crossref","unstructured":"Wickham, H. Reshaping data with the reshape package. J. Stat. Softw. 21, 1\u201320 (2007).","DOI":"10.18637\/jss.v021.i12"},{"key":"401_CR33","unstructured":"Bakdash, J., Marusich, L. rmcorr: Repeated Measures Correlation v. R package version 0.3.0 (2018)."},{"key":"401_CR34","first-page":"5","volume":"1","author":"R Cairo","year":"2019","unstructured":"Cairo, R. Graphics device using cairo graphics library for creating high-quality bitmap (PNG, JPEG, TIFF), Vector (PDF, SVG, PostScript) and display (X11 and Win32) output v. R. Package Version 1, 5\u201310 (2019).","journal-title":"R. Package Version"},{"key":"401_CR35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v039.i05","volume":"39","author":"N Simon","year":"2011","unstructured":"Simon, N., Friedman, J., Hastie, T. & Tibshirani, R. Regularization Paths for Cox\u2019s proportional hazards model via coordinate descent. J. Stat. Softw. 39, 1\u201313 (2011).","journal-title":"J. Stat. Softw."},{"key":"401_CR36","unstructured":"Kuhn, M. et al. caret: Classification and Regression Training v. R package version 6.0-85 (2020)."},{"key":"401_CR37","unstructured":"Pinheiro, J., Bates, D., Debroy, S., Sarkar, D., R Core Team. Linear and Nonlinear Mixed Effects Models v. R package version 3.1-137 (2018)."},{"key":"401_CR38","unstructured":"Fox, J. & Weisberg, S. An {R} Companion to Applied Regression. 3rd edn. (Sage, 2019)."},{"key":"401_CR39","unstructured":"Kassambara, A. ggpubr: \u2018ggplot2\u2019 Based Publication Ready Plots v. R package version 0.2.4 (2019)."},{"key":"401_CR40","doi-asserted-by":"crossref","unstructured":"Wickham, H. ggplot2: Elegant Graphics for Data Analysis. (Springer-Verlag New York, 2016).","DOI":"10.1007\/978-3-319-24277-4"},{"key":"401_CR41","doi-asserted-by":"crossref","unstructured":"Grolemund, G. & Wickham, H. Dates and times made easy with {lubridate}. 40, 1\u201325 (2011).","DOI":"10.18637\/jss.v040.i03"},{"key":"401_CR42","unstructured":"Wickham, H., RStudio. tidyr: Tidy Messy Data v. R package version 1.0.2 (2020)."},{"key":"401_CR43","unstructured":"Wickham, H. dplyr: A. Grammar of Data Manipulation v. R package version 0.8.3 (2019)."},{"key":"401_CR44","unstructured":"Gandrud, C. DataCombine: Tools for Easily Combining and Cleaning Data Sets v. R package version 0.2.21 (2016)."},{"key":"401_CR45","unstructured":"Wickham, H. et al. readr: Read Rectangular Text Data v. R package version 1.3.1 (2018)."},{"key":"401_CR46","unstructured":"Ram, K., Wickham, H., Richards, C., Baggett, A. wesanderson: A Wes Anderson Palette Generator (2018)."},{"key":"401_CR47","doi-asserted-by":"publisher","first-page":"255","DOI":"10.2307\/2532051","volume":"45","author":"LI Lin","year":"1989","unstructured":"Lin, L. I. A concordance correlation coefficient to evaluate reproducibility. Biometrics 45, 255\u2013268 (1989).","journal-title":"Biometrics"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00401-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00401-y","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00401-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,3]],"date-time":"2022-12-03T13:45:28Z","timestamp":1670075128000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00401-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,24]]},"references-count":47,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["401"],"URL":"https:\/\/doi.org\/10.1038\/s41746-021-00401-y","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2020.03.09.20033316","asserted-by":"object"}]},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,24]]},"assertion":[{"value":"20 August 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 January 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"36"}}