{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T09:43:11Z","timestamp":1777369391208,"version":"3.51.4"},"reference-count":51,"publisher":"Oxford University Press (OUP)","issue":"11","license":[{"start":{"date-parts":[[2024,9,30]],"date-time":"2024-09-30T00:00:00Z","timestamp":1727654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>This article describes the design and evaluation of MS Pattern Explorer, a novel visual tool that uses interactive machine learning to analyze fitness wearables\u2019 data. Applied to a clinical study of multiple sclerosis (MS) patients, the tool addresses key challenges: managing activity signals, accelerating insight generation, and rapidly contextualizing identified patterns. By analyzing sensor measurements, it aims to enhance understanding of MS symptomatology and improve the broader problem of clinical exploratory sensor data analysis.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>Following a user-centered design approach, we learned that clinicians have 3 priorities for generating insights for the Barka-MS study data: exploration and search for, and contextualization of, sequences and patterns in patient sleep and activity. We compute meaningful sequences for patients using clustering and proximity search, displaying these with an interactive visual interface composed of coordinated views. Our evaluation posed both closed and open-ended tasks to participants, utilizing a scoring system to gauge the tool\u2019s usability, and effectiveness in supporting insight generation across 15 clinicians, data scientists, and non-experts.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results and Discussion<\/jats:title>\n                  <jats:p>We present MS Pattern Explorer, a visual analytics system that helps clinicians better address complex data-centric challenges by facilitating the understanding of activity patterns. It enables innovative analysis that leads to rapid insight generation and contextualization of temporal activity data, both within and between patients of a cohort. Our evaluation results indicate consistent performance across participant groups and effective support for insight generation in MS patient fitness tracker data. Our implementation offers broad applicability in clinical research, allowing for potential expansion into cohort-wide comparisons or studies of other chronic conditions.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>MS Pattern Explorer successfully reduces the signal overload clinicians currently experience with activity data, introducing novel opportunities for data exploration, sense-making, and hypothesis generation.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocae230","type":"journal-article","created":{"date-parts":[[2024,9,30]],"date-time":"2024-09-30T17:00:37Z","timestamp":1727715637000},"page":"2496-2506","source":"Crossref","is-referenced-by-count":2,"title":["MS Pattern Explorer: interactive visual exploration of temporal activity patterns for multiple sclerosis"],"prefix":"10.1093","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4762-8797","authenticated-orcid":false,"given":"Gabriela","family":"Morgenshtern","sequence":"first","affiliation":[{"name":"Institute for Informatics, University of Z\u00fcrich , 8050 Z\u00fcrich,","place":["Switzerland"]},{"name":"Digital Society Initiative, University of Z\u00fcrich , 8001 Z\u00fcrich,","place":["Switzerland"]}]},{"given":"Yves","family":"Rutishauser","sequence":"additional","affiliation":[{"name":"Institute for Informatics, University of Z\u00fcrich , 8050 Z\u00fcrich,","place":["Switzerland"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9662-5245","authenticated-orcid":false,"given":"Christina","family":"Haag","sequence":"additional","affiliation":[{"name":"Institute for Implementation Science, University of Z\u00fcrich , 8006 Z\u00fcrich,","place":["Switzerland"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8754-9797","authenticated-orcid":false,"given":"Viktor","family":"von Wyl","sequence":"additional","affiliation":[{"name":"Digital Society Initiative, University of Z\u00fcrich , 8001 Z\u00fcrich,","place":["Switzerland"]},{"name":"Institute for Implementation Science, University of Z\u00fcrich , 8006 Z\u00fcrich,","place":["Switzerland"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8741-9709","authenticated-orcid":false,"given":"J\u00fcrgen","family":"Bernard","sequence":"additional","affiliation":[{"name":"Institute for Informatics, University of Z\u00fcrich , 8050 Z\u00fcrich,","place":["Switzerland"]},{"name":"Digital Society Initiative, University of Z\u00fcrich , 8001 Z\u00fcrich,","place":["Switzerland"]}]}],"member":"286","published-online":{"date-parts":[[2024,9,30]]},"reference":[{"issue":"1","key":"2024102107530908500_ocae230-B1","doi-asserted-by":"crossref","first-page":"1006932","DOI":"10.3389\/fdgth.2023.1006932","article-title":"Feasibility and scalability of a fitness tracker study: results from a longitudinal analysis of persons with multiple sclerosis","volume":"5","author":"Sieber","year":"2023","journal-title":"Front Digit Health"},{"issue":"4","key":"2024102107530908500_ocae230-B2","first-page":"2055217319888660","article-title":"Validation of a consumer-grade activity monitor for continuous daily activity monitoring in individuals with multiple sclerosis","volume":"5","author":"Block","year":"2019","journal-title":"Mult Scler J Exp Transl Clin"},{"issue":"1","key":"2024102107530908500_ocae230-B3","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1038\/s41746-021-00533-1","article-title":"Passive detection of COVID-19 with wearable sensors and explainable machine learning algorithms","volume":"4","author":"Gadaleta","year":"2021","journal-title":"NPJ Digit Med"},{"issue":"1","key":"2024102107530908500_ocae230-B4","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1146\/annurev-clinpsy-032816-044949","article-title":"Personal sensing: understanding mental health using ubiquitous sensors and machine learning","volume":"13","author":"Mohr","year":"2017","journal-title":"Annu Rev Clin Psychol"},{"key":"2024102107530908500_ocae230-B5","doi-asserted-by":"publisher","author":"Seals","year":"2022","DOI":"10.1145\/3491102.3501989"},{"issue":"1","key":"2024102107530908500_ocae230-B6","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1111\/cgf.13891","article-title":"A survey of visual analytics for public health","volume":"39","author":"Preim","year":"2020","journal-title":"Comput Graph Forum"},{"issue":"3","key":"2024102107530908500_ocae230-B7","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1561\/1100000039","article-title":"Interactive information visualization to explore and query electronic health records","volume":"5","author":"Rind","year":"2013","journal-title":"FNT Human\u2013Comput Interact"},{"issue":"6","key":"2024102107530908500_ocae230-B8","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1002\/eji.202149757","article-title":"Multiple sclerosis: immunopathological heterogeneity and its implications","volume":"52","author":"Engelhardt","year":"2022","journal-title":"Eur J Immunol"},{"issue":"8","key":"2024102107530908500_ocae230-B9","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1001\/jama.2020.26858","article-title":"Diagnosis and treatment of multiple sclerosis: a review","volume":"325","author":"McGinley","year":"2021","journal-title":"JAMA"},{"issue":"1","key":"2024102107530908500_ocae230-B10","first-page":"CD012732","article-title":"Rehabilitation for people with multiple sclerosis: an overview of cochrane reviews","volume":"1","author":"Amatya","year":"2019","journal-title":"Cochrane Database Syst Rev"},{"issue":"5","key":"2024102107530908500_ocae230-B11","doi-asserted-by":"crossref","first-page":"e0251659","DOI":"10.1371\/journal.pone.0251659","article-title":"Characterizing and predicting person-specific, day-to-day, fluctuations in walking behavior","volume":"16","author":"Chevance","year":"2021","journal-title":"PLoS One"},{"issue":"3","key":"2024102107530908500_ocae230-B12","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1007\/s00422-009-0362-1","article-title":"The identification of critical fluctuations and phase transitions in short term and coarse-grained time series-a method for the real-time monitoring of human change processes","volume":"102","author":"Schiepek","year":"2010","journal-title":"Biol Cybern"},{"issue":"1","key":"2024102107530908500_ocae230-B13","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1177\/2167702619865969","article-title":"Critical fluctuations as an early-warning signal for sudden gains and losses in patients receiving psychotherapy for mood disorders","volume":"8","author":"Olthof","year":"2020","journal-title":"Clin Psychol Sci"},{"key":"2024102107530908500_ocae230-B14","author":"Therneau"},{"issue":"9","key":"2024102107530908500_ocae230-B15","doi-asserted-by":"crossref","first-page":"e17977","DOI":"10.2196\/17977","article-title":"Data imputation and body weight variability calculation using linear and nonlinear methods in data collected from digital smart scales: simulation and validation study","volume":"8","author":"Turicchi","year":"2020","journal-title":"JMIR Mhealth Uhealth"},{"issue":"8","key":"2024102107530908500_ocae230-B16","doi-asserted-by":"crossref","first-page":"e1009303","DOI":"10.1371\/journal.pcbi.1009303","article-title":"Bayesian structural time series for biomedical sensor data: a flexible modeling framework for evaluating interventions","volume":"17","author":"Liu","year":"2021","journal-title":"PLoS Comput Biol"},{"issue":"426","key":"2024102107530908500_ocae230-B17","first-page":"611","article-title":"Estimation, prediction, and interpolation for nonstationary series with the Kalman filter","volume":"89","author":"Gomez","year":"1994","journal-title":"J Am Stat Assoc"},{"issue":"12","key":"2024102107530908500_ocae230-B18","doi-asserted-by":"crossref","first-page":"5091","DOI":"10.1109\/TVCG.2021.3100413","article-title":"Survey on visual analysis of event sequence data","volume":"28","author":"Guo","year":"2022","journal-title":"IEEE Trans Vis Comput Graph"},{"issue":"10","key":"2024102107530908500_ocae230-B19","doi-asserted-by":"crossref","first-page":"e0000104","DOI":"10.1371\/journal.pdig.0000104","article-title":"Challenges and recommendations for wearable devices in digital health: data quality, interoperability, health equity, fairness","volume":"1","author":"Canali","year":"2022","journal-title":"PLOS Digit Health"},{"key":"2024102107530908500_ocae230-B20","first-page":"76","article-title":"LifeLines: using visualization to enhance navigation and analysis of patient records","author":"Plaisant","year":"1998","journal-title":"Proc AMIA Symp"},{"key":"2024102107530908500_ocae230-B21","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2864905","article-title":"Doccurate: a curation-based approach for clinical text visualization","author":"Sultanum","year":"2018","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"2024102107530908500_ocae230-B22","author":"Rind"},{"issue":"1","key":"2024102107530908500_ocae230-B23","doi-asserted-by":"crossref","first-page":"13460","DOI":"10.1109\/ACCESS.2022.3146884","article-title":"A data analytics suite for exploratory predictive, and visual analysis of type 2 diabetes","volume":"10","author":"Philip","year":"2022","journal-title":"IEEE Access"},{"key":"2024102107530908500_ocae230-B24","doi-asserted-by":"publisher","author":"Federico","year":"2015","DOI":"10.2312\/eurova.20151108"},{"issue":"1","key":"2024102107530908500_ocae230-B25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3344258","article-title":"CarePre","volume":"1","author":"Jin","year":"2020","journal-title":"ACM Trans Comput Healthcare"},{"issue":"3","key":"2024102107530908500_ocae230-B26","doi-asserted-by":"crossref","first-page":"1528","DOI":"10.1109\/TVCG.2017.2785271","article-title":"KAVAGait: knowledge-assisted visual analytics for clinical gait analysis","volume":"25","author":"Wagner","year":"2019","journal-title":"IEEE Trans Vis Comput Graph"},{"issue":"2","key":"2024102107530908500_ocae230-B27","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1093\/jamia\/ocy143","article-title":"Designing a medication timeline for patients and physicians","volume":"26","author":"Belden","year":"2019","journal-title":"J Am Med Inform Assoc"},{"issue":"12","key":"2024102107530908500_ocae230-B28","doi-asserted-by":"crossref","first-page":"2237","DOI":"10.1109\/TVCG.2013.222","article-title":"Visual analytics for model selection in time series analysis","volume":"19","author":"B\u00f6gl","year":"2013","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"2024102107530908500_ocae230-B29","doi-asserted-by":"publisher","author":"Ahmad","year":"2021.","DOI":"10.1109\/VAHC53616.2021.00009"},{"key":"2024102107530908500_ocae230-B30","doi-asserted-by":"publisher","author":"Bernard","year":"2012","DOI":"10.1145\/2362456.2362485"},{"key":"2024102107530908500_ocae230-B31","doi-asserted-by":"publisher","author":"Bernard","year":"2012","DOI":"10.1145\/2232817.2232844"},{"issue":"1","key":"2024102107530908500_ocae230-B32","first-page":"342","article-title":"Dashboard design patterns","volume":"29","author":"Bach","year":"2023","journal-title":"IEEE Trans Vis Comput Graph"},{"issue":"4","key":"2024102107530908500_ocae230-B33","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1109\/TVCG.2005.53","article-title":"An insight-based methodology for evaluating bioinformatics visualizations","volume":"11","author":"Saraiya","year":"2005","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"2024102107530908500_ocae230-B34","author":"Bendeck"},{"issue":"1","key":"2024102107530908500_ocae230-B35","doi-asserted-by":"crossref","first-page":"e0000171","DOI":"10.1371\/journal.pdig.0000171","article-title":"Non-equivalent, but still valid: establishing the construct validity of a consumer fitness tracker in persons with multiple sclerosis","volume":"2","author":"Polhemus","year":"2023","journal-title":"PLoS Digit Health"},{"key":"2024102107530908500_ocae230-B36","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1007\/0-387-22440-8_13","volume-title":"Principal component analysis for special types of data. Principal Component Analysis","author":"Jolliffe","year":"2006"},{"key":"2024102107530908500_ocae230-B37","doi-asserted-by":"crossref","DOI":"10.3389\/fresc.2022.989658","article-title":"Methodological heterogeneity biases physical activity metrics derived from the Actigraph GT3X in multiple sclerosis: a rapid review and comparative study","author":"Polhemus","year":"2022","journal-title":"Front Rehabil Sci"},{"key":"2024102107530908500_ocae230-B38","author":"von Wyl"},{"key":"2024102107530908500_ocae230-B39","doi-asserted-by":"crossref","DOI":"10.2196\/52733","article-title":"Exploring the major barriers to physical activity in persons with multiple sclerosis: observational longitudinal study","author":"Sieber","year":"2024","journal-title":"JMIR Rehabil Assist Technol"},{"key":"2024102107530908500_ocae230-B40"},{"key":"2024102107530908500_ocae230-B41","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-97610-0","volume-title":"Self-Organizing Maps","author":"Kohonen","year":"1995"},{"key":"2024102107530908500_ocae230-B42","first-page":"481","article-title":"Visual cluster analysis in support of clinical decision intelligence","volume":"2011","author":"Gotz","year":"2011","journal-title":"AMIA Annu Symp Proc"},{"issue":"1","key":"2024102107530908500_ocae230-B43","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.neunet.2012.09.018","article-title":"Essentials of the self-organizing map","volume":"37","author":"Kohonen","year":"2013","journal-title":"Neural Netw."},{"issue":"3","key":"2024102107530908500_ocae230-B44","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1080\/00031305.1992.10475879","article-title":"An introduction to kernel and nearest-neighbor nonparametric regression","volume":"46","author":"Altman","year":"1992","journal-title":"Am Stat"},{"issue":"3","key":"2024102107530908500_ocae230-B45","doi-asserted-by":"crossref","first-page":"1615","DOI":"10.1109\/TVCG.2018.2803829","article-title":"Using dashboard networks to visualize multiple patient histories: a design study on post-operative prostate cancer","volume":"25","author":"Bernard","year":"2019","journal-title":"IEEE Trans Vis Comput Graph"},{"issue":"3","key":"2024102107530908500_ocae230-B46","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/MCG.2006.70","article-title":"Toward measuring visualization insight","volume":"26","author":"North","year":"2006","journal-title":"IEEE Comput Graph Appl"},{"issue":"2","key":"2024102107530908500_ocae230-B47","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1109\/MCG.2009.22","article-title":"Defining insight for visual analytics","volume":"29","author":"Chang","year":"2009","journal-title":"IEEE Comput Graph Appl"},{"issue":"1","key":"2024102107530908500_ocae230-B48","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1038\/s41746-024-01151-3","article-title":"From wearable sensor data to digital biomarker development: ten lessons learned and a framework proposal","volume":"7","author":"Daniore","year":"2024","journal-title":"NPJ Digit Med"},{"key":"2024102107530908500_ocae230-B49","doi-asserted-by":"publisher","author":"Bernard","year":"2015","DOI":"10.1145\/2836034.2836035"},{"issue":"1","key":"2024102107530908500_ocae230-B50","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1111\/cgf.14424","article-title":"EHR STAR: the state-of-the-art in interactive EHR visualization","volume":"41","author":"Wang","year":"2022","journal-title":"Comput Graph Forum"},{"key":"2024102107530908500_ocae230-B51","author":"Levy-Fix","year":"2019"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/11\/2496\/59813533\/ocae230.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/11\/2496\/59813533\/ocae230.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,21]],"date-time":"2024-10-21T07:53:38Z","timestamp":1729497218000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/11\/2496\/7796608"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,30]]},"references-count":51,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,9,30]]},"published-print":{"date-parts":[[2024,11,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae230","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,11]]},"published":{"date-parts":[[2024,9,30]]}}}