{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T03:46:21Z","timestamp":1787024781584,"version":"build-2736575974"},"reference-count":50,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Sensor-based digital health technologies (sDHTs) are increasingly used to support scientific and clinical decision-making. The digital measures (DMs) they generate offer significant potential to accelerate the drug development timeline, decrease clinical trial costs, and improve access to care. However, choosing an appropriate statistical methodology when conducting analytical validation (AV) of a DM is complicated, particularly for novel DMs, for which appropriate, established reference measures (RMs) may not exist. More understanding of, and a standardized approach to, AV in these scenarios is needed.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n In a prior simulation study, 3 statistical methods were tested for their ability to estimate a simulated relationship between a sDHT-derived DM and several clinical outcome assessment (COA) RMs. The aim of this work was to assess the feasibility of implementation of these methods in real data and to examine the impact of AV study design factors on the relationships estimated.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n Four real-world datasets, captured using sDHTs, were used to prepare hypothetical AV studies representing a range of scenarios with respect to 3 key study design properties: temporal coherence, construct coherence, and data completeness. The datasets analyzed were as follows: Urban Poor (comparing nighttime awakenings to measures of psychological well-being), STAGES (comparing daily step count to psychological and fatigue measures), mPower (comparing daily smartphone screen taps to measures of function in Parkinson\u2019s disease), and Brighten (comparing smartphone communication activity to measures of psychological well-being). For each hypothetical AV study, 3 statistical methods were leveraged: the Pearson correlation coefficient (PCC) between DM and RM, simple linear regression (SLR) between DM and RM, multiple linear regression (MLR) between DMs and combinations of RMs, and 2-factor, correlated-factor confirmatory factor analysis (CFA) models. Performance measures were the PCC magnitudes (for PCC), R\n 2<\/jats:sup>\n and adjusted R\n 2<\/jats:sup>\n statistics (for SLR and MLR, respectively), and factor correlations (for CFA).\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Most of the CFA models exhibited an acceptable fit according to the majority of the fit statistics employed, and each model was able to estimate a factor correlation. For each model, these correlations were greater than or equal to the corresponding PCC in magnitude. Correlations were the strongest in the hypothetical studies with strong temporal and construct coherence.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n The performance of the selected statistical methods shown in this work supports their feasibility when implemented in real-world data. Our findings, in particular, support the use of CFA to assess the relationship between a novel DM and a COA RM. The observed impact of AV study design factors on the relationships estimated allowed the authors to determine practical recommendations for study design in AV of novel DMs. By using a standardized methodology for evaluating novel DMs, sDHT developers, biostatisticians, and clinical researchers can navigate the complex validation landscape more easily, with more certainty, and with more tools at their disposal.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/70314","type":"journal-article","created":{"date-parts":[[2025,7,4]],"date-time":"2025-07-04T08:20:06Z","timestamp":1751617206000},"page":"e70314-e70314","source":"Crossref","is-referenced-by-count":1,"title":["Methods for Analytical Validation of Novel Digital Clinical Measures: Implementation Feasibility Evaluation Using Real-World Datasets"],"prefix":"10.2196","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-3801-1875","authenticated-orcid":false,"given":"Simon","family":"Turner","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0584-6227","authenticated-orcid":false,"given":"Lysbeth","family":"Floden","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1234-5335","authenticated-orcid":false,"given":"Leif","family":"Simmatis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-7751-7226","authenticated-orcid":false,"given":"Piper","family":"Fromy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3204-5866","authenticated-orcid":false,"given":"Joss","family":"Langford","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8376-1600","authenticated-orcid":false,"given":"Eric J","family":"Daza","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0823-8035","authenticated-orcid":false,"given":"Andrew","family":"Potter","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6716-1632","authenticated-orcid":false,"given":"Kathleen","family":"Troeger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"name":"the STAGES cohort investigator group","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,11,17]]},"reference":[{"issue":"8","key":"R1","doi-asserted-by":"publisher","DOI":"10.1111\/cts.13902","article-title":"Assessing the net financial benefits of employing digital endpoints in clinical trials","volume":"17","author":"DiMasi","journal-title":"Clin Transl Sci"},{"key":"R2","unstructured":"European Medicines Agency. Qualification opinion for stride velocity 95th centile as primary endpoint in studies in ambulatory Duchenne muscular dystrophy studies. Feb 20, 2023. URL: https:\/\/tinyurl.com\/hshp3pn3 [Accessed 19-12-2024]"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","DOI":"10.3390\/diseases7010018","article-title":"Assessing gait in Parkinson\u2019s disease using wearable motion sensors: a systematic review","volume":"7","author":"Brognara","journal-title":"Diseases"},{"key":"R4","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-020-0260-4","article-title":"Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs)","volume":"3","author":"Goldsack","journal-title":"NPJ Digit Med"},{"key":"R5","doi-asserted-by":"crossref","unstructured":"Bakker JP Barge R Centra J et al. V3+: An extension to the V3 framework to ensure user-centricity and scalability of sensor-based digital health technologies. Digital Medicine Society; 2024. URL: https:\/\/datacc.dimesociety.org\/resources\/v3-an-extension-to-the-v3-framework-to-ensure-user-centricity-and-scalability-of-sensor-based-digital-health-technologies\/ [Accessed 19-12-2024]","DOI":"10.1038\/s41746-024-01322-2"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1159\/000531054","article-title":"Clinical validation of novel digital measures: statistical methods for reliability evaluation","volume":"7","author":"Ratitch","journal-title":"Digit Biomark"},{"issue":"12","key":"R7","doi-asserted-by":"publisher","first-page":"4736","DOI":"10.1044\/2021_JSLHR-21-00201","article-title":"Validation of an acoustic-based framework of speech motor control: assessing criterion and construct validity using kinematic and perceptual measures","volume":"64","author":"Rowe","journal-title":"J Speech Lang Hear Res"},{"issue":"3","key":"R8","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1159\/000526471","article-title":"Validation of the remote automated ki:e speech Biomarker for cognition in mild cognitive impairment: verification and validation following DiME V3 framework","volume":"6","author":"Tr\u00f6ger","journal-title":"Digit Biomark"},{"key":"R9","doi-asserted-by":"crossref","unstructured":"Turner S Chen C Acosta R et al. Methods for analytical validation of novel digital clinical measures: a simulation study. Health Informatics. Preprint posted online on 2024. [doi: 10.1101\/2024.11.29.24318211]","DOI":"10.1101\/2024.11.29.24318211"},{"issue":"10","key":"R10","doi-asserted-by":"publisher","first-page":"1351","DOI":"10.1093\/jamia\/ocy064","article-title":"The National Sleep Research Resource: towards a sleep data commons","volume":"25","author":"Zhang","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"R11","doi-asserted-by":"publisher","first-page":"1887","DOI":"10.1093\/qje\/qjab013","article-title":"The economic consequences of increasing sleep among the urban poor","volume":"136","author":"Bessone","journal-title":"Q J Econ"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2016.11","article-title":"The mPower study, Parkinson disease mobile data collected using ResearchKit","volume":"3","author":"Bot","journal-title":"Sci Data"},{"issue":"12","key":"R13","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.6482","article-title":"The use and effectiveness of mobile apps for depression: results from a fully remote clinical trial","volume":"18","author":"Arean","journal-title":"J Med Internet Res"},{"issue":"12","key":"R14","doi-asserted-by":"publisher","first-page":"3706","DOI":"10.1016\/j.jaip.2023.08.046","article-title":"Patient global impression of severity scale in chronic cough: validation and formulation of symptom severity categories","volume":"11","author":"Rhatigan","journal-title":"J Allergy Clin Immunol Pract"},{"issue":"9","key":"R15","doi-asserted-by":"publisher","first-page":"606","DOI":"10.1046\/j.1525-1497.2001.016009606.x","article-title":"The PHQ-9: validity of a brief depression severity measure","volume":"16","author":"Kroenke","journal-title":"J Gen Intern Med"},{"key":"R16","doi-asserted-by":"crossref","unstructured":"Rosenberg M . Rosenberg Self-Esteem Scale. APA PsycTests; 1965. [doi: 10.1037\/t01038-000]","DOI":"10.1037\/t01038-000"},{"issue":"10","key":"R17","doi-asserted-by":"publisher","first-page":"1092","DOI":"10.1001\/archinte.166.10.1092","article-title":"A brief measure for assessing generalized anxiety disorder: the GAD-7","volume":"166","author":"Spitzer","journal-title":"Arch Intern Med"},{"issue":"10","key":"R18","doi-asserted-by":"publisher","first-page":"1121","DOI":"10.1001\/archneur.1989.00520460115022","article-title":"The fatigue severity scale. Application to patients with multiple sclerosis and systemic lupus erythematosus","volume":"46","author":"Krupp","journal-title":"Arch Neurol"},{"issue":"2","key":"R19","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1016\/j.otohns.2003.09.016","article-title":"Development and validation of the Nasal Obstruction Symptom Evaluation (NOSE) scale","volume":"130","author":"Stewart","journal-title":"Otolaryngol Head Neck Surg"},{"key":"R20","unstructured":"Fahn S Elton RL . Unified Parkinson\u2019s disease rating scale. In: Fahn S Marsden CD Calne, DB Goldstein M , editors. Recent Developments in Parkinson\u2019s Disease. Vol 2. Macmillan Health Care Information; 1987:153-164. URL: https:\/\/www.movementdisorders.org\/MDS-Files1\/PDFs\/Task-Force-Papers\/unified.pdf [Accessed 29-10-2025]"},{"issue":"6","key":"R21","doi-asserted-by":"publisher","first-page":"805","DOI":"10.1080\/08870449708406741","article-title":"The PDQ-8: development and validation of a short-form Parkinson\u2019s disease questionnaire","volume":"12","author":"Jenkinson","journal-title":"Psychol Health"},{"issue":"11","key":"R22","doi-asserted-by":"publisher","first-page":"1284","DOI":"10.1097\/01.MLR.0000093487.78664.3C","article-title":"The Patient Health Questionnaire-2: validity of a two-item depression screener","volume":"41","author":"Kroenke","journal-title":"Med Care"},{"issue":"1","key":"R23","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-89141-3","article-title":"Number of daily measurements needed to estimate habitual step count levels using wrist-worn trackers and smartphones in 212,048 adults","volume":"11","author":"Yao","journal-title":"Sci Rep"},{"issue":"1","key":"R24","doi-asserted-by":"publisher","DOI":"10.1186\/1479-5868-8-62","article-title":"How many days of monitoring predict physical activity and sedentary behaviour in older adults?","volume":"8","author":"Hart","journal-title":"Int J Behav Nutr Phys Act"},{"issue":"5","key":"R25","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0109913","article-title":"Number of days required to estimate habitual activity using wrist-worn GENEActiv accelerometer: a cross-sectional study","volume":"11","author":"Dillon","journal-title":"PLoS ONE"},{"issue":"1","key":"R26","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1007\/BF02294210","article-title":"A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators","volume":"49","author":"Muth\u00e9n","journal-title":"Psychometrika"},{"issue":"4","key":"R27","doi-asserted-by":"publisher","first-page":"466","DOI":"10.1037\/1082-989X.9.4.466","article-title":"An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data","volume":"9","author":"Flora","journal-title":"Psychol Methods"},{"issue":"1","key":"R28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/10705519909540118","article-title":"Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives","volume":"6","author":"Hu","journal-title":"Struct Equ Modeling"},{"key":"R29","unstructured":"Kline RB . Principles and Practice of Structural Equation Modeling. 5th ed. Guilford Press; 2023. ISBN: ISBN-10 1462551912"},{"key":"R30","unstructured":"R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing. 2024. URL: https:\/\/www.r-project.org\/ [Accessed 19-12-2024]"},{"key":"R31","doi-asserted-by":"crossref","unstructured":"Comrey AL Lee HB . A First Course in Factor Analysis. 2nd ed. Psychology Press; 2013. [doi: 10.4324\/9781315827506]","DOI":"10.4324\/9781315827506"},{"issue":"2","key":"R32","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1177\/0971890716672933","article-title":"Confirmatory factor analysis (CFA) as an analytical technique to assess measurement error in survey research","volume":"20","author":"Mishra","journal-title":"Paradigm: A Management Research Journal"},{"issue":"1","key":"R33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00442-018-4233-0","article-title":"Underestimation of Pearson\u2019s product moment correlation statistic","volume":"189","author":"Humphreys","journal-title":"Oecologia"},{"key":"R34","doi-asserted-by":"crossref","unstructured":"Kline P . An Easy Guide to Factor Analysis. 1st ed. Routledge; 2014. [doi: 10.4324\/9781315788135]","DOI":"10.4324\/9781315788135"},{"issue":"2","key":"R35","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1037\/1082-989X.3.2.231","article-title":"Affects of variable and subject sampling on factor pattern recovery","volume":"3","author":"Velicer","journal-title":"Psychol Methods"},{"issue":"1","key":"R36","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1037\/\/1082-989X.4.1.84","article-title":"Sample size in factor analysis","volume":"4","author":"MacCallum","journal-title":"Psychol Methods"},{"issue":"10","key":"R37","doi-asserted-by":"publisher","first-page":"1394","DOI":"10.1136\/bjophthalmol-2019-315446","article-title":"Development and validation of a machine learning, smartphone-based tonometer","volume":"104","author":"Wu","journal-title":"Br J Ophthalmol"},{"issue":"1","key":"R38","doi-asserted-by":"publisher","DOI":"10.3390\/s22010054","article-title":"Predicting fall counts using wearable sensors: a novel digital biomarker for Parkinson\u2019s disease","volume":"22","author":"Greene","journal-title":"Sensors (Basel)"},{"key":"R39","doi-asserted-by":"publisher","DOI":"10.1109\/TNSRE.2021.3089613","article-title":"Quantification of motor function post-stroke using novel combination of wearable inertial and mechanomyographic sensors","volume":"29","author":"Formstone","journal-title":"IEEE Trans Neural Syst Rehabil Eng"},{"issue":"2","key":"R40","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1097\/PSY.0000000000001034","article-title":"The relationship between objectively measured step count, clinical characteristics, and quality of life among depressed patients recently hospitalized with systolic heart failure","volume":"84","author":"Holber","journal-title":"Psychosom Med"},{"issue":"12","key":"R41","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.51208","article-title":"Daily step count and depression in adults: a systematic review and meta-analysis","volume":"7","author":"Bizzozero-Peroni","journal-title":"JAMA Netw Open"},{"issue":"3_suppl","key":"R42","doi-asserted-by":"publisher","first-page":"1319","DOI":"10.2466\/pms.1976.43.3f.1319","article-title":"Robustness of the Pearson correlation against violations of assumptions","volume":"43","author":"Havlicek","journal-title":"Percept Mot Skills"},{"issue":"1","key":"R43","doi-asserted-by":"publisher","first-page":"40","DOI":"10.20982\/tqmp.10.1.p040","article-title":"Robust factor analysis in the presence of normality violations, missing data, and outliers: Empirical questions and possible solutions","volume":"10","author":"Zygmont","journal-title":"TQMP"},{"issue":"1","key":"R44","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1504\/IJQRE.2013.055642","article-title":"Confirmatory factor analysis under violations of distributional and structural assumptions","volume":"1","author":"Yang","journal-title":"IJQRE"},{"issue":"2-3","key":"R45","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1080\/00273171.2015.1134306","article-title":"The problem with having two watches: assessment of fit when RMSEA and CFI disagree","volume":"51","author":"Lai","journal-title":"Multivariate Behav Res"},{"issue":"12","key":"R46","doi-asserted-by":"publisher","first-page":"1340","DOI":"10.1136\/jnnp-2012-304701","article-title":"Evidence of multidimensionality in the ALSFRS-R Scale: a critical appraisal on its measurement properties using Rasch analysis","volume":"84","author":"Franchignoni","journal-title":"J Neurol Neurosurg Psychiatry"},{"key":"R47","doi-asserted-by":"publisher","DOI":"10.1016\/j.psychres.2018.12.048","article-title":"PHQ-9: One factor or two?","volume":"271","author":"Boothroyd","journal-title":"Psychiatry Res"},{"key":"R48","doi-asserted-by":"publisher","DOI":"10.1016\/j.cct.2020.106219","article-title":"Using digital technologies in clinical trials: current and future applications","volume":"100","author":"Rosa","journal-title":"Contemp Clin Trials"},{"key":"R49","unstructured":"MPower public researcher portal. mPower mobile Parkinson Disease study. URL: https:\/\/www.synapse.org\/Synapse:syn4993293\/wiki\/247859 [Accessed 29-10-2025]"},{"key":"R50","unstructured":"Brighten: bridging research innovations for greater health in technology, emotion, and neuroscience. Brighten Study Public Researcher Portal. URL: https:\/\/www.synapse.org\/Synapse:syn10848316\/wiki\/548727 [Accessed 29-10-2025]"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2025\/1\/e70314","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:58:55Z","timestamp":1782842335000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2025\/1\/e70314"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,17]]},"references-count":50,"URL":"https:\/\/doi.org\/10.2196\/70314","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,17]]},"article-number":"v27i8e70314"}}