{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T18:45:36Z","timestamp":1790621136083,"version":"4.1.0"},"reference-count":46,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Diabetes"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec sec-type=\"background\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Diabetes-related lower extremity complications, such as foot ulceration and amputation, are on the rise, currently affecting nearly 131 million people worldwide. Methods for early detection of individuals at high risk remain elusive. While data-driven diabetic polyneuropathy algorithms exist, high-performing, clinically useful tools to assess risk are needed to improve clinical care.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"objective\">\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aimed to develop an electronic medical record\u2013based machine learning algorithm that would predict lower extremity complications.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"methods\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We conducted a retrospective longitudinal cohort study to predict the risk of lower extremity complications within 24 months of an initial diagnosis of diabetic polyneuropathy. From an initial cohort of 468,162 individuals with at least 1 diagnosis of diabetic polyneuropathy at one of 2 multispecialty health care systems (based in northern California and Colorado) between April 2012 and December 2016, we created an analytic cohort of 48,209 adults with continuous enrollment, who were newly diagnosed with no evidence of end-of-life care. The outcome was any lower extremity complication, including foot ulceration, osteomyelitis, gangrene, or lower extremity amputation. We randomly split the data into training (38,569\/48209; 80%) and testing (9,640\/48209; 20%) datasets. In the training dataset, we used super Learner (SL), an ensemble learning method that employs cross-validation and combines multiple candidate risk predictors, into a single risk predictor. We evaluated the performance of the SL risk predictor in the testing dataset using the receiver operating characteristic curve and a calibration plot.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"results\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Of the 48,209 individuals in the cohort, 2327 developed a lower extremity complication during follow-up. The SL risk estimator exhibited good discrimination (AUC=0.845, 95% CI 0.826-0.863) and calibration. A modified version of our SL algorithm, simplified to facilitate real-world adoption, had only slightly reduced discrimination (AUC=0.817, 95%CI 0.797-0.837). The modified version slightly outperformed the na\u00efve logistic regression model (AUC=0.804, 95% CI 0.783-0.825) in terms of precision gained relative to the frequency of alerts and number of patients that needed to be evaluated.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"conclusions\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>We have built a machine learning\u2013based risk estimator with the potential to improve clinical detection of diabetic patients at high risk for lower extremity complications at the time of an initial diabetic polyneuropathy diagnosis. The algorithm exhibited good discriminant validity and calibration using only data from the electronic medical record. Additional research will be needed to identify optimal contexts and strategies for maximizing algorithmic fairness in both interpretation and deployment.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/60141","type":"journal-article","created":{"date-parts":[[2025,5,29]],"date-time":"2025-05-29T16:22:23Z","timestamp":1748535743000},"page":"e60141-e60141","source":"Crossref","is-referenced-by-count":2,"title":["Estimating the Risk of Lower Extremity Complications in Adults Newly Diagnosed With Diabetic Polyneuropathy: Retrospective Cohort Study"],"prefix":"10.2196","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0146-4996","authenticated-orcid":false,"given":"Alyce S","family":"Adams","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0008-9052","authenticated-orcid":false,"given":"Catherine","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2540-3327","authenticated-orcid":false,"given":"Gabriel","family":"Escobar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5264-0670","authenticated-orcid":false,"given":"Elizabeth A","family":"Bayliss","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8885-6748","authenticated-orcid":false,"given":"Brian","family":"Callaghan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6971-7646","authenticated-orcid":false,"given":"Michael","family":"Horberg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8995-5345","authenticated-orcid":false,"given":"Julie A","family":"Schmittdiel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4799-1994","authenticated-orcid":false,"given":"Connie","family":"Trinacty","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7054-1409","authenticated-orcid":false,"given":"Lisa K","family":"Gilliam","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-5179-505X","authenticated-orcid":false,"given":"Eileen","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7127-2789","authenticated-orcid":false,"given":"Nima S","family":"Hejazi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7009-8213","authenticated-orcid":false,"given":"Lin","family":"Ma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6085-4564","authenticated-orcid":false,"given":"Romain","family":"Neugebauer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,5,29]]},"reference":[{"issue":"6","key":"R1","doi-asserted-by":"publisher","first-page":"1790","DOI":"10.2337\/diacare.26.6.1790","article-title":"The health care costs of diabetic peripheral neuropathy in the US","volume":"26","author":"Gordois","journal-title":"Diabetes Care"},{"issue":"6","key":"R2","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1016\/S1474-4422(12)70065-0","article-title":"Diabetic neuropathy: clinical manifestations and current treatments","volume":"11","author":"Callaghan","journal-title":"Lancet Neurol"},{"issue":"8","key":"R3","doi-asserted-by":"publisher","first-page":"1430","DOI":"10.2337\/dc19-0296","article-title":"Trends of nontraumatic lower-extremity amputation in end-stage renal disease and diabetes: United States, 2000-2015","volume":"42","author":"Harding","journal-title":"Diabetes Care"},{"issue":"1","key":"R4","doi-asserted-by":"publisher","first-page":"50","DOI":"10.2337\/dc18-1380","article-title":"Resurgence of diabetes-related nontraumatic lower-extremity amputation in the young and middle-aged adult U.S. population","volume":"42","author":"Geiss","journal-title":"Diabetes Care"},{"issue":"Supplement_1","key":"R5","doi-asserted-by":"publisher","first-page":"S252","DOI":"10.2337\/dc25-S012","article-title":"12. Retinopathy, neuropathy, and foot care: standards of care in diabetes-2025","volume":"48","author":"American Diabetes Association Professional Practice Committee","journal-title":"Diabetes Care"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.2196\/64400","article-title":"Effects of digital intelligent interventions on self-management of patients with diabetic foot: systematic review","volume":"27","author":"Zhou","journal-title":"J Med Internet Res"},{"issue":"11","key":"R7","doi-asserted-by":"publisher","first-page":"2077","DOI":"10.1007\/s001250100013","article-title":"Prevention of diabetes-related foot ulcers and amputations: a cost-utility analysis based on Markov model simulations","volume":"44","author":"Ragnarson Tennvall","journal-title":"Diabetologia"},{"issue":"4","key":"R8","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1016\/j.hjdsi.2016.10.003","article-title":"Overcoming barriers to diabetic polyneuropathy management in primary care","volume":"5","author":"Adams","journal-title":"Healthc (Amst)"},{"issue":"1","key":"R9","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1053\/j.semvascsurg.2023.01.001","article-title":"Disparities in preventative diabetic foot examination","volume":"36","author":"Fermawi","journal-title":"Semin Vasc Surg"},{"issue":"18-180","key":"R10","doi-asserted-by":"publisher","DOI":"10.7812\/TPP\/18-180","article-title":"Clinical response to real-time patient-reported diabetic peripheral neuropathy symptoms","volume":"23","author":"Banerjee","journal-title":"Perm J"},{"issue":"5","key":"R11","doi-asserted-by":"publisher","first-page":"477","DOI":"10.1016\/j.clineuro.2005.08.003","article-title":"Validation of Michigan neuropathy screening instrument for diabetic peripheral neuropathy","volume":"108","author":"Moghtaderi","journal-title":"Clin Neurol Neurosurg"},{"issue":"7","key":"R12","doi-asserted-by":"publisher","first-page":"610","DOI":"10.1002\/dmrr.2535","article-title":"Classification systems for lower extremity amputation prediction in subjects with active diabetic foot ulcer: a systematic review and meta-analysis","volume":"30","author":"Monteiro-Soares","journal-title":"Diabetes Metab Res Rev"},{"key":"R13","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.i1450","article-title":"Diabetes treatments and risk of amputation, blindness, severe kidney failure, hyperglycaemia, and hypoglycaemia: open cohort study in primary care","volume":"352","author":"Hippisley-Cox","journal-title":"BMJ"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","first-page":"163","DOI":"10.3928\/0147-7447-19870101-28","article-title":"The diabetic foot","volume":"10","author":"Wagner","journal-title":"Orthopedics"},{"issue":"6","key":"R15","doi-asserted-by":"publisher","first-page":"528","DOI":"10.1016\/S1067-2516(96)80125-6","article-title":"Classification of diabetic foot wounds","volume":"35","author":"Lavery","journal-title":"J Foot Ankle Surg"},{"issue":"5","key":"R16","doi-asserted-by":"publisher","first-page":"964","DOI":"10.2337\/dc07-2367","article-title":"Use of the SINBAD classification system and score in comparing outcome of foot ulcer management on three continents","volume":"31","author":"Ince","journal-title":"Diabetes Care"},{"issue":"2","key":"R17","doi-asserted-by":"publisher","first-page":"143","DOI":"10.2174\/1573399816666200511003357","article-title":"A review of statistical and machine learning techniques for microvascular complications in type 2 diabetes","volume":"17","author":"Sambyal","journal-title":"Curr Diabetes Rev"},{"issue":"4","key":"R18","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1111\/jch.12803","article-title":"The Kaiser Permanente Northern California story: improving hypertension control from 44% to 90% in 13 years (2000 to 2013)","volume":"18","author":"Jaffe","journal-title":"J Clin Hypertens (Greenwich)"},{"issue":"3","key":"R19","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1370\/afm.1117","article-title":"System-based participatory research in health care: an approach for sustainable translational research and quality improvement","volume":"8","author":"Schmittdiel","journal-title":"Ann Fam Med"},{"issue":"1","key":"R20","doi-asserted-by":"publisher","DOI":"10.13063\/2327-9214.1049","article-title":"The HMO research network virtual data warehouse: a public data model to support collaboration","volume":"2","author":"Ross","journal-title":"EGEMS (Wash DC)"},{"issue":"16","key":"R21","doi-asserted-by":"publisher","first-page":"1644","DOI":"10.1212\/WNL.0000000000001492","article-title":"Impairments and comorbidities of polyneuropathy revealed by population-based analyses","volume":"84","author":"Hoffman","journal-title":"Neurology (ECronicon)"},{"issue":"1","key":"R22","doi-asserted-by":"publisher","first-page":"98","DOI":"10.2337\/dc09-1497","article-title":"Lower-extremity amputation risk after charcot arthropathy and diabetic foot ulcer","volume":"33","author":"Sohn","journal-title":"Diabetes Care"},{"issue":"9","key":"R23","doi-asserted-by":"publisher","first-page":"1333","DOI":"10.2337\/diacare.23.9.1333","article-title":"A cost analysis of diabetic lower-extremity ulcers","volume":"23","author":"Harrington","journal-title":"Diabetes Care"},{"issue":"1","key":"R24","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1016\/s0149-2918(98)80044-1","article-title":"Costs and duration of care for lower extremity ulcers in patients with diabetes","volume":"20","author":"Holzer","journal-title":"Clin Ther"},{"issue":"8","key":"R25","doi-asserted-by":"publisher","first-page":"779","DOI":"10.1097\/01.mlr.0000218793.74558.0e","article-title":"Initial nontraumatic lower-extremity amputations among veterans with diabetes","volume":"44","author":"Sambamoorthi","journal-title":"Med Care"},{"issue":"1","key":"R26","doi-asserted-by":"publisher","first-page":"24","DOI":"10.2337\/diacare.12.1.24","article-title":"Lower-extremity amputation in people with diabetes. Epidemiology and prevention","volume":"12","author":"Bild","journal-title":"Diabetes Care"},{"issue":"7","key":"R27","doi-asserted-by":"publisher","first-page":"1642","DOI":"10.2337\/dc10-2150","article-title":"Peripheral vascular disease and peripheral neuropathy in individuals with cardiometabolic clustering and obesity: National Health and Nutrition Examination Survey 2001-2004","volume":"34","author":"Ylitalo","journal-title":"Diabetes Care"},{"issue":"1-2","key":"R28","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1900\/RDS.2015.12.48","article-title":"Risk factors and comorbidities in diabetic neuropathy: an update 2015","volume":"12","author":"Papanas","journal-title":"Rev Diabet Stud"},{"issue":"5","key":"R29","doi-asserted-by":"publisher","first-page":"446","DOI":"10.1097\/MLR.0b013e3182881c8e","article-title":"Risk-adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system","volume":"51","author":"Escobar","journal-title":"Med Care"},{"issue":"3","key":"R30","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1093\/bib\/bbr001","article-title":"Using cross-validation to evaluate predictive accuracy of survival risk classifiers based on high-dimensional data","volume":"12","author":"Simon","journal-title":"Brief Bioinform"},{"key":"R31","doi-asserted-by":"publisher","DOI":"10.2202\/1544-6115.1309","article-title":"Super learner","volume":"6","author":"van der Laan","journal-title":"Stat Appl Genet Mol Biol"},{"issue":"1","key":"R32","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1007\/BF00117832","article-title":"Stacked regressions","volume":"24","author":"Breiman","journal-title":"Mach Learn"},{"issue":"1","key":"R33","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/S2213-2600(14)70239-5","article-title":"Mortality prediction in intensive care units with the Super ICU Learner Algorithm (SICULA): a population-based study","volume":"3","author":"Pirracchio","journal-title":"Lancet Respir Med"},{"key":"R34","doi-asserted-by":"crossref","unstructured":"Polley EC Laan MJ . Super learning for right-censored data. In: van der Laan MJ Rose S , editors. Targeted Learning: Causal Inference for Observational and Experimental Data. Springer; 2011:249-258. [doi: 10.1007\/978-1-4419-9782-1_16] ISBN: 9781441997821","DOI":"10.1007\/978-1-4419-9782-1_16"},{"key":"R35","unstructured":"Coyle J Hejazi N Malencia I Phillips R Sofrygin O . R\/sl3: super machine learning with pipelines. GitHub. 2021. URL: https:\/\/github.com\/tlverse\/sl3 [Accessed 19-05-2025]"},{"issue":"2","key":"R36","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1111\/j.2517-6161.1972.tb00899.x","article-title":"Regression models and life-tables","volume":"34","author":"Cox","journal-title":"Journal of the Royal Statistical Society Series B"},{"issue":"14","key":"R37","doi-asserted-by":"publisher","first-page":"1377","DOI":"10.1001\/jama.2017.12126","article-title":"Discrimination and calibration of clinical prediction models: users\u2019 guides to the medical literature","volume":"318","author":"Alba","journal-title":"JAMA"},{"issue":"7","key":"R38","doi-asserted-by":"publisher","first-page":"560","DOI":"10.1097\/MLR.0000000000001140","article-title":"Training and interpreting machine learning algorithms to evaluate fall risk after emergency department visits","volume":"57","author":"Patterson","journal-title":"Med Care"},{"issue":"12","key":"R39","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2019.16769","article-title":"Multiyear rehospitalization rates and hospital outcomes in an integrated health care system","volume":"2","author":"Escobar","journal-title":"JAMA Netw Open"},{"key":"R40","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2020.103616","article-title":"Recognition of ischaemia and infection in diabetic foot ulcers: dataset and techniques","volume":"117","author":"Goyal","journal-title":"Comput Biol Med"},{"issue":"5","key":"R41","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics11050801","article-title":"Performance analysis of conventional machine learning algorithms for diabetic sensorimotor polyneuropathy severity classification","volume":"11","author":"Haque","journal-title":"Diagnostics (Basel)"},{"key":"R42","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2020.103744","article-title":"Competitive neural layer-based method to identify people with high risk for diabetic foot","volume":"120","author":"Ferreira","journal-title":"Comput Biol Med"},{"issue":"601602","key":"R43","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2020.601602","article-title":"Toward machine-learning-based decision support in diabetes care: a risk stratification study on diabetic foot ulcer and amputation","volume":"7","author":"Sch\u00e4fer","journal-title":"Front Med (Lausanne)"},{"issue":"6464","key":"R44","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1126\/science.aax2342","article-title":"Dissecting racial bias in an algorithm used to manage the health of populations","volume":"366","author":"Obermeyer","journal-title":"Science"},{"issue":"1","key":"R45","doi-asserted-by":"publisher","DOI":"10.2196\/41030","article-title":"Chronic disease prediction using the common data model: development study","volume":"1","author":"Lee","journal-title":"JMIR AI"},{"key":"R46","doi-asserted-by":"publisher","DOI":"10.2196\/45032","article-title":"An assessment of how clinicians and staff members use a diabetes artificial intelligence prediction tool: mixed methods study","volume":"2","author":"Liaw","journal-title":"JMIR AI"}],"container-title":["JMIR Diabetes"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/diabetes.jmir.org\/2025\/1\/e60141","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:32:04Z","timestamp":1785537124000},"score":1,"resource":{"primary":{"URL":"https:\/\/diabetes.jmir.org\/2025\/1\/e60141"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,29]]},"references-count":46,"URL":"https:\/\/doi.org\/10.2196\/60141","relation":{},"ISSN":["2371-4379"],"issn-type":[{"value":"2371-4379","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,29]]},"article-number":"v10i6e60141"}}