{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,14]],"date-time":"2026-11-14T02:16:37Z","timestamp":1794622597561,"version":"4.9.1"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2017,8,25]],"date-time":"2017-08-25T00:00:00Z","timestamp":1503619200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2017,8,25]],"date-time":"2017-08-25T00:00:00Z","timestamp":1503619200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Despite their potential for telemedicine in diabetic foot ulcer treatment, diagnostic accuracy of assessment of diabetic foot ulcers using mobile phone images is unknown. Our aim was to determine the validity and reliability of remote diabetic foot ulcer assessment using mobile phone images. Fifty diabetic foot ulcers were assessed live and photographed. Five independent observers remotely assessed the mobile phone images twice for presence of nine clinical characteristics and three treatment decisions. Positive likelihood (LLR+) and negative likelihood (LLR\u2212) ratios were calculated for validity. Multirater Randolph\u2019s and bi-rater Bennet kappa values were calculated for reliability. LLR+ ranged from 1.3\u20134.2; LLR\u2212 ranged from 0.13\u20130.88; the treatment decision \u2018peri-wound debridement\u2019 was the only item with \u2018strong diagnostic evidence\u2019. Inter-observer reliability kappa ranged from 0.09\u20130.71; test-retest reliability from 0.45\u20130.86; the treatment decision \u2018peri-wound debridement\u2019 was the only item with \u2018adequate agreement\u2019. In conclusion, mobile phone images had low validity and reliability for remote assessment of diabetic foot ulcers and should not be used as a stand-alone diagnostic instrument. Clinicians who use mobile phone images in clinical practice should obtain as much additional information as possible when making treatment decisions based on these images, and be cautious of the low diagnostic accuracy.<\/jats:p>","DOI":"10.1038\/s41598-017-09828-4","type":"journal-article","created":{"date-parts":[[2017,8,21]],"date-time":"2017-08-21T13:49:14Z","timestamp":1503323354000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":68,"title":["The validity and reliability of remote diabetic foot ulcer assessment using mobile phone images"],"prefix":"10.1038","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6420-6046","authenticated-orcid":false,"given":"Jaap J.","family":"van Netten","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Damien","family":"Clark","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter A.","family":"Lazzarini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1728-8085","authenticated-orcid":false,"given":"Monika","family":"Janda","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lloyd F.","family":"Reed","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2017,8,25]]},"reference":[{"key":"9828_CR1","doi-asserted-by":"publisher","first-page":"1719","DOI":"10.1016\/S0140-6736(05)67698-2","volume":"366","author":"AJ Boulton","year":"2005","unstructured":"Boulton, A. J., Vileikyte, L., Ragnarson-Tennvall, G. & Apelqvist, J. The global burden of diabetic foot disease. Lancet \n                           366, 1719\u20131724 (2005).","journal-title":"Lancet"},{"issue":"Suppl 1","key":"9828_CR2","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1002\/dmrr.2695","volume":"32","author":"NC Schaper","year":"2016","unstructured":"Schaper, N. C. et al. Prevention and management of foot problems in diabetes: a Summary Guidance for Daily Practice 2015, based on the IWGDF Guidance Documents. Diabetes Metab. Res. Rev. \n                           32(Suppl 1), 7\u201315 (2016).","journal-title":"Diabetes Metab. Res. Rev."},{"key":"9828_CR3","doi-asserted-by":"crossref","unstructured":"Lazzarini, P. A. et al. Prevalence of foot disease and risk factors in general inpatient populations: a systematic review and meta-analysis. BMJ Open \n                           5, e008544-2015-008544 (2015).","DOI":"10.1136\/bmjopen-2015-008544"},{"key":"9828_CR4","doi-asserted-by":"publisher","first-page":"1498","DOI":"10.1111\/dme.12545","volume":"31","author":"M Kerr","year":"2014","unstructured":"Kerr, M., Rayman, G. & Jeffcoate, W. J. Cost of diabetic foot disease to the National Health Service in England. Diabet. Med. \n                           31, 1498\u20131504 (2014).","journal-title":"Diabet. Med."},{"key":"9828_CR5","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0134914","volume":"10","author":"GH Skrepnek","year":"2015","unstructured":"Skrepnek, G. H., Mills, J. L., S. & Armstrong, D. G. A Diabetic Emergency One Million Feet Long: Disparities and Burdens of Illness among Diabetic Foot Ulcer Cases within Emergency Departments in the United States, 2006\u20132010. PLoS One \n                           10, e0134914 (2015).","journal-title":"PLoS One"},{"key":"9828_CR6","doi-asserted-by":"publisher","first-page":"1906","DOI":"10.1007\/s00125-005-1856-6","volume":"48","author":"MH Nabuurs-Franssen","year":"2005","unstructured":"Nabuurs-Franssen, M. H., Huijberts, M. S., Nieuwenhuijzen Kruseman, A. C., Willems, J. & Schaper, N. C. Health-related quality of life of diabetic foot ulcer patients and their caregivers. Diabetologia \n                           48, 1906\u20131910 (2005).","journal-title":"Diabetologia"},{"key":"9828_CR7","doi-asserted-by":"publisher","first-page":"226","DOI":"10.5694\/mja11.10347","volume":"197","author":"SM Bergin","year":"2012","unstructured":"Bergin, S. M. et al. Australian Diabetes Foot Network: management of diabetes-related foot ulceration - a clinical update. Med. J. Aust. \n                           197, 226\u2013229 (2012).","journal-title":"Med. J. Aust."},{"key":"9828_CR8","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1007\/s00125-006-0491-1","volume":"50","author":"L Prompers","year":"2007","unstructured":"Prompers, L. et al. High prevalence of ischaemia, infection and serious comorbidity in patients with diabetic foot disease in Europe. Baseline results from the Eurodiale study. Diabetologia \n                           50, 18\u201325 (2007).","journal-title":"Diabetologia"},{"issue":"Suppl 1","key":"9828_CR9","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1002\/dmrr.2699","volume":"32","author":"BA Lipsky","year":"2016","unstructured":"Lipsky, B. A. et al. IWGDF guidance on the diagnosis and management of foot infections in persons with diabetes. Diabetes Metab. Res. Rev. \n                           32(Suppl 1), 45\u201374 (2016).","journal-title":"Diabetes Metab. Res. Rev."},{"key":"9828_CR10","volume":"369","author":"M Tobalem","year":"2013","unstructured":"Tobalem, M. & Uckay, I. Images in clinical medicine. Evolution of a diabetic foot infection. N. Engl. J. Med. \n                           369, 2252 (2013).","journal-title":"Evolution of a diabetic foot infection. N. Engl. J. Med."},{"key":"9828_CR11","doi-asserted-by":"publisher","first-page":"360","DOI":"10.1097\/01.ASW.0000431329.50869.6f","volume":"26","author":"N Thompson","year":"2013","unstructured":"Thompson, N., Gordey, L., Bowles, H., Parslow, N. & Houghton, P. Reliability and validity of the revised photographic wound assessment tool on digital images taken of various types of chronic wounds. Adv. Skin Wound Care \n                           26, 360\u2013373 (2013).","journal-title":"Adv. Skin Wound Care"},{"issue":"14\u20135","key":"9828_CR12","first-page":"17","volume":"7","author":"N Santamaria","year":"2000","unstructured":"Santamaria, N. & Clayton, L. Cleaning up. The development of the Alfred\/Medseed Wound Imaging System. Collegian \n                           7(14\u20135), 17\u20138 (2000).","journal-title":"Collegian"},{"key":"9828_CR13","first-page":"164","volume":"18","author":"PA Lazzarini","year":"2010","unstructured":"Lazzarini, P. A. et al. Does the use of store-and-forward telehealth systems improve outcomes for clinicians managing diabetic foot ulcers? A pilot study. Wound Practice and Research \n                           18, 164\u2013172 (2010).","journal-title":"Wound Practice and Research"},{"key":"9828_CR14","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1111\/j.1524-475X.2010.00645.x","volume":"19","author":"FL Bowling","year":"2011","unstructured":"Bowling, F. L. et al. Remote assessment of diabetic foot ulcers using a novel wound imaging system. Wound Repair Regen. \n                           19, 25\u201330 (2011).","journal-title":"Wound Repair Regen."},{"key":"9828_CR15","doi-asserted-by":"publisher","first-page":"1011","DOI":"10.1089\/dia.2010.0088","volume":"12","author":"CE Hazenberg","year":"2010","unstructured":"Hazenberg, C. E., van Baal, J. G., Manning, E., Bril, A. & Bus, S. A. The validity and reliability of diagnosing foot ulcers and pre-ulcerative lesions in diabetes using advanced digital photography. Diabetes Technol. Ther. \n                           12, 1011\u20131017 (2010).","journal-title":"Diabetes Technol. Ther."},{"key":"9828_CR16","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1089\/dia.2011.0004","volume":"13","author":"P Foltynski","year":"2011","unstructured":"Foltynski, P. et al. A new imaging and data transmitting device for telemonitoring of diabetic foot syndrome patients. Diabetes Technol. Ther. \n                           13, 861\u2013867 (2011).","journal-title":"Diabetes Technol. Ther."},{"key":"9828_CR17","doi-asserted-by":"publisher","first-page":"1101","DOI":"10.1089\/dia.2011.0082","volume":"13","author":"P Ladyzynski","year":"2011","unstructured":"Ladyzynski, P. et al. Area of the diabetic ulcers estimated applying a foot scanner-based home telecare system and three reference methods. Diabetes Technol. Ther. \n                           13, 1101\u20131107 (2011).","journal-title":"Diabetes Technol. Ther."},{"key":"9828_CR18","doi-asserted-by":"publisher","first-page":"370","DOI":"10.1089\/dia.2013.0251","volume":"16","author":"CE Hazenberg","year":"2014","unstructured":"Hazenberg, C. E., van Netten, J. J., van Baal, S. G. & Bus, S. A. Assessment of signs of foot infection in diabetes patients using photographic foot imaging and infrared thermography. Diabetes Technol. Ther. \n                           16, 370\u2013377 (2014).","journal-title":"Diabetes Technol. Ther."},{"key":"9828_CR19","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1111\/aor.12169","volume":"38","author":"P Foltynski","year":"2014","unstructured":"Foltynski, P., Ladyzynski, P. & Wojcicki, J. M. A new smartphone-based method for wound area measurement. Artif. Organs \n                           38, 346\u2013352 (2014).","journal-title":"Artif. Organs"},{"key":"9828_CR20","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1111\/srt.12218","volume":"21","author":"BS Rasmussen","year":"2015","unstructured":"Rasmussen, B. S., Froekjaer, J., Joergensen, L. B., Halekoh, U. & Yderstraede, K. B. Validation of a new imaging device for telemedical ulcer monitoring. Skin Res. Technol. \n                           21, 485\u2013492 (2015).","journal-title":"Skin Res. Technol."},{"key":"9828_CR21","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1136\/bmj.323.7305.157","volume":"323","author":"JJ Deeks","year":"2001","unstructured":"Deeks, J. J. Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests. BMJ \n                           323, 157\u2013162 (2001).","journal-title":"BMJ"},{"key":"9828_CR22","doi-asserted-by":"publisher","first-page":"1500","DOI":"10.1016\/S0140-6736(05)66422-7","volume":"365","author":"DA Grimes","year":"2005","unstructured":"Grimes, D. A. & Schulz, K. F. Refining clinical diagnosis with likelihood ratios. Lancet \n                           365, 1500\u20131505 (2005).","journal-title":"Lancet"},{"key":"9828_CR23","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.h5527","volume":"351","author":"PM Bossuyt","year":"2015","unstructured":"Bossuyt, P. M. et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ \n                           351, h5527 (2015).","journal-title":"BMJ"},{"key":"9828_CR24","unstructured":"Queensland Industrial Relations Commission. Health Practitioners (Queensland health) Certified Agreement (No. 2). http:\/\/www.qirc.qld.gov.au\/resources\/pdf\/certified_agreements\/cert_agreements\/2011\/ca_106_2011.pdf (2011)."},{"key":"9828_CR25","doi-asserted-by":"crossref","unstructured":"Lazzarini, P. A. et al. The Queensland high risk foot form (QHRFF) - is it a reliable and valid clinical research tool for foot disease? J. Foot Ankle Res. 7, 7-1146-7-7 (2014).","DOI":"10.1186\/1757-1146-7-7"},{"key":"9828_CR26","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1093\/ptj\/85.3.257","volume":"85","author":"J Sim","year":"2005","unstructured":"Sim, J. & Wright, C. C. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys. Ther. \n                           85, 257\u2013268 (2005).","journal-title":"Phys. Ther."},{"key":"9828_CR27","doi-asserted-by":"publisher","first-page":"781","DOI":"10.1016\/S2213-8587(16)30012-2","volume":"4","author":"WJ Jeffcoate","year":"2016","unstructured":"Jeffcoate, W. J. et al. Reporting standards of studies and papers on the prevention and management of foot ulcers in diabetes: required details and markers of good quality. Lancet Diabetes Endocrinol. \n                           4, 781\u2013788 (2016).","journal-title":"Lancet Diabetes Endocrinol."},{"key":"9828_CR28","unstructured":"Randolph, J. J. Free-marginal multirater kappa: An alternative to Fleiss\u2019fixed-marginal multirater kappa. (Joensuu University Learning and Instruction Symposium, https:\/\/eric.ed.gov\/?id=ED490661, 2005)."},{"key":"9828_CR29","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1007\/s11634-010-0073-4","volume":"4","author":"MJ Warrens","year":"2010","unstructured":"Warrens, M. J. Inequalities between multi-rater kappas. Advances in Data Analysis and Classification \n                           4, 271\u2013286 (2010).","journal-title":"Advances in Data Analysis and Classification"},{"key":"9828_CR30","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","volume":"33","author":"JR Landis","year":"1977","unstructured":"Landis, J. R. & Koch, G. G. Measurement of observer agreement for categorical data. Biometrics \n                           33, 159\u2013174 (1977).","journal-title":"Biometrics"},{"key":"9828_CR31","first-page":"62","volume":"12","author":"N Santamaria","year":"2004","unstructured":"Santamaria, N., Carville, K., Ellis, I. & Prentice, J. The effectiveness of digital imaging and remote expert wound consultation on healing rates in chronic lower leg ulcers in the Kimberley region of Western Australia. Primary Intention \n                           12, 62\u201370 (2004).","journal-title":"Primary Intention"},{"key":"9828_CR32","doi-asserted-by":"publisher","first-page":"1723","DOI":"10.2337\/dc15-0332","volume":"38","author":"BS Rasmussen","year":"2015","unstructured":"Rasmussen, B. S. et al. A Randomized Controlled Trial Comparing Telemedical and Standard Outpatient Monitoring of Diabetic Foot Ulcers. Diabetes Care \n                           38, 1723\u20131729 (2015).","journal-title":"Diabetes Care"},{"key":"9828_CR33","doi-asserted-by":"crossref","unstructured":"Muller, M., David-Tchouda, S., Margier, J., Oreglia, M. & Benhamou, P. Y. Comment on Rasmussen et al. A Randomized Controlled Trial Comparing Telemedical and Standard Outpatient Monitoring of Diabetic Foot Ulcers. Diabetes Care 2015, 38, 172\u20131729, Diabetes Care \n                           39, e9\u201310 (2016).","DOI":"10.2337\/dc15-1659"},{"key":"9828_CR34","doi-asserted-by":"publisher","first-page":"714","DOI":"10.1089\/dia.2014.0052","volume":"16","author":"JJ Van Netten","year":"2014","unstructured":"Van Netten, J. J. et al. Diagnostic Values for Skin Temperature Assessment to Detect Diabetes-Related Foot Complications. Diabetes Technol. Ther. \n                           16, 714\u2013721 (2014).","journal-title":"Diabetes Technol. Ther."},{"issue":"177\u201380","key":"9828_CR35","volume":"25","author":"T Kanazawa","year":"2016","unstructured":"Kanazawa, T. et al. Use of smartphone attached mobile thermography assessing subclinical inflammation: a pilot study. J. Wound Care \n                           25(177\u201380), 182 (2016).","journal-title":"J. Wound Care"},{"key":"9828_CR36","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","volume":"542","author":"A Esteva","year":"2017","unstructured":"Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature \n                           542, 115\u2013118 (2017).","journal-title":"Nature"},{"key":"9828_CR37","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1177\/1534734616654567","volume":"15","author":"RO Forsythe","year":"2016","unstructured":"Forsythe, R. O., Ozdemir, B. A., Chemla, E. S., Jones, K. G. & Hinchliffe, R. J. Interobserver Reliability of Three Validated Scoring Systems in the Assessment of Diabetic Foot Ulcers. Int. J. Low Extrem Wounds \n                           15, 213\u2013219 (2016).","journal-title":"Int. J. Low Extrem Wounds"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-017-09828-4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-017-09828-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-017-09828-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,23]],"date-time":"2022-12-23T09:30:48Z","timestamp":1671787848000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-017-09828-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,8,25]]},"references-count":37,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2017,12]]}},"alternative-id":["9828"],"URL":"https:\/\/doi.org\/10.1038\/s41598-017-09828-4","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,8,25]]},"assertion":[{"value":"11 May 2017","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 July 2017","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 August 2017","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare that they have no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}],"article-number":"9480"}}