{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,2]],"date-time":"2026-11-02T23:12:51Z","timestamp":1793661171158,"version":"4.5.1"},"reference-count":18,"publisher":"American Medical Association (AMA)","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Dermatol"],"published-print":{"date-parts":[[2023,5,1]]},"abstract":"<jats:sec id=\"ab-doi230004-4\"><jats:title>Importance<\/jats:title><jats:p>Telemedicine use accelerated during the COVID-19 pandemic, and skin conditions were a common use case. However, many images submitted may be of insufficient quality for making a clinical determination.<\/jats:p><\/jats:sec><jats:sec id=\"ab-doi230004-5\"><jats:title>Objective<\/jats:title><jats:p>To determine whether an artificial intelligence (AI) decision support tool, a machine learning algorithm, could improve the quality of images submitted for telemedicine by providing real-time feedback and explanations to patients.<\/jats:p><\/jats:sec><jats:sec id=\"ab-doi230004-6\"><jats:title>Design, Setting, and Participants<\/jats:title><jats:p>This quality improvement study with an AI performance component and single-arm clinical pilot study component was conducted from March 2020 to October 2021. After training, the AI decision support tool was tested on 357 retrospectively collected telemedicine images from Stanford telemedicine from March 2020 to June 2021. Subsequently, a single-arm clinical pilot study was conducted to assess feasibility with 98 patients in the Stanford Department of Dermatology across 2 clinical sites from July 2021 to October 2021. For the clinical pilot study, inclusion criteria for patients included being adults (aged \u226518 years), presenting to clinic for a skin condition, and being able to photograph their own skin with a smartphone.<\/jats:p><\/jats:sec><jats:sec id=\"ab-doi230004-7\"><jats:title>Interventions<\/jats:title><jats:p>During the clinical pilot study, patients were given a handheld smartphone device with a machine learning algorithm interface loaded and were asked to take images of any lesions of concern. Patients were able to review and retake photos prior to submitting, so each submitted photo met the patient\u2019s assumed standard of clinical acceptability. A machine learning algorithm then gave the patient feedback on whether the image was acceptable. If the image was rejected, the patient was provided a reason by the AI decision support tool and allowed to retake the photos.<\/jats:p><\/jats:sec><jats:sec id=\"ab-doi230004-8\"><jats:title>Main Outcomes and Measures<\/jats:title><jats:p>The main outcome of the retrospective image analysis was the receiver operator curve area under the curve (ROC-AUC). The main outcome of the clinical pilot study was the image quality difference between the baseline images and the images approved by AI decision support.<\/jats:p><\/jats:sec><jats:sec id=\"ab-doi230004-9\"><jats:title>Results<\/jats:title><jats:p>Of the 98 patients included, the mean (SD) age was 49.8 (17.6) years, and 50 (51%) of the patients were male. On retrospective telemedicine images, the machine learning algorithm effectively identified poor-quality images (ROC-AUC of 0.78) and the reason for poor quality (blurry ROC-AUC of 0.84; lighting issues ROC-AUC of 0.70). The performance was consistent across age and sex. In the clinical pilot study, patient use of the machine learning algorithm was associated with improved image quality. An AI algorithm was associated with reduction in the number of patients with a poor-quality image by 68.0%.<\/jats:p><\/jats:sec><jats:sec id=\"ab-doi230004-10\"><jats:title>Conclusions and Relevance<\/jats:title><jats:p>In this quality improvement study, patients use of the AI decision support with a machine learning algorithm was associated with improved quality of skin disease photographs submitted for telemedicine use.<\/jats:p><\/jats:sec>","DOI":"10.1001\/jamadermatol.2023.0091","type":"journal-article","created":{"date-parts":[[2023,3,15]],"date-time":"2023-03-15T15:12:40Z","timestamp":1678893160000},"page":"496","source":"Crossref","is-referenced-by-count":27,"title":["Development and Clinical Evaluation of an Artificial Intelligence Support Tool for Improving Telemedicine Photo Quality"],"prefix":"10.1001","volume":"159","author":[{"given":"Kailas","family":"Vodrahalli","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Stanford University, Stanford, California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Justin","family":"Ko","sequence":"additional","affiliation":[{"name":"Department of Dermatology, Stanford School of Medicine, Redwood City, California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Albert S.","family":"Chiou","sequence":"additional","affiliation":[{"name":"Department of Dermatology, Stanford School of Medicine, Redwood City, California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roberto","family":"Novoa","sequence":"additional","affiliation":[{"name":"Department of Dermatology, Stanford School of Medicine, Redwood City, California"},{"name":"Department of Pathology, Stanford School of Medicine, Stanford, California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abubakar","family":"Abid","sequence":"additional","affiliation":[{"name":"Hugging Face, New York, New York"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michelle","family":"Phung","sequence":"additional","affiliation":[{"name":"Department of Dermatology, Stanford School of Medicine, Redwood City, California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kiana","family":"Yekrang","sequence":"additional","affiliation":[{"name":"Department of Dermatology, Stanford School of Medicine, Redwood City, California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paige","family":"Petrone","sequence":"additional","affiliation":[{"name":"Department of Dermatology, Stanford School of Medicine, Redwood City, California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James","family":"Zou","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Stanford University, Stanford, California"},{"name":"Department of Biomedical Data Science, Stanford School of Medicine, Stanford, California"},{"name":"Department of Computer Science, Stanford University, Stanford, California"},{"name":"Chan-Zuckerberg Biohub, San Francisco, California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roxana","family":"Daneshjou","sequence":"additional","affiliation":[{"name":"Department of Dermatology, Stanford School of Medicine, Redwood City, California"},{"name":"Department of Biomedical Data Science, Stanford School of Medicine, Stanford, California"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10","reference":[{"issue":"6","key":"doi230004r1","first-page":"285","article-title":"Most common dermatologic conditions encountered by dermatologists and nondermatologists.","volume":"94","author":"Wilmer","year":"2014","journal-title":"Cutis"},{"issue":"2","key":"doi230004r2","doi-asserted-by":"publisher","DOI":"10.2196\/33833","article-title":"Using implementation science to understand teledermatology implementation early in the COVID-19 pandemic: cross-sectional study.","volume":"5","author":"Briggs","year":"2022","journal-title":"JMIR Dermatol"},{"issue":"1","key":"doi230004r4","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1177\/15589447211006834","article-title":"The ability of patients to provide standardized, patient-taken photographs for the remote assessment of Dupuytren disease.","volume":"18","author":"Irvine","year":"2023","journal-title":"Hand (N Y)"},{"issue":"10","key":"doi230004r5","doi-asserted-by":"publisher","first-page":"1183","DOI":"10.1001\/jamadermatol.2022.2815","article-title":"Quality and perceived usefulness of patient-submitted store-and-forward teledermatology images.","volume":"158","author":"Jiang","year":"2022","journal-title":"JAMA Dermatol"},{"issue":"2","key":"doi230004r6","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1001\/jamadermatol.2019.3849","article-title":"Improved medical photography: key tips for creating images of lasting value.","volume":"156","author":"Muraco","year":"2020","journal-title":"JAMA Dermatol"},{"issue":"6","key":"doi230004r9","doi-asserted-by":"publisher","first-page":"1657","DOI":"10.1109\/TIP.2010.2044957","article-title":"A completed modeling of local binary pattern operator for texture classification.","volume":"19","author":"Guo","year":"2010","journal-title":"IEEE Trans Image Process"},{"issue":"3","key":"doi230004r11","doi-asserted-by":"publisher","first-page":"837","DOI":"10.2307\/2531595","article-title":"Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.","volume":"44","author":"DeLong","year":"1988","journal-title":"Biometrics"},{"issue":"11","key":"doi230004r12","doi-asserted-by":"publisher","first-page":"1389","DOI":"10.1109\/LSP.2014.2337313","article-title":"Fast implementation of DeLong\u2019s algorithm for comparing the areas under correlated receiver operating characteristic curves.","volume":"21","author":"Sun","year":"2014","journal-title":"IEEE Signal Process Lett"},{"issue":"2","key":"doi230004r15","doi-asserted-by":"publisher","first-page":"492","DOI":"10.1016\/j.jaad.2021.09.026","article-title":"The disproportionate burden of electronic health record messages with image attachments in dermatology.","volume":"86","author":"Borre","year":"2022","journal-title":"J Am Acad Dermatol"},{"issue":"9","key":"doi230004r16","doi-asserted-by":"publisher","first-page":"2353","DOI":"10.1016\/j.jid.2022.02.003","article-title":"Evaluation of artificial intelligence-assisted diagnosis of skin neoplasms: a single-center, paralleled, unmasked, randomized controlled trial.","volume":"142","author":"Han","year":"2022","journal-title":"J Invest Dermatol"},{"issue":"1","key":"doi230004r17","doi-asserted-by":"publisher","first-page":"e64","DOI":"10.1016\/S2589-7500(21)00252-1","article-title":"Characteristics of publicly available skin cancer image datasets: a systematic review.","volume":"4","author":"Wen","year":"2022","journal-title":"Lancet Digit Health"},{"issue":"11","key":"doi230004r18","doi-asserted-by":"publisher","first-page":"1362","DOI":"10.1001\/jamadermatol.2021.3129","article-title":"Lack of transparency and potential bias in artificial intelligence data sets and algorithms: a scoping review.","volume":"157","author":"Daneshjou","year":"2021","journal-title":"JAMA Dermatol"},{"key":"doi230004r3","first-page":"220","volume-title":"TrueImage: A Machine Learning Algorithm To Improve the Quality of Telehealth Photos","author":"Vodrahalli","year":"2021"},{"key":"doi230004r7","unstructured":"TensorFlow.js. How DermAssist uses TensorFlow.js for on-device image quality checks. October 11, 2021. Accessed May 9, 2022. https:\/\/blog.tensorflow.org\/2021\/10\/how-DermAssist-uses-TensorFlowJS.html"},{"key":"doi230004r8","doi-asserted-by":"crossref","unstructured":"He? K, Zhang? X, Ren? S, Sun? J. Deep Residual Learning for Image Recognition. Accessed February 1, 2023. https:\/\/www.cv-foundation.org\/openaccess\/content_cvpr_2016\/papers\/He_Deep_Residual_Learning_CVPR_2016_paper.pdf","DOI":"10.1109\/CVPR.2016.90"},{"key":"doi230004r10","unstructured":"Abid? A, Abdalla? A, Abid? A, Khan? D, Alfozan? A, Zou? J. Gradio: hassle-free sharing and testing of ML models in the wild.? ArXiv. Preprint posted online June 6, 2019. doi:10.48550\/arXiv.1906.02569"},{"key":"doi230004r13","unstructured":"Kazeev? N. DeLong Test Implementation. GitHub. Accessed February 1, 2023. https:\/\/github.com\/yandexdataschool\/roc_comparison"},{"key":"doi230004r14","unstructured":"McKinsey & Co. Telehealth: a quarter-trillion-dollar post-COVID-19 reality? July 9, 2021. Accessed July 1, 2022. https:\/\/www.mckinsey.com\/industries\/healthcare-systems-and-services\/our-insights\/telehealth-a-quarter-trillion-dollar-post-covid-19-reality"}],"container-title":["JAMA Dermatology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamadermatology\/articlepdf\/2802154\/jamadermatology_vodrahalli_2023_oi_230004_1684162779.76932.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,17]],"date-time":"2023-05-17T15:09:34Z","timestamp":1684336174000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamadermatology\/fullarticle\/2802154"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,1]]},"references-count":18,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,5,1]]}},"URL":"https:\/\/doi.org\/10.1001\/jamadermatol.2023.0091","relation":{},"ISSN":["2168-6068"],"issn-type":[{"value":"2168-6068","type":"print"}],"subject":[],"published":{"date-parts":[[2023,5,1]]}}}