{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T13:59:34Z","timestamp":1787320774559,"version":"build-2736575974"},"reference-count":36,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2024,9,9]],"date-time":"2024-09-09T00:00:00Z","timestamp":1725840000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Institute of Innovation and Technology (EIT) Health and the European Union","award":["230123"],"award-info":[{"award-number":["230123"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background: This study aimed to test an artificial intelligence-based reading system (AIRS) capable of reading retinographies of type 2 diabetic (T2DM) patients and a predictive algorithm (DRPA) that predicts the risk of each patient with T2DM of developing diabetic retinopathy (DR). Methods: We tested the ability of the AIRS to read and classify 15,297 retinal photographs from our database of diabetics and 1200 retinal images taken with Messidor-2 into the different DR categories. We tested the DRPA in a sample of 40,129 T2DM patients. The results obtained by the AIRS and the DRPA were then compared with those provided by four retina specialists regarding sensitivity (S), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), accuracy (ACC), and area under the curve (AUC). Results: The results of testing the AIRS for identifying referral DR (RDR) in our database were ACC = 98.6, S = 96.7, SP = 99.8, PPV = 99.0, NPV = 98.0, and AUC = 0.958, and in Messidor-2 were ACC = 96.78%, S = 94.64%, SP = 99.14%, PPV = 90.54%, NPV = 99.53%, and AUC = 0.918. The results of our DRPA when predicting the presence of any type of DR were ACC = 0.97, S = 0.89, SP = 0.98, PPV = 0.79, NPV = 0.98, and AUC = 0.92. Conclusions: The AIRS performed well when reading and classifying the retinographies of T2DM patients with RDR. The DRPA performed well in predicting the absence of DR based on some clinical variables.<\/jats:p>","DOI":"10.3390\/diagnostics14171992","type":"journal-article","created":{"date-parts":[[2024,9,9]],"date-time":"2024-09-09T07:14:04Z","timestamp":1725866044000},"page":"1992","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Artificial Intelligence-Based Screening System for Diabetic Retinopathy in Primary Care"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5014-6329","authenticated-orcid":false,"given":"Marc","family":"Baget-Bernaldiz","sequence":"first","affiliation":[{"name":"Ophthalmology Service, Hospital Universitari Sant Joan, Institut d\u2019Investigaci\u00f3 Sanit\u00e0ria Pere Virgili [IISPV], Universitat Rovira i Virgili, 43204 Reus, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2518-9973","authenticated-orcid":false,"given":"Benilde","family":"Fontoba-Poveda","sequence":"additional","affiliation":[{"name":"Responsible for Diabetic Retinopathy Eye Screening Program in Primary Care in Baix Llobregat Barcelona (Spain), Institut d\u2019Investigaci\u00f3 Sanitaria Pere Virgili [IISPV], 43204 Reus, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7061-8987","authenticated-orcid":false,"given":"Pedro","family":"Romero-Aroca","sequence":"additional","affiliation":[{"name":"Ophthalmology Service, Hospital Universitari Sant Joan, Institut d\u2019Investigaci\u00f3 Sanit\u00e0ria Pere Virgili [IISPV], Universitat Rovira i Virgili, 43204 Reus, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2970-205X","authenticated-orcid":false,"given":"Raul","family":"Navarro-Gil","sequence":"additional","affiliation":[{"name":"Ophthalmology Service, Hospital Universitari Sant Joan, Institut d\u2019Investigaci\u00f3 Sanit\u00e0ria Pere Virgili [IISPV], Universitat Rovira i Virgili, 43204 Reus, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adriana","family":"Hernando-Comerma","sequence":"additional","affiliation":[{"name":"Ophthalmology Service, Hospital Universitari Sant Joan, Institut d\u2019Investigaci\u00f3 Sanit\u00e0ria Pere Virgili [IISPV], Universitat Rovira i Virgili, 43204 Reus, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Angel","family":"Bautista-Perez","sequence":"additional","affiliation":[{"name":"Ophthalmology Service, Hospital Universitari Sant Joan, Institut d\u2019Investigaci\u00f3 Sanit\u00e0ria Pere Virgili [IISPV], Universitat Rovira i Virgili, 43204 Reus, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Monica","family":"Llagostera-Serra","sequence":"additional","affiliation":[{"name":"Ophthalmology Service, Hospital Universitari Sant Joan, Institut d\u2019Investigaci\u00f3 Sanit\u00e0ria Pere Virgili [IISPV], Universitat Rovira i Virgili, 43204 Reus, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cristian","family":"Morente-Lorenzo","sequence":"additional","affiliation":[{"name":"Ophthalmology Service, Hospital Universitari Sant Joan, Institut d\u2019Investigaci\u00f3 Sanit\u00e0ria Pere Virgili [IISPV], Universitat Rovira i Virgili, 43204 Reus, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Montse","family":"Vizcarro","sequence":"additional","affiliation":[{"name":"Ophthalmology Service, Hospital Universitari Sant Joan, Institut d\u2019Investigaci\u00f3 Sanit\u00e0ria Pere Virgili [IISPV], Universitat Rovira i Virgili, 43204 Reus, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8548-0290","authenticated-orcid":false,"given":"Alejandra","family":"Mira-Puerto","sequence":"additional","affiliation":[{"name":"Ophthalmology Service, Hospital Universitari Sant Joan, Institut d\u2019Investigaci\u00f3 Sanit\u00e0ria Pere Virgili [IISPV], Universitat Rovira i Virgili, 43204 Reus, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,9,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1016\/j.diabres.2017.03.024","article-title":"IDF Diabetes Atlas: Global estimates for the prevalence of diabetes for 2015 and 2040","volume":"128","author":"Ogurtsova","year":"2017","journal-title":"Diabetes Res. Clin. Pract."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1007\/s00125-011-2336-9","article-title":"Prevalence of diabetes mellitus and impaired glucose regulation in Spain: The Di@bet.es Study","volume":"55","author":"Soriguer","year":"2012","journal-title":"Diabetologia"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1580","DOI":"10.1016\/j.ophtha.2021.04.027","article-title":"Global Prevalence of Diabetic Retinopathy and Projection of Burden through 2045: Systematic Review and Meta-analysis","volume":"128","author":"Teo","year":"2021","journal-title":"Ophthalmology"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1549","DOI":"10.1185\/030079908X297303","article-title":"Direct and indirect costs among employees with diabetic retinopathy in the United States","volume":"24","author":"Lee","year":"2008","journal-title":"Curr. Med Res. Opin."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/s00417-007-0573-x","article-title":"The economic burden of diabetic retinopathy in Germany in 2002","volume":"246","author":"Happich","year":"2008","journal-title":"Graefe\u2019s Arch. Clin. Exp. Ophthalmol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1038\/eye.2012.287","article-title":"The Royal College of Ophthalmologists\u2019 clinical guidelines for diabetic retinopathy: A summary","volume":"27","author":"Ghanchi","year":"2013","journal-title":"Eye"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1177\/1062860615574863","article-title":"An Appraisal of Clinical Practice Guidelines for Diabetic Retinopathy","volume":"31","author":"Wu","year":"2016","journal-title":"Am. J. Med Qual."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1076\/opep.9.3.205.1516","article-title":"Prevalence of diabetic retinopathy and eye care in a rural area of Spain","volume":"9","author":"Velilla","year":"2002","journal-title":"Ophthalmic Epidemiol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2366","DOI":"10.1001\/jama.2016.17563","article-title":"Artificial intelligence with deep learning technology looks into diabetic retinopathy screening","volume":"316","author":"Wong","year":"2016","journal-title":"JAMA"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2211","DOI":"10.1001\/jama.2017.18152","article-title":"Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes","volume":"318","author":"Ting","year":"2017","journal-title":"JAMA"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1038\/s41746-018-0040-6","article-title":"Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices","volume":"1","author":"Lavin","year":"2018","journal-title":"NPJ Digit. Med."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1518","DOI":"10.1136\/bjo.2007.120972","article-title":"Cost-effectiveness of implementing automated grading within the national screening programme for diabetic retinopathy in Scotland","volume":"91","author":"Scotland","year":"2007","journal-title":"Br. J. Ophthalmol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1016\/j.ophtha.2016.11.014","article-title":"Automated Diabetic Retinopathy Image Assessment Software: Diagnostic Accuracy and Cost-Effectiveness Compared with Human Graders","volume":"124","author":"Tufail","year":"2017","journal-title":"Ophthalmology"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1016\/j.pcd.2021.08.004","article-title":"Predictors of attendance at diabetic retinopathy screening among people with type 2 diabetes: Secondary analysis of data from primary care","volume":"15","author":"Riordan","year":"2021","journal-title":"Prim. Care Diabetes"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1111\/aos.13613","article-title":"Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System","volume":"96","author":"Abramoff","year":"2018","journal-title":"Acta Ophthalmol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.preteyeres.2018.07.004","article-title":"Artificial intelligence in retina","volume":"67","author":"Sadeghipour","year":"2018","journal-title":"Prog. Retin. Eye Res."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1089\/dia.2019.0164","article-title":"The value of automated diabetic retinopathy screening with the EyeArt system: A study of more than 100,000 consecutive encounters from people with diabetes","volume":"21","author":"Bhaskaranand","year":"2019","journal-title":"Diabetes Technol. Ther."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1254","DOI":"10.2337\/dc12-1491","article-title":"Microaneurysm turnover at the macula predicts risk of development of clinically significant macular edema in persons with mild nonproliferative diabetic retinopathy","volume":"36","author":"Ribeiro","year":"2013","journal-title":"Diabetes Care"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1111\/aos.14306","article-title":"Evaluation of a deep learning system for the joint automated detection of diabetic retinopathy and age-related macular degeneration","volume":"98","author":"Contreras","year":"2020","journal-title":"Acta Ophthalmol."},{"key":"ref_20","unstructured":"Grzybowski, A. (2021). Singapore Eye Lesions Analyzer (SELENA): The Deep Learning System for Retinal Diseases. Artificial Intelligence in Ophthalmology, Springer International Publishing. [1st ed.]."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2509","DOI":"10.2337\/dc18-0147","article-title":"An automated grading System for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs","volume":"41","author":"Li","year":"2018","journal-title":"Diabetes Care"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1038\/s41746-019-0099-8","article-title":"Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program","volume":"2","author":"Ruamviboonsuk","year":"2019","journal-title":"NPJ Digit. Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1168","DOI":"10.2337\/dc20-1877","article-title":"Multicenter, head-to-head, real-world validation study of seven automated artificial intelligence diabetic retinopathy screening systems","volume":"44","author":"Soliz","year":"2021","journal-title":"Diabetes Care"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2525","DOI":"10.1007\/s00125-011-2257-7","article-title":"Individual risk assessment and information technology to optimise screening frequency for diabetic retinopathy","volume":"54","author":"Aspelund","year":"2011","journal-title":"Diabetologia"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"508","DOI":"10.1016\/j.jdiacomp.2014.12.020","article-title":"External validation of a risk assessment model to adjust the frequency of eye-screening visits in patients with diabetes mellitus","volume":"29","author":"Pinies","year":"2015","journal-title":"J. Diabetes Its Complicat."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1136\/bjophthalmol-2015-307341","article-title":"Individualised risk assessment for diabetic retinopathy and optimisation of screening intervals: A scientific approach to reducing healthcare costs","volume":"100","author":"Lund","year":"2016","journal-title":"Br. J. Ophthalmol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3310\/hta19740","article-title":"Development of a cost-effectiveness model for optimisation of the screening interval in diabetic retinopathy screening","volume":"19","author":"Scanlon","year":"2015","journal-title":"Heal. Technol. Assess."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e025788","DOI":"10.1136\/bmjopen-2018-025788","article-title":"Individualised screening for diabetic retinopathy: The ISDR study\u2014rationale, design and methodology for a randomised controlled trial comparing annual and individualised risk-based variable-interval screening","volume":"9","author":"Broadbent","year":"2019","journal-title":"BMJ Open"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"231","DOI":"10.5566\/ias.1155","article-title":"feedback on a publicly distributed image database: The Messidor database","volume":"33","author":"Zhang","year":"2014","journal-title":"Image Anal. Ster."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1177\/193229680900300315","article-title":"EyePACS: An adaptable telemedicine system for diabetic retinopathy screening","volume":"3","author":"Cuadros","year":"2009","journal-title":"J. Diabetes Sci. Technol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.patrec.2017.05.018","article-title":"Weighted kappa loss function for multi-class classification of ordinal data in deep learning","volume":"105","author":"Puig","year":"2018","journal-title":"Pattern Recognit. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1001","DOI":"10.1089\/tmj.2019.0137","article-title":"Validation of a Deep Learning Algorithm for Diabetic Retinopathy","volume":"26","author":"Valls","year":"2020","journal-title":"Telemed. e-Health"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1089\/tmj.2017.0282","article-title":"A Clinical Decision Support System for Diabetic Retinopathy Screening: Creating a Clinical Support Application","volume":"25","author":"Valls","year":"2019","journal-title":"Telemed. e-Health"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1167\/tvst.10.3.17","article-title":"Validation of a diagnostic support system for diabetic retinopathy based on clinical parameters","volume":"10","author":"Maarof","year":"2021","journal-title":"Transl. Vis. Sci. Technol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1159\/000368426","article-title":"Screening for diabetic retinopathy in the central region of portugal. added value of automated \u2018disease\/no disease\u2019 grading","volume":"233","author":"Ribeiro","year":"2015","journal-title":"Ophthalmologica"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1332","DOI":"10.1007\/s00125-014-3246-4","article-title":"Validation of a model to estimate personalised screening frequency to monitor diabetic retinopathy","volume":"57","author":"Walraven","year":"2014","journal-title":"Diabetologia"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/14\/17\/1992\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:52:05Z","timestamp":1760111525000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/14\/17\/1992"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,9]]},"references-count":36,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2024,9]]}},"alternative-id":["diagnostics14171992"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics14171992","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,9]]}}}