{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T01:21:00Z","timestamp":1789435260741,"version":"build-2803163510"},"reference-count":36,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T00:00:00Z","timestamp":1776297600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Pretrained foundation models are increasingly adopted for diabetic retinopathy (DR) screening, yet it remains unclear how much of their performance derives from the learned representations versus the adaptation procedure. Most benchmarks report discrimination metrics alone, neglecting probability calibration.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We compared the frozen representations of three pretrained encoders: MedSigLIP (medical vision\u2013language; ViT-B\/16, 448\u202f\u00d7\u202f448), RETFound (retinal self-supervised; ViT-L\/16, 224\u202f\u00d7\u202f224), and EfficientNet-B0 (ImageNet-supervised; 224\u202f\u00d7\u202f224). All encoder weights were frozen; only an identical lightweight multilayer perceptron head was trained. Models were developed on APTOS 2019 (3,662 fundus images; five-fold cross-validation) and externally validated on MESSIDOR-2 (1,744 images). Binary referable DR detection and five-class severity grading were evaluated. AUC, expected calibration error (ECE), and Brier score served as co-primary endpoints. External-set tests used patient-level cluster-robust bootstrap to account for bilateral correlation.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      On the development set, all three encoders achieved near-identical binary AUC (0.980\u20130.985). MedSigLIP showed superior calibration, with a lower Brier score than RETFound (0.044 vs. 0.049;\n                      <jats:italic>p<\/jats:italic>\n                      \u202f=\u202f0.030) and EfficientNet-B0 (0.044 vs. 0.052;\n                      <jats:italic>p<\/jats:italic>\n                      \u202f=\u202f0.006). External validation on MESSIDOR-2 revealed divergence: MedSigLIP maintained an AUC of 0.915 (drop 0.070), whereas RETFound fell to 0.697 (drop 0.286) and EfficientNet-B0 to 0.745 (drop 0.236). Retina-specific RETFound performed below the ImageNet baseline (\u0394AUC\u202f=\u202f\u22120.051;\n                      <jats:italic>p<\/jats:italic>\n                      \u202f=\u202f0.016, cluster-robust bootstrap). For five-class grading, MedSigLIP achieved an external macro-F1 of 0.450 versus 0.247 (RETFound) and 0.291 (EfficientNet-B0). Temperature scaling reduced development ECE to 0.014\u20130.022 but proved ineffective under domain shift (external ECE 0.086\u20130.149). All encoders exhibited catastrophic failure on mild DR (grade 1) externally, with RETFound and EfficientNet-B0 achieving F1\u202f=\u202f0.000 and MedSigLIP only 0.153.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Under frozen transfer, the MedSigLIP encoder package produced more generalisable and better calibrated representations than both retinal self-supervised (RETFound) and ImageNet-supervised (EfficientNet-B0) encoders. Domain-specific pretraining did not guarantee domain-general frozen representations. These findings demonstrate that development-set discrimination alone is insufficient for encoder evaluation and that calibration metrics\u2014particularly the Brier score\u2014should be reported as standard practice.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fmed.2026.1815982","type":"journal-article","created":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T04:20:57Z","timestamp":1776313257000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["How well do frozen foundation models transfer? A calibration-focused benchmark for diabetic retinopathy grading"],"prefix":"10.3389","volume":"13","author":[{"given":"Mehmet","family":"Poyrazer","sequence":"first","affiliation":[{"name":"Division of Endocrinology and Metabolism, University of Health Sciences Ankara Training and Research Hospital","place":["Ankara, T\u00fcrkiye"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"H\u00fcseyin","family":"Ya\u011fc\u0131","sequence":"additional","affiliation":[{"name":"Division of Endocrinology and Metabolism, University of Health Sciences Ankara Training and Research Hospital","place":["Ankara, T\u00fcrkiye"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"R\u0131dvan","family":"Erten","sequence":"additional","affiliation":[{"name":"Division of Geriatrics, Department of Internal Medicine, Faculty of Medicine, Kocaeli University","place":["Kocaeli, T\u00fcrkiye"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,4,16]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"1580","DOI":"10.1016\/j.ophtha.2021.04.027","article-title":"Global prevalence of diabetic retinopathy and projection of burden through 2045: systematic review and meta-analysis","volume":"128","author":"Teo","year":"2021","journal-title":"Ophthalmology"},{"key":"ref2","volume-title":"IDF Diabetes Atlas","year":"2021"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs","volume":"316","author":"Gulshan","year":"2016","journal-title":"JAMA"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"2211","DOI":"10.1001\/jama.2017.18152","article-title":"Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes","volume":"318","author":"Ting","year":"2017","journal-title":"JAMA"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"5200","DOI":"10.1167\/iovs.16-19964","article-title":"Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning","volume":"57","author":"Abr\u00e0moff","year":"2016","journal-title":"Invest Ophthalmol Vis Sci"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1038\/s41746-018-0040-6","article-title":"Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices","volume":"1","author":"Abr\u00e0moff","year":"2018","journal-title":"NPJ Digit Med"},{"key":"ref7","article-title":"On the opportunities and risks of foundation models","author":"Bommasani","year":"2021"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1038\/s41586-023-05881-4","article-title":"Foundation models for generalist medical artificial intelligence","volume":"616","author":"Moor","year":"2023","journal-title":"Nature"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1038\/s41586-023-06555-x","article-title":"A foundation model for generalizable disease detection from retinal images","volume":"622","author":"Zhou","year":"2023","journal-title":"Nature"},{"key":"ref10","year":""},{"key":"ref11","first-page":"11975","author":"Zhai","year":"2023"},{"key":"ref12","first-page":"6105","author":"Tan","year":"2019"},{"key":"ref13","author":"Hu","year":"2022"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1136\/amiajnl-2011-000291","article-title":"Calibrating predictive model estimates to support personalized medicine","volume":"19","author":"Jiang","year":"2012","journal-title":"J Am Med Inform Assoc"},{"key":"ref15","first-page":"1321","author":"Guo","year":"2017"},{"key":"ref16","year":"2019"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"231","DOI":"10.5566\/ias.1155","article-title":"Feedback on a publicly distributed image database: the Messidor database","volume":"33","author":"Decenci\u00e8re","year":"2014","journal-title":"Image Anal Stereol"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"1264","DOI":"10.1016\/j.ophtha.2018.01.034","article-title":"Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy","volume":"125","author":"Krause","year":"2018","journal-title":"Ophthalmology"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"1677","DOI":"10.1016\/S0161-6420(03)00475-5","article-title":"Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales","volume":"110","author":"Wilkinson","year":"2003","journal-title":"Ophthalmology"},{"key":"ref20","author":"Dosovitskiy","year":"2021"},{"key":"ref21","first-page":"15979","author":"He","year":"2022"},{"key":"ref22","first-page":"248","author":"Deng","year":"2009"},{"key":"ref23","author":"Loshchilov","year":"2019"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2","article-title":"Verification of forecasts expressed in terms of probability","volume":"78","author":"Brier","year":"1950","journal-title":"Mon Weather Rev"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"837","DOI":"10.2307\/2531595","article-title":"Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach","volume":"44","author":"DeLong","year":"1988","journal-title":"Biometrics"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1007\/BF02295996","article-title":"Note on the sampling error of the difference between correlated proportions or percentages","volume":"12","author":"McNemar","year":"1947","journal-title":"Psychometrika"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1037\/h0026256","article-title":"Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit","volume":"70","author":"Cohen","year":"1968","journal-title":"Psychol Bull"},{"key":"ref28","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4899-4541-9","volume-title":"An Introduction to the Bootstrap","author":"Efron","year":"1993"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"J R Stat Soc Series B Stat Methodol"},{"key":"ref30","author":"Alain","year":"2017"},{"key":"ref31","first-page":"512","author":"Neyshabur","year":"2020"},{"key":"ref32","first-page":"13991","author":"Ovadia","year":"2019"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2507.07780","article-title":"Where are we with calibration under dataset shift in image classification","volume":"2025","author":"Roschewitz","year":"2025","journal-title":"Trans Mach Learn Res"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"e23213","DOI":"10.1002\/ima.23213","article-title":"Multi-deep learning approach with transfer learning for 7-stages diabetic retinopathy classification","volume":"34","author":"Khan","year":"2024","journal-title":"Int J Imaging Syst Technol"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"2481","DOI":"10.32604\/cmc.2023.043239","article-title":"Optimizing fully convolutional encoder-decoder network for segmentation of diabetic eye disease","volume":"77","author":"Khan","year":"2023","journal-title":"Comput Mater Contin"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"78125","DOI":"10.1007\/s11042-024-18627-9","article-title":"Towards improved fundus disease detection using Swin transformers","volume":"83","author":"Jawad","year":"2024","journal-title":"Multimed Tools Appl"}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1815982\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T04:20:59Z","timestamp":1776313259000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1815982\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,16]]},"references-count":36,"alternative-id":["10.3389\/fmed.2026.1815982"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2026.1815982","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,16]]},"article-number":"1815982"}}