{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T11:16:59Z","timestamp":1772191019762,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,11,22]],"date-time":"2025-11-22T00:00:00Z","timestamp":1763769600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>This study evaluated the generalization and reliability of machine learning models for multiclass classification of retinal pathologies using a diverse set of images representing eight disease categories. Images were aggregated from two public datasets and divided into training, validation, and test sets, with an additional independent dataset used for external validation. Multiple modeling approaches were compared, including classical machine learning algorithms, convolutional neural networks with and without data augmentation, and a deep neural network using pre-trained feature extraction. Analysis of learning dynamics revealed that classical models and unaugmented convolutional neural networks exhibited overfitting and poor generalization, while models with data augmentation and the deep neural network showed healthy, parallel convergence of training and validation performance. Only the deep neural network demonstrated a consistent, monotonic decrease in accuracy, F1-score, and recall from training through external validation, indicating robust generalization. These results underscore the necessity of evaluating learning dynamics (not just summary metrics) to ensure model reliability and patient safety. Typically, model performance is expected to decrease gradually as data becomes less familiar. Therefore, models that do not exhibit these healthy learning dynamics, or that show unexpected improvements in performance on subsequent datasets, should not be considered for clinical application, as such patterns may indicate methodological flaws or data leakage rather than true generalization.<\/jats:p>","DOI":"10.3390\/informatics12040128","type":"journal-article","created":{"date-parts":[[2025,11,24]],"date-time":"2025-11-24T09:02:07Z","timestamp":1763974927000},"page":"128","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Learning Dynamics Analysis: Assessing Generalization of Machine Learning Models for Optical Coherence Tomography Multiclass Classification"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-6317-0066","authenticated-orcid":false,"given":"Michael","family":"Sher","sequence":"first","affiliation":[{"name":"Department of Osteopathic Manipulative Medicine, New York Institute of Technology, College of Osteopathic Medicine, Old Westbury, Nassau County, Long Island, NY 11568, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5453-8101","authenticated-orcid":false,"given":"David","family":"Remyes","sequence":"additional","affiliation":[{"name":"Department of Osteopathic Manipulative Medicine, New York Institute of Technology, College of Osteopathic Medicine, Old Westbury, Nassau County, Long Island, NY 11568, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-0272-1501","authenticated-orcid":false,"given":"Riah","family":"Sharma","sequence":"additional","affiliation":[{"name":"Department of Osteopathic Manipulative Medicine, New York Institute of Technology, College of Osteopathic Medicine, Old Westbury, Nassau County, Long Island, NY 11568, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2154-3728","authenticated-orcid":false,"given":"Milan","family":"Toma","sequence":"additional","affiliation":[{"name":"Department of Osteopathic Manipulative Medicine, New York Institute of Technology, College of Osteopathic Medicine, Old Westbury, Nassau County, Long Island, NY 11568, USA"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1178","DOI":"10.1126\/science.1957169","article-title":"Optical Coherence Tomography","volume":"254","author":"Huang","year":"1991","journal-title":"Science"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1745","DOI":"10.1364\/OL.28.001745","article-title":"Real-time in vivo imaging by high-speed spectral optical coherence tomography","volume":"28","author":"Wojtkowski","year":"2003","journal-title":"Opt. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Yi, J., Chen, S., Shu, X., Fawzi, A.A., and Zhang, H.F. (2015). Human retinal imaging using visible-light optical coherence tomography guided by scanning laser ophthalmoscopy. Biomed. Opt. Express, 6.","DOI":"10.1364\/BOE.6.003701"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1035","DOI":"10.1016\/j.jcin.2009.06.019","article-title":"Intracoronary Optical Coherence Tomography: A Comprehensive Review","volume":"2","author":"Bezerra","year":"2009","journal-title":"JACC Cardiovasc. Interv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Tsai, T.H., Leggett, C.L., and Trindade, A.J. (2017). Optical coherence tomography in gastroenterology: A review and future outlook. J. Biomed. Opt., 22.","DOI":"10.1117\/1.JBO.22.12.121716"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"S56","DOI":"10.3928\/15428877-20110627-05","article-title":"The Role of Spectral-Domain OCT in the Diagnosis and Management of Neovascular Age-Related Macular Degeneration","volume":"42","author":"Regatieri","year":"2011","journal-title":"Ophthalmic Surg. Lasers Imaging Retin."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"20713","DOI":"10.1155\/2013\/920713","article-title":"Structural Changes in Individual Retinal Layers in Diabetic Macular Edema","volume":"2013","author":"Murakami","year":"2013","journal-title":"J. Diabetes Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2368","DOI":"10.1016\/j.ophtha.2016.07.010","article-title":"Optical Coherence Tomography Angiography of DME and Its Association with Anti-VEGF Treatment Response","volume":"123","author":"Lee","year":"2016","journal-title":"Ophthalmology"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"693","DOI":"10.1146\/annurev-vision-100419-111350","article-title":"Optical Coherence Tomography and Glaucoma","volume":"7","author":"Geevarghese","year":"2021","journal-title":"Annu. Rev. Vis. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"ii15","DOI":"10.1136\/bjophthalmol-2013-304326","article-title":"OCT for glaucoma diagnosis, screening and detection of glaucoma progression","volume":"98","author":"Bussel","year":"2013","journal-title":"Br. J. Ophthalmol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1817","DOI":"10.1016\/j.ophtha.2018.05.008","article-title":"Spectral-Domain OCT: Helping the Clinician Diagnose Glaucoma","volume":"125","author":"Chen","year":"2018","journal-title":"Ophthalmology"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"123","DOI":"10.4103\/JOCO.JOCO_83_20","article-title":"Artifacts in Macular Optical Coherence Tomography","volume":"32","author":"Bazvand","year":"2020","journal-title":"J. Curr. Ophthalmol."},{"key":"ref_13","first-page":"e108","article-title":"Pitfalls in Retinal OCT Imaging","volume":"1","author":"Brinkmann","year":"2017","journal-title":"Ophthalmol. Point Care"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1001\/jamaophthalmol.2019.5099","article-title":"Pearls and Pitfalls of Optical Coherence Tomography Angiography Image Interpretation","volume":"138","author":"Sarraf","year":"2020","journal-title":"JAMA Ophthalmol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Karn, P.K., and Abdulla, W.H. (2023). On Machine Learning in Clinical Interpretation of Retinal Diseases Using OCT Images. Bioengineering, 10.","DOI":"10.3390\/bioengineering10040407"},{"key":"ref_16","first-page":"264","article-title":"Promising Artificial Intelligence-Machine Learning-Deep Learning Algorithms in Ophthalmology","volume":"8","author":"Balyen","year":"2019","journal-title":"Asia-Pac. J. Ophthalmol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1080\/08820538.2021.1890151","article-title":"Machine Learning Applications in Pediatric Ophthalmology: AUTHORS","volume":"36","author":"Oke","year":"2021","journal-title":"Semin. Ophthalmol."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Alsaih, K., Lemaitre, G., Rastgoo, M., Massich, J., Sidib\u00e9, D., and Meriaudeau, F. (2017). Machine learning techniques for diabetic macular edema (DME) classification on SD-OCT images. BioMed. Eng. OnLine, 16.","DOI":"10.1186\/s12938-017-0352-9"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1016\/j.oret.2016.12.009","article-title":"Deep Learning Is Effective for Classifying Normal Versus Age-Related Macular Degeneration OCT Images","volume":"1","author":"Lee","year":"2017","journal-title":"Ophthalmol. Retin."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"BIO141","DOI":"10.1167\/iovs.17-21789","article-title":"Machine Learning of the Progression of Intermediate Age-Related Macular Degeneration Based on OCT Imaging","volume":"58","author":"Bogunovic","year":"2017","journal-title":"Investig. Opthalmol. Vis. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Banerjee, I., de Sisternes, L., Hallak, J.A., Leng, T., Osborne, A., Rosenfeld, P.J., Gregori, G., Durbin, M., and Rubin, D. (2020). Prediction of age-related macular degeneration disease using a sequential deep learning approach on longitudinal SD-OCT imaging biomarkers. Sci. Rep., 10.","DOI":"10.1038\/s41598-020-72359-y"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e51","DOI":"10.1016\/S2589-7500(20)30240-5","article-title":"A global review of publicly available datasets for ophthalmological imaging: Barriers to access, usability, and generalisability","volume":"3","author":"Khan","year":"2021","journal-title":"Lancet Digit. Health"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1167\/tvst.9.2.11","article-title":"Methodological Challenges of Deep Learning in Optical Coherence Tomography for Retinal Diseases: A Review","volume":"9","author":"Yanagihara","year":"2020","journal-title":"Transl. Vis. Sci. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1111\/ceo.14258","article-title":"Deep learning in optical coherence tomography: Where are the gaps?","volume":"51","author":"Li","year":"2023","journal-title":"Clin. Exp. Ophthalmol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1080\/08820538.2021.1901123","article-title":"Artificial Intelligence (AI) and Retinal Optical Coherence Tomography (OCT)","volume":"36","author":"Dahrouj","year":"2021","journal-title":"Semin. Ophthalmol."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Miladinovi\u0107, A., Biscontin, A., Aj\u010devi\u0107, M., Kresevic, S., Accardo, A., Marangoni, D., Tognetto, D., and Inferrera, L. (2024). Evaluating deep learning models for classifying OCT images with limited data and noisy labels. Sci. Rep., 14.","DOI":"10.1038\/s41598-024-81127-1"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"43537","DOI":"10.1109\/ACCESS.2020.2977355","article-title":"Effect of Altered OCT Image Quality on Deep Learning Boundary Segmentation","volume":"8","author":"Kugelman","year":"2020","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/s11517-021-02469-w","article-title":"Synthetic OCT data in challenging conditions: Three-dimensional OCT and presence of abnormalities","volume":"60","author":"Danesh","year":"2021","journal-title":"Med Biol. Eng. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1167\/tvst.9.2.29","article-title":"Assessment of Generative Adversarial Networks Model for Synthetic Optical Coherence Tomography Images of Retinal Disorders","volume":"9","author":"Zheng","year":"2020","journal-title":"Transl. Vis. Sci. Technol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sher, M., Sharma, R., Remyes, D., Nasef, D., Nasef, D., and Toma, M. (2025). Stratified Multisource Optical Coherence Tomography Integration and Cross-Pathology Validation Framework for Automated Retinal Diagnostics. Appl. Sci., 15.","DOI":"10.3390\/app15094985"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Remyes, D., Nasef, D., Remyes, S., Tawfellos, J., Sher, M., Nasef, D., and Toma, M. (2025). Clinical Applicability and Cross-Dataset Validation of Machine Learning Models for Binary Glaucoma Detection. Information, 16.","DOI":"10.3390\/info16060432"},{"key":"ref_32","unstructured":"Ravelli, F. (2025, September 20). Retinal OCT Images: Retinal OCT Images Without Duplicates and Splitted. Version 1, 5.25 GB. Re-Edition of Original Dataset with Duplicates Removed. Contains 76,607 Images Total: 61,284 Training, 7659 Validation, 7664 Testing Images. Available online: https:\/\/www.kaggle.com\/datasets\/fabrizioravelli\/retinal-oct-images-splitted."},{"key":"ref_33","unstructured":"Kermany, D. (2025, September 20). Retinal OCT Images (Optical Coherence Tomography). Available online: https:\/\/www.kaggle.com\/datasets\/paultimothymooney\/kermany2018."},{"key":"ref_34","unstructured":"Naren, O.S. (2021). Retinal OCT Image Classification\u2014C8: High-Quality Multi-Class Dataset of OCT Images Across 8 Retinal Conditions. Version 3, 1.53 GB. Contains 24,000 High-Quality Retinal OCT Images Across 8 Classes (AMD, CNV, CSR, DME, DR, DRUSEN, MH, NORMAL)."},{"key":"ref_35","unstructured":"Gholami, P., Roy, P., Parthasarathy, M.K., and Lakshminarayanan, V. (2018). OCTID: Optical Coherence Tomography Image Database. arXiv."},{"key":"ref_36","unstructured":"Toma, M. (2025). AI-Assisted Medical Diagnostics: A Clinical Guide to Next-Generation Diagnostics, Dawning Research Press. [1st ed.]. Available online: https:\/\/openlibrary.org\/books\/OL60165315M\/."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Pang, S., Zou, B., Xiao, X., Peng, Q., Yan, J., Zhang, W., and Yue, K. (2024). A novel approach for automatic classification of macular degeneration OCT images. Sci. Rep., 14.","DOI":"10.1038\/s41598-024-70175-2"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Elkholy, M., and Marzouk, M.A. (2024). Deep learning-based classification of eye diseases using Convolutional Neural Network for OCT images. Front. Comput. Sci., 5.","DOI":"10.3389\/fcomp.2023.1252295"},{"key":"ref_39","unstructured":"U.S. Food and Drug Administration (FDA) (2025, May 22). Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products, Available online: https:\/\/www.fda.gov\/media\/184830\/download."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/12\/4\/128\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,24]],"date-time":"2025-11-24T09:47:01Z","timestamp":1763977621000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/12\/4\/128"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,22]]},"references-count":39,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["informatics12040128"],"URL":"https:\/\/doi.org\/10.3390\/informatics12040128","relation":{},"ISSN":["2227-9709"],"issn-type":[{"value":"2227-9709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,22]]}}}