{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T03:49:34Z","timestamp":1793332174427,"version":"4.5.0"},"reference-count":39,"publisher":"Public Library of Science (PLoS)","issue":"8","license":[{"start":{"date-parts":[[2023,8,7]],"date-time":"2023-08-07T00:00:00Z","timestamp":1691366400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004837","name":"Spanish Ministry of Science and Innovation","doi-asserted-by":"crossref","award":["PID2019-104358RB-I00"],"award-info":[{"award-number":["PID2019-104358RB-I00"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004587","name":"Instituto de Salud Carlos III","doi-asserted-by":"publisher","award":["PI17\/01726, PI20\/00437, RD21\/0002\/0050"],"award-info":[{"award-number":["PI17\/01726, PI20\/00437, RD21\/0002\/0050"]}],"id":[{"id":"10.13039\/501100004587","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010067","name":"Aragon Government","doi-asserted-by":"crossref","award":["T64_20R"],"award-info":[{"award-number":["T64_20R"]}],"id":[{"id":"10.13039\/501100010067","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100010067","name":"Aragon Government","doi-asserted-by":"crossref","award":["Ph. D. grant"],"award-info":[{"award-number":["Ph. D. grant"]}],"id":[{"id":"10.13039\/501100010067","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovacion","doi-asserted-by":"crossref","award":["PID2022-138703OB-I00"],"award-info":[{"award-number":["PID2022-138703OB-I00"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovacion","doi-asserted-by":"crossref","award":["PID2022-138703OB-I00"],"award-info":[{"award-number":["PID2022-138703OB-I00"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:sec id=\"sec001\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Several studies indicate that the anterior visual pathway provides information about the dynamics of axonal degeneration in Multiple Sclerosis (MS). Current research in the field is focused on the quest for the most discriminative features among patients and controls and the development of machine learning models that yield computer-aided solutions widely usable in clinical practice. However, most studies are conducted with small samples and the models are used as black boxes. Clinicians should not trust machine learning decisions unless they come with comprehensive and easily understandable explanations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec002\">\n                    <jats:title>Materials and methods<\/jats:title>\n                    <jats:p>A total of 216 eyes from 111 healthy controls and 100 eyes from 59 patients with relapsing-remitting MS were enrolled. The feature set was obtained from the thickness of the ganglion cell layer (GCL) and the retinal nerve fiber layer (RNFL). Measurements were acquired by the novel Posterior Pole protocol from Spectralis Optical Coherence Tomography (OCT) device. We compared two black-box methods (gradient boosting and random forests) with a glass-box method (explainable boosting machine). Explainability was studied using SHAP for the black-box methods and the scores of the glass-box method.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec003\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The best-performing models were obtained for the GCL layer. Explainability pointed out to the temporal location of the GCL layer that is usually broken or thinning in MS and the relationship between low thickness values and high probability of MS, which is coherent with clinical knowledge.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec004\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>The insights on how to use explainability shown in this work represent a first important step toward a trustworthy computer-aided solution for the diagnosis of MS with OCT.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1371\/journal.pone.0289495","type":"journal-article","created":{"date-parts":[[2023,8,7]],"date-time":"2023-08-07T13:27:17Z","timestamp":1691414837000},"page":"e0289495","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":24,"title":["Explainable artificial intelligence toward usable and trustworthy computer-aided diagnosis of multiple sclerosis from Optical Coherence Tomography"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1270-5852","authenticated-orcid":true,"given":"Monica","family":"Hernandez","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7996-4587","authenticated-orcid":true,"given":"Ubaldo","family":"Ramon-Julvez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elisa","family":"Vilades","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5470-8977","authenticated-orcid":true,"given":"Beatriz","family":"Cordon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elvira","family":"Mayordomo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elena","family":"Garcia-Martin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2023,8,7]]},"reference":[{"issue":"2","key":"pone.0289495.ref001","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/S1474-4422(17)30470-2","article-title":"Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria","volume":"17","author":"AJ Thompson","year":"2018","journal-title":"The Lancet Neurology"},{"issue":"9","key":"pone.0289495.ref002","doi-asserted-by":"crossref","first-page":"921","DOI":"10.1016\/S1474-4422(10)70168-X","article-title":"Optical coherence tomography in multiple sclerosis: a systematic review and meta-analysis","volume":"9","author":"A Petzold","year":"2010","journal-title":"Lancet Neurol"},{"issue":"1","key":"pone.0289495.ref003","first-page":"167","article-title":"Progressive changes in the retinal nerve fiber layer in patients with multiple sclerosis","volume":"20","author":"E Garcia-Martin","year":"2010","journal-title":"FJEur J Ophthalmol"},{"issue":"7","key":"pone.0289495.ref004","doi-asserted-by":"crossref","first-page":"893","DOI":"10.1177\/1352458515604496","article-title":"Optical coherence tomography indicates disease activity prior to clinical onset of central nervous system demyelination","volume":"22","author":"B Knier","year":"2015","journal-title":"Mult Scler"},{"issue":"6","key":"pone.0289495.ref005","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1016\/S1474-4422(16)00068-5","article-title":"Retinal thickness measured with optical coherence tomography and risk of disability worsening in multiple sclerosis: a cohort study","volume":"15","author":"IMSVISUAL consortium","year":"2016","journal-title":"Lancet Neurol"},{"issue":"5","key":"pone.0289495.ref006","doi-asserted-by":"crossref","first-page":"884","DOI":"10.1038\/s41433-017-0010-2","article-title":"Optical coherence tomography in multiple sclerosis","volume":"32","author":"J Britze","year":"2018","journal-title":"Eye (Lond)"},{"issue":"6","key":"pone.0289495.ref007","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1177\/1352458517753722","article-title":"OCT is an alternative to MRI for monitoring MS\u2013YES","volume":"24","author":"ES Sotirchos","year":"2018","journal-title":"Mult Scler"},{"issue":"4","key":"pone.0289495.ref008","doi-asserted-by":"crossref","first-page":"903","DOI":"10.1093\/brain\/awz038","article-title":"Optical coherence tomography: a window to the optic nerve in clinically isolated syndrome","volume":"142","author":"F London","year":"2019","journal-title":"Brain"},{"issue":"1","key":"pone.0289495.ref009","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1186\/s40478-019-0768-5","article-title":"Retinal pathology in experimental optic neuritis is characterized by retrograde degeneration and gliosis","volume":"7","author":"P Manogaran","year":"2019","journal-title":"Acta Neuropathol Commun"},{"issue":"11","key":"pone.0289495.ref010","doi-asserted-by":"crossref","first-page":"2217","DOI":"10.1111\/ene.14416","article-title":"Inner retinal layer thinning in radiologically isolated syndrome predicts conversion to multiple sclerosis","volume":"27","author":"L Aly","year":"2020","journal-title":"Eur J Neurol"},{"issue":"1","key":"pone.0289495.ref011","doi-asserted-by":"crossref","first-page":"93","DOI":"10.3390\/jcm12010093","article-title":"The Usefulness of Optical Coherence Tomography in Disease Progression Monitoring in Younger Patients with Relapsing-Remitting Multiple Sclerosis: A Single-Centre Study","volume":"12","author":"M Orbus","year":"2022","journal-title":"J Clin Med"},{"issue":"7","key":"pone.0289495.ref012","doi-asserted-by":"crossref","first-page":"1323","DOI":"10.1007\/s00415-016-8127-y","article-title":"Timing of retinal neuronal and axonal loss in MS: a longitudinal OCT study","volume":"263","author":"LJ Balk","year":"2016","journal-title":"J Neurol"},{"issue":"11","key":"pone.0289495.ref013","doi-asserted-by":"crossref","DOI":"10.1212\/WNL.0000000000200862","article-title":"Machine Learning Utility for Optical Coherence Tomography in Multiple Sclerosis. Is the Future Now?","volume":"99","author":"AT Toosy","year":"2019","journal-title":"Neurology"},{"issue":"9","key":"pone.0289495.ref014","doi-asserted-by":"crossref","first-page":"1837","DOI":"10.1007\/s00415-017-8531-y","article-title":"Retinal ganglion cell analysis in multiple sclerosis and optic neuritis: a systematic review and meta-analysis","volume":"264","author":"J Britze","year":"2017","journal-title":"J Neurol"},{"issue":"5","key":"pone.0289495.ref015","doi-asserted-by":"crossref","first-page":"662","DOI":"10.1097\/WCO.0000000000000604","article-title":"Optical coherence tomography as a means to characterize visual pathway involvement in multiple sclerosis","volume":"31","author":"CA Wicki","year":"2018","journal-title":"Current Opinion in Neurology"},{"key":"pone.0289495.ref016","doi-asserted-by":"crossref","DOI":"10.3389\/fneur.2017.00675","article-title":"The Temporal Retinal Nerve Fiber Layer Thickness Is the Most Important Optical Coherence Tomography Estimate in Multiple Sclerosis","volume":"8","author":"U Birkeldh","year":"2017","journal-title":"Front Neurol"},{"issue":"12","key":"pone.0289495.ref017","first-page":"1857","article-title":"A new strategy to interpret OCT posterior pole asymmetry analysis for glaucoma\u00ef\u00bf\u00bddiagnosis","volume":"10","author":"Y Zhang","year":"2017","journal-title":"Int J Ophthalmol"},{"issue":"9","key":"pone.0289495.ref018","doi-asserted-by":"crossref","first-page":"e0222347","DOI":"10.1371\/journal.pone.0222347","article-title":"Topographic correlation and asymmetry analysis of ganglion cell layer thinning and the retinal nerve fiber layer with localized visual field defects","volume":"14","author":"A Casado","year":"2019","journal-title":"PLoS One"},{"issue":"20","key":"pone.0289495.ref019","doi-asserted-by":"crossref","first-page":"4693","DOI":"10.3390\/jcm10204693","article-title":"Complex Rearrangement of the Entire Retinal Posterior Pole in Patients with Relapsing Remitting Multiple Sclerosis","volume":"10","author":"A Martucci","year":"2021","journal-title":"J Clin Med"},{"issue":"7","key":"pone.0289495.ref020","doi-asserted-by":"crossref","first-page":"e0288581","DOI":"10.1371\/journal.pone.0288581","article-title":"Evaluation of multiple sclerosis severity using a new OCT tool","volume":"18","author":"E Vilades","year":"2023","journal-title":"PLOS ONE"},{"key":"pone.0289495.ref021","doi-asserted-by":"crossref","first-page":"7856","DOI":"10.3390\/s22207856","article-title":"Multiple Sclerosis Diagnosis Using Machine Learning and Deep Learning: Challenges and Opportunities","volume":"22","author":"N Aslam","year":"2022","journal-title":"Sensors (Basel)"},{"issue":"8","key":"pone.0289495.ref022","doi-asserted-by":"crossref","first-page":"628","DOI":"10.1111\/aos.12156","article-title":"Neural networks to identify multiple sclerosis with optical coherence tomography","volume":"91","author":"E Garcia-Martin","year":"2013","journal-title":"Acta Ophtalmol"},{"issue":"5","key":"pone.0289495.ref023","doi-asserted-by":"crossref","first-page":"e0216410","DOI":"10.1371\/journal.pone.0216410","article-title":"Swept source optical coherence tomography to early detect multiple sclerosis disease. The use of machine learning techniques","volume":"14","author":"AP del Palomar","year":"2019","journal-title":"PLoS ONE"},{"issue":"23","key":"pone.0289495.ref024","doi-asserted-by":"crossref","first-page":"5323","DOI":"10.3390\/s19235323","article-title":"Computer-Aided Diagnosis of Multiple Sclerosis Using a Support Vector Machine and Optical Coherence Tomography Features","volume":"19","author":"C Cavaliere","year":"2019","journal-title":"Sensors (Basel)"},{"key":"pone.0289495.ref025","doi-asserted-by":"crossref","first-page":"104416","DOI":"10.1016\/j.compbiomed.2021.104416","article-title":"Machine learning in diagnosis and disability prediction of multiple sclerosis using optical coherence tomography","volume":"133","author":"A Montolio","year":"2021","journal-title":"Comput Biol Med"},{"key":"pone.0289495.ref026","doi-asserted-by":"crossref","first-page":"104165","DOI":"10.1016\/j.compbiomed.2020.104165","article-title":"Early diagnosis of multiple sclerosis by OCT analysis using Cohen\u2019s d method and a neural network as classifier","volume":"129","author":"E Garcia-Martin","year":"2021","journal-title":"Comput Biol Med"},{"issue":"5","key":"pone.0289495.ref027","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1007\/s10439-022-02930-3","article-title":"Comparison of Machine Learning Methods Using Spectralis OCT for Diagnosis and Disability Progression Prognosis in Multiple Sclerosis","volume":"50","author":"A Montolio","year":"2022","journal-title":"Ann Biomed Eng"},{"key":"pone.0289495.ref028","doi-asserted-by":"crossref","first-page":"167","DOI":"10.3390\/s22010167","article-title":"Early Diagnosis of Multiple Sclerosis Using Swept-Source Optical Coherence Tomography and Convolutional Neural Networks Trained with Data Augmentation","volume":"22","author":"A Lopez-Dorado","year":"2022","journal-title":"Sensors (Basel)"},{"key":"pone.0289495.ref029","unstructured":"Marinescu RV, Oxtoby NP, Young AL, Bron EE, AW AWT, et al. The Alzheimer\u2019s Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge: Results after 1 Year Follow-up. ArXiv. 2020;."},{"key":"pone.0289495.ref030","article-title":"Explainable AI toward understanding the performance of the top three TADPOLE Challenge methods in the forecast of Alzheimer\u2019s disease diagnosis","author":"ADNI","year":"2022","journal-title":"PLoS ONE"},{"issue":"11","key":"pone.0289495.ref031","doi-asserted-by":"crossref","first-page":"e1100","DOI":"10.1212\/WNL.0000000000200883","article-title":"The Role of Optical Coherence Tomography Criteria and Machine Learning in Multiple Sclerosis and Optic Neuritis Diagnosis","volume":"99","author":"RC Kenney","year":"2022","journal-title":"Neurology"},{"key":"pone.0289495.ref032","doi-asserted-by":"crossref","first-page":"2660","DOI":"10.1038\/s41598-021-82098-3","article-title":"A multilayer multimodal detection and prediction model based on explainable artificial intelligence for Alzheimer\u2019s disease","volume":"11","author":"S ElSappagh","year":"2021","journal-title":"Nature Scientific Reports"},{"key":"pone.0289495.ref033","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1007\/978-3-030-57321-8_2","article-title":"The explanation game: explaining machine learning models using shapley values","author":"L Merrick","year":"2020","journal-title":"Machine Learning and Knowledge Extraction"},{"key":"pone.0289495.ref034","unstructured":"Grinsztajn L, Oyallon E, Varoquaux G. Why do tree-based models still outperform deep leraning on tabular data? ArXiv. 2022;."},{"issue":"8","key":"pone.0289495.ref035","doi-asserted-by":"crossref","first-page":"1473","DOI":"10.3390\/diagnostics11081473","article-title":"Artificial Intelligence for Alzheimer\u2019s Disease: Promise or Challenge?","volume":"11","author":"C Fabrizio","year":"2021","journal-title":"Diagnostics (Basel)"},{"key":"pone.0289495.ref036","doi-asserted-by":"crossref","first-page":"52138","DOI":"10.1109\/ACCESS.2018.2870052","article-title":"Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)","volume":"6","author":"A Adadi","year":"2017","journal-title":"IEEE Access"},{"key":"pone.0289495.ref037","unstructured":"Molnar C. A guide for making black box models explainable. https:\/\/christophmgithubio\/interpretable-ml-book. 2021;."},{"key":"pone.0289495.ref038","unstructured":"Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Proc of the 31st Conference on Neural Information Processing Systems. 2017;."},{"issue":"11","key":"pone.0289495.ref039","doi-asserted-by":"crossref","first-page":"6320","DOI":"10.1167\/iovs.15-17450","article-title":"Influence of the Retinal Blood Vessel Topography on the Variability of the Retinal Nerve Fiber Bundle Trajectories in the Human Retina","volume":"56","author":"K Qiu","year":"2015","journal-title":"Invest Ophthalmol Vis Sci"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0289495","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,7]],"date-time":"2023-08-07T13:27:52Z","timestamp":1691414872000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0289495"}},"subtitle":[],"editor":[{"given":"Karim Adly","family":"Raafat","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2023,8,7]]},"references-count":39,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2023,8,7]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0289495","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,7]]}}}