{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T07:47:10Z","timestamp":1776757630716,"version":"3.51.2"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","license":[{"start":{"date-parts":[[2022,3,31]],"date-time":"2022-03-31T00:00:00Z","timestamp":1648684800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,31]],"date-time":"2022-03-31T00:00:00Z","timestamp":1648684800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003484","name":"Heinrich-Heine-Universit\u00e4t D\u00fcsseldorf","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003484","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Graefes Arch Clin Exp Ophthalmol"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n              <jats:title>Background and purpose<\/jats:title>\n              <jats:p>Corneal fluorescein staining is one of the most important diagnostic tests in dry eye disease (DED). Nevertheless, the result of this examination is depending on the grader. So far, there is no method for an automated quantification of corneal staining commercially available. Aim of this study was to develop a software-assisted grading algorithm and to compare it with a group of human graders with variable clinical experience in patients with DED.<\/jats:p>\n            <\/jats:sec><jats:sec>\n              <jats:title>Methods<\/jats:title>\n              <jats:p>Fifty images of eyes stained with 2\u00a0\u00b5l of 2% fluorescein presenting different severity of superficial punctate keratopathy in patients with DED were taken under standardized conditions. An algorithm for detecting and counting superficial punctate keratitis was developed using ImageJ with a training dataset of 20 randomly picked images. Then, the test\u00a0dataset of 30 images was analyzed (1) by the ImageJ algorithm and (2) by 22 graders, all ophthalmologists with different levels of experience. All graders evaluated the images using the Oxford grading scheme for corneal staining at baseline and after 6\u20138\u00a0weeks. Intrarater agreement was also evaluated by adding a mirrored version of all original images into the set of images during the 2nd grading.<\/jats:p>\n            <\/jats:sec><jats:sec>\n              <jats:title>Results<\/jats:title>\n              <jats:p>The count of particles detected by the algorithm correlated significantly (<jats:italic>n<\/jats:italic>\u2009=\u200930; <jats:italic>p<\/jats:italic>\u2009&lt;\u20090.01) with the estimated true Oxford grade (Sr\u2009=\u20090,91). Overall human graders showed only moderate intrarater agreement (<jats:italic>K<\/jats:italic>\u2009=\u20090,426), while software-assisted grading was always the same (<jats:italic>K<\/jats:italic>\u2009=\u20091,0). Little difference was found between specialists and non-specialists in terms of intrarater agreement (<jats:italic>K<\/jats:italic>\u2009=\u20090,436 specialists; <jats:italic>K<\/jats:italic>\u2009=\u20090,417 non-specialists). The highest interrater agreement was seen with 75,6% in the most experienced grader, a cornea specialist with 29\u00a0years of experience, and the lowest was seen in a resident with 25,6% who had only 2\u00a0years of experience.<\/jats:p>\n            <\/jats:sec><jats:sec>\n              <jats:title>Conclusion<\/jats:title>\n              <jats:p>The variance in human grading of corneal staining\u00a0-\u00a0if only small\u00a0-\u00a0is likely to have only little impact on clinical management and thus seems to be acceptable. While human graders give results sufficient for clinical application, software-assisted grading of corneal staining ensures higher consistency and thus is preferrable for re-evaluating patients, e.g., in clinical trials.<\/jats:p>\n            <\/jats:sec>","DOI":"10.1007\/s00417-022-05574-0","type":"journal-article","created":{"date-parts":[[2022,3,31]],"date-time":"2022-03-31T08:04:41Z","timestamp":1648713881000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Automated vs. human evaluation of corneal staining"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5997-9215","authenticated-orcid":false,"given":"R.","family":"Kourukmas","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"M.","family":"Roth","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"G.","family":"Geerling","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,3,31]]},"reference":[{"key":"5574_CR1","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1111\/aos.12012","volume":"92","author":"BD Sullivan","year":"2014","unstructured":"Sullivan BD, Crews LA, Messmer EM et al (2014) Correlations between commonly used objective signs and symptoms for the diagnosis of dry eye disease: clinical implications. Acta Ophthalmol 92:161\u2013166. https:\/\/doi.org\/10.1111\/aos.12012","journal-title":"Acta Ophthalmol"},{"key":"5574_CR2","doi-asserted-by":"publisher","unstructured":"Pellegrini M, Bernabei F, Moscardelli F, et al (2019) Assessment of corneal fluorescein staining in different dry eye subtypes using digital image analysis. Transl Vis Sci Technol 8: https:\/\/doi.org\/10.1167\/tvst.8.6.34","DOI":"10.1167\/tvst.8.6.34"},{"key":"5574_CR3","doi-asserted-by":"crossref","unstructured":"Aumann S, Donner S, Fischer J, M\u00fcller F (2019) Optical coherence tomography (OCT): principle and technical realization. In: Bille JF (ed). Cham (CH), pp 59\u201385","DOI":"10.1007\/978-3-030-16638-0_3"},{"key":"5574_CR4","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1111\/ceo.13136","volume":"46","author":"R Fan","year":"2018","unstructured":"Fan R, Chan TC, Prakash G, Jhanji V (2018) Applications of corneal topography and tomography: a review. Clin Experiment Ophthalmol 46:133\u2013146. https:\/\/doi.org\/10.1111\/ceo.13136","journal-title":"Clin Experiment Ophthalmol"},{"issue":"129\u201333","key":"5574_CR5","doi-asserted-by":"publisher","first-page":"v","DOI":"10.1016\/j.ohc.2004.02.007","volume":"17","author":"F Carones","year":"2004","unstructured":"Carones F (2004) Diagnostic use of ocular wavefront sensing. Ophthalmol Clin North Am 17(129\u201333):v. https:\/\/doi.org\/10.1016\/j.ohc.2004.02.007","journal-title":"Ophthalmol Clin North Am"},{"key":"5574_CR6","doi-asserted-by":"publisher","first-page":"1264","DOI":"10.1016\/j.ophtha.2018.01.034","volume":"125","author":"J Krause","year":"2018","unstructured":"Krause J, Gulshan V, Rahimy E et al (2018) Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy. Ophthalmology 125:1264\u20131272. https:\/\/doi.org\/10.1016\/j.ophtha.2018.01.034","journal-title":"Ophthalmology"},{"key":"5574_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1259\/bjr.20130245","volume":"86","author":"B Mucci","year":"2013","unstructured":"Mucci B, Murray H, Downie A, Osborne K (2013) Interrater variation in scoring radiological discrepancies. Br J Radiol 86:1\u20135. https:\/\/doi.org\/10.1259\/bjr.20130245","journal-title":"Br J Radiol"},{"key":"5574_CR8","unstructured":"Liu Y, Gadepalli K, Norouzi M, et al (2017) Detecting Cancer metastases on gigapixel pathology images. 1\u201313"},{"key":"5574_CR9","doi-asserted-by":"publisher","first-page":"1948","DOI":"10.1109\/TMI.2019.2891305","volume":"38","author":"H Lin","year":"2019","unstructured":"Lin H, Chen H, Graham S et al (2019) Fast ScanNet: fast and dense analysis of multi-gigapixel whole-slide images for cancer metastasis detection. IEEE Trans Med Imaging 38:1948\u20131958. https:\/\/doi.org\/10.1109\/TMI.2019.2891305","journal-title":"IEEE Trans Med Imaging"},{"key":"5574_CR10","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1002\/acr2.1033","volume":"1","author":"A Rasmussen","year":"2019","unstructured":"Rasmussen A, Stone DU, Kaufman CE et al (2019) Reproducibility of ocular surface staining in the assessment of Sj\u00f6gren syndrome\u2013related keratoconjunctivitis sicca: implications on disease classification. ACR Open Rheumatol 1:292\u2013302. https:\/\/doi.org\/10.1002\/acr2.1033","journal-title":"ACR Open Rheumatol"},{"key":"5574_CR11","doi-asserted-by":"publisher","first-page":"707","DOI":"10.1089\/jop.2017.0021","volume":"33","author":"JS Eaton","year":"2017","unstructured":"Eaton JS, Miller PE, Bentley E et al (2017) Slit lamp-based ocular scoring systems in toxicology and drug development: a literature survey. J Ocul Pharmacol Ther 33:707\u2013717. https:\/\/doi.org\/10.1089\/jop.2017.0021","journal-title":"J Ocul Pharmacol Ther"},{"key":"5574_CR12","first-page":"422","volume":"32","author":"IL Bailey","year":"1991","unstructured":"Bailey IL, Bullimore MA, Raasch TW, Taylor HR (1991) Clinical grading and the effects of scaling. Invest Ophthalmol Vis Sci 32:422\u2013432","journal-title":"Invest Ophthalmol Vis Sci"},{"key":"5574_CR13","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1076\/0928-6586(200003)711-2FT049","volume":"7","author":"NA Sparrow","year":"2000","unstructured":"Sparrow NA, Frost NA, Pantelides EP, Laidlaw DA (2000) Decimalization of the oxford clinical cataract classification and grading system. Ophthalmic Epidemiol 7:49\u201360","journal-title":"Ophthalmic Epidemiol"},{"key":"5574_CR14","doi-asserted-by":"publisher","first-page":"1097","DOI":"10.1016\/j.ajo.2014.02.012","volume":"157","author":"Y Sook Chun","year":"2014","unstructured":"Sook Chun Y, Park IK (2014) Reliability of 4 clinical grading systems for corneal staining. Am J Ophthalmol 157:1097\u20131102. https:\/\/doi.org\/10.1016\/j.ajo.2014.02.012","journal-title":"Am J Ophthalmol"},{"key":"5574_CR15","doi-asserted-by":"publisher","first-page":"2369","DOI":"10.2147\/OPTH.S178113","volume":"12","author":"J Woods","year":"2018","unstructured":"Woods J, Varikooty J, Fonn D, Jones LW (2018) A novel scale for describing corneal staining. Clin Ophthalmol 12:2369\u20132375. https:\/\/doi.org\/10.2147\/OPTH.S178113","journal-title":"Clin Ophthalmol"},{"key":"5574_CR16","first-page":"340","volume":"43","author":"P Fieguth","year":"2002","unstructured":"Fieguth P, Simpson T (2002) Automated measurement of bulbar redness. Invest Ophthalmol Vis Sci 43:340\u2013347","journal-title":"Invest Ophthalmol Vis Sci"},{"key":"5574_CR17","doi-asserted-by":"publisher","unstructured":"Schindelin J, Arganda-Carrera I, Frise E, et al (2009) Fiji - an open platform for biological image analysis. Nat Methods 9: https:\/\/doi.org\/10.1038\/nmeth.2019.Fiji","DOI":"10.1038\/nmeth.2019.Fiji"},{"key":"5574_CR18","doi-asserted-by":"publisher","first-page":"741","DOI":"10.1177\/25.7.70454","volume":"25","author":"GW Zack","year":"1977","unstructured":"Zack GW, Rogers WE, Latt SA (1977) Automatic measurement of sister chromatid exchange frequency. J Histochem Cytochem Off J Histochem Soc 25:741\u2013753. https:\/\/doi.org\/10.1177\/25.7.70454","journal-title":"J Histochem Cytochem Off J Histochem Soc"},{"key":"5574_CR19","doi-asserted-by":"publisher","first-page":"640","DOI":"10.1097\/00003226-200310000-00008","volume":"22","author":"AJ Bron","year":"2003","unstructured":"Bron AJ, Evans VE, Smith JA (2003) Grading of corneal and conjunctival staining in the context of other dry eye tests. Cornea 22:640\u2013650. https:\/\/doi.org\/10.1097\/00003226-200310000-00008","journal-title":"Cornea"},{"key":"5574_CR20","doi-asserted-by":"publisher","first-page":"1401","DOI":"10.1038\/eye.2011.172","volume":"25","author":"S Andersson","year":"2011","unstructured":"Andersson S, Heijl A, Bengtsson B (2011) Optic disc classification by the Heidelberg Retina Tomograph and by physicians with varying experience of glaucoma. Eye 25:1401\u20131407. https:\/\/doi.org\/10.1038\/eye.2011.172","journal-title":"Eye"},{"key":"5574_CR21","doi-asserted-by":"publisher","first-page":"4548","DOI":"10.1167\/iovs.13-11804","volume":"54","author":"RP Danis","year":"2013","unstructured":"Danis RP, Domalpally A, Chew EY et al (2013) Methods and reproducibility of grading optimized digital color fundus photographs in the age-related eye disease study 2 (AREDS2 Report Number 2). Investig Ophthalmol Vis Sci 54:4548\u20134554. https:\/\/doi.org\/10.1167\/iovs.13-11804","journal-title":"Investig Ophthalmol Vis Sci"},{"key":"5574_CR22","doi-asserted-by":"publisher","first-page":"675","DOI":"10.1001\/jamaophthalmol.2015.0460","volume":"133","author":"E Daniel","year":"2015","unstructured":"Daniel E, Quinn GE, Hildebrand PL et al (2015) Validated system for centralized grading of retinopathy of prematurity: telemedicine approaches to evaluating acute-phase retinopathy of prematurity (e-ROP) study. JAMA Ophthalmol 133:675\u2013682. https:\/\/doi.org\/10.1001\/jamaophthalmol.2015.0460","journal-title":"JAMA Ophthalmol"},{"key":"5574_CR23","doi-asserted-by":"publisher","first-page":"272","DOI":"10.1097\/00003226-200404000-00010","volume":"23","author":"KK Nichols","year":"2004","unstructured":"Nichols KK, Mitchell GL, Zadnik K (2004) The repeatability of clinical measurements of dry eye. Cornea 23:272\u2013285. https:\/\/doi.org\/10.1097\/00003226-200404000-00010","journal-title":"Cornea"},{"key":"5574_CR24","doi-asserted-by":"publisher","first-page":"2340","DOI":"10.1167\/iovs.14-15318","volume":"56","author":"JD Rodriguez","year":"2015","unstructured":"Rodriguez JD, Lane KJ, Ousler GW et al (2015) Automated grading system for evaluation of superficial punctate keratitis associated with dry eye. Investig Ophthalmol Vis Sci 56:2340\u20132347. https:\/\/doi.org\/10.1167\/iovs.14-15318","journal-title":"Investig Ophthalmol Vis Sci"},{"key":"5574_CR25","doi-asserted-by":"publisher","unstructured":"Amparo F, Wang H, Yin J, et al (2017) Evaluating corneal fluorescein staining using a novel automated method. Invest Ophthalmol Vis Sci 58:BIO168\u2013BIO173. https:\/\/doi.org\/10.1167\/iovs.17-21831","DOI":"10.1167\/iovs.17-21831"},{"key":"5574_CR26","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1097\/OPX.0000000000000496","volume":"92","author":"L Sorbara","year":"2015","unstructured":"Sorbara L, Peterson R, Schneider S, Woods C (2015) Comparison between live and photographed slit lamp grading of corneal staining. Optom Vis Sci Off Publ Am Acad Optom 92:312\u2013317. https:\/\/doi.org\/10.1097\/OPX.0000000000000496","journal-title":"Optom Vis Sci Off Publ Am Acad Optom"},{"key":"5574_CR27","doi-asserted-by":"publisher","first-page":"7896","DOI":"10.1167\/iovs.14-15618","volume":"55","author":"YS Chun","year":"2014","unstructured":"Chun YS, Yoon WB, Gi Kim K, Ki Park I (2014) Objective assessment of corneal staining using digital image analysis. Investig Ophthalmol Vis Sci 55:7896\u20137903. https:\/\/doi.org\/10.1167\/iovs.14-15618","journal-title":"Investig Ophthalmol Vis Sci"},{"key":"5574_CR28","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1167\/tvst.6.4.13","volume":"6","author":"F Amparo","year":"2017","unstructured":"Amparo F, Yin J, Di Zazzo A et al (2017) Evaluating changes in ocular redness using a novel automated method. Transl Vis Sci Technol 6:13. https:\/\/doi.org\/10.1167\/tvst.6.4.13","journal-title":"Transl Vis Sci Technol"},{"key":"5574_CR29","doi-asserted-by":"publisher","first-page":"1078","DOI":"10.1001\/jamaophthalmol.2017.3346","volume":"135","author":"VY Bunya","year":"2017","unstructured":"Bunya VY, Chen M, Zheng Y et al (2017) Development and evaluation of semiautomated quantification of lissamine green staining of the bulbar conjunctiva from digital images. JAMA Ophthalmol 135:1078\u20131085. https:\/\/doi.org\/10.1001\/jamaophthalmol.2017.3346","journal-title":"JAMA Ophthalmol"},{"key":"5574_CR30","doi-asserted-by":"publisher","first-page":"1464","DOI":"10.1136\/bjo.2006.112680","volume":"91","author":"RC Peterson","year":"2007","unstructured":"Peterson RC, Wolffsohn JS (2007) Sensitivity and reliability of objective image analysis compared to subjective grading of bulbar hyperaemia. Br J Ophthalmol 91:1464\u20131466. https:\/\/doi.org\/10.1136\/bjo.2006.112680","journal-title":"Br J Ophthalmol"},{"key":"5574_CR31","doi-asserted-by":"publisher","first-page":"287","DOI":"10.5301\/ejo.5000779","volume":"26","author":"A Leonardi","year":"2016","unstructured":"Leonardi A, Van Setten G, Amrane M et al (2016) Efficacy and safety of 0.1% cyclosporine A cationic emulsion in the treatment of severe dry eye disease: a multicenter randomized trial. Eur J Ophthalmol 26:287\u2013296. https:\/\/doi.org\/10.5301\/ejo.5000779","journal-title":"Eur J Ophthalmol"},{"key":"5574_CR32","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/j.preteyeres.2018.04.003","volume":"66","author":"HS Dua","year":"2018","unstructured":"Dua HS, Said DG, Messmer EM et al (2018) Neurotrophic keratopathy. Prog Retin Eye Res 66:107\u2013131. https:\/\/doi.org\/10.1016\/j.preteyeres.2018.04.003","journal-title":"Prog Retin Eye Res"}],"container-title":["Graefe's Archive for Clinical and Experimental Ophthalmology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00417-022-05574-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00417-022-05574-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00417-022-05574-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,3,31]],"date-time":"2022-03-31T08:08:35Z","timestamp":1648714115000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00417-022-05574-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,31]]},"references-count":32,"alternative-id":["5574"],"URL":"https:\/\/doi.org\/10.1007\/s00417-022-05574-0","relation":{},"ISSN":["0721-832X","1435-702X"],"issn-type":[{"value":"0721-832X","type":"print"},{"value":"1435-702X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,31]]},"assertion":[{"value":"7 August 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 December 2021","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 January 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 March 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest\/Competing interests."}}]}}