{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T11:27:37Z","timestamp":1775820457530,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2021,6,27]],"date-time":"2021-06-27T00:00:00Z","timestamp":1624752000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Eye tracking techniques based on deep learning are rapidly spreading in a wide variety of application fields. With this study, we want to exploit the potentiality of eye tracking techniques in ocular proton therapy (OPT) applications. We implemented a fully automatic approach based on two-stage convolutional neural networks (CNNs): the first stage roughly identifies the eye position and the second one performs a fine iris and pupil detection. We selected 707 video frames recorded during clinical operations during OPT treatments performed at our institute. 650 frames were used for training and 57 for a blind test. The estimations of iris and pupil were evaluated against the manual labelled contours delineated by a clinical operator. For iris and pupil predictions, Dice coefficient (median = 0.94 and 0.97), Szymkiewicz\u2013Simpson coefficient (median = 0.97 and 0.98), Intersection over Union coefficient (median = 0.88 and 0.94) and Hausdorff distance (median = 11.6 and 5.0 (pixels)) were quantified. Iris and pupil regions were found to be comparable to the manually labelled ground truths. Our proposed framework could provide an automatic approach to quantitatively evaluating pupil and iris misalignments, and it could be used as an additional support tool for clinical activity, without impacting in any way with the consolidated routine.<\/jats:p>","DOI":"10.3390\/s21134400","type":"journal-article","created":{"date-parts":[[2021,6,27]],"date-time":"2021-06-27T23:57:22Z","timestamp":1624838242000},"page":"4400","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Convolutional Neural Networks Cascade for Automatic Pupil and Iris Detection in Ocular Proton Therapy"],"prefix":"10.3390","volume":"21","author":[{"given":"Luca","family":"Antonioli","sequence":"first","affiliation":[{"name":"Bioengineering Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, Italy"}]},{"given":"Andrea","family":"Pella","sequence":"additional","affiliation":[{"name":"Bioengineering Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, Italy"}]},{"given":"Rosalinda","family":"Ricotti","sequence":"additional","affiliation":[{"name":"Bioengineering Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2519-0720","authenticated-orcid":false,"given":"Matteo","family":"Rossi","sequence":"additional","affiliation":[{"name":"Department of Electronics, Information and Bioengineering, Politecnico di Milano University, 20133 Milan, Italy"}]},{"given":"Maria Rosaria","family":"Fiore","sequence":"additional","affiliation":[{"name":"Radiotherapy Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2630-745X","authenticated-orcid":false,"given":"Gabriele","family":"Belotti","sequence":"additional","affiliation":[{"name":"Department of Electronics, Information and Bioengineering, Politecnico di Milano University, 20133 Milan, Italy"}]},{"given":"Giuseppe","family":"Magro","sequence":"additional","affiliation":[{"name":"Medical Physics Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, Italy"}]},{"given":"Chiara","family":"Paganelli","sequence":"additional","affiliation":[{"name":"Department of Electronics, Information and Bioengineering, Politecnico di Milano University, 20133 Milan, Italy"}]},{"given":"Ester","family":"Orlandi","sequence":"additional","affiliation":[{"name":"Radiotherapy Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, Italy"}]},{"given":"Mario","family":"Ciocca","sequence":"additional","affiliation":[{"name":"Medical Physics Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, Italy"}]},{"given":"Guido","family":"Baroni","sequence":"additional","affiliation":[{"name":"Bioengineering Unit, Clinical Department, National Center for Oncological Hadrontherapy (CNAO), 27100 Pavia, Italy"},{"name":"Department of Electronics, Information and Bioengineering, Politecnico di Milano University, 20133 Milan, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Dieckmann, K., Dunavoelgyi, R., Langmann, G., Ma, R., Poetter, R., Schmutzer, M., Wackernagel, W., and Zehetmayer, M. (2014). Stereotactic Radiotherapy of Uveal Melanoma. Clinical Ophthalmic Oncology, Springer.","DOI":"10.1007\/978-3-642-54255-8_15"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.ijrobp.2012.08.026","article-title":"Charged particle radiation therapy for uveal melanoma: A systematic review and meta-analysis","volume":"86","author":"Wang","year":"2013","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1016\/j.clon.2015.08.007","article-title":"Tumor response in uveal melanomas treated with proton beam therapy","volume":"28","author":"Maschi","year":"2015","journal-title":"Clin. Oncol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"061708","DOI":"10.1118\/1.4807090","article-title":"Quantification of dose perturbations induced by external and internal accessories in ocular proton therapy and evaluation of their dosimetric impact","volume":"40","author":"Carnicer","year":"2013","journal-title":"Med. Phys."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1016\/j.ijrobp.2016.01.040","article-title":"Practice Patterns Analysis of Ocular Proton Therapy Centers: The International OPTIC Survey","volume":"95","author":"Hrbacek","year":"2016","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.ejmp.2019.02.020","article-title":"A platform for patient positioning and motion monitoring in ocular proton therapy with a non-dedicated beamline","volume":"59","author":"Via","year":"2019","journal-title":"Phys. Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1852","DOI":"10.1002\/mp.13389","article-title":"Design and commissioning of the non-dedicated scanning proton beamline for ocular treatment at the synchrotron-based CNAO facility","volume":"46","author":"Ciocca","year":"2019","journal-title":"Med. Phys."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4265","DOI":"10.1118\/1.4729708","article-title":"Eye tracking and gating system for proton therapy of orbital tumors","volume":"39","author":"Shin","year":"2012","journal-title":"Med. Phys."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3521","DOI":"10.1118\/1.1824195","article-title":"Automatic real-time surveillance of eye position and gating for stereotactic radiotherapy of uveal melanoma","volume":"31","author":"Petersch","year":"2004","journal-title":"Med. Phys."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1128","DOI":"10.1016\/S0360-3016(03)00280-3","article-title":"A noninvasive eye fixation and computer-aided eye monitoring system for linear accelerator-based stereotactic radiotherapy of uveal melanoma","volume":"56","author":"Bogner","year":"2003","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1120\/jacmp.v4i2.2531","article-title":"Stereotactic radiotherapy in the treatment of ocular melanoma: A noninvasive eye fixation aid and tracking system","volume":"4","author":"Jaywant","year":"2003","journal-title":"J. Appl. Clin. Med. Phys."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Punde, P.A., Jadhav, M.E., and Manza, R.R. (2017, January 5\u20136). A study of eye tracking technology and its applications. Proceedings of the 2017 1st International Conference on Intelligent Systems and Information Management (ICISIM), Aurangabad, India.","DOI":"10.1109\/ICISIM.2017.8122153"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kim, J., Stengel, M., Majercik, A., de Mello, S., Dunn, D., Laine, S., McGuire, M., and Luebke, D. (2019, January 4). NVGaze: An anatomically-informed dataset for low-latency, near-eye gaze estimation. Proceedings of the Conference on Human Factors in Computing Systems\u2014Proceedings, Glasgow, UK.","DOI":"10.1145\/3290605.3300780"},{"key":"ref_14","first-page":"1297","article-title":"The effect of training set size for the performance of neural networks of classification","volume":"9","author":"Sug","year":"2010","journal-title":"W. Trans. Comp."},{"key":"ref_15","unstructured":"Cho, J., Lee, K., Shin, E., Choy, G., and Do, S. (2015). How much data is needed to train a medical image deep learning system to achieve necessary high accuracy. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"108307","DOI":"10.1016\/j.jneumeth.2019.05.016","article-title":"DeepVOG: Open-source pupil segmentation and gaze estimation in neuroscience using deep learning","volume":"324","author":"Yiu","year":"2019","journal-title":"J. Neurosci. Methods"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chaudhary, A.K., Kothari, R., Acharya, M., Dangi, S., Nair, N., Bailey, R., Kanan, C., Diaz, G., and Pelz, J.B. (2019, January 27\u201328). RITnet: Real-time Semantic Segmentation of the Eye for Gaze Tracking. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea.","DOI":"10.1109\/ICCVW.2019.00568"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Lee, Y.W., Kim, K.W., Hoang, T.M., Arsalan, M., and Park, K.R. (2019). Deep Residual CNN-Based Ocular Recognition Based on Rough Pupil Detection in the Images by NIR Camera Sensor. Sensors, 19.","DOI":"10.3390\/s19040842"},{"key":"ref_19","unstructured":"Fuhl, W., Santini, T., Kasneci, G., and Kasneci, E. (2016). PupilNet: Convolutional neural networks for robust pupil detection. arXiv, Available online: http:\/\/arxiv.org\/abs\/1601.04902."},{"key":"ref_20","first-page":"1","article-title":"CNN-based pupil center detection for wearable gaze estimation system","volume":"2017","author":"Chinsatit","year":"2017","journal-title":"Appl. Comput. Intell. Soft Comput."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kondo, N., Chinsatit, W., and Saitoh, T. (2017, January 19\u201322). Pupil center detection for infrared irradiation eye image using CNN. Proceedings of the 2017 56th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), Kanazawa, Japan.","DOI":"10.23919\/SICE.2017.8105630"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"7251280","DOI":"10.1155\/2020\/7251280","article-title":"Driver Fatigue Detection Based on Convolutional Neural Networks Using EM-CNN","volume":"2020","author":"Zhao","year":"2020","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Arsalan, M., Naqvi, R.A., Kim, D.S., Nguyen, P.H., Owais, M., and Park, K.R. (2018). IrisDenseNet: Robust Iris Segmentation Using Densely Connected Fully Convolutional Networks in the Images by Visible Light and Near-Infrared Light Camera Sensors. Sensors, 18.","DOI":"10.3390\/s18051501"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"64739","DOI":"10.1109\/ACCESS.2020.2985095","article-title":"Noise-Robust Pupil Center Detection Through CNN-Based Segmentation with Shape-Prior Loss","volume":"8","author":"Han","year":"2020","journal-title":"IEEE Access"},{"key":"ref_25","unstructured":"Kayalibay, B., Jensen, G., and Smagt, P. (2021, March 21). CNN-Based Segmentation of Medical Imaging Data. Available online: https:\/\/arxiv.org\/abs\/1701.03056."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ricotti, R., Pella, A., Elisei, G., Tagaste, B., Bello, F., Fontana, G., Fiore, M.R., Ciocca, M., Mastella, E., and Orlandi, E. (2021). Gaze Stability During Ocular Proton Therapy: Quantitative Evaluation Based on Eye Surface Surveillance Videos. Pattern Recognition. ICPR International Workshops and Challenges, Springer.","DOI":"10.1007\/978-3-030-68796-0_32"},{"key":"ref_28","unstructured":"Chollet, F. (2021, May 01). Kera. Available online: https:\/\/github.com\/fchollet\/keras."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Aziz, T.A., and Hanbury, A. (2015). Metrics for evaluating 3D medical image segmentation: Analysis, selection, and tool. BMC Med. Imaging, 15.","DOI":"10.1186\/s12880-015-0068-x"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wang, C., MacGillivray, T., Macnaught, G., Yang, G., and Newby, D.E. (2018). A two-stage 3D U-net framework for multi-class segmentation on full resolution image. arXiv, Available online: https:\/\/arxiv.org\/abs\/1804.04341.","DOI":"10.1007\/978-3-030-12029-0_21"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Galea, R.-R., Diosan, L., Andreica, A., Popa, L., Manole, S., and B\u00e1lint, Z. (2021). Region-of-Interest-Based Cardiac Image Segmentation with Deep Learning. Appl. Sci., 11.","DOI":"10.3390\/app11041965"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/13\/4400\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:25:15Z","timestamp":1760163915000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/13\/4400"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,27]]},"references-count":31,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2021,7]]}},"alternative-id":["s21134400"],"URL":"https:\/\/doi.org\/10.3390\/s21134400","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,27]]}}}