{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T06:13:33Z","timestamp":1773814413741,"version":"3.50.1"},"reference-count":34,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T00:00:00Z","timestamp":1573171200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2019,12,31]]},"abstract":"<jats:p>We present a novel, biomimetic model of the eye for realistic virtual human animation. We also introduce a deep learning approach to oculomotor control that is compatible with our biomechanical eye model. Our eye model consists of the following functional components: (i) submodels of the 6 extraocular muscles that actuate realistic eye movements, (ii) an iris submodel, actuated by pupillary muscles, that accommodates to incoming light intensity, (iii) a corneal submodel and a deformable, ciliary-muscle-actuated lens submodel, which refract incoming light rays for focal accommodation, and (iv) a retina with a multitude of photoreceptors arranged in a biomimetic, foveated distribution. The light intensity captured by the photoreceptors is computed using ray tracing from the photoreceptor positions through the finite aperture pupil into the 3D virtual environment, and the visual information from the retina is output via an optic nerve vector. Our oculomotor control system includes a foveation controller implemented as a locally-connected, irregular Deep Neural Network (DNN), or \"LiNet\", that conforms to the nonuniform retinal photoreceptor distribution, and a neuromuscular motor controller implemented as a fully-connected DNN, plus auxiliary Shallow Neural Networks (SNNs) that control the accommodation of the pupil and lens. The DNNs are trained offline through deep learning from data synthesized by the eye model itself. Once trained, the oculomotor control system operates robustly and efficiently online. It innervates the intraocular muscles to perform illumination and focal accommodation and the extraocular muscles to produce natural eye movements in order to foveate and pursue moving visual targets. We additionally demonstrate the operation of our eye model (binocularly) within our recently introduced sensorimotor control framework involving an anatomically-accurate biomechanical human musculoskeletal model.<\/jats:p>","DOI":"10.1145\/3355089.3356558","type":"journal-article","created":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T20:27:58Z","timestamp":1573244878000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Biomimetic eye modeling &amp; deep neuromuscular oculomotor control"],"prefix":"10.1145","volume":"38","author":[{"given":"Masaki","family":"Nakada","sequence":"first","affiliation":[{"name":"University of California"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arjun","family":"Lakshmipathy","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Honglin","family":"Chen","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nina","family":"Ling","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tao","family":"Zhou","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Demetri","family":"Terzopoulos","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,11,8]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/0025-5564(75)90075-9"},{"key":"e_1_2_2_2_1","first-page":"13","article-title":"The neurobiology of saccadic eye movements","volume":"3","author":"Becker W.","year":"1989","unstructured":"W. Becker. 1989. The neurobiology of saccadic eye movements: Metrics. Reviews of Oculomotor Research 3 (1989), 13.","journal-title":"Metrics. Reviews of Oculomotor Research"},{"key":"e_1_2_2_3_1","doi-asserted-by":"crossref","unstructured":"I. Bekerman P. Gottlieb and M. Vaiman. 2014. Variations in eyeball diameters of the healthy adults. Journal of Ophthalmology 2014 Article 503645 (2014) 5 pages.","DOI":"10.1155\/2014\/503645"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1167\/8.16.14"},{"key":"e_1_2_2_6_1","volume-title":"Visual prosthetics: Physiology, bioengineering, rehabilitation","author":"Dagnelie G.","unstructured":"G. Dagnelie. 2011. Visual prosthetics: Physiology, bioengineering, rehabilitation. Springer."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073243"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1117\/1.2357734"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0002-9394(14)72611-X"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/0042-6989(94)00257-M"},{"key":"e_1_2_2_12_1","volume-title":"Proc. IEEE International Conference on Computer Vision","author":"He K.","unstructured":"K. He, X. Zhang, S. Ren, and J. Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proc. IEEE International Conference on Computer Vision. Santiago, Chile, 1026--1034."},{"key":"e_1_2_2_13_1","volume-title":"Optics","author":"Hecht E.","unstructured":"E. Hecht. 1987. Optics. Addison Wesley."},{"key":"e_1_2_2_14_1","first-page":"27","article-title":"2D linear oculomotor plant mathematical model: Verification and biometric applications","volume":"10","author":"Komogortsev O.","year":"2013","unstructured":"O. Komogortsev, C. Holland, S. Jayarathna, and A. Karpov. 2013. 2D linear oculomotor plant mathematical model: Verification and biometric applications. ACM Transactions on Applied Perception (TAP) 10, 4 (2013), 27.","journal-title":"ACM Transactions on Applied Perception (TAP)"},{"key":"e_1_2_2_15_1","volume-title":"Proceedings ACM SIGGRAPH. ACM","author":"Lee S.P.","unstructured":"S.P. Lee, J.B. Badler, and N.I. Badler. 2002. Eyes alive. In Proceedings ACM SIGGRAPH. ACM, New York, NY, USA, 637--644."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1559755.1559756"},{"key":"e_1_2_2_17_1","volume-title":"Proc. ACM SIGGRAPH 2006","author":"Lee S.-H.","year":"2006","unstructured":"S.-H. Lee and D. Terzopoulos. 2006. Heads up! Biomechanical modeling and neuromuscular control of the neck. ACM Transactions on Graphics 25, 3 (2006), 1188--1198. Proc. ACM SIGGRAPH 2006, Boston, MA, August 2006."},{"key":"e_1_2_2_18_1","doi-asserted-by":"crossref","unstructured":"R.J. Leigh and D.S. Zee. 2015. The Neurology of Eye Movements. Oxford Univ Press.","DOI":"10.1093\/med\/9780199969289.001.0001"},{"key":"e_1_2_2_19_1","volume-title":"Proc. Indian Conf. on Computer Vision, Graphics and Image Processing. 81","author":"Lesmana M.","unstructured":"M. Lesmana, A. Landgren, P.-E. Forss\u00e9n, and D.K. Pai. 2014. Active gaze stabilization. In Proc. Indian Conf. on Computer Vision, Graphics and Image Processing. 81."},{"key":"e_1_2_2_20_1","volume-title":"2011 IEEE International Conference on Robotics and Automation. 3670--3675","author":"Lesmana M.","unstructured":"M. Lesmana and D.K. Pai. 2011. A biologically inspired controller for fast eye movements. In 2011 IEEE International Conference on Robotics and Automation. 3670--3675."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1177\/0956797610388044"},{"key":"e_1_2_2_22_1","first-page":"1","article-title":"Deep learning of biomimetic visual perception for virtual humans. In ACM Symposium on Applied Perception (SAP 18). Vancouver","volume":"20","author":"Nakada M.","year":"2018","unstructured":"M. Nakada, H. Chen, and D. Terzopoulos. 2018a. Deep learning of biomimetic visual perception for virtual humans. In ACM Symposium on Applied Perception (SAP 18). Vancouver, Canada, 20:1--8.","journal-title":"Canada"},{"key":"e_1_2_2_23_1","volume-title":"Proc. ACM SIGGRAPH 2018","author":"Nakada M.","year":"2018","unstructured":"M. Nakada, T. Zhou, H. Chen, T. Weiss, and D. Terzopoulos. 2018b. Deep learning of biomimetic sensorimotor control for biomechanical human animation. ACM Transactions on Graphics 37, 4, Article 56 (August 2018), 15 pages. Proc. ACM SIGGRAPH 2018, Vancouver, Canada, August 2018."},{"key":"e_1_2_2_24_1","unstructured":"P. Riordan-Eva and E.T. Cunningham. 2011. Vaughan & Asbury's General Ophthalmology. McGraw Hill Professional."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.1964.sp007485"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1097\/00006324-199705000-00027"},{"key":"e_1_2_2_27_1","unstructured":"K. Ruhland S. Andrist J. Badler C. Peters N. Badler M. Gleicher B. Mutlu and R. Mcdonnell. 2014. Look me in the eyes: A survey of eye and gaze animation for virtual agents and artificial systems. In Eurographics State-of-the-Art Reports. 69--91."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00221-007-1176-9"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01885636"},{"key":"e_1_2_2_30_1","volume-title":"Proceedings of the Royal Society of London. Series B, Containing Papers of a Biological Character 112","author":"Stiles W.S.","year":"1933","unstructured":"W.S. Stiles, B.H. Crawford, and J.H. Parsons. 1933. The luminous efficiency of rays entering the eye pupil at different points. Proceedings of the Royal Society of London. Series B, Containing Papers of a Biological Character 112 (1933), 428--450."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.1969.sp008684"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/0042-6989(90)90130-D"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1111\/ejn.12641"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2014.02.003"},{"key":"e_1_2_2_35_1","volume-title":"Biomechanical Simulation. Lecture Notes in Computer Science","volume":"5985","author":"Wei Q.","unstructured":"Q. Wei, S. Sueda, and D.K. Pai. 2010. Biomechanical simulation of human eye movement. In Biomechanical Simulation. Lecture Notes in Computer Science, Vol. 5985. Springer, Berlin, 108--118."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185538"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3355089.3356558","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3355089.3356558","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:44:41Z","timestamp":1750203881000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3355089.3356558"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,8]]},"references-count":34,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2019,12,31]]}},"alternative-id":["10.1145\/3355089.3356558"],"URL":"https:\/\/doi.org\/10.1145\/3355089.3356558","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,8]]},"assertion":[{"value":"2019-11-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}