{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T22:18:25Z","timestamp":1776464305635,"version":"3.51.2"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"6","funder":[{"DOI":"10.13039\/501100004052","name":"King Abdullah University of Science and Technology","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004052","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2025,12]]},"abstract":"<jats:p>The desire for cameras with smaller form factors has recently led to a push for exploring computational imaging systems with reduced optical complexity such as a smaller number of lens elements. Unfortunately such simplified optical systems usually suffer from severe aberrations, especially in off-axis regions, which can be difficult to correct purely in software.<\/jats:p>\n                  <jats:p>In this paper we introduce Fovea Stacking, a new type of imaging system that utilizes an emerging dynamic optical component called the deformable phase plate (DPP) for localized aberration correction anywhere on the image sensor. By optimizing DPP deformations through a differentiable optical model, off-axis aberrations are corrected locally, producing a foveated image with enhanced sharpness at the fixation point - analogous to the eye's fovea. Stacking multiple such foveated images, each with a different fixation point, yields a composite image free from aberrations. To efficiently cover the entire field of view, we propose joint optimization of DPP deformations under imaging budget constraints. Due to the DPP device's non-linear behavior, we introduce a neural network-based control model for improved agreement between simulation and hardware performance.<\/jats:p>\n                  <jats:p>We further demonstrated that for extended depth-of-field imaging, Fovea Stacking outperforms traditional focus stacking in image quality. By integrating object detection or eye-tracking, the system can dynamically adjust the lens to track the object of interest-enabling real-time foveated video suitable for downstream applications such as surveillance or foveated virtual reality displays.<\/jats:p>","DOI":"10.1145\/3763278","type":"journal-article","created":{"date-parts":[[2025,12,4]],"date-time":"2025-12-04T17:15:39Z","timestamp":1764868539000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Fovea Stacking: Imaging with Dynamic Localized Aberration Correction"],"prefix":"10.1145","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9275-4632","authenticated-orcid":false,"given":"Shi","family":"Mao","sequence":"first","affiliation":[{"name":"King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2063-2200","authenticated-orcid":false,"given":"Yogeshwar Nath","family":"Mishra","sequence":"additional","affiliation":[{"name":"King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4227-8508","authenticated-orcid":false,"given":"Wolfgang","family":"Heidrich","sequence":"additional","affiliation":[{"name":"King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,12,4]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Optofluidic adaptive optics. Applied optics 57, 22","author":"Banerjee Kaustubh","year":"2018","unstructured":"Kaustubh Banerjee, Pouya Rajaeipour, \u00c7a\u011flar Ataman, and Hans Zappe. 2018. Optofluidic adaptive optics. Applied optics 57, 22 (2018), 6338\u20136344."},{"key":"e_1_2_2_2_1","volume-title":"MEMS deformable mirrors. Nature photonics 5, 1","author":"Bifano Thomas","year":"2011","unstructured":"Thomas Bifano. 2011. MEMS deformable mirrors. Nature photonics 5, 1 (2011), 21\u201323."},{"key":"e_1_2_2_3_1","volume-title":"Xiaomi to launch the world's first liquid lens smartphone camera. https:\/\/newatlas.com\/mobile-technology\/xiaomi-mi-mix-liquid-lens\/ Accessed","author":"Blain Loz","year":"2025","unstructured":"Loz Blain. 2021. Xiaomi to launch the world's first liquid lens smartphone camera. https:\/\/newatlas.com\/mobile-technology\/xiaomi-mi-mix-liquid-lens\/ Accessed: May 19, 2025."},{"key":"e_1_2_2_4_1","volume-title":"Visual object tracking using adaptive correlation filters. In 2010 IEEE computer society conference on computer vision and pattern recognition","author":"Bolme David S","unstructured":"David S Bolme, J Ross Beveridge, Bruce A Draper, and Yui Man Lui. 2010. Visual object tracking using adaptive correlation filters. In 2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, 2544\u20132550."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICECS.1996.584453"},{"key":"e_1_2_2_6_1","volume-title":"Multi-aperture foveated imaging. Optics letters 41, 8","author":"Carles Guillem","year":"2016","unstructured":"Guillem Carles, Shouqian Chen, Nicholas Bustin, James Downing, Duncan McCall, Andrew Wood, and Andrew R Harvey. 2016. Multi-aperture foveated imaging. Optics letters 41, 8 (2016), 1869\u20131872."},{"key":"e_1_2_2_7_1","volume-title":"Current Developments in Lens Design and Optical Engineering VI","author":"Curatu George","unstructured":"George Curatu, David V Wick, Don M Payne, Ty Martinez, Jamie Harriman, and James E Harvey. 2005. Wide field-of-view imaging system using a liquid crystal spatial light modulator. In Current Developments in Lens Design and Optical Engineering VI, Vol. 5874. SPIE, 80\u201386."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3581783.3612261"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1038\/s43586-021-00066-7"},{"key":"e_1_2_2_10_1","volume-title":"Liquid Crystal Materials, Devices, and Applications XI","author":"Harriman Jamie","unstructured":"Jamie Harriman, Sebastian Gauza, Shin-Tson Wu, David Wick, Brett Bagwell, Ty Martinez, Don Payne, and Steven Serati. 2006. Transmissive spatial light modulators with high figure-of-merit liquid crystals for foveated imaging applications. In Liquid Crystal Materials, Devices, and Applications XI, Vol. 6135. SPIE, 80\u201392."},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1364\/OE.442662"},{"key":"e_1_2_2_12_1","volume-title":"Evaluation of focus measures in multi-focus image fusion. Pattern recognition letters 28, 4","author":"Huang Wei","year":"2007","unstructured":"Wei Huang and Zhongliang Jing. 2007. Evaluation of focus measures in multi-focus image fusion. Pattern recognition letters 28, 4 (2007), 493\u2013500."},{"key":"e_1_2_2_13_1","volume-title":"Flexible depth of field photography","author":"Kuthirummal Sujit","year":"2010","unstructured":"Sujit Kuthirummal, Hajime Nagahara, Changyin Zhou, and Shree K Nayar. 2010. Flexible depth of field photography. IEEE transactions on pattern analysis and machine intelligence 33, 1 (2010), 58\u201371."},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1002\/lpor.202300274"},{"key":"e_1_2_2_15_1","volume-title":"Third-order aberration theory. https:\/\/www.thepulsar.be\/article\/-devoptical-part-14\u2013third-order-aberration-theory Accessed","year":"2025","unstructured":"Luc. 2022. Third-order aberration theory. https:\/\/www.thepulsar.be\/article\/-devoptical-part-14\u2013third-order-aberration-theory Accessed: May 19, 2025."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1364\/OE.8.000555"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCPhot.2013.6528302"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3680528.3687584"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.308479"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356526"},{"key":"e_1_2_2_21_1","volume-title":"2013 international conference on intelligent systems and signal processing (ISSP). IEEE, 77\u201381","author":"Mirajkar Pradnya P","year":"2013","unstructured":"P Mirajkar Pradnya and D Ruikar Sachin. 2013. Wavelet based image fusion techniques. In 2013 international conference on intelligent systems and signal processing (ISSP). IEEE, 77\u201381."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1117\/1.AP.2.6.066005"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1117\/1.JOM.1.3.034502"},{"key":"e_1_2_2_24_1","volume-title":"Sensors and sensing in biology and engineering","author":"Sandini Giulio","unstructured":"Giulio Sandini and Giorgio Metta. 2003. Retina-like sensors: motivations, technology and applications. In Sensors and sensing in biology and engineering. Springer, 251\u2013262."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3687976"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00882"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201333"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSAA.35.000480"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459674"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417821"},{"key":"e_1_2_2_31_1","doi-asserted-by":"crossref","unstructured":"Larry N Thibos Raymond A Applegate James T Schwiegerling and Robert Webb. 2002. Standards for reporting the optical aberrations of eyes. S652\u2013S660 pages.","DOI":"10.3928\/1081-597X-20020901-30"},{"key":"e_1_2_2_32_1","volume-title":"3D-printed eagle eye: Compound microlens system for foveated imaging. Science advances 3, 2","author":"Thiele Simon","year":"2017","unstructured":"Simon Thiele, Kathrin Arzenbacher, Timo Gissibl, Harald Giessen, and Alois M Herkommer. 2017. 3D-printed eagle eye: Compound microlens system for foveated imaging. Science advances 3, 2 (2017), e1602655."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3446791"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCI.2022.3212837"},{"key":"e_1_2_2_35_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3197517.3201299","article-title":"Megapixel adaptive optics: towards correcting large-scale distortions in computational cameras","volume":"37","author":"Wang Congli","year":"2018","unstructured":"Congli Wang, Qiang Fu, Xiong Dun, and Wolfgang Heidrich. 2018. Megapixel adaptive optics: towards correcting large-scale distortions in computational cameras. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1\u201312.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-50264-3"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01239"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.4304\/jcp.6.12.2559-2566"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1364\/OE.10.000060"},{"key":"e_1_2_2_40_1","volume-title":"Curriculum learning for ab initio deep learned refractive optics. Nature communications 15, 1","author":"Yang Xinge","year":"2024","unstructured":"Xinge Yang, Qiang Fu, and Wolfgang Heidrich. 2024a. Curriculum learning for ab initio deep learned refractive optics. Nature communications 15, 1 (2024), 6572."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3680528.3687640"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.29026\/oes.2023.230026"},{"key":"e_1_2_2_43_1","volume-title":"Light propagation with phase discontinuities: generalized laws of reflection and refraction. science 334, 6054","author":"Yu Nanfang","year":"2011","unstructured":"Nanfang Yu, Patrice Genevet, Mikhail A Kats, Francesco Aieta, Jean-Philippe Tetienne, Federico Capasso, and Zeno Gaburro. 2011. Light propagation with phase discontinuities: generalized laws of reflection and refraction. science 334, 6054 (2011), 333\u2013337."},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.3390\/jimaging6070060"},{"key":"e_1_2_2_45_1","first-page":"4819","article-title":"Deep learning-based multi-focus image fusion: A survey and a comparative study","volume":"44","author":"Zhang Xingchen","year":"2021","unstructured":"Xingchen Zhang. 2021. Deep learning-based multi-focus image fusion: A survey and a comparative study. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 9 (2021), 4819\u20134838.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.07.011"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1117\/1.3000432"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.3788\/COL20080608.0561"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3763278","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T21:17:31Z","timestamp":1764969451000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3763278"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12]]},"references-count":48,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,12]]}},"alternative-id":["10.1145\/3763278"],"URL":"https:\/\/doi.org\/10.1145\/3763278","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12]]},"assertion":[{"value":"2025-05-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-04","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}