{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T15:35:49Z","timestamp":1778081749831,"version":"3.51.4"},"reference-count":77,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004052","name":"KAUST","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004052","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:p>\n            Unwanted camera obstruction can severely degrade captured images, including both scene occluders near the camera and partial occlusions of the camera cover glass. Such occlusions can cause catastrophic failures for various scene understanding tasks such as semantic segmentation, object detection, and depth estimation. Existing camera arrays capture multiple redundant views of a scene to see around thin occlusions. Such multi-camera systems effectively form a large synthetic aperture, which can suppress nearby occluders with a large defocus blur, but significantly increase the overall form factor of the imaging setup. In this work, we propose a\n            <jats:italic>monocular single-shot<\/jats:italic>\n            imaging approach that\n            <jats:italic>optically cloaks<\/jats:italic>\n            obstructions by emulating a large array. Instead of relying on different camera views, we learn a diffractive optical element (DOE) that performs depth-dependent optical encoding, scattering nearby occlusions while allowing paraxial wavefronts to be focused. We computationally reconstruct unobstructed images from these superposed measurements with a neural network that is trained jointly with the optical layer of the proposed imaging system. We assess the proposed method in simulation and with an experimental prototype, validating that the proposed computational camera is capable of recovering occluded scene information in the presence of severe camera obstruction.\n          <\/jats:p>","DOI":"10.1145\/3528223.3530185","type":"journal-article","created":{"date-parts":[[2022,7,22]],"date-time":"2022-07-22T21:06:27Z","timestamp":1658523987000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["Seeing through obstructions with diffractive cloaking"],"prefix":"10.1145","volume":"41","author":[{"given":"Zheng","family":"Shi","sequence":"first","affiliation":[{"name":"Princeton University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuval","family":"Bahat","sequence":"additional","affiliation":[{"name":"Princeton University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seung-Hwan","family":"Baek","sequence":"additional","affiliation":[{"name":"Princeton University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qiang","family":"Fu","sequence":"additional","affiliation":[{"name":"King Abdullah University of Science and Technology, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hadi","family":"Amata","sequence":"additional","affiliation":[{"name":"King Abdullah University of Science and Technology, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiao","family":"Li","sequence":"additional","affiliation":[{"name":"Princeton University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Praneeth","family":"Chakravarthula","sequence":"additional","affiliation":[{"name":"Princeton University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wolfgang","family":"Heidrich","sequence":"additional","affiliation":[{"name":"King Abdullah University of Science and Technology, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Felix","family":"Heide","sequence":"additional","affiliation":[{"name":"Princeton University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,7,22]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00570"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00265"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01164"},{"key":"e_1_2_2_4_1","volume-title":"Optical cloaking with metamaterials. Nature photonics 1, 4","author":"Cai Wenshan","year":"2007","unstructured":"Wenshan Cai, Uday K Chettiar, Alexander V Kildishev, and Vladimir M Shalaev. 2007. Optical cloaking with metamaterials. Nature photonics 1, 4 (2007), 224--227."},{"key":"e_1_2_2_5_1","volume-title":"Learning sensor multiplexing design through back-propagation. Advances in Neural Information Processing Systems 29","author":"Chakrabarti Ayan","year":"2016","unstructured":"Ayan Chakrabarti. 2016. Learning sensor multiplexing design through back-propagation. Advances in Neural Information Processing Systems 29 (2016)."},{"key":"e_1_2_2_6_1","volume-title":"Deep Optics for Monocular Depth Estimation and 3D Object Detection. ArXiv abs\/1904.08601","author":"Chang Julie","year":"2019","unstructured":"Julie Chang and Gordon Wetzstein. 2019. Deep Optics for Monocular Depth Estimation and 3D Object Detection. ArXiv abs\/1904.08601 (2019)."},{"key":"e_1_2_2_7_1","volume-title":"Ray-optics cloaking devices for large objects in incoherent natural light. Nature communications 4, 1","author":"Chen Hongsheng","year":"2013","unstructured":"Hongsheng Chen, Bin Zheng, Lian Shen, Huaping Wang, Xianmin Zhang, Nikolay I Zheludev, and Baile Zhang. 2013. Ray-optics cloaking devices for large objects in incoherent natural light. Nature communications 4, 1 (2013), 1--6."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1038\/ncomms1176"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00900"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.350"},{"key":"e_1_2_2_11_1","volume-title":"Multiple imaging phase-encoded optical elements written as programmable spatial light modulators. Applied optics 29, 17","author":"Cottrell Don M","year":"1990","unstructured":"Don M Cottrell, Jeffrey A Davis, Theodore R Hedman, and Rodger A Lilly. 1990. Multiple imaging phase-encoded optical elements written as programmable spatial light modulators. Applied optics 29, 17 (1990), 2505--2509."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2018.8486522"},{"key":"e_1_2_2_13_1","volume-title":"Three-dimensional invisibility cloak at optical wavelengths. science 328, 5976","author":"Ergin Tolga","year":"2010","unstructured":"Tolga Ergin, Nicolas Stenger, Patrice Brenner, John B Pendry, and Martin Wegener. 2010. Three-dimensional invisibility cloak at optical wavelengths. science 328, 5976 (2010), 337--339."},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11760-016-0876-7"},{"key":"e_1_2_2_15_1","volume-title":"Accessed","year":"2017","unstructured":"Ficosa. 2017. Ficosa Sensor and Camera Cleaning System. Accessed Oct 18, 2021. (2017). https:\/\/www.ficosa.com\/products\/underhood\/sensor-and-camera-cleaning\/"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1177\/0954407017695141"},{"key":"e_1_2_2_17_1","volume-title":"Introduction to Fourier Optics","author":"Goodman J.W.","unstructured":"J.W. Goodman. 2005. Introduction to Fourier Optics. W. H. Freeman. https:\/\/books.google.com\/books?id=ow5xs_Rtt9AC"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01387"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11760-020-01749-6"},{"key":"e_1_2_2_20_1","volume-title":"3D single-molecule super-resolution microscopy with a tilted light sheet. Nature communications 9, 1","author":"Gustavsson Anna-Karin","year":"2018","unstructured":"Anna-Karin Gustavsson, Petar N Petrov, Maurice Y Lee, Yoav Shechtman, and WE Moerner. 2018. 3D single-molecule super-resolution microscopy with a tilted light sheet. Nature communications 9, 1 (2018), 1--8."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCI.2018.2849326"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2019.00534"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2832296"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2020.2984801"},{"key":"e_1_2_2_25_1","volume-title":"Convolutional neural networks that teach microscopes how to image. ArXiv abs\/1709.07223","author":"Horstmeyer Roarke","year":"2017","unstructured":"Roarke Horstmeyer, Richard Y. Chen, Barbara Kappes, and Benjamin Judkewitz. 2017. Convolutional neural networks that teach microscopes how to image. ArXiv abs\/1709.07223 (2017)."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1364\/ao.53.001958"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344929"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073589"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1002\/adom.202000827"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACPR.2015.7486506"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49409-8_68"},{"key":"e_1_2_2_32_1","volume-title":"Data-Driven Design for Fourier Ptychographic Microscopy. In 2019 IEEE International Conference on Computational Photography (ICCP). IEEE, 1--8.","author":"Kellman Michael","year":"2019","unstructured":"Michael Kellman, Emrah Bostan, Michael Chen, and Laura Waller. 2019. Data-Driven Design for Fourier Ptychographic Microscopy. In 2019 IEEE International Conference on Computational Photography (ICCP). IEEE, 1--8."},{"key":"e_1_2_2_33_1","volume-title":"The IEEE International Conference on Computer Vision (ICCV).","author":"Kupyn Orest","year":"2019","unstructured":"Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang. 2019. DeblurGANv2: Deblurring (Orders-of-Magnitude) Faster and Better. In The IEEE International Conference on Computer Vision (ICCV)."},{"key":"e_1_2_2_34_1","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV). 2011--2020","author":"Li Xiaoyu","unstructured":"Xiaoyu Li, Bo Zhang, Jing Liao, and Pedro V. Sander. 2021. Let's See Clearly: Contaminant Artifact Removal for Moving Cameras. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV). 2011--2020."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2008.4587493"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01422"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130884"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2960087"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2960087"},{"key":"e_1_2_2_40_1","volume-title":"Deep Optics for Single-shot High-dynamic-range Imaging. arXiv preprint arXiv:1908.00620","author":"Metzler Christopher A","year":"2019","unstructured":"Christopher A Metzler, Hayato Ikoma, Yifan Peng, and Gordon Wetzstein. 2019. Deep Optics for Single-shot High-dynamic-range Imaging. arXiv preprint arXiv:1908.00620 (2019)."},{"key":"e_1_2_2_41_1","volume-title":"Accessed","author":"Monrad Rolf","year":"2017","unstructured":"Rolf Monrad. 2017. Mostad Mekaniske - Apparatus for Cleaning Object Surface. Accessed Oct 18, 2021. (2017). https:\/\/uspto.report\/patent\/app\/20200188965"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSAA.398828"},{"key":"e_1_2_2_43_1","volume-title":"Deep-STORM3D: dense 3D localization microscopy and PSF design by deep learning. Nature methods 17, 7","author":"Nehme Elias","year":"2020","unstructured":"Elias Nehme, Daniel Freedman, Racheli Gordon, Boris Ferdman, Lucien E Weiss, Onit Alalouf, Tal Naor, Reut Orange, Tomer Michaeli, and Yoav Shechtman. 2020. Deep-STORM3D: dense 3D localization microscopy and PSF design by deep learning. Nature methods 17, 7 (2020), 734--740."},{"key":"e_1_2_2_44_1","volume-title":"Accessed","year":"2013","unstructured":"Orlaco. 2013. Orlaco All Time Vision Camera. Accessed Oct 18, 2021. (2013). https:\/\/rmtequip.com\/en\/products\/orlaco-all-time-vision-camera"},{"key":"e_1_2_2_45_1","volume-title":"Three dimensional tracking of fluorescent microparticles using a photon-limited double-helix response system. Optics express 16, 26","author":"Prasanna Pavani Sri Rama","year":"2008","unstructured":"Sri Rama Prasanna Pavani and Rafael Piestun. 2008. Three dimensional tracking of fluorescent microparticles using a photon-limited double-helix response system. Optics express 16, 26 (2008), 22048--22057."},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2012.06.014"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356526"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/325165.325247"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00263"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_2_51_1","volume-title":"Accessed","author":"Rotoclear","year":"2016","unstructured":"Rotoclear S3. 2016. Rotoclear S3 Spin Window. Accessed Oct 18, 2021. (2016). https:\/\/cromar.co.uk\/products\/accessories\/roto-clear-spin-window\/"},{"key":"e_1_2_2_52_1","volume-title":"Multicolour localization microscopy by point-spread-function engineering. Nature photonics 10","author":"Shechtman Yoav","year":"2016","unstructured":"Yoav Shechtman, Lucien E Weiss, Adam S. Backer, Maurice Y. Lee, and W E Moerner. 2016. Multicolour localization microscopy by point-spread-function engineering. Nature photonics 10 (2016), 590--594."},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201333"},{"key":"e_1_2_2_54_1","volume-title":"Deep End-to-End Time-of-Flight Imaging. 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Su Shuochen","year":"2018","unstructured":"Shuochen Su, Felix Heide, Gordon Wetzstein, and Wolfgang Heidrich. 2018. Deep End-to-End Time-of-Flight Imaging. 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (2018), 6383--6392."},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00146"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459674"},{"key":"e_1_2_2_57_1","volume-title":"Resolution-robust Large Mask Inpainting with Fourier Convolutions. arXiv preprint arXiv:2109.07161","author":"Suvorov Roman","year":"2021","unstructured":"Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. 2021. Resolution-robust Large Mask Inpainting with Fourier Convolutions. arXiv preprint arXiv:2109.07161 (2021)."},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-26443-0"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3446791"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV48630.2021.00081"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.537"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.244"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2004.1315006"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV45572.2020.9093448"},{"key":"e_1_2_2_65_1","unstructured":"Waymo. 2019. Waymo Open Dataset: An autonomous driving dataset. (2019)."},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073259"},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCPHOT.2019.8747330"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2017.2715012"},{"key":"e_1_2_2_69_1","volume-title":"Freeman","author":"Xue Tianfan","year":"2015","unstructured":"Tianfan Xue, Michael Rubinstein, Ce Liu, and William T. Freeman. 2015. A Computational Approach for Obstruction-Free Photography. ACM Transactions on Graphics (Proc. SIGGRAPH) 34, 4 (2015)."},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR.2010.1101"},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01264-9_1"},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10599-4_1"},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.1117\/1.JEI.30.2.023028"},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00940"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.138"},{"key":"e_1_2_2_76_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00457"},{"key":"e_1_2_2_77_1","volume-title":"Places: A 10 million Image Database for Scene Recognition","author":"Zhou Bolei","year":"2017","unstructured":"Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Places: A 10 million Image Database for Scene Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (2017)."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530185","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530185","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:49Z","timestamp":1750186969000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530185"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":77,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10.1145\/3528223.3530185"],"URL":"https:\/\/doi.org\/10.1145\/3528223.3530185","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7]]},"assertion":[{"value":"2022-07-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}