{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T18:23:53Z","timestamp":1773771833537,"version":"3.50.1"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NSF","award":["1840131, 1405847"],"award-info":[{"award-number":["1840131, 1405847"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:p>Recent deep learning approaches have shown remarkable promise to enable high fidelity holographic displays. However, lightweight wearable display devices cannot afford the computation demand and energy consumption for hologram generation due to the limited onboard compute capability and battery life. On the other hand, if the computation is conducted entirely remotely on a cloud server, transmitting lossless hologram data is not only challenging but also result in prohibitively high latency and storage.<\/jats:p>\n          <jats:p>In this work, by distributing the computation and optimizing the transmission, we propose the first framework that jointly generates and compresses high-quality phase-only holograms. Specifically, our framework asymmetrically separates the hologram generation process into high-compute remote encoding (on the server), and low-compute decoding (on the edge) stages. Our encoding enables light weight latent space data, thus faster and efficient transmission to the edge device. With our framework, we observed a reduction of 76% computation and consequently 83% in energy cost on edge devices, compared to the existing hologram generation methods. Our framework is robust to transmission and decoding errors, and approach high image fidelity for as low as 2 bits-per-pixel, and further reduced average bit-rates and decoding time for holographic videos.<\/jats:p>","DOI":"10.1145\/3528223.3530070","type":"journal-article","created":{"date-parts":[[2022,7,22]],"date-time":"2022-07-22T21:06:27Z","timestamp":1658523987000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["Joint neural phase retrieval and compression for energy- and computation-efficient holography on the edge"],"prefix":"10.1145","volume":"41","author":[{"given":"Yujie","family":"Wang","sequence":"first","affiliation":[{"name":"Shandong University and Peking University, China"}]},{"given":"Praneeth","family":"Chakravarthula","sequence":"additional","affiliation":[{"name":"Princeton University and UNC"}]},{"given":"Qi","family":"Sun","sequence":"additional","affiliation":[{"name":"New York University"}]},{"given":"Baoquan","family":"Chen","sequence":"additional","affiliation":[{"name":"Peking University, China"}]}],"member":"320","published-online":{"date-parts":[[2022,7,22]]},"reference":[{"key":"e_1_2_2_1_1","first-page":"1","article-title":"FLIP: A Difference Evaluator for Alternating Images. In Proceedings of the ACM on Computer Graphics and Interactive Techniques, Vol. 3","volume":"15","author":"Andersson Pontus","year":"2020","unstructured":"Pontus Andersson, Jim Nilsson, Tomas Akenine-M\u00f6ller, Magnus Oskarsson, Kalle \u00c5str\u00f6m, and Mark D. Fairchild. 2020. FLIP: A Difference Evaluator for Alternating Images. In Proceedings of the ACM on Computer Graphics and Interactive Techniques, Vol. 3. Article 15, 15:1--15:23 pages.","journal-title":"Article"},{"key":"e_1_2_2_2_1","volume-title":"Carbon-tracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models. ICML Workshop on Challenges in Deploying and monitoring Machine Learning Systems.","author":"Wolff Anthony Lasse F.","year":"2020","unstructured":"Lasse F. Wolff Anthony, Benjamin Kanding, and Raghavendra Selvan. 2020. Carbon-tracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models. ICML Workshop on Challenges in Deploying and monitoring Machine Learning Systems."},{"key":"e_1_2_2_3_1","volume-title":"International Conference on Learning Representations (ICLR).","author":"Ball\u00e9 Johannes","year":"2018","unstructured":"Johannes Ball\u00e9, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. 2018. Variational image compression with a scale hyperprior. In International Conference on Learning Representations (ICLR)."},{"key":"e_1_2_2_4_1","volume-title":"Picture Coding Symposium (PCS). IEEE Signal Processing Society, 1--5.","author":"Ball\u00e9 Johannes","unstructured":"Johannes Ball\u00e9, Valero Laparra, and Eero P. Simoncelli. 2016. End-to-end optimization of nonlinear transform codes for perceptual quality. In Picture Coding Symposium (PCS). IEEE Signal Processing Society, 1--5."},{"key":"e_1_2_2_5_1","volume-title":"International Conference on Learning Representations (ICLR).","author":"Ball\u00e9 Johannes","unstructured":"Johannes Ball\u00e9, Valero Laparra, and Eero P. Simoncelli. 2017. End-to-end optimized image compression. In International Conference on Learning Representations (ICLR)."},{"key":"e_1_2_2_6_1","volume-title":"CompressAI: a PyTorch library and evaluation platform for end-to-end compression research. arXiv preprint arXiv:2011.03029","author":"B\u00e9gaint Jean","year":"2020","unstructured":"Jean B\u00e9gaint, Fabien Racap\u00e9, Simon Feltman, and Akshay Pushparaja. 2020. CompressAI: a PyTorch library and evaluation platform for end-to-end compression research. arXiv preprint arXiv:2011.03029 (2020)."},{"key":"e_1_2_2_7_1","volume-title":"Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv preprint arXiv:1308.3432","author":"Bengio Yoshua","year":"2013","unstructured":"Yoshua Bengio, Nicholas L\u00e9onard, and Aaron Courville. 2013. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv preprint arXiv:1308.3432 (2013)."},{"key":"e_1_2_2_8_1","volume-title":"Holographic imaging","author":"Benton Stephen A","unstructured":"Stephen A Benton and V Michael Bove Jr. 2008. Holographic imaging. John Wiley & Sons."},{"key":"e_1_2_2_9_1","volume-title":"Phase retrieval for Fourier Ptychography under varying amount of measurements. arXiv preprint arXiv:1805.03593","author":"Boominathan Lokesh","year":"2018","unstructured":"Lokesh Boominathan, Mayug Maniparambil, Honey Gupta, Rahul Baburajan, and Kaushik Mitra. 2018. Phase retrieval for Fourier Ptychography under varying amount of measurements. arXiv preprint arXiv:1805.03593 (2018)."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356539"},{"key":"e_1_2_2_11_1","volume-title":"ACM Transactions on Graphics (TOG)","author":"Chakravarthula Praneeth","year":"2022","unstructured":"Praneeth Chakravarthula, Ethan Tseng, Henry Fuchs, and Felix Heide. 2022. Hogel-free Holography. ACM Transactions on Graphics (TOG) (2022)."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417846"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417846"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3106433"},{"key":"e_1_2_2_15_1","volume-title":"Computer generated hologram from point cloud using graphics processor. Applied optics 48, 36","author":"Chen Rick H-Y","year":"2009","unstructured":"Rick H-Y Chen and Timothy D Wilkinson. 2009. Computer generated hologram from point cloud using graphics processor. Applied optics 48, 36 (2009), 6841--6850."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00852"},{"key":"e_1_2_2_17_1","volume-title":"Youssef SG Nashed, and Ross J Harder","author":"Cherukara Mathew J","year":"2018","unstructured":"Mathew J Cherukara, Youssef SG Nashed, and Ross J Harder. 2018. Real-time coherent diffraction inversion using deep generative networks. Scientific reports 8, 1 (2018), 1--8."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478513.3480542"},{"key":"e_1_2_2_19_1","volume-title":"Thomas","author":"Cover Thomas M.","year":"2006","unstructured":"Thomas M. Cover and Joy A. Thomas. 2006. Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience, USA."},{"key":"e_1_2_2_20_1","first-page":"2051","article-title":"A Loss Function for Generative Neural Networks Based on Watson's Perceptual Model","volume":"33","author":"Czolbe Steffen","year":"2020","unstructured":"Steffen Czolbe, Oswin Krause, Ingemar Cox, and Christian Igel. 2020. A Loss Function for Generative Neural Networks Based on Watson's Perceptual Model. Advances in Neural Information Processing Systems 33, 2051--2061.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_2_21_1","volume-title":"On the Effectiveness of Weight-Encoded Neural Implicit 3D Shapes. arXiv preprint arXiv:2009.09808","author":"Davies Thomas","year":"2021","unstructured":"Thomas Davies, Derek Nowrouzezahrai, and Alec Jacobson. 2021. On the Effectiveness of Weight-Encoded Neural Implicit 3D Shapes. arXiv preprint arXiv:2009.09808 (2021)."},{"key":"e_1_2_2_22_1","volume-title":"Asymmetric numeral systems: entropy coding combining speed of Huffman coding with compression rate of arithmetic coding. arXiv preprint arXiv:1311.2540","author":"Duda Jarek","year":"2014","unstructured":"Jarek Duda. 2014. Asymmetric numeral systems: entropy coding combining speed of Huffman coding with compression rate of arithmetic coding. arXiv preprint arXiv:1311.2540 (2014)."},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1364\/OE.399624"},{"key":"e_1_2_2_24_1","volume-title":"Low photon count phase retrieval using deep learning. Physical review letters 121, 24","author":"Goy Alexandre","year":"2018","unstructured":"Alexandre Goy, Kwabena Arthur, Shuai Li, and George Barbastathis. 2018. Low photon count phase retrieval using deep learning. Physical review letters 121, 24 (2018)."},{"key":"e_1_2_2_25_1","volume-title":"Entropy and information theory","author":"Gray Robert M","unstructured":"Robert M Gray. 2011. Entropy and information theory. Springer Science & Business Media."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3065339"},{"key":"e_1_2_2_27_1","volume-title":"Article 1258","author":"Jiao Shuming","year":"2018","unstructured":"Shuming Jiao, Zhi Jin, Chenliang Chang, Changyuan Zhou, Wenbin Zou, and Xia Li. 2018. Compression of Phase-Only Holograms with JPEG Standard and Deep Learning. Applied Sciences 8, 8, Article 1258 (2018)."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCI.2019.2905434"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1088\/2040-8986\/aab02f"},{"key":"e_1_2_2_30_1","volume-title":"Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography. Applied optics 47, 19","author":"Kim Hwi","year":"2008","unstructured":"Hwi Kim, Joonku Hahn, and Byoungho Lee. 2008. Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography. Applied optics 47, 19 (2008), D117--D127."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1364\/AO.47.000D55"},{"key":"e_1_2_2_32_1","volume-title":"Cloud-Based Video Streaming Services: A Survey. arXiv preprint arXiv:2011.14976","author":"Li Xiangbo","year":"2020","unstructured":"Xiangbo Li, Mahmoud Darwich, Magdy Bayoumi, and Mohsen Amini Salehi. 2020. Cloud-Based Video Streaming Services: A Survey. arXiv preprint arXiv:2011.14976 (2020)."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2637166.2637230"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2019.2910119"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073624"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459831"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459785"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1364\/OPEX.14.000603"},{"key":"e_1_2_2_39_1","volume-title":"Computer-generated holograms for three-dimensional surface objects with shade and texture. Applied optics 44, 22","author":"Matsushima Kyoji","year":"2005","unstructured":"Kyoji Matsushima. 2005. Computer-generated holograms for three-dimensional surface objects with shade and texture. Applied optics 44, 22 (2005), 4607--4614."},{"key":"e_1_2_2_40_1","volume-title":"Band-Limited Angular Spectrum Method for Numerical Simulation of Free-Space Propagation in Far and Near Fields. Optics express 17, 22","author":"Matsushima Kyoji","year":"2009","unstructured":"Kyoji Matsushima and Tomoyoshi Shimobaba. 2009. Band-Limited Angular Spectrum Method for Numerical Simulation of Free-Space Propagation in Far and Near Fields. Optics express 17, 22 (2009), 19662--19673."},{"key":"e_1_2_2_41_1","first-page":"11913","article-title":"High-Fidelity Generative Image Compression","volume":"33","author":"Mentzer Fabian","year":"2020","unstructured":"Fabian Mentzer, George Toderici, Michael Tschannen, and Eirikur Agustsson. 2020. High-Fidelity Generative Image Compression. In Advances in Neural Information Processing Systems, Vol. 33. 11913--11924.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_2_42_1","unstructured":"David Minnen Johannes Ball\u00e9 and George Toderici. 2018. Joint Autoregressive and Hierarchical Priors for Learned Image Compression. In Advances in neural information processing systems. 10794--10803."},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1364\/AO.54.000A76"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356517"},{"key":"e_1_2_2_45_1","volume-title":"PyTorch: An Imperative Style","author":"Paszke Adam","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems. 8024--8035."},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417802"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.476879"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1981.1056282"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1038\/lsa.2017.141"},{"key":"e_1_2_2_50_1","volume-title":"Towards real-time photorealistic 3D holography with deep neural networks. Nature 591, 7849","author":"Shi Liang","year":"2021","unstructured":"Liang Shi, Beichen Li, Changil Kim, Petr Kellnhofer, and Wojciech Matusik. 2021. Towards real-time photorealistic 3D holography with deep neural networks. Nature 591, 7849 (2021), 234--239."},{"key":"e_1_2_2_51_1","volume-title":"Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane. Optics letters 34, 20","author":"Shimobaba Tomoyoshi","year":"2009","unstructured":"Tomoyoshi Shimobaba, Nobuyuki Masuda, and Tomoyoshi Ito. 2009. Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane. Optics letters 34, 20 (2009), 3133--3135."},{"key":"e_1_2_2_52_1","volume-title":"Very Deep Convolutional Networks for Large-Scale Image Recognition. In International Conference on Learning Representations (ICLR).","author":"Simonyan K.","unstructured":"K. Simonyan and A. Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In International Conference on Learning Representations (ICLR)."},{"key":"e_1_2_2_53_1","volume-title":"JPEG2000 Image Compression Fundamentals, Standards and Practice","author":"Taubman David","unstructured":"David Taubman and Michael Marcellin. 2013. JPEG2000 Image Compression Fundamentals, Standards and Practice. Springer Publishing Company, Incorporated."},{"key":"e_1_2_2_54_1","volume-title":"Lossy Image Compression with Compressive Autoencoders. In International Conference on Learning Representations (ICLR).","author":"Theis Lucas","year":"2017","unstructured":"Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Husz\u00e1r. 2017. Lossy Image Compression with Compressive Autoencoders. In International Conference on Learning Representations (ICLR)."},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2017.149"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/30.125072"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvcir.2017.04.009"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_2_2_59_1","first-page":"1398","article-title":"Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals","volume":"2","author":"Wang Z.","year":"2003","unstructured":"Z. Wang, E. P. Simoncelli, and A. C. Bovik. 2003. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems Computers, Vol. 2. 1398--1402.","journal-title":"Systems Computers"},{"key":"e_1_2_2_60_1","first-page":"13","article-title":"Computer-generated hologram with occlusion effect using layer-based processing","volume":"56","author":"Zhang Hao","year":"2017","unstructured":"Hao Zhang, Liangcai Cao, and Guofan Jin. 2017. Computer-generated hologram with occlusion effect using layer-based processing. Appl. Opt. 56, 13 (May 2017), F138--F143.","journal-title":"Appl. Opt."},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00068"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530070","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530070","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530070","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:25Z","timestamp":1750186945000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530070"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":61,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10.1145\/3528223.3530070"],"URL":"https:\/\/doi.org\/10.1145\/3528223.3530070","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7]]},"assertion":[{"value":"2022-07-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}