{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T07:11:25Z","timestamp":1768288285979,"version":"3.49.0"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2020,2,17]],"date-time":"2020-02-17T00:00:00Z","timestamp":1581897600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100007567","name":"City University of Hong Kong","doi-asserted-by":"crossref","award":["ACIM-SCM"],"award-info":[{"award-number":["ACIM-SCM"]}],"id":[{"id":"10.13039\/100007567","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Research Grants Council","award":["CityU 11206216"],"award-info":[{"award-number":["CityU 11206216"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2020,2,29]]},"abstract":"<jats:p>\n            We present a deep neural network called the\n            <jats:italic>light field generative adversarial network<\/jats:italic>\n            (LFGAN) that synthesizes a 4D light field from a single 2D RGB image. We generate light fields using a single image super-resolution (SISR) technique based on two important observations. First, the small baseline gives rise to the high similarity between the full light field image and each sub-aperture view. Second, the occlusion edge at any spatial coordinate of a sub-aperture view has the same orientation as the occlusion edge at the corresponding angular patch, implying that the occlusion information in the angular domain can be inferred from the sub-aperture local information. We employ the Wasserstein GAN with gradient penalty (WGAN-GP) to learn the color and geometry information from the light field datasets. The network can generate a plausible 4D light field comprising 8\u00d78 angular views from a single sub-aperture 2D image. We propose new loss terms, namely epipolar plane image (EPI) and brightness regularization (BRI) losses, as well as a novel multi-stage training framework to feed the loss terms at different time to generate superior light fields. The\n            <jats:italic>EPI loss<\/jats:italic>\n            can reinforce the network to learn the geometric features of the light fields, and the\n            <jats:italic>BRI loss<\/jats:italic>\n            can preserve the brightness consistency across different sub-aperture views. Two datasets have been used to evaluate our method: in addition to an existing light field dataset capturing scenes of flowers and plants, we have built a large dataset of toy animals consisting of 2,100 light fields captured with a plenoptic camera. We have performed comprehensive ablation studies to evaluate the effects of individual loss terms and the multi-stage training strategy, and have compared LFGAN to other state-of-the-art techniques. Qualitative and quantitative evaluation demonstrates that LFGAN can effectively estimate complex occlusions and geometry in challenging scenes, and outperform other existing techniques.\n          <\/jats:p>","DOI":"10.1145\/3366371","type":"journal-article","created":{"date-parts":[[2020,3,4]],"date-time":"2020-03-04T10:23:32Z","timestamp":1583317412000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["LFGAN"],"prefix":"10.1145","volume":"16","author":[{"given":"Bin","family":"Chen","sequence":"first","affiliation":[{"name":"City University of Hong Kong, Kowloon, Hong Kong SAR, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lingyan","family":"Ruan","sequence":"additional","affiliation":[{"name":"City University of Hong Kong, Kowloon, Hong Kong SAR, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Miu-Ling","family":"Lam","sequence":"additional","affiliation":[{"name":"City University of Hong Kong, Kowloon, Hong Kong SAR, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,2,17]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Arjovsky Martin","year":"2017"},{"key":"e_1_2_2_2_1","volume-title":"Proceedings of the International Conference on Machine Learning. 214--223","author":"Arjovsky Martin","year":"2017"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2011.168"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCPHOT.2009.5559010"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00128525"},{"key":"e_1_2_2_6_1","unstructured":"Angel X. Chang Thomas Funkhouser Leonidas Guibas Pat Hanrahan Qixing Huang Zimo Li Silvio Savarese etal 2015. ShapeNet: An information-rich 3D model repository. arXiv:1512.03012.  Angel X. Chang Thomas Funkhouser Leonidas Guibas Pat Hanrahan Qixing Huang Zimo Li Silvio Savarese et al. 2015. ShapeNet: An information-rich 3D model repository. arXiv:1512.03012."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487228.2487238"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618470"},{"key":"e_1_2_2_9_1","unstructured":"Emily L. Denton Soumith Chintala Arthur Szlam and Rob Fergus. 2015. Deep generative image models using a Laplacian pyramid of adversarial networks. In Advances in Neural Information Processing Systems. 1486--1494.  Emily L. Denton Soumith Chintala Arthur Szlam and Rob Fergus. 2015. Deep generative image models using a Laplacian pyramid of adversarial networks. In Advances in Neural Information Processing Systems. 1486--1494."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508376"},{"key":"e_1_2_2_11_1","volume-title":"Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. ACM","author":"Alexei"},{"key":"e_1_2_2_12_1","volume-title":"Retrieved","author":"Finley Darel Rex","year":"2006"},{"key":"e_1_2_2_13_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201916)","author":"Flynn John","year":"2016"},{"key":"e_1_2_2_15_1","first-page":"263","article-title":"Spatio-angular resolution tradeoffs in integral photography","volume":"2006","author":"Georgiev Todor","year":"2006","journal-title":"Rendering Techniques"},{"key":"e_1_2_2_16_1","unstructured":"Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680.  Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680."},{"key":"e_1_2_2_17_1","unstructured":"Ishaan Gulrajani Faruk Ahmed Martin Arjovsky Vincent Dumoulin and Aaron C. Courville. 2017. Improved training of Wasserstein GANs. In Advances in Neural Information Processing Systems. 5769--5779.  Ishaan Gulrajani Faruk Ahmed Martin Arjovsky Vincent Dumoulin and Aaron C. Courville. 2017. Improved training of Wasserstein GANs. In Advances in Neural Information Processing Systems. 5769--5779."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.267"},{"key":"e_1_2_2_19_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201917)","author":"Isola Phillip"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980251"},{"key":"e_1_2_2_21_1","unstructured":"Tero Karras Timo Aila Samuli Laine and Jaakko Lehtinen. 2017. Progressive growing of GANs for improved quality stability and variation. arXiv:1710.10196.  Tero Karras Timo Aila Samuli Laine and Jaakko Lehtinen. 2017. Progressive growing of GANs for improved quality stability and variation. arXiv:1710.10196."},{"key":"e_1_2_2_22_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Diederik"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.19"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5539854"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-88693-8_7"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531403"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/237170.237199"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCPHOT.2009.5559008"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.304"},{"key":"e_1_2_2_30_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Metz Luke","year":"2016"},{"key":"e_1_2_2_31_1","unstructured":"Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv:1411.1784.  Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv:1411.1784."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2012.6239346"},{"key":"e_1_2_2_33_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Radford Alec","year":"2015"},{"key":"e_1_2_2_34_1","volume-title":"Proceedings of the 33rd International Conference on Machine Learning","volume":"48","author":"Reed Scott","year":"2016"},{"key":"e_1_2_2_35_1","unstructured":"Scott E. Reed Zeynep Akata Santosh Mohan Samuel Tenka Bernt Schiele and Honglak Lee. 2016. Learning what and where to draw. In Advances in Neural Information Processing Systems. 217--225.  Scott E. Reed Zeynep Akata Santosh Mohan Samuel Tenka Bernt Schiele and Honglak Lee. 2016. Learning what and where to draw. In Advances in Neural Information Processing Systems. 217--225."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.723"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2682631"},{"key":"e_1_2_2_38_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Simonyan Karen","year":"2014"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.246"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46478-7_20"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.30"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2017.2738617"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2653101"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.398"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33715-4_44"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.147"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.178"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00882"},{"key":"e_1_2_2_49_1","unstructured":"Jimei Yang Scott E. Reed Ming-Hsuan Yang and Honglak Lee. 2015. Weakly-supervised disentangling with recurrent transformations for 3D view synthesis. In Advances in Neural Information Processing Systems. 1099--1107.  Jimei Yang Scott E. Reed Ming-Hsuan Yang and Honglak Lee. 2015. Weakly-supervised disentangling with recurrent transformations for 3D view synthesis. In Advances in Neural Information Processing Systems. 1099--1107."},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2015.17"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.629"},{"key":"e_1_2_2_52_1","volume-title":"Proceedings of the European Conference on Computer Vision. 649--666","author":"Zhang Richard"},{"key":"e_1_2_2_53_1","doi-asserted-by":"crossref","unstructured":"Bo Zhao Xiao Wu Zhi-Qi Cheng Hao Liu Zequn Jie and Jiashi Feng. 2017. Multi-view image generation from a single-view. arXiv:1704.04886.  Bo Zhao Xiao Wu Zhi-Qi Cheng Hao Liu Zequn Jie and Jiashi Feng. 2017. Multi-view image generation from a single-view. arXiv:1704.04886.","DOI":"10.1145\/3240508.3240536"},{"key":"e_1_2_2_54_1","volume-title":"Proceedings of the European Conference on Computer Vision. 286--301","author":"Zhou Tinghui"},{"key":"e_1_2_2_55_1","volume-title":"Proceedings of the IEEE International Conference on Computer Vision (ICCV\u201917)","author":"Zhu Jun-Yan"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3366371","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3366371","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:13:42Z","timestamp":1750202022000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3366371"}},"subtitle":["4D Light Field Synthesis from a Single RGB Image"],"short-title":[],"issued":{"date-parts":[[2020,2,17]]},"references-count":54,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,2,29]]}},"alternative-id":["10.1145\/3366371"],"URL":"https:\/\/doi.org\/10.1145\/3366371","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,17]]},"assertion":[{"value":"2018-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-02-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}