{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T22:30:33Z","timestamp":1784413833655,"version":"3.55.0"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2017,12,5]],"date-time":"2017-12-05T00:00:00Z","timestamp":1512432000000},"content-version":"vor","delay-in-days":389,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1451830 and 1617234"],"award-info":[{"award-number":["1451830 and 1617234"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["N00014152013"],"award-info":[{"award-number":["N00014152013"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004356","name":"Nokia","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004356","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Draper Lab"},{"DOI":"10.13039\/100006785","name":"Google","doi-asserted-by":"publisher","award":["Research award"],"award-info":[{"award-number":["Research award"]}],"id":[{"id":"10.13039\/100006785","id-type":"DOI","asserted-by":"publisher"}]},{"name":"UC San Diego Center for Visual Computing"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2016,11,11]]},"abstract":"<jats:p>With the introduction of consumer light field cameras, light field imaging has recently become widespread. However, there is an inherent trade-off between the angular and spatial resolution, and thus, these cameras often sparsely sample in either spatial or angular domain. In this paper, we use machine learning to mitigate this trade-off. Specifically, we propose a novel learning-based approach to synthesize new views from a sparse set of input views. We build upon existing view synthesis techniques and break down the process into disparity and color estimation components. We use two sequential convolutional neural networks to model these two components and train both networks simultaneously by minimizing the error between the synthesized and ground truth images. We show the performance of our approach using only four corner sub-aperture views from the light fields captured by the Lytro Illum camera. Experimental results show that our approach synthesizes high-quality images that are superior to the state-of-the-art techniques on a variety of challenging real-world scenes. We believe our method could potentially decrease the required angular resolution of consumer light field cameras, which allows their spatial resolution to increase.<\/jats:p>","DOI":"10.1145\/2980179.2980251","type":"journal-article","created":{"date-parts":[[2016,11,11]],"date-time":"2016-11-11T12:02:54Z","timestamp":1478865774000},"page":"1-10","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":595,"title":["Learning-based view synthesis for light field cameras"],"prefix":"10.1145","volume":"35","author":[{"given":"Nima Khademi","family":"Kalantari","sequence":"first","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ting-Chun","family":"Wang","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ravi","family":"Ramamoorthi","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2016,12,5]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.121783"},{"key":"e_1_2_2_2_1","doi-asserted-by":"crossref","unstructured":"Bishop T. E. Zanetti S. and Favaro P. 2009. Light field superresolution. In IEEE ICCP 1--9.","DOI":"10.1109\/ICCPHOT.2009.5559010"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","unstructured":"Burger H. C. Schuler C. J. and Harmeling S. 2012. Image denoising: Can plain neural networks compete with BM3D? In IEEE CVPR 2392--2399.","DOI":"10.5555\/2354409.2354805"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","unstructured":"Chaurasia G. Sorkine O. and Drettakis G. 2011. Silhouette-aware warping for image-based rendering. In EGSR 1223--1232. 10.1111\/j.1467-8659.2011.01981.x","DOI":"10.1111\/j.1467-8659.2011.01981.x"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","unstructured":"Chaurasia G. Duchene S. Sorkine-Hornung O. and Drettakis G. 2013. Depth synthesis and local warps for plausible image-based navigation. ACM TOG 32 3 30:1--30:12. 10.1145\/2487228.2487238","DOI":"10.1145\/2487228.2487238"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.407"},{"key":"e_1_2_2_7_1","doi-asserted-by":"crossref","unstructured":"Dong C. Loy C. C. He K. and Tang X. 2014. Learning a deep convolutional network for image super-resolution. In ECCV 184--199.","DOI":"10.1007\/978-3-319-10593-2_13"},{"key":"e_1_2_2_8_1","doi-asserted-by":"crossref","unstructured":"Dosovitskiy A. Springenberg J. T. and Brox T. 2015. Learning to generate chairs with convolutional neural networks. In IEEE CVPR 1538--1546.","DOI":"10.1109\/CVPR.2015.7298761"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2008.01138.x"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","unstructured":"Fitzgibbon A. Wexler Y. and Zisserman A. 2003. Image-based rendering using image-based priors. In IEEE ICCV 1176--1183 vol.2.","DOI":"10.5555\/946247.946764"},{"key":"e_1_2_2_11_1","volume-title":"Deepstereo: Learning to predict new views from the worlds imagery","author":"Flynn J.","year":"2016","unstructured":"Flynn, J., Neulander, I., Philbin, J., and Snavely, N. 2016. Deepstereo: Learning to predict new views from the worlds imagery. In IEEE CVPR, 5515--5524."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.161"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","unstructured":"Georgiev T. Zheng K. C. Curless B. Salesin D. Nayar S. and Intwala C. 2006. Spatio-angular resolution tradeoffs in integral photography. In EGSR 263--272. 10.2312\/EGWR\/EGSR06\/263-272","DOI":"10.2312\/EGWR\/EGSR06\/263-272"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/1170745.1171481"},{"key":"e_1_2_2_15_1","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"9","author":"Glorot X.","year":"2010","unstructured":"Glorot, X., and Bengio, Y. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS, vol. 9, 249--256.","journal-title":"AISTATS"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778832"},{"key":"e_1_2_2_17_1","doi-asserted-by":"crossref","unstructured":"Heber S. and Pock T. 2016. Convolutional networks for shape from light field. In IEEE CVPR.","DOI":"10.1109\/CVPR.2016.407"},{"key":"e_1_2_2_18_1","doi-asserted-by":"crossref","unstructured":"Jeon H. G. Park J. Choe G. Park J. Bok Y. Tai Y. W. and Kweon I. S. 2015. Accurate depth map estimation from a lenslet light field camera. In IEEE CVPR 1547--1555.","DOI":"10.1109\/CVPR.2015.7298762"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601209"},{"key":"e_1_2_2_20_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.","author":"Kingma D.","year":"2014","unstructured":"Kingma, D., and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980."},{"key":"e_1_2_2_21_1","volume-title":"IEEE CVPR","author":"Levin A.","unstructured":"Levin, A., and Durand, F. 2010. Linear view synthesis using a dimensionality gap light field prior. In IEEE CVPR, 1831--1838."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","unstructured":"Levoy M. and Hanrahan P. 1996. Light field rendering. In ACM SIGGRAPH 31--42. 10.1145\/237170.237199","DOI":"10.1145\/237170.237199"},{"key":"e_1_2_2_23_1","unstructured":"Lytro 2016. https:\/\/www.lytro.com\/."},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531348"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","unstructured":"Marwah K. Wetzstein G. Bando Y. and Raskar R. 2013. Compressive light field photography using overcomplete dictionaries and optimized projections. ACM TOG 32 4 46:1--46:12. 10.1145\/2461912.2461914","DOI":"10.1145\/2461912.2461914"},{"key":"e_1_2_2_26_1","doi-asserted-by":"crossref","unstructured":"Mitra K. and Veeraraghavan A. 2012. Light field de-noising light field superresolution and stereo camera based re-focussing using a GMM light field patch prior. In IEEE CVPRW 22--28.","DOI":"10.1109\/CVPRW.2012.6239346"},{"key":"e_1_2_2_27_1","first-page":"1","article-title":"Light field photography with a hand-held plenoptic camera","volume":"2","author":"Ng R.","year":"2005","unstructured":"Ng, R., Levoy, M., Br\u00e9dif, M., Duval, G., Horowitz, M., and Hanrahan, P. 2005. Light field photography with a hand-held plenoptic camera. Computer Science Technical Report CSTR 2, 11, 1--11.","journal-title":"Computer Science Technical Report CSTR"},{"key":"e_1_2_2_28_1","unstructured":"Pelican Imaging 2016. Capture life in 3D. http:\/\/www.pelicanimaging.com\/."},{"key":"e_1_2_2_29_1","unstructured":"Raj A. Lowney M. Shah R. and Wetzstein G. 2016. Stanford lytro light field archive. http:\/\/lightfields.stanford.edu\/."},{"key":"e_1_2_2_30_1","unstructured":"RayTrix 2016. 3D light field camera technology. https:\/\/www.raytrix.de\/."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1038\/323533a0"},{"key":"e_1_2_2_32_1","doi-asserted-by":"crossref","unstructured":"Schedl D. C. Birklbauer C. and Bimber O. 2015. Directional super-resolution by means of coded sampling and guided upsampling. In IEEE ICCP 1--10.","DOI":"10.1109\/ICCPHOT.2015.7168365"},{"key":"e_1_2_2_33_1","doi-asserted-by":"crossref","unstructured":"Shechtman E. Rav-Acha A. Irani M. and Seitz S. 2010. Regenerative morphing. In IEEE CVPR 615--622.","DOI":"10.1109\/CVPR.2010.5540159"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","unstructured":"Shi L. Hassanieh H. Davis A. Katabi D. and Du-rand F. 2014. Light field reconstruction using sparsity in the continuous fourier domain. ACM TOG 34 1 12:1--12:13. 10.1145\/2682631","DOI":"10.1145\/2682631"},{"key":"e_1_2_2_35_1","doi-asserted-by":"crossref","unstructured":"Su H. Wang F. Yi L. and Guibas L. 2014. 3D-assisted image feature synthesis for novel views of an object. arXiv preprint arXiv:1412.0003.","DOI":"10.1109\/ICCV.2015.307"},{"key":"e_1_2_2_36_1","doi-asserted-by":"crossref","unstructured":"Sun J. Cao W. Xu Z. and Ponce J. 2015. Learning a convolutional neural network for non-uniform motion blur removal. In IEEE CVPR 769--777.","DOI":"10.1109\/CVPR.2015.7298677"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","unstructured":"Tao M. W. Hadap S. Malik J. and Ramamoorthi R. 2013. Depth from combining defocus and correspondence using light-field cameras. In IEEE ICCV 673--680. 10.1109\/ICCV.2013.89","DOI":"10.1109\/ICCV.2013.89"},{"key":"e_1_2_2_38_1","volume-title":"IEEE CVPR","author":"Tao M. W.","unstructured":"Tao, M. W., Srinivasan, P. P., Malik, J., Rusinkiewicz, S., and Ramamoorthi, R. 2015. Depth from shading, defocus, and correspondence using light-field angular coherence. In IEEE CVPR, 1940--1948."},{"key":"e_1_2_2_39_1","unstructured":"Tatarchenko M. Dosovitskiy A. and Brox T. 2015. Single-view to multi-view: Reconstructing unseen views with a convolutional network. CoRR abs\/1511.06702."},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2003.817626"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","unstructured":"Vedaldi A. and Lenc K. 2015. MatConvNet: Convolutional neural networks for Matlab. In ACMMM 689--692. 10.1145\/2733373.2807412","DOI":"10.1145\/2733373.2807412"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","unstructured":"Wang T. C. Efros A. A. and Ramamoorthi R. 2015. Occlusion-aware depth estimation using light-field cameras. In IEEE ICCV 3487--3495. 10.1109\/ICCV.2015.398","DOI":"10.1109\/ICCV.2015.398"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","unstructured":"Wanner S. and Goldluecke B. 2012. Globally consistent depth labeling of 4D light fields. In IEEE CVPR 41--48.","DOI":"10.5555\/2354409.2355069"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.147"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073259"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","unstructured":"Yang J. Reed S. E. Yang M.-H. and Lee H. 2015. Weakly-supervised disentangling with recurrent transformations for 3D view synthesis. In NIPS 1099--1107.","DOI":"10.5555\/2969239.2969362"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2015.17"},{"key":"e_1_2_2_49_1","doi-asserted-by":"crossref","unstructured":"Zhang Z. Liu Y. and Dai Q. 2015. Light field from micro-baseline image pair. In IEEE CVPR 3800--3809.","DOI":"10.1109\/CVPR.2015.7299004"},{"key":"e_1_2_2_50_1","first-page":"1","article-title":"PlenoPatch: Patch-based plenoptic image manipulation","volume":"99","author":"Zhang F. L.","year":"2016","unstructured":"Zhang, F. L., Wang, J., Shechtman, E., Zhou, Z. Y., Shi, J. X., and Hu, S. M. 2016. PlenoPatch: Patch-based plenoptic image manipulation. IEEE TVCG PP, 99, 1--1.","journal-title":"IEEE TVCG PP"},{"key":"e_1_2_2_51_1","doi-asserted-by":"crossref","unstructured":"Zhou T. Tulsiani S. Sun W. Malik J. and Efros A. A. 2016. View synthesis by appearance flow. CoRR abs\/1605.03557.","DOI":"10.1007\/978-3-319-46493-0_18"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2980179.2980251","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2980179.2980251","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2980179.2980251","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T09:29:54Z","timestamp":1763458194000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2980179.2980251"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,11,11]]},"references-count":51,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2016,11,11]]}},"alternative-id":["10.1145\/2980179.2980251"],"URL":"https:\/\/doi.org\/10.1145\/2980179.2980251","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,11,11]]},"assertion":[{"value":"2016-12-05","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}