{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T17:04:21Z","timestamp":1780765461318,"version":"3.54.1"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2017,7,20]],"date-time":"2017-07-20T00:00:00Z","timestamp":1500508800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100002418","name":"Intel Corporation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100002418","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004356","name":"Nokia","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004356","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004358","name":"Samsung","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004358","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Draper Lab"},{"DOI":"10.13039\/100006785","name":"Google","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006785","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["N00014152013"],"award-info":[{"award-number":["N00014152013"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004359","name":"Sony Electronics","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004359","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2017,8,31]]},"abstract":"<jats:p>\n            Light field cameras have many advantages over traditional cameras, as they allow the user to change various camera settings\n            <jats:italic>after<\/jats:italic>\n            capture. However, capturing light fields requires a huge bandwidth to record the data: a modern light field camera can only take three images per second. This prevents current consumer light field cameras from capturing light field videos. Temporal interpolation at such extreme scale (10x, from 3 fps to 30 fps) is infeasible as too much information will be entirely missing between adjacent frames. Instead, we develop a hybrid imaging system, adding another standard video camera to capture the temporal information. Given a 3 fps light field sequence and a standard 30 fps 2D video, our system can then generate a full light field video at 30 fps. We adopt a learning-based approach, which can be decomposed into two steps: spatio-temporal flow estimation and appearance estimation. The flow estimation propagates the angular information from the light field sequence to the 2D video, so we can warp input images to the target view. The appearance estimation then combines these warped images to output the final pixels. The whole process is trained end-to-end using convolutional neural networks. Experimental results demonstrate that our algorithm outperforms current video interpolation methods, enabling consumer light field videography, and making applications such as refocusing and parallax view generation achievable on videos for the first time.\n          <\/jats:p>","DOI":"10.1145\/3072959.3073614","type":"journal-article","created":{"date-parts":[[2017,7,21]],"date-time":"2017-07-21T12:24:07Z","timestamp":1500639847000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":91,"title":["Light field video capture using a learning-based hybrid imaging system"],"prefix":"10.1145","volume":"36","author":[{"given":"Ting-Chun","family":"Wang","sequence":"first","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jun-Yan","family":"Zhu","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nima Khademi","family":"Kalantari","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexei A.","family":"Efros","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ravi","family":"Ramamoorthi","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2017,7,20]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-010-0390-2"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2003.1211416"},{"key":"e_1_2_2_3_1","volume-title":"Eurographics Symposium on Rendering (EGSR). 327--338","author":"Bhat Pravin","year":"2007","unstructured":"Pravin Bhat , C Lawrence Zitnick , Noah Snavely , Aseem Agarwala , Maneesh Agrawala , Michael Cohen , Brian Curless , and Sing Bing Kang . 2007 . Using photographs to enhance videos of a static scene . In Eurographics Symposium on Rendering (EGSR). 327--338 . Pravin Bhat, C Lawrence Zitnick, Noah Snavely, Aseem Agarwala, Maneesh Agrawala, Michael Cohen, Brian Curless, and Sing Bing Kang. 2007. Using photographs to enhance videos of a static scene. In Eurographics Symposium on Rendering (EGSR). 327--338."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCPHOT.2009.5559010"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCPHOT.2014.6831814"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995418"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487228.2487238"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10593-2_7"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.407"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.316"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2008.01138.x"},{"key":"e_1_2_2_12_1","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5515--5524","author":"Flynn John","year":"2016","unstructured":"John Flynn , Ivan Neulander , James Philbin , and Noah Snavely . 2016 . DeepStereo: Learning to Predict New Views from the World's Imagery . In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5515--5524 . John Flynn, Ivan Neulander, James Philbin, and Noah Snavely. 2016. DeepStereo: Learning to Predict New Views from the World's Imagery. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5515--5524."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778832"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964965"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.123"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1127647"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298762"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980251"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2014.2316835"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995457"},{"key":"e_1_2_2_21_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik","year":"2014","unstructured":"Diederik Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2012.6238903"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2013.2270375"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5539854"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.359"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2629494"},{"key":"e_1_2_2_28_1","unstructured":"Lytro Cinema. 2017. The ultimate creative tool for cinema and broadcast. https:\/\/www.lytro.com\/cinema. (2017).  Lytro Cinema. 2017. The ultimate creative tool for cinema and broadcast. https:\/\/www.lytro.com\/cinema. (2017)."},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531348"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461914"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298747"},{"key":"e_1_2_2_32_1","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop. 22--28","author":"Mitra Kaushik","year":"2012","unstructured":"Kaushik Mitra and Ashok Veeraraghavan . 2012 . Light field denoising, light field super-resolution and stereo camera based refocussing using a GMM light field patch prior . In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop. 22--28 . Kaushik Mitra and Ashok Veeraraghavan. 2012. Light field denoising, light field super-resolution and stereo camera based refocussing using a GMM light field patch prior. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop. 22--28."},{"key":"e_1_2_2_33_1","unstructured":"RayTrix. 2017. 3D Light Field Camera Technology. https:\/\/www.raytrix.de\/. (2017).  RayTrix. 2017. 3D Light Field Camera Technology. https:\/\/www.raytrix.de\/. (2017)."},{"key":"e_1_2_2_34_1","unstructured":"Red Camera. 2017. Red Digital Cinema Camera. http:\/\/www.red.com\/. (2017).  Red Camera. 2017. Red Digital Cinema Camera. http:\/\/www.red.com\/. (2017)."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298720"},{"key":"e_1_2_2_36_1","doi-asserted-by":"crossref","unstructured":"Harpreet S Sawhney Yanlin Guo Keith Hanna Rakesh Kumar Sean Adkins and Samuel Zhou. 2001. Hybrid stereo camera: an IBR approach for synthesis of very high resolution stereoscopic image sequences. In ACM SIGGRAPH. 451--460.  Harpreet S Sawhney Yanlin Guo Keith Hanna Rakesh Kumar Sean Adkins and Samuel Zhou. 2001. Hybrid stereo camera: an IBR approach for synthesis of very high resolution stereoscopic image sequences. In ACM SIGGRAPH. 451--460.","DOI":"10.1145\/383259.383312"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2682631"},{"key":"e_1_2_2_38_1","volume-title":"Occlusion-aware Depth Estimation Using Light-field Cameras. In IEEE International Conference on Computer Vision (ICCV). 3487--3495","author":"Wang Ting-Chun","year":"2015","unstructured":"Ting-Chun Wang , Alexei A Efros , and Ravi Ramamoorthi . 2015 . Occlusion-aware Depth Estimation Using Light-field Cameras. In IEEE International Conference on Computer Vision (ICCV). 3487--3495 . Ting-Chun Wang, Alexei A Efros, and Ravi Ramamoorthi. 2015. Occlusion-aware Depth Estimation Using Light-field Cameras. In IEEE International Conference on Computer Vision (ICCV). 3487--3495."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.404"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46487-9_8"},{"key":"e_1_2_2_41_1","volume-title":"The Light Field Attachment: Turning a DSLR into a Light Field Camera Using a Low Budget Camera Ring","author":"Wang Yuwang","year":"2016","unstructured":"Yuwang Wang , Yebin Liu , Wolfgang Heidrich , and Qionghai Dai . 2016a. The Light Field Attachment: Turning a DSLR into a Light Field Camera Using a Low Budget Camera Ring . IEEE Transactions on Visualization and Computer Graphics (TVCG) ( 2016 ). Yuwang Wang, Yebin Liu, Wolfgang Heidrich, and Qionghai Dai. 2016a. The Light Field Attachment: Turning a DSLR into a Light Field Camera Using a Low Budget Camera Ring. IEEE Transactions on Visualization and Computer Graphics (TVCG) (2016)."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.147"},{"key":"e_1_2_2_43_1","volume-title":"HH Kelin Lee, and Mark Horowitz","author":"Wilburn Bennett","year":"2002","unstructured":"Bennett Wilburn , Michael Smulski , HH Kelin Lee, and Mark Horowitz . 2002 . The light field video camera. In SPIE Proc . Media Processors, Vol. 4674 . Bennett Wilburn, Michael Smulski, HH Kelin Lee, and Mark Horowitz. 2002. The light field video camera. In SPIE Proc. Media Processors, Vol. 4674."},{"key":"e_1_2_2_44_1","volume-title":"Light Field Reconstruction Using Deep Convolutional Network on EPI. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Wu Gaochang","year":"2017","unstructured":"Gaochang Wu , Mandan Zhao , Liangyong Wang , Qionghai Dai , Tianyou Chai , and Yebin Liu . 2017 . Light Field Reconstruction Using Deep Convolutional Network on EPI. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Gaochang Wu, Mandan Zhao, Liangyong Wang, Qionghai Dai, Tianyou Chai, and Yebin Liu. 2017. Light Field Reconstruction Using Deep Convolutional Network on EPI. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2015.17"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299004"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_18"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3072959.3073614","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3072959.3073614","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3072959.3073614","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T03:37:23Z","timestamp":1750217843000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3072959.3073614"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,7,20]]},"references-count":47,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2017,8,31]]}},"alternative-id":["10.1145\/3072959.3073614"],"URL":"https:\/\/doi.org\/10.1145\/3072959.3073614","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,7,20]]},"assertion":[{"value":"2017-07-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}