{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:09:41Z","timestamp":1760234981475,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2021,7,15]],"date-time":"2021-07-15T00:00:00Z","timestamp":1626307200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008316","name":"Baden-W\u00fcrttemberg Stiftung","doi-asserted-by":"publisher","award":["Komo3D"],"award-info":[{"award-number":["Komo3D"]}],"id":[{"id":"10.13039\/100008316","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Plenoptic camera based system captures the light-field that can be exploited to estimate the 3D depth of the scene. This process generally consists of a significant number of recurrent operations, and thus requires high computation power. General purpose processor based system, due to its sequential architecture, consequently results in the problem of large execution time. A desktop graphics processing unit (GPU) can be employed to resolve this problem. However, it is an expensive solution with respect to power consumption and therefore cannot be used in mobile applications with low energy requirements. In this paper, we propose a modified plenoptic depth estimation algorithm that works on a single frame recorded by the camera and respective FPGA based hardware design. For this purpose, the algorithm is modified for parallelization and pipelining. In combination with efficient memory access, the results show good performance and lower power consumption compared to other systems.<\/jats:p>","DOI":"10.3390\/a14070215","type":"journal-article","created":{"date-parts":[[2021,7,15]],"date-time":"2021-07-15T09:32:07Z","timestamp":1626341527000},"page":"215","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Design of an FPGA Hardware Optimizing the Performance and Power Consumption of a Plenoptic Camera Depth Estimation Algorithm"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4056-3647","authenticated-orcid":false,"given":"Faraz","family":"Bhatti","sequence":"first","affiliation":[{"name":"Institute of Smart Systems and Services (IoS<sup>3<\/sup>), Pforzheim University, Tiefenbronner Strasse 65, 75175 Pforzheim, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6202-3345","authenticated-orcid":false,"given":"Thomas","family":"Greiner","sequence":"additional","affiliation":[{"name":"Institute of Smart Systems and Services (IoS<sup>3<\/sup>), Pforzheim University, Tiefenbronner Strasse 65, 75175 Pforzheim, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1109\/34.121783","article-title":"Single lens stereo with a plenoptic camera","volume":"14","author":"Adelson","year":"1992","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Fiss, J., Curless, B., and Szeliski, R. (2014, January 2\u20134). Refocusing plenoptic images using depth-adaptive splatting. Proceedings of the 2014 IEEE International Conference on Computational Photography (ICCP), Santa Clara, CA, USA.","DOI":"10.1109\/ICCPHOT.2014.6831809"},{"key":"ref_3","unstructured":"Vaish, V., Wilburn, B., Joshi, N., and Levoy, M. (July, January 27). Using plane + parallax for calibrating dense camera arrays. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA."},{"key":"ref_4","unstructured":"Yang, J.C., Everett, M., Buehler, C., and McMillan, L. (2002). A Real-time distributed light field camera. Proceedings of the 13th Eurographics Workshop on Rendering, Eurographics Association."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Bizai, G., Peiretti, F., Salvatelli, A., and Drozdowicz, B. (2017, January 4\u20138). Algorithms for codification, multiperspective and refocusing of light fields: Implementation and evaluation. Proceedings of the 2017 XLIII Latin American Computer Conference (CLEI), Cordoba, Argentina.","DOI":"10.1109\/CLEI.2017.8226403"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zeller, N., Quint, F., and Stilla, U. (2015, January 19\u201322). Establishing a probabilistic depth map from focused plenoptic cameras. Proceedings of the 2015 International Conference on 3D Vision, Lyon, France.","DOI":"10.1109\/3DV.2015.18"},{"key":"ref_7","first-page":"45","article-title":"Single lens 3D-camera with extended depth-of-field","volume":"Volume 8291","author":"Rogowitz","year":"2012","journal-title":"Human Vision and Electronic Imaging XVII"},{"key":"ref_8","unstructured":"Zeller, N. (2018). Direct Plenoptic Odometry-Robust Tracking and Mapping with a Light Field Camera. [Ph.D. Thesis, Technische Universit\u00e4t M\u00fcnchen]. Available online: https:\/\/dgk.badw.de\/fileadmin\/user_upload\/Files\/DGK\/docs\/c-825.pdf."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Collange, S., Defour, D., and Tisserand, A. (2009, January 25\u201327). Power Consumption of GPUs from a Software Perspective. Proceedings of the International Conference on Computational Science, Baton Rouge, LA, USA.","DOI":"10.1007\/978-3-642-01970-8_92"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Engel, G., Bhatti, F., Greiner, T., Heizmann, M., and Quint, F. (2020, January 16\u201321). Distributed and context aware application of deep neural networks in mobile 3D-multi-sensor systems based on cloud-, edge- and FPGA-computing. Proceedings of the 2020 IEEE 7th International Conference on Industrial Engineering and Applications (ICIEA), Bangkok, Thailand.","DOI":"10.1109\/ICIEA49774.2020.9101915"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wang, G., Xiong, Y., Yun, J., and Cavallaro, J.R. (2013, January 26\u201331). Accelerating computer vision algorithms using OpenCL framework on the mobile GPU\u2014A case study. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638132"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Rister, B., Wang, G., Wu, M., and Cavallaro, J.R. (2013, January 26\u201331). A Fast and Efficient Sift Detector Using the Mobile GPU. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638141"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1016\/j.mejo.2005.11.003","article-title":"FPGA vs. ASIC for low power applications","volume":"37","author":"Amara","year":"2006","journal-title":"Microelectron. J."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1109\/TCAD.2006.884574","article-title":"Measuring the Gap Between FPGAs and ASICs","volume":"26","author":"Kuon","year":"2007","journal-title":"IEEE Trans. Comput. Aided Des. Integr. Circuits Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Page, D.S. (2009). A Practical Introduction to Computer Architecture, Springer. Texts in Computer Science.","DOI":"10.1007\/978-1-84882-256-6"},{"key":"ref_16","unstructured":"Johnston, C.T., Gribbon, K.T., and Bailey, D.G. (2021, June 17). Implementing Image Processing Algorithms on FPGAs. Available online: https:\/\/crisp.massey.ac.nz\/pdfs\/2004_ENZCON_118.pdf."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1049\/ecej:19980307","article-title":"Parallel architectures for image processing","volume":"10","author":"Downton","year":"1998","journal-title":"Electron. Commun. Eng. J."},{"key":"ref_18","unstructured":"Xu, Y., Long, Q., Mita, S., Tehrani, H., Ishimaru, K., and Shirai, N. (2016, January 19\u201322). Real-time stereo vision system at nighttime with noise reduction using simplified non-local matching cost. Proceedings of the 2016 IEEE Intelligent Vehicles Symposium (IV), Gothenburg, Sweden."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1696","DOI":"10.1109\/TCSVT.2015.2397196","article-title":"Real-Time High-Quality Stereo Vision System in FPGA","volume":"25","author":"Wang","year":"2015","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_20","first-page":"265","article-title":"Implementing image applications on FPGAs. Object recognition supported by user interaction for service robots","volume":"3","author":"Draper","year":"2002","journal-title":"IEEE Comput. Soc."},{"key":"ref_21","unstructured":"Sugimura, T., Shim, J., Kurino, H., and Koyanagi, M. (2003, January 17). Parallel image processing field programmable gate array for real time image processing system. Proceedings of the 2003 IEEE International Conference on Field-Programmable Technology (FPT) (IEEE Cat. No.03EX798), Tokyo, Japan."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jinghong, D., Yaling, D., and Kun, L. (2007, January 16\u201318). Development of image processing system based on DSP and FPGA. Proceedings of the 2007 8th International Conference on Electronic Measurement and Instruments, Xi\u2019an, China.","DOI":"10.1109\/ICEMI.2007.4350799"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Birla, M. (2006, January 7\u201310). FPGA based reconfigurable platform for complex image processing. Proceedings of the 2006 IEEE International Conference on Electro\/Information Technology, East Lansing, MI, USA.","DOI":"10.1109\/EIT.2006.252111"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Bhatti, F., Greiner, T., Heizmann, M., and Ziebarth, M. (2017, January 5\u20137). A new FPGA based architecture to improve performance of deflectometry image processing algorithm. Proceedings of the 2017 40th International Conference on Telecommunications and Signal Processing (TSP), Barcelona, Spain.","DOI":"10.1109\/TSP.2017.8076049"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"929","DOI":"10.1109\/TCSVT.2013.2290575","article-title":"High Performance Stereo System for Dense 3-D Reconstruction","volume":"24","author":"Michailidis","year":"2014","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1179","DOI":"10.1109\/TCSVT.2018.2825022","article-title":"A Real-Time High-Quality Complete System for Depth Image-Based Rendering on FPGA","volume":"29","author":"Li","year":"2019","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hahne, C., Lumsdaine, A., Aggoun, A., and Velisavljevic, V. (2018). Real-Time Refocusing using an FPGA-based Standard Plenoptic Camera. IEEE Trans. Ind. Electron., 1.","DOI":"10.1109\/TIE.2018.2818644"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"8669","DOI":"10.3390\/s140508669","article-title":"Super-resolution in plenoptic cameras using FPGAs","volume":"14","author":"Magdaleno","year":"2014","journal-title":"Sensors"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Perez Nava, F., and Luke, J.P. (2009, January 4\u20136). Simultaneous estimation of super-resolved depth and all-in-focus images from a plenoptic camera. Proceedings of the 2009 3DTV Conference: The True Vision\u2014 Capture, Transmission and Display of 3D Video, Potsdam, Germany.","DOI":"10.1109\/3DTV.2009.5069675"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Bebis, G., Boyle, R., Parvin, B., Koracin, D., Pavlidis, I., Feris, R., McGraw, T., Elendt, M., Kopper, R., and Ragan, E. (2015). A real-time depth estimation approach for a focused plenoptic camera. Advances in Visual Computing, Springer International Publishing.","DOI":"10.1007\/978-3-319-27857-5"},{"key":"ref_31","first-page":"318","article-title":"Plenoptic rendering with interactive performance using GPUs","volume":"Volume 8295","author":"Egiazarian","year":"2012","journal-title":"Image Processing: Algorithms and Systems X; and Parallel Processing for Imaging Applications II"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chang, C.W., Chen, M.R., Hsu, P.H., and Lu, Y.C. (2014, January 1\u20135). A pixel-based depth estimation algorithm and its hardware implementation for 4-D light field data. Proceedings of the Circuits and Systems (ISCAS), 2014 IEEE International Symposium on, Melbourne, Australia.","DOI":"10.1109\/ISCAS.2014.6865253"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Dom\u00ednguez Conde, C., L\u00fcke, J.P., and Rosa Gonz\u00e1lez, F. (2019). Implementation of a Depth from Light Field Algorithm on FPGA. Sensors, 19.","DOI":"10.3390\/s19163562"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Jiang, X., Hornegger, J., and Koch, R. (2014). Lens-based depth estimation for multi-focus plenoptic cameras. Pattern Recognition, Springer International Publishing.","DOI":"10.1007\/978-3-319-11752-2"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1109\/TCI.2017.2710906","article-title":"An Image Rendering Pipeline for Focused Plenoptic Cameras","volume":"3","author":"Hog","year":"2017","journal-title":"IEEE Trans. Comput. Imaging"},{"key":"ref_36","unstructured":"Georgiev, T., and Lumsdaine, A. (2021, June 17). Depth of Field in Plenoptic Cameras. Available online: https:\/\/diglib.eg.org\/bitstream\/handle\/10.2312\/egs.20091035.005-008\/005-008.pdf?sequence=1&isAllowed=y."},{"key":"ref_37","first-page":"69","article-title":"The multifocus plenoptic camera","volume":"Volume 8299","author":"Battiato","year":"2012","journal-title":"Digital Photography VIII"},{"key":"ref_38","unstructured":"(2021, June 17). Raytrix 3D Light-Field Vision. Raytrix|3D Light Field Camera Technology. Available online: https:\/\/raytrix.de\/."},{"key":"ref_39","unstructured":"Eklundh, J.O. (1994). Non-parametric local transforms for computing visual correspondence. Computer Vision\u2014ECCV \u201994, Springer."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Lai, S.H., Lepetit, V., Nishino, K., and Sato, Y. (2017). A dataset and evaluation methodology for depth estimation on 4D light fields. Computer Vision\u2014ACCV 2016, Springer International Publishing.","DOI":"10.1007\/978-3-319-54190-7"},{"key":"ref_41","unstructured":"(2021, June 17). Synthetic Light Field Archive. Available online: https:\/\/web.media.mit.edu\/~gordonw\/SyntheticLightFields\/."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/7\/215\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:31:06Z","timestamp":1760164266000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/7\/215"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,15]]},"references-count":41,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,7]]}},"alternative-id":["a14070215"],"URL":"https:\/\/doi.org\/10.3390\/a14070215","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2021,7,15]]}}}