{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,18]],"date-time":"2026-01-18T13:21:45Z","timestamp":1768742505002,"version":"3.49.0"},"reference-count":51,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,3,19]],"date-time":"2021-03-19T00:00:00Z","timestamp":1616112000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["NRF-2018R1D1A3B07044041"],"award-info":[{"award-number":["NRF-2018R1D1A3B07044041"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["NRF-2020R1A2C1101258"],"award-info":[{"award-number":["NRF-2020R1A2C1101258"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion","doi-asserted-by":"publisher","award":["IITP-2020-0-01462"],"award-info":[{"award-number":["IITP-2020-0-01462"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The integral imaging microscopy system provides a three-dimensional visualization of a microscopic object. However, it has a low-resolution problem due to the fundamental limitation of the F-number (the aperture stops) by using micro lens array (MLA) and a poor illumination environment. In this paper, a generative adversarial network (GAN)-based super-resolution algorithm is proposed to enhance the resolution where the directional view image is directly fed as input. In a GAN network, the generator regresses the high-resolution output from the low-resolution input image, whereas the discriminator distinguishes between the original and generated image. In the generator part, we use consecutive residual blocks with the content loss to retrieve the photo-realistic original image. It can restore the edges and enhance the resolution by \u00d72, \u00d74, and even \u00d78 times without seriously hampering the image quality. The model is tested with a variety of low-resolution microscopic sample images and successfully generates high-resolution directional view images with better illumination. The quantitative analysis shows that the proposed model performs better for microscopic images than the existing algorithms.<\/jats:p>","DOI":"10.3390\/s21062164","type":"journal-article","created":{"date-parts":[[2021,3,21]],"date-time":"2021-03-21T23:47:41Z","timestamp":1616370461000},"page":"2164","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Super-Resolution Enhancement Method Based on Generative Adversarial Network for Integral Imaging Microscopy"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8413-5428","authenticated-orcid":false,"given":"Md. Shahinur","family":"Alam","sequence":"first","affiliation":[{"name":"Department of Computer and Communication Engineering, Chungbuk National University, Cheongju, Chungbuk 28644, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0334-504X","authenticated-orcid":false,"given":"Ki-Chul","family":"Kwon","sequence":"additional","affiliation":[{"name":"Department of Computer and Communication Engineering, Chungbuk National University, Cheongju, Chungbuk 28644, Korea"}]},{"given":"Munkh-Uchral","family":"Erdenebat","sequence":"additional","affiliation":[{"name":"Department of Computer and Communication Engineering, Chungbuk National University, Cheongju, Chungbuk 28644, Korea"}]},{"given":"Mohammed","family":"Y. Abbass","sequence":"additional","affiliation":[{"name":"Department of Computer and Communication Engineering, Chungbuk National University, Cheongju, Chungbuk 28644, Korea"}]},{"given":"Md. Ashraful","family":"Alam","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, BRAC University, Dhaka 1212, Bangladesh"}]},{"given":"Nam","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Computer and Communication Engineering, Chungbuk National University, Cheongju, Chungbuk 28644, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1215","DOI":"10.1038\/s41592-019-0458-z","article-title":"Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction","volume":"16","author":"Belthangady","year":"2019","journal-title":"Nat. Methods"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Palmieri, L., Scrofani, G., Incardona, N., Saavedra, G., Mart\u00ednez-Corral, M., and Koch, R. (2019). Robust Depth Estimation for Light Field Microscopy. Sensors, 19.","DOI":"10.3390\/s19030500"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.mimet.2016.03.002","article-title":"Confocal microscopy imaging of the biofilm matrix","volume":"138","author":"Schlafer","year":"2017","journal-title":"J. Microbiol. Methods"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1323","DOI":"10.1038\/s41592-019-0622-5","article-title":"Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning","volume":"16","author":"Wu","year":"2019","journal-title":"Nat. Methods"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/JPHOT.2018.2890429","article-title":"Resolution-Enhancement for an Integral Imaging Microscopy Using Deep Learning","volume":"11","author":"Kwon","year":"2019","journal-title":"IEEE Photonics J."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"10210","DOI":"10.1364\/OE.22.010210","article-title":"Real-time integral imaging system for light field microscopy","volume":"22","author":"Kim","year":"2014","journal-title":"Opt. Express"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"060005","DOI":"10.3788\/COL201412.060005","article-title":"Advances in the light field displays based on integral imaging and holographic techniques (Invited Paper)","volume":"12","author":"Kim","year":"2014","journal-title":"Chin. Opt. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1230","DOI":"10.1364\/OL.29.001230","article-title":"Three-dimensional integral imaging of micro-objects","volume":"29","author":"Jang","year":"2004","journal-title":"Opt. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Levoy, M., Ng, R., Adams, A., Footer, M., and Horowitz, M. (August, January 30). Light field microscopy. Proceedings of the ACM SIGGRAPH 2006 Papers, SIGGRAPH\u201906, Boston, MA, USA.","DOI":"10.1145\/1179352.1141976"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"19253","DOI":"10.1364\/OE.17.019253","article-title":"Resolution-enhanced integral imaging microscopy that uses lens array shifting","volume":"17","author":"Lim","year":"2009","journal-title":"Opt. Express"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"30503","DOI":"10.1364\/OE.25.030503","article-title":"Enhancement of the depth-of-field of integral imaging microscope by using switchable bifocal liquid-crystalline polymer micro lens array","volume":"25","author":"Kwon","year":"2017","journal-title":"Opt. Express"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4450","DOI":"10.1364\/AO.53.004450","article-title":"Real-time interactive display for integral imaging microscopy","volume":"53","author":"Kwon","year":"2014","journal-title":"Appl. Opt."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1364\/OL.27.000324","article-title":"Improved viewing resolution of three-dimensional integral imaging by use of nonstationary micro-optics","volume":"27","author":"Jang","year":"2002","journal-title":"Opt. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3528","DOI":"10.1364\/OE.11.003528","article-title":"Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging","volume":"11","author":"Kishk","year":"2003","journal-title":"Opt. Express"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.1364\/OPTICA.4.001437","article-title":"Deep learning microscopy","volume":"4","author":"Rivenson","year":"2017","journal-title":"Optica"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"825","DOI":"10.1109\/JPROC.2017.2655260","article-title":"Recent Advances in the Capture and Display of Macroscopic and Microscopic 3-D Scenes by Integral Imaging","volume":"105","author":"Dorado","year":"2017","journal-title":"Proc. IEEE"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Alam, S., Kwon, K.-C., Erdenebat, M.-U., Lim, Y.-T., Imtiaz, S., Sufian, M.A., Jeon, S.-H., and Kim, N. (2020, January 2\u20136). Resolution Enhancement of an Integral Imaging Microscopy Using Generative Adversarial Network. Proceedings of the 14th Pacific Rim Conference on Lasers and Electro-Optics (CLEO PR 2020) (2020), paper C3G_4, The Optical Society, Sydney, Australia.","DOI":"10.1364\/CLEOPR.2020.C3G_4"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5592","DOI":"10.1364\/AO.40.005592","article-title":"High-resolution digital integral photography by use of a scanning microlens array","volume":"40","author":"Erdmann","year":"2001","journal-title":"Appl. Opt."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"736","DOI":"10.1364\/BOE.6.000736","article-title":"Resolution-enhancement for an orthographic-view image display in an integral imaging microscope system","volume":"6","author":"Kwon","year":"2015","journal-title":"Biomed. Opt. Express"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kwon, H., Yoon, H., and Park, K.-W. (2020). CAPTCHA Image Generation: Two-Step Style-Transfer Learning in Deep Neural Networks. Sensors, 20.","DOI":"10.3390\/s20051495"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"879","DOI":"10.1587\/transinf.2019EDL8194","article-title":"Robust CAPTCHA image generation enhanced with adversarial example methods","volume":"103","author":"Kwon","year":"2020","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.inffus.2017.10.006","article-title":"A survey on deep learning for big data","volume":"42","author":"Zhang","year":"2018","journal-title":"Inf. Fusion"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Alam, M.S., Kwon, K.-C., Alam, M.A., Abbass, M.Y., Imtiaz, S.M., and Kim, N. (2020). Trajectory-Based Air-Writing Recognition Using Deep Neural Network and Depth Sensor. Sensors, 20.","DOI":"10.3390\/s20020376"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Xie, J., Girshick, R., and Farhadi, A. (2016). Deep3D: Fully Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networks, Springer. European Conference on Computer Vision.","DOI":"10.1007\/978-3-319-46493-0_51"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Nguyen-Phuoc, T., Li, C., Theis, L., Richardt, C., and Yang, Y.L. (November, January 27). HoloGAN: Unsupervised learning of 3D representations from natural images. Proceedings of the 2019 International Conference on Computer Vision Workshop, ICCVW 2019, Seoul, Korea.","DOI":"10.1109\/ICCVW.2019.00255"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3106","DOI":"10.1109\/TMM.2019.2919431","article-title":"Deep Learning for Single Image Super-Resolution: A Brief Review","volume":"21","author":"Yang","year":"2019","journal-title":"IEEE Trans. Multimed."},{"key":"ref_28","first-page":"1","article-title":"Image super resolution based on residual dense CNN and guided filters","volume":"80","author":"Abbass","year":"2020","journal-title":"Multimed. Tools Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Mei, Y., Fan, Y., Zhou, Y., Huang, L., Huang, T.S., and Shi, H. (2020, January 13\u201319). Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00573"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"107475","DOI":"10.1016\/j.patcog.2020.107475","article-title":"Hierarchical dense recursive network for image super-resolution","volume":"107","author":"Jiang","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image Super-Resolution Using Deep Convolutional Networks","volume":"38","author":"Dong","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"5835","DOI":"10.1109\/CVPR.2017.618","article-title":"Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution","volume":"Volume 2017","author":"Lai","year":"2017","journal-title":"Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhang, K., Zuo, W., and Zhang, L. (2018, January 18\u201323). Learning a Single Convolutional Super-Resolution Network for Multiple Degradations. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00344"},{"key":"ref_35","unstructured":"Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative Adversarial Nets. Proceedings of the International Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_36","unstructured":"Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., and Abbeel, P. (2016). InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. arXiv, (preprint)."},{"key":"ref_37","unstructured":"Radford, A., Metz, L., and Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv, (preprint)."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zhu, J.-Y., Park, T., Isola, P., and Efros, A.A. (2017). Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. Proc. IEEE Int. Conf. Comput. Vis., 2223\u20132232.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (July, January 26). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2017.19"},{"key":"ref_40","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified Linear Units Improve Restricted Boltzmann Machines. Proceedings of the 27th International Conference on International Conference on Machine Learning, Haifa, Israel."},{"key":"ref_41","unstructured":"Xu, B., Wang, N., Chen, T., and Li, M. (2015). Empirical Evaluation of Rectified Activations in Convolutional Network. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7\u201313). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref_43","first-page":"448","article-title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","volume":"Volume 1","author":"Ioffe","year":"2015","journal-title":"Proceedings of the 32nd International Conference on Machine Learning, ICML 2015"},{"key":"ref_44","first-page":"1874","article-title":"Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network","volume":"Volume 2016","author":"Shi","year":"2016","journal-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition"},{"key":"ref_45","unstructured":"Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013, January 16\u201321). Rectifier nonlinearities improve neural network acoustic models. Proceedings of the in ICML Workshop on Deep Learning for Audio, Speech and Language Processing, Atlanta, GA, USA."},{"key":"ref_46","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_47","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_48","unstructured":"(2020, June 15). The PASCAL Visual Object Classes Challenge (VOC2012). Available online: http:\/\/host.robots.ox.ac.uk\/pascal\/VOC\/voc2012\/."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1007\/978-81-322-2247-7_38","article-title":"Analysis and Evaluation of Image Quality Metrics","volume":"340","author":"Mandal","year":"2015","journal-title":"Adv. Intell. Syst. Comput."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Wang, Z., Chen, J., and Hoi, S.C.H. (2020). Deep Learning for Image Super-resolution: A Survey. IEEE Trans. Pattern Anal. Mach. Intell., 1.","DOI":"10.1109\/TPAMI.2021.3069908"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2164\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:38:21Z","timestamp":1760161101000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2164"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,19]]},"references-count":51,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["s21062164"],"URL":"https:\/\/doi.org\/10.3390\/s21062164","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,19]]}}}