{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T15:38:00Z","timestamp":1778600280538,"version":"3.51.4"},"reference-count":53,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2021,5,8]],"date-time":"2021-05-08T00:00:00Z","timestamp":1620432000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2019YFB2204300"],"award-info":[{"award-number":["2019YFB2204300"]}]},{"name":"National Natural Science Foundation of China","award":["Grant U20A20205"],"award-info":[{"award-number":["Grant U20A20205"]}]},{"name":"Strategic Priority Research Program of Chinese Academy of Science","award":["XDB32050200"],"award-info":[{"award-number":["XDB32050200"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Image demosaicking has been an essential and challenging problem among the most crucial steps of image processing behind image sensors. Due to the rapid development of intelligent processors based on deep learning, several demosaicking methods based on a convolutional neural network (CNN) have been proposed. However, it is difficult for their networks to run in real-time on edge computing devices with a large number of model parameters. This paper presents a compact demosaicking neural network based on the UNet++ structure. The network inserts densely connected layer blocks and adopts Gaussian smoothing layers instead of down-sampling operations before the backbone network. The densely connected blocks can extract mosaic image features efficiently by utilizing the correlation between feature maps. Furthermore, the block adopts depthwise separable convolutions to reduce the model parameters; the Gaussian smoothing layer can expand the receptive fields without down-sampling image size and discarding image information. The size constraints on the input and output images can also be relaxed, and the quality of demosaicked images is improved. Experiment results show that the proposed network can improve the running speed by 42% compared with the fastest CNN-based method and achieve comparable reconstruction quality as it on four mainstream datasets. Besides, when we carry out the inference processing on the demosaicked images on typical deep CNN networks, Mobilenet v1 and SSD, the accuracy can also achieve 85.83% (top 5) and 75.44% (mAP), which performs comparably to the existing methods. The proposed network has the highest computing efficiency and lowest parameter number through all methods, demonstrating that it is well suitable for applications on modern edge computing devices.<\/jats:p>","DOI":"10.3390\/s21093265","type":"journal-article","created":{"date-parts":[[2021,5,10]],"date-time":"2021-05-10T02:54:58Z","timestamp":1620615298000},"page":"3265","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["A Compact High-Quality Image Demosaicking Neural Network for Edge-Computing Devices"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4855-7943","authenticated-orcid":false,"given":"Shuyu","family":"Wang","sequence":"first","affiliation":[{"name":"State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"},{"name":"Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5148-4218","authenticated-orcid":false,"given":"Mingxin","family":"Zhao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"},{"name":"Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Runjiang","family":"Dou","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"},{"name":"Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuangming","family":"Yu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"},{"name":"Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liyuan","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"},{"name":"Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nanjian","family":"Wu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"},{"name":"Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing 100083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,5,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1260","DOI":"10.1109\/TCE.2005.1561853","article-title":"Color filter arrays: Design and performance analysis","volume":"51","author":"Lukac","year":"2005","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_2","first-page":"44","article-title":"Demosaicking: Color filter array interpolation","volume":"17","author":"Wang","year":"2005","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_3","unstructured":"Malvar, H.S., He, L.W., and Cutler, R. (2004, January 17\u201321). High-quality linear interpolation for demosaicing of Bayer-patterned color images. Proceedings of the 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Montreal, QC, Canada."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wang, D., Yu, G., Zhou, X., and Wang, C. (2017, January 16\u201319). Image demosaicking for Bayer-patterned CFA images using improved linear interpolation. Proceedings of the 7th International Conference on Innovation in Science and Technology, Da Nang, Vietnam.","DOI":"10.1109\/ICIST.2017.7926804"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kiku, D., Monno, Y., Tanaka, M., and Okutomi, M. (2013, January 15\u201318). Residual interpolation for color image demosaicking. Proceedings of the 2013 IEEE International Conference on Image Processing, Melbourne, Australia.","DOI":"10.1109\/ICIP.2013.6738475"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"90230L","DOI":"10.1117\/12.2038425","article-title":"Minimized-Laplacian residual interpolation for color image demosaicking","volume":"9023","author":"Kiku","year":"2014","journal-title":"Digit. Photogr. X"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Monno, Y., Kiku, D., Tanaka, M., and Okutomi, M. (2017). Adaptive residual interpolation for color and multispectral image demosaicking. Sensors, 17.","DOI":"10.3390\/s17122787"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2356","DOI":"10.1109\/TIP.2013.2244215","article-title":"A compressive framework for demosaicing of natural images","volume":"22","author":"Moghadam","year":"2013","journal-title":"IEEE Trans. Image Process."},{"key":"ref_9","first-page":"9","article-title":"A Survey of Demosaicing: Issues and Challenges","volume":"2","author":"Kaur","year":"2015","journal-title":"Int. J. Sci. Eng. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2980179.2982399","article-title":"Deep joint demosaicking and denoising","volume":"35","author":"Gharbi","year":"2016","journal-title":"ACM Trans. Graph."},{"key":"ref_11","unstructured":"Tan, R., Zhang, K., Zuo, W., and Zhang, L. (2017, January 10\u201314). Color image demosaicking via deep residual learning. Proceedings of the 2017 IEEE International Conference on Multimedia and Expo (ICME), Hong Kong, China."},{"key":"ref_12","unstructured":"Syu, N.-S., Chen, Y.-S., and Chuang, Y.-Y. (2018, February 11). Learning Deep Convolutional Networks for Demosaicing. Available online: http:\/\/arxiv.org\/abs\/1802.03769."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kokkinos, F., and Lefkimmiatis, S. (2018, January 8\u201314). Deep image demosaicking using a cascade of convolutional residual denoising networks. Proceedings of the 2018 European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_19"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Cui, K., Jin, Z., and Steinbach, E. (2018, January 7\u201310). Color image demosaicking using a 3-stage convolutional neural network structure. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.","DOI":"10.1109\/ICIP.2018.8451020"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Podili, A., Zhang, C., and Prasanna, V. (2017, January 10\u201312). Fast and efficient implementation of Convolutional Neural Networks on FPGA. Proceedings of the 2017 IEEE 28th International Conference on Application-Specific Systems, Architectures and Processor, Seatlle, WA, USA.","DOI":"10.1109\/ASAP.2017.7995253"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Shafiee, A., Nag, A., Muralimanohar, N., Balasubramonian, R., Strachan, J.P., Hu, M., Williams, R.S., and Srikumar, V. (2016, January 18\u201322). ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars. Proceedings of the 2016 ACM\/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), Seoul, Korea.","DOI":"10.1109\/ISCA.2016.12"},{"key":"ref_17","unstructured":"Li, H., Fan, X., Jiao, L., Cao, W., Zhou, X., and Wang, L. (September, January 29). A high performance FPGA-based accelerator for large-scale convolutional neural networks. Proceedings of the 2016 26th International Conference on Field Programmable Logic and Applications (FPL), Lausanne, Switzerland."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2067","DOI":"10.1109\/JSSC.2014.2332134","article-title":"A 1000 fps vision chip based on a dynamically reconfigurable hybrid architecture comprising a PE array processor and self-organizing map neural network","volume":"49","author":"Shi","year":"2014","journal-title":"IEEE J. Solid-State Circuits"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Li, H., Zhang, Z., Yang, J., Liu, L., and Wu, N. (2015, January 3\u20135). A novel vision chip architecture for image recognition based on convolutional neural network. Proceedings of the 2015 IEEE 11th International Conference on ASIC, Chengdu, China.","DOI":"10.1109\/ASICON.2015.7516878"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11432-017-9303-0","article-title":"Neuromorphic vision chips","volume":"61","author":"Wu","year":"2018","journal-title":"Sci. China Inf. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1109\/TCSVT.2016.2618753","article-title":"A heterogeneous parallel processor for high-speed vision chip","volume":"28","author":"Yang","year":"2018","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2415","DOI":"10.1109\/TIP.2018.2883815","article-title":"Low Cost Edge Sensing for High Quality Demosaicking","volume":"28","author":"Niu","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1007\/978-3-319-10593-2_13","article-title":"Learning a deep convolutional network for image super-resolution","volume":"Volume 8692","author":"Dong","year":"2014","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1007\/978-3-319-46475-6_25","article-title":"Accelerating the super-resolution convolutional neural network","volume":"Volume 9906","author":"Dong","year":"2016","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Habtegebrial, T.A., Reis, G., and Stricker, D. (2019, January 24\u201326). Deep convolutional networks for snapshot hypercpectral demosaicking. Proceedings of the 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS), Amterdam, The Netherlands.","DOI":"10.1109\/WHISPERS.2019.8921273"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s00138-018-0965-4","article-title":"Hyperspectral demosaicking and crosstalk correction using deep learning","volume":"30","author":"Dijkstra","year":"2019","journal-title":"Mach. Vis. Appl."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Pan, Z., Li, B., Bao, Y., and Cheng, H. (2019, January 24\u201326). Deep panchromatic image guided residual interpolation for multispectral image demosaicking. Proceedings of the 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS), Amterdam, The Netherlands.","DOI":"10.1109\/WHISPERS.2019.8920868"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhou, Z., and Siddiquee, M.R. (2018). UNet++: A Nested U-Net architecture for medical image segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-030-00889-5_1"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Park, B., Yu, S., and Jeong, J. (2019, January 16\u201320). Densely connected hierarchical network for image denoising. Proceedings of the 2019 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.","DOI":"10.1109\/CVPRW.2019.00263"},{"key":"ref_31","first-page":"2899","article-title":"RGB-NIR image demosaicing based on deep learning","volume":"39","author":"Xie","year":"2019","journal-title":"J. Comput. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Shopovska, I., Jovanov, L., and Philips, W. (2018, January 20\u201321). RGB-NIR Demosaicing Using Deep Residual U-Net. Proceedings of the 2018 26th Telecommunications Forum (TELFOR), Belgrade, Serbia.","DOI":"10.1109\/TELFOR.2018.8611819"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"4177","DOI":"10.1109\/TIP.2019.2905991","article-title":"Iterative joint image demosaicking and denoising using a residual denoising network","volume":"28","author":"Kokkinos","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhou, R., Achanta, R., and S\u00fcsstrunk, S. (2018, January 12\u201316). Deep residual network for joint demosaicing and super-resolutionin. Proceedings of the 26th Color and Imaging Conference, Society for Imaging Science and Technology, Vancouver, BC, Canada.","DOI":"10.2352\/ISSN.2169-2629.2018.26.75"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1109\/TCI.2016.2644865","article-title":"Loss functions for image restoration with neural networks","volume":"3","author":"Zhao","year":"2016","journal-title":"IEEE Trans. Comput. Imaging"},{"key":"ref_36","first-page":"1","article-title":"Perceptual Image Quality Assessment: A Survey","volume":"63","author":"Guangtao","year":"2020","journal-title":"Sci. China Inf. Sci."},{"key":"ref_37","first-page":"12","article-title":"Appropriate number of standard 2 \u00d7 2 max pooling layers and their allocation in convolutional neural networks for diverse and heterogeneous datasets","volume":"20","author":"Romanuke","year":"2018","journal-title":"Inf. Technol. Manag. Sci."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_39","unstructured":"Wang, X., Yu, K., Wu, S., Gu, J., and Liu, Y. (2018, January 8\u201314). ESRGAN: Enhanced super-resolution generative adversarial networks. Proceedings of the 2018 European Conference on Computer Vision, Munich, Germany."},{"key":"ref_40","unstructured":"Howard, A.G., Zku, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017, April 17). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. Available online: http:\/\/arxiv.org\/abs\/1704.04861."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7\u201313). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Fei-fei, L. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_43","unstructured":"Kingma, D.P., and Ba, J.L. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_44","first-page":"68221J","article-title":"Image demosaicing: A systematic survey","volume":"6822","author":"Li","year":"2008","journal-title":"Vis. Commun. Image Process."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"023016","DOI":"10.1117\/1.3600632","article-title":"Color demosaicking by local directional interpolation and nonlocal adaptive thresholding","volume":"20","author":"Zhang","year":"2011","journal-title":"J. Electron. Imaging"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Huang, J.B., Singh, A., and Ahuja, N. (2015, January 7\u201312). Single Image Super-resolution from Transformed Self-Exemplars. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299156"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"21811","DOI":"10.1007\/s11042-016-4020-z","article-title":"Sketch-based manga retrieval using manga109 dataset","volume":"76","author":"Matsui","year":"2017","journal-title":"Multimed. Tools Appl."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1109\/TIP.2004.838691","article-title":"Adaptive homogeneity-directed demosaicing algorithm","volume":"14","author":"Hirakawa","year":"2005","journal-title":"IEEE Trans. Image Process."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"2167","DOI":"10.1109\/TIP.2005.857260","article-title":"Color demosaicking via directional linear minimum mean square-error estimation","volume":"14","author":"Zhang","year":"2005","journal-title":"IEEE Trans. Image Process."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1016\/j.image.2011.04.003","article-title":"Color image demosaicking: An overview","volume":"26","author":"Menon","year":"2011","journal-title":"Signal Process. Image Commun."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1016\/j.sysarc.2019.01.011","article-title":"A Survey on optimized implementation of deep learning models on the NVIDIA Jetson platform","volume":"97","author":"Mittal","year":"2019","journal-title":"J. Syst. Archit."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016). SSD: Single shot multibox detector. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Cartucho, J., Ventura, R., and Veloso, M. (2018, January 1\u20135). Robust object recognition through symbiotic deep learning in mobile robots. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594067"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/9\/3265\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:58:16Z","timestamp":1760162296000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/9\/3265"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,8]]},"references-count":53,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,5]]}},"alternative-id":["s21093265"],"URL":"https:\/\/doi.org\/10.3390\/s21093265","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,8]]}}}