{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,6]],"date-time":"2025-12-06T05:02:05Z","timestamp":1764997325591,"version":"build-2065373602"},"reference-count":54,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2020,10,22]],"date-time":"2020-10-22T00:00:00Z","timestamp":1603324800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2018YFB1600400","2020YFE0201500"],"award-info":[{"award-number":["2018YFB1600400","2020YFE0201500"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"publisher","award":["2020M670730"],"award-info":[{"award-number":["2020M670730"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Mixed Poisson\u2013Gaussian noise exists in the star images and is difficult to be effectively suppressed via maximum likelihood estimation (MLE) method due to its complicated likelihood function. In this article, the MLE method is incorporated with a state-of-the-art machine learning algorithm in order to achieve accurate restoration results. By applying the mixed Poisson\u2013Gaussian likelihood function as the reward function of a reinforcement learning algorithm, an agent is able to form the restored image that achieves the maximum value of the complex likelihood function through the Markov Decision Process (MDP). In order to provide the appropriate parameter settings of the denoising model, the key hyperparameters of the model and their influences on denoising results are tested through simulated experiments. The model is then compared with two existing star image denoising methods so as to verify its performance. The experiment results indicate that this algorithm based on reinforcement learning is able to suppress the mixed Poisson\u2013Gaussian noise in the star image more accurately than the traditional MLE method, as well as the method based on the deep convolutional neural network (DCNN).<\/jats:p>","DOI":"10.3390\/s20215983","type":"journal-article","created":{"date-parts":[[2020,10,22]],"date-time":"2020-10-22T20:51:00Z","timestamp":1603399860000},"page":"5983","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Multi-Frame Star Image Denoising Algorithm Based on Deep Reinforcement Learning and Mixed Poisson\u2013Gaussian Likelihood"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8013-7274","authenticated-orcid":false,"given":"Ming","family":"Xie","sequence":"first","affiliation":[{"name":"Navigation College, Dalian Maritime University, Dalian 116026, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhenduo","family":"Zhang","sequence":"additional","affiliation":[{"name":"Navigation College, Dalian Maritime University, Dalian 116026, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenbo","family":"Zheng","sequence":"additional","affiliation":[{"name":"Navigation College, Dalian Maritime University, Dalian 116026, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ying","family":"Li","sequence":"additional","affiliation":[{"name":"Navigation College, Dalian Maritime University, Dalian 116026, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai","family":"Cao","sequence":"additional","affiliation":[{"name":"Navigation College, Dalian Maritime University, Dalian 116026, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,10,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1016\/j.infrared.2017.08.002","article-title":"Noise suppression algorithm of short-wave infrared star image for daytime star sensor","volume":"85","author":"Wang","year":"2017","journal-title":"Infrared Phys. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1713","DOI":"10.1109\/TIM.2003.818735","article-title":"Modeling and analysis of soft-test\/repair for CCD-based digital X-ray systems","volume":"52","author":"Jin","year":"2003","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1109\/83.370679","article-title":"Adaptive median filters: New algorithms and results","volume":"4","author":"Hwang","year":"1995","journal-title":"IEEE Trans. Image Process."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Yugander, P., Tejaswini, C.H., Meenakshi, J., Samapath kumar, K., Suresh Varma, B.V.N., and Jagannath, M. (2019, January 20\u201323). MR image enhancement using adaptive weighted mean filtering and homomorphic filtering. Proceedings of the International Conference on Computational Intelligence and Data Science, Gurgaon, India. Procedia Computer Science.","DOI":"10.1016\/j.procs.2020.03.334"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"99231","DOI":"10.1109\/ACCESS.2019.2929364","article-title":"Nonlocal means filtering based speckle removal utilizing the maximum a posteriori estimation and the total variation image prior","volume":"7","author":"Zhou","year":"2019","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1364\/OE.25.000168","article-title":"Fourier ptychographic microscopy using a generalized Anscombe transform approximation of the mixed Poisson-Gaussian likelihood","volume":"25","author":"Zhang","year":"2017","journal-title":"Opt. Express"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"5768","DOI":"10.1364\/AO.32.005768","article-title":"Maximum entropy restoration of blurred and oversaturated Hubble space telescope imagery","volume":"32","author":"Bonavito","year":"1993","journal-title":"Appl. Opt."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"729","DOI":"10.1109\/78.80894","article-title":"Maximum likelihood blur identification and image restoration using the EM algorithm","volume":"39","author":"Katsaggelos","year":"1991","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_9","unstructured":"White, R.L., and Allen, R.J. (1990). Iterative maximum-likelihood and Bayesian algorithms for image reconstruction in astronomy. Restoration of Hubble Space Telescope Images, Space Telescope Science Institute."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1014","DOI":"10.1364\/JOSAA.10.001014","article-title":"Image recovery from data acquired with a charge-coupled-device camera","volume":"10","author":"Snyder","year":"1993","journal-title":"J. Opt. Soc. Am. A"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1088\/0266-5611\/24\/3\/035016","article-title":"The study of an iterative method for the reconstruction of images corrupted by Poisson and Gaussian noise","volume":"24","author":"Benvenuto","year":"2008","journal-title":"Inverse Probl."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Li, D., Sun, C., Yang, J., Liu, H., Peng, J., and Zhang, L. (2017). Robust Multi-Frame Adaptive Optics Image Restoration Algorithm Using Maximum Likelihood Estimation with Poisson Statistics. Sensors, 17.","DOI":"10.3390\/s17040785"},{"key":"ref_13","unstructured":"Omidvar, O., and Elliott, D.L. (1997). Reinforcement Learning. Neural Systems for Control, Academic Press."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An. Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_15","first-page":"679","article-title":"A Markov Decision Process","volume":"6","author":"Bellman","year":"1957","journal-title":"J. Math. Fluid Mech."},{"key":"ref_16","unstructured":"Puterman, M.L. (2014). Markov Decision Processes: Discrete Stochastic Programing, John Wiley & Sons, Inc."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016, January 8\u201316). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_19","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7\u201312). Faster-RCNN: Towards real-time object detection with region proposal networks. Proceedings of the Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhang, P., Cheng, H., Zou, B., Dai, P., and Ye, C. (2019, January 22\u201324). Load data mining based on deep learning method. Proceedings of the 3rd International Conference on Computer Science and Application Engineering, Sanya, China.","DOI":"10.1145\/3331453.3361279"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1007\/s10462-018-09679-z","article-title":"Machine learning and deep learning for frameworks and libraries for large-scale data mining: A survey","volume":"52","author":"Nguyen","year":"2019","journal-title":"Artif. Intell. Rev."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4608","DOI":"10.1109\/TIP.2018.2839891","article-title":"FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising","volume":"27","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"172231","DOI":"10.1109\/ACCESS.2019.2956508","article-title":"A survey on the new generation of deep learning in image processing","volume":"7","author":"Jiao","year":"2019","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Nah, J., Kim, T.H., and Lee, K.M. (2017, January 21\u201326). Deep multi-scale convolutional neural network for dynamic scene deblurring. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.35"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sun, J., Cao, W., Xu, Z., and Ponce, J. (2015, January 7\u201312). Learning a convolutional neural network for non-uniform motion blur removal. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298677"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Xu, L., Tao, X., and Jia, J. (2014, January 5\u201312). Inverse kernels for fast spatial deconvolution. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_3"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, Y., Yu, W., and Pock, T. (2015, January 7\u201312). On learning optimized reaction diffusion processes for effective image restoration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299163"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Lefkimmiatis, S. (2017, January 21\u201326). Non-local color image denoising with convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.623"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3142","DOI":"10.1109\/TIP.2017.2662206","article-title":"Beyond a Gaussian denoiser: Residual learning of Deep CNN for image denoising","volume":"26","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Dong, C., Deng, Y., Loy, C.C., and Tang, X. (2015, January 7\u201312). Compression artifacts reduction by a deep convolutional network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/ICCV.2015.73"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Guo, J., and Chao, H. (2016, January 8\u201316). Building dual-domain representations for compression artifacts reduction. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_38"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wang, Z., Liu, D., Chang, S., Ling, Q., Yang, Y., and Huang, T.S. (2016, January 27\u201330). D3: Deep dual-domain based fast restoration of JPEG-compressed images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.302"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","article-title":"Image super-resolution using deep convolutional networks","volume":"38","author":"Dong","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Hui, T.W., Loy, C.C., and Tang, X. (2016, January 8\u201316). Depth map super-resolution by deep multi-scale guidance. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46487-9_22"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Kim, J., Lee, J.K., and Lee, K.M. (2016, January 27\u201330). Accurate image super-resolution using very deep convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.182"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Kim, J., Lee, J.K., and Lee, K.M. (2016, January 27\u201330). Deeply-recursive convolutional network for image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.181"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Haris, M., Shakhnarovich, G., and Ukita, N. (2018, January 18\u201323). Deep back-projection networks for super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00179"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Tai, Y., Yang, J., and Liu, X. (2017, January 21\u201326). Image super-resolution via deep recursive residual network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.298"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Tai, Y., Yang, J., Liu, X., and Xu, C. (2017, January 22\u201329). Memnet: A persistent memory network for image restoration. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.486"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wang, X., Yu, K., Dong, C., and Loy, C.C. (2018, January 18\u201323). Recovering realistic texture in image super-resolution by deep spatial feature transform. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00070"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1109\/TMM.2019.2951462","article-title":"Deep reinforcement learning for image hashing","volume":"22","author":"Peng","year":"2020","journal-title":"IEEE Trans. Multimed."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Yu, K., Dong, C., Lin, L., and Loy, C.C. (2018, January 18\u201323). Crafting a toolchain for image restoration by deep reinforcement learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00259"},{"key":"ref_45","first-page":"1","article-title":"Neural network-based noise suppression algorithm for star images captured during daylight hours","volume":"39","author":"Liu","year":"2019","journal-title":"Acta Opt. Sin."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"2662","DOI":"10.1137\/15M1014395","article-title":"A convex approach for image restoration with exact Poisson\u2013Gaussian likelihood","volume":"8","author":"Chouzenoux","year":"2015","journal-title":"SIAM J. Imaging Sci."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Marnissi, Y., Zheng, Y., and Pesquet, J.C. (2016, January 20\u201325). Fast variational Bayesian signal recovery in the presence of Poisson-Gaussian noise. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Shanghai, China.","DOI":"10.1109\/ICASSP.2016.7472421"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Zhang, J., Hirakawa, K., and Jin, X. (2015, January 19\u201324). Quantile analysis of image sensor noise distribution. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Brisbane, QLD, Australia.","DOI":"10.1109\/ICASSP.2015.7178240"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Xu, Q., Zhao, C., and Li, X. (2016, January 20\u201325). Stellar radiation modeling and image simulation for airborne daytime star sensor. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Shanghai, China.","DOI":"10.1109\/SIPROCESS.2016.7888339"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Sundermeyer, M., Schluter, R., and Ney, H. (2012, January 9\u201313). LSTM neural networks for language modeling. Proceedings of the InterSpeech, Portland, OR, USA.","DOI":"10.21437\/Interspeech.2012-65"},{"key":"ref_52","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Ghemawat, S., Irving, G., Isard, M., and Kudlur, M. (2016, January 2\u20134). TensorFlow: A system for large-scale machine learning. Proceedings of the 12th USENIX Symposium on Operating System Design and Implementation (OSDI), Savannah, GA, USA."},{"key":"ref_53","unstructured":"Kingma, D., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the IEEE International Conference on Learning Representation, San Diego, CA, USA."},{"key":"ref_54","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted Boltzmann machines. Proceedings of the International Conference on Machine Learning, Haifa, Israel."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/5983\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:26:13Z","timestamp":1760178373000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/5983"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,22]]},"references-count":54,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["s20215983"],"URL":"https:\/\/doi.org\/10.3390\/s20215983","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,10,22]]}}}