{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T12:26:59Z","timestamp":1782131219737,"version":"3.54.5"},"reference-count":50,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T00:00:00Z","timestamp":1722988800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2022YFF0904400"],"award-info":[{"award-number":["2022YFF0904400"]}]},{"name":"Peach and Plum Program of the Palace Museum","award":["2022YFF0904400"],"award-info":[{"award-number":["2022YFF0904400"]}]},{"name":"Vanke Foundation","award":["2022YFF0904400"],"award-info":[{"award-number":["2022YFF0904400"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Painted cultural relics hold significant historical value and are crucial in transmitting human culture. However, mold is a common issue for paper or silk-based relics, which not only affects their preservation and longevity but also conceals the texture, patterns, and color information, hindering cultural value and heritage. Currently, the virtual restoration of painting relics primarily involves filling in the RGB based on neighborhood information, which might cause color distortion and other problems. Another approach considers mold as noise and employs maximum noise separation for its removal; however, eliminating the mold components and implementing the inverse transformation often leads to more loss of information. To effectively acquire virtual restoration for mold removal from ancient paintings, the spectral characteristics of mold were analyzed. Based on the spectral features of mold and the cultural relic restoration philosophy of maintaining originality, a 3D CNN artifact restoration network was proposed. This network is capable of learning features in the near-infrared spectrum (NIR) and spatial dimensions to reconstruct the reflectance of visible spectrum, achieving the virtual restoration for mold removal of calligraphic and art relics. Using an ancient painting from the Qing Dynasty as a test subject, the proposed method was compared with the Inpainting, Criminisi, and inverse MNF transformation methods across three regions. Visual analysis, quantitative evaluation (the root mean squared error (RMSE), mean absolute percentage error (MAPE), mean absolute error (MEA), and a classification application were used to assess the restoration accuracy. The visual results and quantitative analyses demonstrated that the proposed 3D CNN method effectively removes or mitigates mold while restoring the artwork to its authentic color in various backgrounds. Furthermore, the color classification results indicated that the images restored with 3D CNN had the highest classification accuracy, with overall accuracies of 89.51%, 92.24%, and 93.63%, and Kappa coefficients of 0.88, 0.91, and 0.93, respectively. This research provides technological support for the digitalization and restoration of cultural artifacts, thereby contributing to the preservation and transmission of cultural heritage.<\/jats:p>","DOI":"10.3390\/rs16162882","type":"journal-article","created":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T11:33:52Z","timestamp":1723030432000},"page":"2882","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Virtual Restoration of Ancient Mold-Damaged Painting Based on 3D Convolutional Neural Network for Hyperspectral Image"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0451-7181","authenticated-orcid":false,"given":"Sa","family":"Wang","sequence":"first","affiliation":[{"name":"The Palace Museum, Beijing 100009, China"},{"name":"China-Greece Belt and Road Joint Laboratory on Cultural Heritage Conservation Technology, Beijing 100009, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1500-8350","authenticated-orcid":false,"given":"Yi","family":"Cen","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100101, China"},{"name":"School of Computer Science, Hubei University of Technology, Wuhan 430068, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liang","family":"Qu","sequence":"additional","affiliation":[{"name":"The Palace Museum, Beijing 100009, China"},{"name":"China-Greece Belt and Road Joint Laboratory on Cultural Heritage Conservation Technology, Beijing 100009, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guanghua","family":"Li","sequence":"additional","affiliation":[{"name":"The Palace Museum, Beijing 100009, China"},{"name":"China-Greece Belt and Road Joint Laboratory on Cultural Heritage Conservation Technology, Beijing 100009, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yao","family":"Chen","sequence":"additional","affiliation":[{"name":"The Palace Museum, Beijing 100009, China"},{"name":"China-Greece Belt and Road Joint Laboratory on Cultural Heritage Conservation Technology, Beijing 100009, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3533-9966","authenticated-orcid":false,"given":"Lifu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100101, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,7]]},"reference":[{"key":"ref_1","unstructured":"Lu, Y., and Kong, M. (2021). A Discussion on the Conservation and Restoration Methods for Mold-Damaged Calligraphy and Paintings. Identification and Appreciation to Cultural Relic, Anhui Publishing Group Co., Ltd."},{"key":"ref_2","first-page":"8","article-title":"Virtual restoration of mildew stains on calligraphy and paintings based on abundance inversion and spectral transformation","volume":"35","author":"Hou","year":"2023","journal-title":"Sci. Conserv. Archaeol."},{"key":"ref_3","first-page":"294","article-title":"Identification of the Contaminated Fungus in the Ancient Books \u201cZhongyongDaxue\u201dand the Removal of Mildew from Paper Samples","volume":"46","author":"Li","year":"2019","journal-title":"J. Liaoning Univ. (Nat. Sci. Ed.)"},{"key":"ref_4","first-page":"61","article-title":"Isolation and identification of fungi from some cultural relics and packaging boxes in Tianjin Museum storerooms","volume":"31","author":"Zhang","year":"2019","journal-title":"Sci. Conserv. Archaeol."},{"key":"ref_5","first-page":"39","article-title":"Preliminary study on the damage mechanism of ancient paintings","volume":"15","author":"Liu","year":"2003","journal-title":"Sci. Conserv. Archaeol."},{"key":"ref_6","first-page":"65","article-title":"Mold identification and remediation of cultural relics based on scanning electron microscopy","volume":"39","author":"Yang","year":"2020","journal-title":"J. Chin. Electron Microsc. Soc."},{"key":"ref_7","unstructured":"Zhang, N., and Chen, X. (2018). Study on the Effect of Cleaning Agent for Paper Cultural Relics Mildew Spot, China Cultural Heritage Scientific Research."},{"key":"ref_8","first-page":"32","article-title":"Research Progress of Bio-diseases on Paper Relics","volume":"27","author":"Zhen","year":"2019","journal-title":"J. Beijing Inst. Graph. Commun."},{"key":"ref_9","unstructured":"Li, M. (2011). Prevention and Control of Mold Damage to Paper-Based Cultural Artifacts, China Cultural Heritage Scientific Research."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1016\/j.culher.2018.04.004","article-title":"Virtual restoration of stains on ancient paintings with maximum noise fraction transformation based on the hyperspectral imaging","volume":"34","author":"Hou","year":"2018","journal-title":"J. Cult. Herit."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ma, Y. (2020, January 11\u201313). Research on Digital Oil Painting Based on Digital Image Processing Technology. Proceedings of the 2020 IEEE Conference on Telecommunications, Optics and Computer Science (TOCS), Shenyang, China.","DOI":"10.1109\/TOCS50858.2020.9339719"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.cag.2022.11.001","article-title":"Restoration of ancient temple murals using cGAN and PConv networks","volume":"109","author":"Rakhimol","year":"2022","journal-title":"Comput. Graph."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhou, P., Hou, M., Lv, S., Zhao, X., and Wu, W. (2019). Virtual Restoration of Stained Chinese Paintings Using Patch-Based Color Constrained Poisson Editing with Selected Hyperspectral Feature Bands. Remote Sens., 11.","DOI":"10.3390\/rs11111384"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mishra, R., Mittal, N., and Khatri, S.K. (2019, January 24\u201326). Digital Image Restoration using Image Filtering Techniques. Proceedings of the 2019 International Conference on Automation, Computational and Technology Management (ICACTM), London, UK.","DOI":"10.1109\/ICACTM.2019.8776813"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1109\/MSP.2013.2273004","article-title":"Image Inpainting: Overview and Recent Advances","volume":"31","author":"Guillemot","year":"2014","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Pushpalwar, R.T., and Bhandari, S.H. (2016, January 27\u201328). Image Inpainting Approaches\u2014A Review. Proceedings of the 2016 IEEE 6th International Conference on Advanced Computing (IACC), Bhimavaram, India.","DOI":"10.1109\/IACC.2016.70"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2007","DOI":"10.1007\/s11063-019-10163-0","article-title":"Image Inpainting: A Review","volume":"51","author":"Elharrouss","year":"2020","journal-title":"Neural Process. Lett."},{"key":"ref_18","first-page":"130","article-title":"Review of digital image restoration techniques","volume":"29","author":"Li","year":"2016","journal-title":"Inf. Commun."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1145\/882262.882269","article-title":"Poisson Image Editing","volume":"22","author":"Gangnet","year":"2003","journal-title":"ACM Trans. Graph."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"300","DOI":"10.5201\/ipol.2016.163","article-title":"Poisson Image Editing","volume":"6","author":"Facciolo","year":"2016","journal-title":"Image Process. Line"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Bertalmio, M., Sapiro, G., Caselles, V., and Ballester, C. (2000, January 23\u201328). Image Inpainting. Proceedings of the 27th annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, USA.","DOI":"10.1145\/344779.344972"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1006\/jvci.2001.0487","article-title":"Nontexture inpainting by curvature-driven diffusions","volume":"12","author":"Chan","year":"2001","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2134","DOI":"10.3788\/AOS20092908.2134","article-title":"Fourier-Curvelet Transform Combined Image Restoration","volume":"29","author":"Deng","year":"2009","journal-title":"Acta Opt. Sin."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"073001","DOI":"10.1088\/2040-8978\/17\/7\/073001","article-title":"Compression, restoration, resampling, 'compressive sensing\u2019: Fast transforms in digital imaging","volume":"17","author":"Yaroslavsky","year":"2015","journal-title":"J. Opt."},{"key":"ref_25","first-page":"582","article-title":"Regularization algorithm for blind image restoration based on wavelet transform","volume":"15","author":"Jiang","year":"2007","journal-title":"Opt. Precis. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1109\/TIP.2006.887733","article-title":"The undecimated wavelet decomposition and its reconstruction","volume":"16","author":"Starck","year":"2007","journal-title":"EEE Trans. Image Process."},{"key":"ref_27","first-page":"451","article-title":"Fast restoration algorithm for turbulence-degraded images based on wavelet decomposition","volume":"22","author":"Hong","year":"2003","journal-title":"J. Infrared Millim. Waves"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.image.2017.01.003","article-title":"Image restoration using structured sparse representation with a novel parametric data-adaptive transformation matrix","volume":"52","author":"Su","year":"2017","journal-title":"Signal Process. Image Commun."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"221015","DOI":"10.3788\/LOP57.221015","article-title":"Algorithm for Dunhuang Mural Inpainting Based on Gabor Transform and Group Sparse Representation","volume":"57","author":"Chen","year":"2020","journal-title":"Laser Optoelectron. Prog."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Li, J., Chen, X., Zou, D., Gao, B., and Teng, W. (2015, January 7\u201313). Conformal and Low-Rank Sparse Representation for Image Restoration. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.35"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Hanif, M., Tonazzini, A., Savino, P., and Salerno, E. (2018). Non-Local Sparse Image Inpainting for Document Bleed-Through Removal. J. Imaging, 4.","DOI":"10.3390\/jimaging4050068"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1145\/1531326.1531330","article-title":"PatchMatch: A randomized correspondence algorithm for structural image editing","volume":"28","author":"Barnes","year":"2009","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"373","DOI":"10.5201\/ipol.2017.189","article-title":"Non-Local Patch-Based Image Inpainting","volume":"7","author":"Newson","year":"2017","journal-title":"Image Process. Line"},{"key":"ref_34","unstructured":"Criminisi, A., P\u00e9rez, P., and Toyama, K. (2003, January 18\u201320). Object removal by exemplar-based inpainting. Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI, USA."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.neucom.2016.08.149","article-title":"Exemplar-based image inpainting using structure consistent patch matching","volume":"269","author":"Wang","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_36","first-page":"1126","article-title":"Progress of hyperspectral remote sensing applications on cultural relics protection","volume":"52","author":"Zhang","year":"2023","journal-title":"Acta Geod. Cartogr. Sin."},{"key":"ref_37","first-page":"113","article-title":"Virtual Restoration of Ancient Painting Stains Based on Classified Linear Regression of Hyper-spectral Image","volume":"24","author":"Zhou","year":"2017","journal-title":"Geomat. World"},{"key":"ref_38","first-page":"11","article-title":"Research on a method for virtual restoration of the colors of tomb mural pigments based on spectral fusion analysis","volume":"35","author":"Yang","year":"2023","journal-title":"Sci. Conserv. Archaeol."},{"key":"ref_39","first-page":"180","article-title":"Study on the application of an automatic hyperspectral scanning system to investigate Chinese paintings","volume":"S2","author":"Li","year":"2021","journal-title":"Chin. Mus."},{"key":"ref_40","unstructured":"Yan, L., Gao, Y., and Jia, D. (2011). Isolation and identification of contaminated mold on ancient painting and calligraphy relics. China Cult. Herit. Sci. Res., 78\u201382."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/j.scitotenv.2007.06.022","article-title":"Bacterial and fungal deterioration of the Milan Cathedral marble treated with protective synthetic resins","volume":"385","author":"Cappitelli","year":"2007","journal-title":"Sci. Total Environ."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.polymdegradstab.2005.01.003","article-title":"Photo-induced degradation of cellulose","volume":"89","author":"Kolar","year":"2005","journal-title":"Polym. Degrad. Stab."},{"key":"ref_43","first-page":"1136","article-title":"Spectral Analysis of Main Mineral Pigments in Thangka","volume":"39","author":"Cen","year":"2019","journal-title":"Spectrosc. Spectr. Anal."},{"key":"ref_44","unstructured":"Yu, Q. (2023). Research on Optical Fiber Sensor for Online Detection of Mold and Disease Process Parameters on Paper Cultural Relics, Chongqing University of Technology."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1016\/j.chroma.2018.10.024","article-title":"Thin layer chromatography combined with surface-enhanced raman spectroscopy for rapid sensing aflatoxins","volume":"1579","author":"Qu","year":"2018","journal-title":"J. Chromatogr. A"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Qi, W., Zhang, X., Wang, N., Zhang, M., and Cen, Y. (2019). A Spectral-Spatial Cascaded 3D Convolutional Neural Network with a Convolutional Long Short-Term Memory Network for Hyperspectral Image Classification. Remote Sens., 11.","DOI":"10.3390\/rs11202363"},{"key":"ref_48","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 7\u20139). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on International Conference on Machine Learning, Lille, France."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1109\/72.279181","article-title":"Learning long-term dependencies with gradient descent is difficult","volume":"5","author":"Bengio","year":"1994","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1080\/10867651.2004.10487596","article-title":"An Image Inpainting Technique Based on the Fast Marching Method","volume":"9","author":"Telea","year":"2004","journal-title":"J. Graph. Tools"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/16\/2882\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:31:26Z","timestamp":1760110286000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/16\/2882"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,7]]},"references-count":50,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["rs16162882"],"URL":"https:\/\/doi.org\/10.3390\/rs16162882","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,7]]}}}