{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T08:10:34Z","timestamp":1770970234683,"version":"3.50.1"},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"10-12","license":[{"start":{"date-parts":[[2020,7,17]],"date-time":"2020-07-17T00:00:00Z","timestamp":1594944000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,7,17]],"date-time":"2020-07-17T00:00:00Z","timestamp":1594944000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003407","name":"Ministero dell\u2019Istruzione, dell\u2019Universit\u00e0 e della Ricerca","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003407","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Vis Comput"],"published-print":{"date-parts":[[2020,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Reflectance transformation imaging (RTI) is a computational photography technique widely used in the cultural heritage and material science domains to characterize relieved surfaces. It basically consists of capturing multiple images from a fixed viewpoint with varying lights. Handling the potentially huge amount of information stored in an RTI acquisition that consists typically of 50\u2013100\u00a0RGB values per pixel, allowing data exchange, interactive visualization, and material analysis, is not easy. The solution used in practical applications consists of creating \u201crelightable images\u201d by approximating the pixel information with a function of the light direction, encoded with a small number of parameters. This encoding allows the estimation of images relighted from novel, arbitrary lights, with a quality that, however, is not always satisfactory. In this paper, we present NeuralRTI, a framework for pixel-based encoding and relighting of RTI data. Using a simple autoencoder architecture, we show that it is possible to obtain a highly compressed representation that better preserves the original information and provides increased quality of virtual images relighted from novel directions, especially in the case of challenging glossy materials. We also address the problem of validating the relight quality on different surfaces, proposing a specific benchmark, SynthRTI, including image collections synthetically created with physical-based rendering and featuring objects with different materials and geometric complexity. On this dataset and as well on a collection of real acquisitions performed on heterogeneous surfaces, we demonstrate the advantages of the proposed relightable image encoding.<\/jats:p>","DOI":"10.1007\/s00371-020-01910-9","type":"journal-article","created":{"date-parts":[[2020,7,17]],"date-time":"2020-07-17T16:19:35Z","timestamp":1595002775000},"page":"2161-2174","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Neural reflectance transformation imaging"],"prefix":"10.1007","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3955-3019","authenticated-orcid":false,"given":"Tinsae G.","family":"Dulecha","sequence":"first","affiliation":[]},{"given":"Filippo A.","family":"Fanni","sequence":"additional","affiliation":[]},{"given":"Federico","family":"Ponchio","sequence":"additional","affiliation":[]},{"given":"Fabio","family":"Pellacini","sequence":"additional","affiliation":[]},{"given":"Andrea","family":"Giachetti","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,7,17]]},"reference":[{"issue":"1","key":"1910_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1561\/2200000006","volume":"2","author":"Y Bengio","year":"2009","unstructured":"Bengio, Y., et al.: Learning deep architectures for ai. Found. Trends\u00ae Mach. Learn. 2(1), 1\u2013127 (2009)","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"1910_CR2","doi-asserted-by":"crossref","unstructured":"Chen, G., Han, K., Shi, B., Matsushita, Y., and Wong, K.-Y.K.: Self-calibrating deep photometric stereo networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8739\u20138747, (2019)","DOI":"10.1109\/CVPR.2019.00894"},{"key":"1910_CR3","unstructured":"CHI. Cultural heritage imaging website, 2019. [Online; accessed-March-2019]"},{"key":"1910_CR4","doi-asserted-by":"crossref","unstructured":"Earl, G., Basford, P., Bischoff, A., Bowman, A, Crowther, C., Dahl, J., Hodgson, M., Isaksen, L., Kotoula, E., Martinez, K, Pagi, H., and Piquette, K.E.: Reflectance transformation imaging systems for ancient documentary artefacts. In Proc. International Conference on Electronic Visualisation and the Arts, pp. 147\u2013154, Swindon, UK, 2011. BCS Learning & Development Ltd","DOI":"10.14236\/ewic\/EVA2011.27"},{"key":"1910_CR5","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1016\/j.cviu.2017.05.014","volume":"168","author":"A Giachetti","year":"2018","unstructured":"Giachetti, A., Ciortan, I.M., Daffara, C., Marchioro, G., Pintus, R., Gobbetti, E.: A novel framework for highlight reflectance transformation imaging. Comput. Vis. Image Underst. 168, 118\u2013131 (2018)","journal-title":"Comput. Vis. Image Underst."},{"issue":"5786","key":"1910_CR6","doi-asserted-by":"publisher","first-page":"504","DOI":"10.1126\/science.1127647","volume":"313","author":"GE Hinton","year":"2006","unstructured":"Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504\u2013507 (2006)","journal-title":"Science"},{"key":"1910_CR7","unstructured":"Kingma, D.P. and Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, (2014)"},{"key":"1910_CR8","doi-asserted-by":"publisher","first-page":"650","DOI":"10.1016\/j.patcog.2016.06.008","volume":"61","author":"KG Lore","year":"2017","unstructured":"Lore, K.G., Akintayo, A., Sarkar, S.: Llnet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recognit. 61, 650\u2013662 (2017)","journal-title":"Pattern Recognit."},{"key":"1910_CR9","doi-asserted-by":"crossref","unstructured":"Malzbender, T., Gelb, D., and Wolters, H.: Polynomial texture maps. In Proceedings SIGGRAPH, pp. 519\u2013528,(2001)","DOI":"10.1145\/383259.383320"},{"key":"1910_CR10","unstructured":"Mudge, M., Malzbender, T., Chalmers, A., Scopigno, R., Davis, J., Wang, O., Gunawardane, P., Ashley, M., Doerr, M., Proenca, A., and Barbosa, J.: Image-based empirical information acquisition, scientific reliability, and long-term digital preservation for the natural sciences and cultural heritage. In Eurographics (Tutorials), (2008)"},{"key":"1910_CR11","unstructured":"Mudge, M., Malzbender, T., Schroer, C., and Lum, M.: New reflection transformation imaging methods for rock art and multiple-viewpoint display. In Proceedings VAST, vol. 6, pp. 195\u2013202, (2006)"},{"issue":"2","key":"1910_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1841317.1841321","volume":"3","author":"G Palma","year":"2010","unstructured":"Palma, G., Corsini, M., Cignoni, P., Scopigno, R., Mudge, M.: Dynamic shading enhancement for reflectance transformation imaging. J. Comput. Cult. Herit. 3(2), 1\u201320 (2010)","journal-title":"J. Comput. Cult. Herit."},{"key":"1910_CR13","unstructured":"Pintus, R., Dulecha, T., Jaspe, A., Giachetti, A., Ciortan, I.M., and Gobbetti, E.: Objective and subjective evaluation of virtual relighting from reflectance transformation imaging data. In GCH, pp. 87\u201396, (2018)"},{"key":"1910_CR14","doi-asserted-by":"crossref","unstructured":"Pintus, R., Dulecha, T.G., Ciortan, I., Gobbetti, E., and Giachetti, A.: State-of-the-art in multi-light image collections for surface visualization and analysis. In Computer Graphics Forum, vol. 38, pp. 909\u2013934. Wiley Online Library, (2019)","DOI":"10.1111\/cgf.13732"},{"key":"1910_CR15","doi-asserted-by":"crossref","unstructured":"Pitard, G., LeGo\u00efc, G., Favreli\u00e8re, H., Samper, S., Desage, S.-F., and Pillet, M.: Discrete modal decomposition for surface appearance modelling and rendering. In Optical Measurement Systems for Industrial Inspection IX, vol. 9525, pp. 952523:1\u2013952523:10. International Society for Optics and Photonics, (2015)","DOI":"10.1117\/12.2184840"},{"issue":"5\u20136","key":"1910_CR16","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1007\/s00138-017-0856-0","volume":"28","author":"G Pitard","year":"2017","unstructured":"Pitard, G., LeGo\u00efc, G., Mansouri, A., Favreli\u00e8re, H., Desage, S.-F., Samper, S., Pillet, M.: Discrete modal decomposition: a new approach for the reflectance modeling and rendering of real surfaces. Mach. Vis. Appl. 28(5\u20136), 607\u2013621 (2017)","journal-title":"Mach. Vis. Appl."},{"key":"1910_CR17","doi-asserted-by":"crossref","unstructured":"Ponchio, F., Corsini, M., and Scopigno, R.: A compact representation of relightable images for the web. In Proceedings ACM Web3D, pp. 1:1\u20131:10, (2018)","DOI":"10.1145\/3208806.3208820"},{"key":"1910_CR18","doi-asserted-by":"crossref","unstructured":"Rainer, G., Jakob, W., Ghosh, A., and Weyrich, T.: Neural btf compression and interpolation. In Computer Graphics Forum, vol. 38, pp. 235\u2013244. Wiley Online Library, (2019)","DOI":"10.1111\/cgf.13633"},{"key":"1910_CR19","doi-asserted-by":"crossref","unstructured":"Ren, P., Dong, Y., Lin, S., Tong, X., and Guo, B.: Image based relighting using neural networks. ACM TOG, 34(4):111:1\u2013111:12, (2015)","DOI":"10.1145\/2766899"},{"key":"1910_CR20","doi-asserted-by":"crossref","unstructured":"Shi, B., Wu, Z., Mo, Z., Duan, D., Yeung, S.-K., and Tan, P.: A benchmark dataset and evaluation for non-lambertian and uncalibrated photometric stereo. In Proceedings CVPR, pp. 3707\u20133716, (2016)","DOI":"10.1109\/CVPR.2016.403"},{"key":"1910_CR21","unstructured":"Smilkov, D., Thorat, N., Assogba, Y., Yuan, A., Kreeger, N., Yu, P., Zhang, K., Cai, S., Nielsen, E., Soergel, D., Bileschi, S., Terry, M., Nicholson, C., Gupta, S.N., Sirajuddin, S., Sculley, D., Monga, R., Corrado, G., Vi\u00e9gas, F.B., and Wattenberg, M.: Tensorflow.js: Machine learning for the web and beyond. CoRR, abs\/1901.05350, (2019)"},{"key":"1910_CR22","unstructured":"Xie, J., Xu, L., and Chen, E.: Image denoising and inpainting with deep neural networks. In F.Pereira, C.J.C. Burges, L.Bottou, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pp. 341\u2013349. Curran Associates, Inc., (2012)"},{"key":"1910_CR23","doi-asserted-by":"crossref","unstructured":"Xu, Z., Sunkavalli, K., Hadap, S., and Ramamoorthi, R.: Deep image-based relighting from optimal sparse samples. ACM TOG, 37(4):126:1\u2013126:13, (2018)","DOI":"10.1145\/3197517.3201313"},{"key":"1910_CR24","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1016\/j.compind.2018.02.006","volume":"98","author":"W Zhang","year":"2018","unstructured":"Zhang, W., Hansen, M.F., Smith, M., Smith, L., Grieve, B.: Photometric stereo for three-dimensional leaf venation extraction. Comput. Ind. 98, 56\u201367 (2018)","journal-title":"Comput. Ind."},{"issue":"4","key":"1910_CR25","first-page":"043037","volume":"27","author":"B Zheng","year":"2018","unstructured":"Zheng, B., Sun, R., Tian, X., Chen, Y.: S-net: a scalable convolutional neural network for jpeg compression artifact reduction. J. Electr. Imaging 27(4), 043037 (2018)","journal-title":"J. Electr. Imaging"},{"key":"1910_CR26","doi-asserted-by":"crossref","unstructured":"Zini, S., Bianco, S., and Schettini, R.: Deep residual autoencoder for quality independent jpeg restoration. arXiv preprint arXiv:1903.06117 (2019)","DOI":"10.1109\/ACCESS.2020.2984387"}],"container-title":["The Visual Computer"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00371-020-01910-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00371-020-01910-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00371-020-01910-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,16]],"date-time":"2021-07-16T23:30:47Z","timestamp":1626478247000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00371-020-01910-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,17]]},"references-count":26,"journal-issue":{"issue":"10-12","published-print":{"date-parts":[[2020,10]]}},"alternative-id":["1910"],"URL":"https:\/\/doi.org\/10.1007\/s00371-020-01910-9","relation":{},"ISSN":["0178-2789","1432-2315"],"issn-type":[{"value":"0178-2789","type":"print"},{"value":"1432-2315","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,17]]},"assertion":[{"value":"17 July 2020","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}