{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T14:32:27Z","timestamp":1777645947069,"version":"3.51.4"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2012,11,1]],"date-time":"2012-11-01T00:00:00Z","timestamp":1351728000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100000921","name":"European Cooperation in Science and Technology","doi-asserted-by":"publisher","award":["STSM-100312-015783"],"award-info":[{"award-number":["STSM-100312-015783"]}],"id":[{"id":"10.13039\/501100000921","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/I006575\/1"],"award-info":[{"award-number":["EP\/I006575\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2012,11]]},"abstract":"<jats:p>Reliable detection of global illumination and rendering artifacts in the form of localized distortion maps is important for many graphics applications. Although many quality metrics have been developed for this task, they are often tuned for compression\/transmission artifacts and have not been evaluated in the context of synthetic CG-images. In this work, we run two experiments where observers use a brush-painting interface to directly mark image regions with noticeable\/objectionable distortions in the presence\/absence of a high-quality reference image, respectively. The collected data shows a relatively high correlation between the with-reference and no-reference observer markings. Also, our demanding per-pixel image-quality datasets reveal weaknesses of both simple (PSNR, MSE, sCIE-Lab) and advanced (SSIM, MS-SSIM, HDR-VDP-2) quality metrics. The most problematic are excessive sensitivity to brightness and contrast changes, the calibration for near visibility-threshold distortions, lack of discrimination between plausible\/implausible illumination, and poor spatial localization of distortions for multi-scale metrics. We believe that our datasets have further potential in improving existing quality metrics, but also in analyzing the saliency of rendering distortions, and investigating visual equivalence given our with- and no-reference data.<\/jats:p>","DOI":"10.1145\/2366145.2366166","type":"journal-article","created":{"date-parts":[[2012,11,14]],"date-time":"2012-11-14T20:36:17Z","timestamp":1352925377000},"page":"1-10","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["New measurements reveal weaknesses of image quality metrics in evaluating graphics artifacts"],"prefix":"10.1145","volume":"31","author":[{"given":"Martin","family":"\u010cad\u00edk","sequence":"first","affiliation":[{"name":"MPI Informatik Saarbr\u00fccken, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert","family":"Herzog","sequence":"additional","affiliation":[{"name":"MPI Informatik Saarbr\u00fccken, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rafa\u0142","family":"Mantiuk","sequence":"additional","affiliation":[{"name":"Bangor University, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Karol","family":"Myszkowski","sequence":"additional","affiliation":[{"name":"MPI Informatik Saarbr\u00fccken, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hans-Peter","family":"Seidel","sequence":"additional","affiliation":[{"name":"MPI Informatik Saarbr\u00fccken, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2012,11]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/16.5.412"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280924"},{"key":"e_1_2_2_3_1","volume-title":"-P","author":"\u010cad\u00edk M.","year":"2011","unstructured":"\u010cad\u00edk , M. , Aydin , T. O. , Myszkowski , K. , and Seidel , H . -P . 2011 . On evaluation of video quality metrics: an HDR dataset for computer graphics applications. In SPIE HVEI XVI. \u010cad\u00edk, M., Aydin, T. O., Myszkowski, K., and Seidel, H.-P. 2011. On evaluation of video quality metrics: an HDR dataset for computer graphics applications. In SPIE HVEI XVI."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2007.901820"},{"key":"e_1_2_2_5_1","volume-title":"Digital Images and Human Vision","author":"Daly S.","unstructured":"Daly , S. 1993. The Visible Differences Predictor: An algorithm for the assessment of image fidelity . In Digital Images and Human Vision , MIT Press , 179--206. Daly, S. 1993. The Visible Differences Predictor: An algorithm for the assessment of image fidelity. In Digital Images and Human Vision, MIT Press, 179--206."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8659.00689"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/566570.566573"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1457515.1409083"},{"key":"e_1_2_2_9_1","volume-title":"-P","author":"Herzog R.","year":"2009","unstructured":"Herzog , R. , Myszkowski , K. , and Seidel , H . -P . 2009 . Anisotropic radiance-cache splatting for efficiently computing high-quality GI with lightcuts. Proc. of Eurographics , 259--268. Herzog, R., Myszkowski, K., and Seidel, H.-P. 2009. Anisotropic radiance-cache splatting for efficiently computing high-quality GI with lightcuts. Proc. of Eurographics, 259--268."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2012.03055.x"},{"key":"e_1_2_2_11_1","volume-title":"Statistical Methods for Psychology","author":"Howell D. C.","unstructured":"Howell , D. C. 2007. Statistical Methods for Psychology , 6 th edition ed. Thomas Wadsworth . Howell, D. C. 2007. Statistical Methods for Psychology, 6th edition ed. Thomas Wadsworth.","edition":"6"},{"key":"e_1_2_2_12_1","unstructured":"ITU-R-BT.500-11 2002. Methodology for the subjective assessment of the quality of television pictures.  ITU-R-BT.500-11 2002. Methodology for the subjective assessment of the quality of television pictures."},{"key":"e_1_2_2_13_1","unstructured":"ITU-T-P.910. 2008. Subjective audiovisual quality assessment methods for multimedia applications. Tech. rep.  ITU-T-P.910. 2008. Subjective audiovisual quality assessment methods for multimedia applications. Tech. rep."},{"key":"e_1_2_2_14_1","doi-asserted-by":"crossref","unstructured":"Jensen H. W. 2001. Realistic Image Synthesis Using Photon Mapping. AK Peters.   Jensen H. W. 2001. Realistic Image Synthesis Using Photon Mapping . AK Peters.","DOI":"10.1201\/b10685"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/258734.258769"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2005.83"},{"key":"e_1_2_2_17_1","article-title":"Most apparent distortion: full-reference image quality assessment and the role of strategy","volume":"19","author":"Larson E. C.","year":"2010","unstructured":"Larson , E. C. , and Chandler , D. M. 2010 . Most apparent distortion: full-reference image quality assessment and the role of strategy . J. Electron. Imaging 19 , 1, 011006:1--21. Larson, E. C., and Chandler, D. M. 2010. Most apparent distortion: full-reference image quality assessment and the role of strategy. J. Electron. Imaging 19, 1, 011006:1--21.","journal-title":"J. Electron. Imaging"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1186822.1073242"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvcir.2011.01.005"},{"key":"e_1_2_2_20_1","doi-asserted-by":"crossref","unstructured":"Lubin J. 1995. Vision Models for Target Detection and Recognition. World Scientific ch. A Visual Discrimination Model for Imaging System Design and Evaluation 245--283.  Lubin J. 1995. Vision Models for Target Detection and Recognition . World Scientific ch. A Visual Discrimination Model for Imaging System Design and Evaluation 245--283.","DOI":"10.1142\/9789812831200_0010"},{"key":"e_1_2_2_21_1","volume-title":"-P","author":"Mantiuk R.","year":"2005","unstructured":"Mantiuk , R. , Daly , S. , Myszkowski , K. , and Seidel , H . -P . 2005 . Predicting visible differences in high dynamic range images - model and its calibration. In SPIE HVEI X. Mantiuk, R., Daly, S., Myszkowski, K., and Seidel, H.-P. 2005. Predicting visible differences in high dynamic range images - model and its calibration. In SPIE HVEI X."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1166087.1166095"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360667"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1964921.1964935"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1561\/0600000037"},{"key":"e_1_2_2_26_1","first-page":"30","article-title":"TID2008 - A database for evaluation of full-reference visual quality assessment metrics","volume":"10","author":"Ponomarenko N.","year":"2009","unstructured":"Ponomarenko , N. , Lukin , V. , Zelensky , A. , Egiazarian , K. , Carli , M. , and Battisti , F. 2009 . TID2008 - A database for evaluation of full-reference visual quality assessment metrics . Advances of Modern Radioelectronics 10 , 30 -- 45 . Ponomarenko, N., Lukin, V., Zelensky, A., Egiazarian, K., Carli, M., and Battisti, F. 2009. TID2008 - A database for evaluation of full-reference visual quality assessment metrics. Advances of Modern Radioelectronics 10, 30--45.","journal-title":"Advances of Modern Radioelectronics"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1275808.1276472"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311543"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/566570.566575"},{"key":"e_1_2_2_30_1","volume-title":"Rendering Techniques '95","author":"Rushmeier H.","unstructured":"Rushmeier , H. , Ward , G. , Piatko , C. , Sanders , P. , and Rust , B . 1995. Comparing real and synthetic images: some ideas about metrics . In Rendering Techniques '95 , 82--91. Rushmeier, H., Ward, G., Piatko, C., Sanders, P., and Rust, B. 1995. Comparing real and synthetic images: some ideas about metrics. In Rendering Techniques '95, 82--91."},{"key":"e_1_2_2_31_1","volume-title":"2007. Encyclopedia of measurement and statistics","author":"Salkind N.","unstructured":"Salkind , N. , Ed. 2007. Encyclopedia of measurement and statistics . A Sage reference publication. SAGE, Thousand Oaks . Salkind, N., Ed. 2007. Encyclopedia of measurement and statistics. A Sage reference publication. SAGE, Thousand Oaks."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2006.881959"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1186822.1073318"},{"key":"e_1_2_2_34_1","doi-asserted-by":"crossref","unstructured":"Wang Z. and Bovik A. C. 2006. Modern Image Quality Assessment. Morgan & Claypool Publishers.  Wang Z. and Bovik A. C. 2006. Modern Image Quality Assessment . Morgan & Claypool Publishers.","DOI":"10.1007\/978-3-031-02238-8"},{"key":"e_1_2_2_35_1","volume-title":"Proc. IEEE Asilomar Conf. on Signals, Systems & Comp., 1398--1402","author":"Wang Z.","unstructured":"Wang , Z. , Simoncelli , E. P. , and Bovik , A. C . 2003. Multi-scale structural similarity for image quality assessment . In Proc. IEEE Asilomar Conf. on Signals, Systems & Comp., 1398--1402 . Wang, Z., Simoncelli, E. P., and Bovik, A. C. 2003. Multi-scale structural similarity for image quality assessment. In Proc. IEEE Asilomar Conf. on Signals, Systems & Comp., 1398--1402."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/54852.378490"},{"key":"e_1_2_2_37_1","unstructured":"Wu H. and Rao K. 2005. Digital Video Image Quality and Perceptual Coding. CRC Press.   Wu H. and Rao K. 2005. Digital Video Image Quality and Perceptual Coding . CRC Press."},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0165-1684(98)00125-X"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2366145.2366166","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2366145.2366166","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T09:34:05Z","timestamp":1750239245000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2366145.2366166"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,11]]},"references-count":38,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2012,11]]}},"alternative-id":["10.1145\/2366145.2366166"],"URL":"https:\/\/doi.org\/10.1145\/2366145.2366166","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,11]]},"assertion":[{"value":"2012-11-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}