{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T15:33:59Z","timestamp":1773329639550,"version":"3.50.1"},"reference-count":36,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2020,8,26]],"date-time":"2020-08-26T00:00:00Z","timestamp":1598400000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Comput. Graph. Interact. Tech."],"published-print":{"date-parts":[[2020,8,26]]},"abstract":"<jats:p>Image quality measures are becoming increasingly important in the field of computer graphics. For example, there is currently a major focus on generating photorealistic images in real time by combining path tracing with denoising, for which such quality assessment is integral. We present FLIP, which is a difference evaluator with a particular focus on the differences between rendered images and corresponding ground truths. Our algorithm produces a map that approximates the difference perceived by humans when alternating between two images. FLIP is a combination of modified existing building blocks, and the net result is surprisingly powerful. We have compared our work against a wide range of existing image difference algorithms and we have visually inspected over a thousand image pairs that were either retrieved from image databases or generated in-house. We also present results of a user study which indicate that our method performs substantially better, on average, than the other algorithms. To facilitate the use of FLIP, we provide source code in C++, MATLAB, NumPy\/SciPy, and PyTorch.<\/jats:p>","DOI":"10.1145\/3406183","type":"journal-article","created":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T17:45:26Z","timestamp":1616521526000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":151,"title":["FLIP"],"prefix":"10.1145","volume":"3","author":[{"given":"Pontus","family":"Andersson","sequence":"first","affiliation":[{"name":"NVIDIA"}]},{"given":"Jim","family":"Nilsson","sequence":"additional","affiliation":[{"name":"NVIDIA"}]},{"given":"Tomas","family":"Akenine-M\u00f6ller","sequence":"additional","affiliation":[{"name":"NVIDIA"}]},{"given":"Magnus","family":"Oskarsson","sequence":"additional","affiliation":[{"name":"Centre for Mathematical Sciences, Lund University"}]},{"given":"Kalle","family":"\u00c5str\u00f6m","sequence":"additional","affiliation":[{"name":"Centre for Mathematical Sciences, Lund University"}]},{"given":"Mark D.","family":"Fairchild","sequence":"additional","affiliation":[{"name":"Munsell Color Science Laboratory, Rochester Institute of Technology"}]}],"member":"320","published-online":{"date-parts":[[2020,8,26]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1002\/col.22451"},{"key":"e_1_2_2_2_1","volume-title":"Guetzli: Perceptually Guided JPEG Encoder. CoRR abs\/1703.04421","author":"Alakuijala Jyrki","year":"2017","unstructured":"Jyrki Alakuijala , Robert Obryk , Ostap Stoliarchuk , Zoltan Szabadka , Lode Vandevenne , and Jan Wassenberg . 2017 . Guetzli: Perceptually Guided JPEG Encoder. CoRR abs\/1703.04421 (2017). arXiv:1703.04421 http:\/\/arxiv.org\/abs\/1703.04421 Jyrki Alakuijala, Robert Obryk, Ostap Stoliarchuk, Zoltan Szabadka, Lode Vandevenne, and Jan Wassenberg. 2017. Guetzli: Perceptually Guided JPEG Encoder. CoRR abs\/1703.04421 (2017). arXiv:1703.04421 http:\/\/arxiv.org\/abs\/1703.04421"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073708"},{"key":"e_1_2_2_4_1","unstructured":"Benedikt Bitterli. 2016. Rendering Resources. https:\/\/benedikt-bitterli.me\/resources\/.  Benedikt Bitterli. 2016. Rendering Resources. https:\/\/benedikt-bitterli.me\/resources\/."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366166"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1155\/2013\/905685"},{"key":"e_1_2_2_7_1","volume-title":"Color-Difference Equations, Psychometric Color Terms. Supplement No.2 to CIE publication No.15 (E.-1.3.1) 1971\/(TC-1.3.)","author":"Comission Internationale","year":"1978","unstructured":"Comission Internationale de l'Eclairage CIE. 1978. Recommendations on Uniform Color Spaces , Color-Difference Equations, Psychometric Color Terms. Supplement No.2 to CIE publication No.15 (E.-1.3.1) 1971\/(TC-1.3.) ( 1978 ). Comission Internationale de l'Eclairage CIE. 1978. Recommendations on Uniform Color Spaces, Color-Difference Equations, Psychometric Color Terms. Supplement No.2 to CIE publication No.15 (E.-1.3.1) 1971\/(TC-1.3.) (1978)."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280898"},{"key":"e_1_2_2_9_1","volume-title":"DeValois","author":"DeValois Russell L.","year":"1990","unstructured":"Russell L. DeValois and Karen K . DeValois . 1990 . Spatial Vision. Oxford University Press . Russell L. DeValois and Karen K. DeValois. 1990. Spatial Vision. Oxford University Press."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1520-6378(199610)21:5<338::AID-COL3>3.0.CO;2-Z"},{"key":"e_1_2_2_11_1","volume-title":"Color Appearance Models","author":"Fairchild Mark D.","unstructured":"Mark D. Fairchild . 2013. Color Appearance Models ( 3 rd ed.). John Wiley & Sons . Mark D. Fairchild. 2013. Color Appearance Models (3rd ed.). John Wiley & Sons.","edition":"3"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1117\/1.1635368"},{"key":"e_1_2_2_13_1","volume-title":"Human Vision, Visual Processing, and Digital Display IV","author":"Flohr Thomas J","unstructured":"Thomas J Flohr , Bernd W Kolpatzik , Raja Balasubramanian , David A Carrara , Charles A Bouman , and Jan P Allebach . 1993. Model-Based Color Image Quantization . In Human Vision, Visual Processing, and Digital Display IV , Vol. 1913 . 270--281. Thomas J Flohr, Bernd W Kolpatzik, Raja Balasubramanian, David A Carrara, Charles A Bouman, and Jan P Allebach. 1993. Model-Based Color Image Quantization. In Human Vision, Visual Processing, and Digital Display IV, Vol. 1913. 270--281."},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2012.03055.x"},{"key":"e_1_2_2_15_1","first-page":"1","article-title":"Power Functions Improving the Performance of Color-Difference Formulas","volume":"23","author":"Huang Min","year":"2015","unstructured":"Min Huang , Guihua Cui , Manuel Melgosa , Manuel S\u00e1nchez-Mara n \u00f3n , Changjun Li , M. Ronnier Luo , and Haoxue Liu . 2015 . Power Functions Improving the Performance of Color-Difference Formulas . Optics Express 23 , 1 (Jan 2015), 597--610. Min Huang, Guihua Cui, Manuel Melgosa, Manuel S\u00e1nchez-Mara n\u00f3n, Changjun Li, M. Ronnier Luo, and Haoxue Liu. 2015. Power Functions Improving the Performance of Color-Difference Formulas. Optics Express 23, 1 (Jan 2015), 597--610.","journal-title":"Optics Express"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSA.42.000190"},{"key":"e_1_2_2_17_1","volume-title":"Darwinism of Color Image Difference Models. In Color and Imaging Conference","volume":"2001","author":"Johnson Garrett M","year":"2001","unstructured":"Garrett M Johnson and Mark D Fairchild . 2001 . Darwinism of Color Image Difference Models. In Color and Imaging Conference , Vol. 2001 . 108--112. Garrett M Johnson and Mark D Fairchild. 2001. Darwinism of Color Image Difference Models. In Color and Imaging Conference, Vol. 2001. 108--112."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/col.10195"},{"key":"e_1_2_2_19_1","article-title":"Most Apparent Distortion: Full-Reference Image Quality Assessment and the Role of Strategy","volume":"19","author":"Larson E. C.","year":"2010","unstructured":"E. C. Larson and D. M. Chandler . 2010 . Most Apparent Distortion: Full-Reference Image Quality Assessment and the Role of Strategy . Journal of Electronic Imaging 19 , 1 (2010). E. C. Larson and D. M. Chandler. 2010. Most Apparent Distortion: Full-Reference Image Quality Assessment and the Role of Strategy. Journal of Electronic Imaging 19, 1 (2010).","journal-title":"Journal of Electronic Imaging"},{"key":"e_1_2_2_20_1","unstructured":"Amazon Lumberyard. 2017. Amazon Lumberyard Bistro Open Research Content Archive (ORCA). http:\/\/developer.nvidia.com\/orca\/amazon-lumberyard-bistro  Amazon Lumberyard. 2017. Amazon Lumberyard Bistro Open Research Content Archive (ORCA). http:\/\/developer.nvidia.com\/orca\/amazon-lumberyard-bistro"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964935"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.visres.2018.09.007"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2002.806708"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSAA.5.002166"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.1985.sp015591"},{"key":"e_1_2_2_26_1","unstructured":"Jim Nilsson and Tomas Akenine-M\u00f6ller. 2020. Understanding SSIM. arXiv:2006.13846 [eess.IV]  Jim Nilsson and Tomas Akenine-M\u00f6ller. 2020. Understanding SSIM. arXiv:2006.13846 [eess.IV]"},{"key":"e_1_2_2_27_1","unstructured":"Charles Poynton. 2003. Digital Video and HDTV -- Algorithms and Interfaces. Morgan Kaufmann.  Charles Poynton. 2003. Digital Video and HDTV -- Algorithms and Interfaces. Morgan Kaufmann."},{"key":"e_1_2_2_28_1","volume-title":"PieAPP: Perceptual Image-Error Assessment through Pairwise Preference. CoRR abs\/1806.02067","author":"Prashnani Ekta","year":"2018","unstructured":"Ekta Prashnani , Hong Cai , Yasamin Mostofi , and Pradeep Sen . 2018. PieAPP: Perceptual Image-Error Assessment through Pairwise Preference. CoRR abs\/1806.02067 ( 2018 ). arXiv:1806.02067 http:\/\/arxiv.org\/abs\/1806.02067 Ekta Prashnani, Hong Cai, Yasamin Mostofi, and Pradeep Sen. 2018. PieAPP: Perceptual Image-Error Assessment through Pairwise Preference. CoRR abs\/1806.02067 (2018). arXiv:1806.02067 http:\/\/arxiv.org\/abs\/1806.02067"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201388"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_2_2_31_1","volume-title":"Multiscale Structural Similarity for Image Quality Assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers","volume":"2","author":"Wang Zhou","year":"2003","unstructured":"Zhou Wang , Eero P Simoncelli , and Alan C Bovik . 2003 . Multiscale Structural Similarity for Image Quality Assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers , Vol. 2 . IEEE, 1398--1402. Zhou Wang, Eero P Simoncelli, and Alan C Bovik. 2003. Multiscale Structural Similarity for Image Quality Assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, Vol. 2. IEEE, 1398--1402."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196493"},{"key":"e_1_2_2_33_1","volume-title":"Color Science","author":"Wyszecki Gunter","unstructured":"Gunter Wyszecki and Walter Stanley Stiles . 1982. Color Science . Vol. 8 . Wiley New York . Gunter Wyszecki and Walter Stanley Stiles. 1982. Color Science. Vol. 8. Wiley New York."},{"key":"e_1_2_2_34_1","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition. 586--595","author":"Zhang R.","unstructured":"R. Zhang , P. Isola , A. A. Efros , E. Shechtman , and O. Wang . 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric . In IEEE Conference on Computer Vision and Pattern Recognition. 586--595 . R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In IEEE Conference on Computer Vision and Pattern Recognition. 586--595."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1889\/1.1985127"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.5555\/2816723.2816781"}],"container-title":["Proceedings of the ACM on Computer Graphics and Interactive Techniques"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3406183","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3406183","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:52Z","timestamp":1750195912000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3406183"}},"subtitle":["A Difference Evaluator for Alternating Images"],"short-title":[],"issued":{"date-parts":[[2020,8,26]]},"references-count":36,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,8,26]]}},"alternative-id":["10.1145\/3406183"],"URL":"https:\/\/doi.org\/10.1145\/3406183","relation":{},"ISSN":["2577-6193"],"issn-type":[{"value":"2577-6193","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,26]]},"assertion":[{"value":"2020-08-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}