{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:16:51Z","timestamp":1750220211167,"version":"3.41.0"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"2s","license":[{"start":{"date-parts":[[2022,6,30]],"date-time":"2022-06-30T00:00:00Z","timestamp":1656547200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"European Commission under European Horizon 2020 Programme","award":["951911 - AI4Media, 101004545 - ReInHerit"],"award-info":[{"award-number":["951911 - AI4Media, 101004545 - ReInHerit"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2022,6,30]]},"abstract":"<jats:p>Image quality assessment is often performed with deep networks that are fine-tuned to regress a human provided quality score of a given image. Usually, this approach may lack generalization capabilities and, while being highly precise on similar image distribution, it may yield lower correlation on unseen distortions. In particular, they show poor performances, whereas images corrupted by noise, blur, or compression have been restored by generative models. As a matter of fact, evaluation of these generative models is often performed providing anecdotal results to the reader. In the case of image enhancement and restoration, reference images are usually available. Nevertheless, using signal based metrics often leads to counterintuitive results: Highly natural crisp images may obtain worse scores than blurry ones. However, blind reference image assessment may rank images reconstructed with GANs higher than the original undistorted images. To avoid time-consuming human-based image assessment, semantic computer vision tasks may be exploited instead.<\/jats:p>\n          <jats:p>In this article, we advocate the use of language generation tasks to evaluate the quality of restored images. We refer to our assessment approach as LANguage-based Blind Image QUality Evaluation (LANBIQUE). We show experimentally that image captioning, used as a downstream task, may serve as a method to score image quality, independently of the distortion process that affects the data. Captioning scores are better aligned with human rankings with respect to classic signal based or No-reference image quality metrics. We show insights on how the corruption, by artefacts, of local image structure may steer image captions in the wrong direction.<\/jats:p>","DOI":"10.1145\/3538649","type":"journal-article","created":{"date-parts":[[2022,5,20]],"date-time":"2022-05-20T12:28:13Z","timestamp":1653049693000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["LANBIQUE: LANguage-based Blind Image QUality Evaluation"],"prefix":"10.1145","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7247-9407","authenticated-orcid":false,"given":"Leonardo","family":"Galteri","sequence":"first","affiliation":[{"name":"MICC - Universit\u00e0 degli Studi di Firenze, Firenze, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4816-0268","authenticated-orcid":false,"given":"Lorenzo","family":"Seidenari","sequence":"additional","affiliation":[{"name":"MICC - Universit\u00e0 degli Studi di Firenze, Firenze, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8217-6266","authenticated-orcid":false,"given":"Pietro","family":"Bongini","sequence":"additional","affiliation":[{"name":"MICC - Universit\u00e0 degli Studi di Firenze, Firenze, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1364-218X","authenticated-orcid":false,"given":"Marco","family":"Bertini","sequence":"additional","affiliation":[{"name":"MICC - Universit\u00e0 degli Studi di Firenze, Firenze, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1052-8322","authenticated-orcid":false,"given":"Alberto Del","family":"Bimbo","sequence":"additional","affiliation":[{"name":"MICC - Universit\u00e0 degli Studi di Firenze, Firenze, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,10,6]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46454-1_24"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00636"},{"key":"e_1_3_2_4_2","first-page":"65","volume-title":"Proceedings of the ACL Workshop","author":"Banerjee Satanjeev","year":"2005","unstructured":"Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop. 65\u201372."},{"key":"e_1_3_2_5_2","first-page":"675","volume-title":"Proceedings of the ICML","author":"Blau Yochai","year":"2019","unstructured":"Yochai Blau and Tomer Michaeli. 2019. Rethinking lossy compression: The rate-distortion-perception tradeoff. In Proceedings of the ICML. PMLR, 675\u2013685."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-30645-8_20"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2018.10.009"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2760518"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2013.2290508"},{"key":"e_1_3_2_10_2","article-title":"Microsoft COCO captions: Data collection and evaluation server","author":"Chen Xinlei","year":"2015","unstructured":"Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Doll\u00e1r, and C. Lawrence Zitnick. 2015. Microsoft COCO captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325 (2015).","journal-title":"arXiv preprint arXiv:1504.00325"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW53098.2021.00054"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01059"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.517"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2019.2895280"},{"key":"e_1_3_2_16_2","first-page":"1","volume-title":"Proceedings of the ACM Multimedia Asia","author":"Galteri Leonardo","year":"2021","unstructured":"Leonardo Galteri, Lorenzo Seidenari, Pietro Bongini, Marco Bertini, and Alberto Del Bimbo. 2021. Language based image quality assessment. In Proceedings of the ACM Multimedia Asia. 1\u20137."},{"key":"e_1_3_2_17_2","first-page":"2672","volume-title":"Proceedings of the NIPS","volume":"27","author":"Goodfellow Ian","year":"2014","unstructured":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the NIPS, Vol. 27. 2672\u20132680."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_19_2","first-page":"6629","volume-title":"Proceedings of the NIPS","volume":"30","author":"Heusel Martin","year":"2017","unstructured":"Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Proceedings of the NIPS, Vol. 30. 6629\u20136640."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.2967829"},{"key":"e_1_3_2_21_2","first-page":"448","volume-title":"Proceedings of the ICML","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the ICML. 448\u2013456."},{"volume-title":"Rec. ITU-R BT.500-13 - Methodology for the Subjective Assessment of the Quality of Television Pictures","year":"2012","key":"e_1_3_2_22_2","unstructured":"ITU 2012. Rec. ITU-R BT.500-13 - Methodology for the Subjective Assessment of the Quality of Television Pictures. ITU."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_3_2_24_2","volume-title":"Proceedings of the CVPR Workshops","author":"Kim Dong-Wook","year":"2019","unstructured":"Dong-Wook Kim, Jae Ryun Chung, and Seung-Won Jung. 2019. GRDN: Grouped residual dense network for real image denoising and GAN-based real-world noise modeling. In Proceedings of the CVPR Workshops."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.213"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.2987180"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-016-0981-7"},{"key":"e_1_3_2_28_2","first-page":"74","volume-title":"Proceedings of the ACL","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Proceedings of the ACL. 74\u201381. Retrieved from https:\/\/aclanthology.org\/W04-1013."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.118"},{"key":"e_1_3_2_30_2","first-page":"172","volume-title":"Proceedings of the Human Vision and Electronic Imaging XX","volume":"9394","author":"Lukin Vladimir V.","year":"2015","unstructured":"Vladimir V. Lukin, Nikolay N. Ponomarenko, Oleg I. Ieremeiev, Karen O. Egiazarian, and Jaakko Astola. 2015. Combining full-reference image visual quality metrics by neural network. In Proceedings of the Human Vision and Electronic Imaging XX, Vol. 9394. SPIE, 172\u2013183."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR48806.2021.9413095"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2012.2214050"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2012.2227726"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.23915\/distill.00003"},{"key":"e_1_3_2_35_2","first-page":"311","volume-title":"Proceedings of the ACL","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the ACL. 311\u2013318. Retrieved from https:\/\/aclanthology.org\/P02-1040."},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00244"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2015.2440172"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00194"},{"key":"e_1_3_2_39_2","first-page":"91","volume-title":"Proceedings of the NIPS","author":"Ren Shaoqing","year":"2015","unstructured":"Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of the NIPS. 91\u201399."},{"key":"e_1_3_2_40_2","first-page":"2234","volume-title":"Proceedings of the NIPS","volume":"29","author":"Salimans Tim","year":"2016","unstructured":"Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training GANs. In Proceedings of the NIPS, Vol. 29. 2234\u20132242."},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2006.881959"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01216-8_14"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR48806.2021.9412584"},{"key":"e_1_3_2_45_2","volume-title":"Proceedings of the ACCV","author":"Tran Linh Duy","year":"2020","unstructured":"Linh Duy Tran, Son Minh Nguyen, and Masayuki Arai. 2020. GAN-based noise model for denoising real images. In Proceedings of the ACCV."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3474085.3475683"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299087"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW54120.2021.00217"},{"key":"e_1_3_2_49_2","volume-title":"Proceedings of the ECCV Workshops","author":"Wang Xintao","year":"2018","unstructured":"Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. 2018. ESRGAN: Enhanced super-resolution generative adversarial networks. In Proceedings of the ECCV Workshops."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2002.5745362"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/QOMEX.2009.5246961"},{"key":"e_1_3_2_52_2","first-page":"6684","volume-title":"Proceedings of the CVPR","author":"Yoo Jaeyoung","year":"2018","unstructured":"Jaeyoung Yoo, Sang-ho Lee, and Nojun Kwak. 2018. Image restoration by estimating frequency distribution of local patches. In Proceedings of the CVPR. 6684\u20136692."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP42928.2021.9506075"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00281"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00068"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3538649","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3538649","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:03:03Z","timestamp":1750186983000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3538649"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,30]]},"references-count":54,"journal-issue":{"issue":"2s","published-print":{"date-parts":[[2022,6,30]]}},"alternative-id":["10.1145\/3538649"],"URL":"https:\/\/doi.org\/10.1145\/3538649","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2022,6,30]]},"assertion":[{"value":"2022-03-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-05-16","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-10-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}