{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T18:30:04Z","timestamp":1782412204289,"version":"3.54.5"},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:p>Automated image captioning has the potential to be a useful tool for people with vision impairments. Images taken by this user group are often noisy,  which leads to incorrect and even unsafe model predictions. In this paper, we propose a quality-agnostic framework to improve the performance and robustness of image captioning models for visually impaired people. We address this problem from three angles: data, model, and evaluation. First, we show how data augmentation techniques for generating synthetic noise can address data sparsity in this domain. Second, we enhance the robustness of the model by expanding a state-of-the-art model to a dual network architecture, using the augmented data and leveraging different consistency losses. Our results demonstrate increased performance, e.g. an absolute improvement of 2.15 on CIDEr, compared to state-of-the-art image captioning networks, as well as increased robustness to noise with up to 3 points improvement on CIDEr in more noisy settings. Finally, we evaluate the prediction reliability using confidence calibration on images with different difficulty \/ noise levels, showing that our models perform more reliably\n\nin safety-critical situations. The improved model is part of an assisted living application, which we develop in partnership with the Royal National Institute of Blind People.<\/jats:p>","DOI":"10.24963\/ijcai.2023\/697","type":"proceedings-article","created":{"date-parts":[[2023,8,11]],"date-time":"2023-08-11T08:31:30Z","timestamp":1691742690000},"page":"6281-6289","source":"Crossref","is-referenced-by-count":8,"title":["Quality-agnostic Image Captioning to Safely Assist People with Vision Impairment"],"prefix":"10.24963","author":[{"given":"Lu","family":"Yu","sequence":"first","affiliation":[{"name":"Tianjin University of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Malvina","family":"Nikandrou","sequence":"additional","affiliation":[{"name":"Heriot Watt University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiali","family":"Jin","sequence":"additional","affiliation":[{"name":"Tianjin University of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Verena","family":"Rieser","sequence":"additional","affiliation":[{"name":"Heriot Watt University"},{"name":"Alana AI"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10584","event":{"name":"Thirty-Second International Joint Conference on Artificial Intelligence {IJCAI-23}","theme":"Artificial Intelligence","location":"Macau, SAR China","acronym":"IJCAI-2023","number":"32","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"start":{"date-parts":[[2023,8,19]]},"end":{"date-parts":[[2023,8,25]]}},"container-title":["Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2023,8,11]],"date-time":"2023-08-11T08:54:05Z","timestamp":1691744045000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2023\/697"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2023,8]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2023\/697","relation":{},"subject":[],"published":{"date-parts":[[2023,8]]}}}