{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T13:23:06Z","timestamp":1758892986697},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,8]]},"abstract":"<jats:p>Image captioning models depend on training with paired image-text corpora, which poses various challenges in describing images containing novel objects absent from the training data. While previous novel object captioning methods rely on external image taggers or object detectors to describe novel objects, we present the Attention-based Novel Object Captioner (ANOC) that complements novel object captioners with human attention features that characterize generally important information independent of tasks. It introduces a gating mechanism that adaptively incorporates human attention with self-learned machine attention, with a Constrained Self-Critical Sequence Training method to address the exposure bias while maintaining constraints of novel object descriptions. Extensive experiments conducted on the nocaps and Held-Out COCO datasets demonstrate that our method considerably outperforms the state-of-the-art novel object captioners.\n\nOur source code is available at https:\/\/github.com\/chenxy99\/ANOC.<\/jats:p>","DOI":"10.24963\/ijcai.2021\/86","type":"proceedings-article","created":{"date-parts":[[2021,8,11]],"date-time":"2021-08-11T11:00:49Z","timestamp":1628679649000},"page":"622-628","source":"Crossref","is-referenced-by-count":13,"title":["Leveraging Human Attention in Novel Object Captioning"],"prefix":"10.24963","author":[{"given":"Xianyu","family":"Chen","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, University of Minnesota"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ming","family":"Jiang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Minnesota"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qi","family":"Zhao","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Minnesota"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"10584","event":{"number":"30","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"acronym":"IJCAI-2021","name":"Thirtieth International Joint Conference on Artificial Intelligence {IJCAI-21}","start":{"date-parts":[[2021,8,19]]},"theme":"Artificial Intelligence","location":"Montreal, Canada","end":{"date-parts":[[2021,8,27]]}},"container-title":["Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2021,8,11]],"date-time":"2021-08-11T11:01:17Z","timestamp":1628679677000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2021\/86"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2021,8]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2021\/86","relation":{},"subject":[],"published":{"date-parts":[[2021,8]]}}}