{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T13:54:06Z","timestamp":1762091646908,"version":"build-2065373602"},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:p>Panoptic narrative grounding (PNG) aims to segment things and stuff objects in an image described by noun phrases of a narrative caption. As a multimodal task, an essential aspect of PNG is the visual-linguistic interaction between image and caption. The previous two-stage method aggregates visual contexts from offline-generated mask proposals to phrase features, which tend to be noisy and fragmentary. The recent one-stage method aggregates only pixel contexts from image features to phrase features, which may incur semantic misalignment due to lacking object priors. To realize more comprehensive visual-linguistic interaction, we propose to enrich phrases with coupled pixel and object contexts by designing a Phrase-Pixel-Object Transformer Decoder (PPO-TD), where both fine-grained part details and coarse-grained entity clues are aggregated to phrase features. In addition, we also propose a Phrase-Object Contrastive Loss (POCL) to pull closer the matched phrase-object pairs and push away unmatched ones for aggregating more precise object contexts from more phrase-relevant object tokens. Extensive experiments on the PNG benchmark show our method achieves new state-of-the-art performance with large margins.<\/jats:p>","DOI":"10.24963\/ijcai.2023\/99","type":"proceedings-article","created":{"date-parts":[[2023,8,11]],"date-time":"2023-08-11T08:31:30Z","timestamp":1691742690000},"page":"893-901","source":"Crossref","is-referenced-by-count":4,"title":["Enriching Phrases with Coupled Pixel and Object Contexts for Panoptic Narrative Grounding"],"prefix":"10.24963","author":[{"given":"Tianrui","family":"Hui","sequence":"first","affiliation":[{"name":"Institute of Information Engineering, Chinese Academy of Sciences"},{"name":"School of Cyber Security, University of Chinese Academy of Sciences"},{"name":"Meituan"}]},{"given":"Zihan","family":"Ding","sequence":"additional","affiliation":[{"name":"Institute of Artificial Intelligence, Beihang University"},{"name":"Meituan"}]},{"given":"Junshi","family":"Huang","sequence":"additional","affiliation":[{"name":"Meituan"}]},{"given":"Xiaoming","family":"Wei","sequence":"additional","affiliation":[{"name":"Meituan"}]},{"given":"Xiaolin","family":"Wei","sequence":"additional","affiliation":[{"name":"Meituan"}]},{"given":"Jiao","family":"Dai","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering, Chinese Academy of Sciences"},{"name":"School of Cyber Security, University of Chinese Academy of Sciences"}]},{"given":"Jizhong","family":"Han","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering, Chinese Academy of Sciences"},{"name":"School of Cyber Security, University of Chinese Academy of Sciences"}]},{"given":"Si","family":"Liu","sequence":"additional","affiliation":[{"name":"Institute of Artificial Intelligence, Beihang University"},{"name":"Hangzhou Innovation Institute, Beihang University"}]}],"member":"10584","event":{"number":"32","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"acronym":"IJCAI-2023","name":"Thirty-Second International Joint Conference on Artificial Intelligence {IJCAI-23}","start":{"date-parts":[[2023,8,19]]},"theme":"Artificial Intelligence","location":"Macau, SAR China","end":{"date-parts":[[2023,8,25]]}},"container-title":["Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2023,8,11]],"date-time":"2023-08-11T08:35:19Z","timestamp":1691742919000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2023\/99"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2023,8]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2023\/99","relation":{},"subject":[],"published":{"date-parts":[[2023,8]]}}}