{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T21:41:41Z","timestamp":1790718101921,"version":"4.1.0"},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,8]]},"abstract":"<jats:p>Recent progress has been made in using attention based encoder-decoder framework for video captioning. However, most existing decoders apply the attention mechanism to every generated words including both visual words (e.g., \u201cgun\u201d and \"shooting\u201c) and non-visual words (e.g. \"the\u201c, \"a\u201d).However, these non-visual words can be easily predicted using natural language model without considering visual signals or attention.Imposing attention mechanism on non-visual words could mislead and decrease the overall performance of video captioning.To address this issue, we propose a hierarchical LSTM with adjusted temporal attention (hLSTMat) approach for video captioning. Specifically, the proposed framework utilizes the temporal attention for selecting specific frames to predict related words, while the adjusted temporal attention is for deciding whether to depend on the visual information or the language context information. Also, a hierarchical LSTMs is designed to simultaneously consider both low-level visual information and deep semantic information to support the video caption generation. To demonstrate the effectiveness of our proposed framework, we test our method on two prevalent datasets: MSVD and MSR-VTT, and experimental results show that our approach outperforms the state-of-the-art methods on both two datasets.<\/jats:p>","DOI":"10.24963\/ijcai.2017\/381","type":"proceedings-article","created":{"date-parts":[[2017,7,28]],"date-time":"2017-07-28T09:14:07Z","timestamp":1501233247000},"page":"2737-2743","source":"Crossref","is-referenced-by-count":115,"title":["Hierarchical LSTM with Adjusted Temporal Attention for Video Captioning"],"prefix":"10.24963","author":[{"given":"Jingkuan","family":"Song","sequence":"first","affiliation":[{"name":"University of Electronic Science and Technology of China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lianli","family":"Gao","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhao","family":"Guo","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wu","family":"Liu","sequence":"additional","affiliation":[{"name":"Beijing University of Posts and Telecommunications"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dongxiang","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Heng Tao","family":"Shen","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10584","event":{"name":"Twenty-Sixth International Joint Conference on Artificial Intelligence","theme":"Artificial Intelligence","location":"Melbourne, Australia","acronym":"IJCAI-2017","number":"26","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)","University of Technology Sydney (UTS)","Australian Computer Society (ACS)"],"start":{"date-parts":[[2017,8,19]]},"end":{"date-parts":[[2017,8,26]]}},"container-title":["Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2017,7,28]],"date-time":"2017-07-28T11:53:39Z","timestamp":1501242819000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2017\/381"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2017,8]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2017\/381","relation":{},"subject":[],"published":{"date-parts":[[2017,8]]}}}