{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T17:30:11Z","timestamp":1791567011272,"version":"4.3.4"},"reference-count":52,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T00:00:00Z","timestamp":1658880000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T00:00:00Z","timestamp":1658880000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100014440","name":"Ministerio de Ciencia, Innovaci\u00f3n y Universidades","doi-asserted-by":"publisher","award":["PID2020-117142GB-I00"],"award-info":[{"award-number":["PID2020-117142GB-I00"]}],"id":[{"id":"10.13039\/100014440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003329","name":"Ministerio de Econom\u00eda y Competitividad","doi-asserted-by":"publisher","award":["RTI2018-095232-B-C22"],"award-info":[{"award-number":["RTI2018-095232-B-C22"]}],"id":[{"id":"10.13039\/501100003329","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014440","name":"Ministerio de Ciencia, Innovaci\u00f3n y Universidades","doi-asserted-by":"publisher","award":["PID2019-107255GB-C22"],"award-info":[{"award-number":["PID2019-107255GB-C22"]}],"id":[{"id":"10.13039\/100014440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006531","name":"Departament d\u2019Universitats, Recerca i Societat de la Informaci\u00f3","doi-asserted-by":"publisher","award":["2017-SGR-1414"],"award-info":[{"award-number":["2017-SGR-1414"]}],"id":[{"id":"10.13039\/501100006531","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100013774","name":"Universitat Oberta de Catalunya","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100013774","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Multimed Tools Appl"],"published-print":{"date-parts":[[2023,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The task of Language-guided Video Object Segmentation (LVOS) aims at generating binary masks for an object referred by a linguistic expression. When this expression unambiguously describes an object in the scene, it is named<jats:italic>referring expression<\/jats:italic>(RE). Our work argues that existing benchmarks used for LVOS are mainly composed of trivial cases, in which referents can be identified with simple phrases. Our analysis relies on a new categorization of the referring expressions in the DAVIS-2017 and Actor-Action datasets into trivial and non-trivial REs, where the non-trivial REs are further annotated with seven RE semantic categories. We leverage these data to analyze the performance of RefVOS, a novel neural network that obtains competitive results for the task of language-guided image segmentation and state of the art results for LVOS. Our study indicates that the major challenges for the task are related to understanding motion and static actions.<\/jats:p>","DOI":"10.1007\/s11042-022-13413-x","type":"journal-article","created":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T19:11:51Z","timestamp":1658949111000},"page":"4419-4438","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["A closer look at referring expressions for video object segmentation"],"prefix":"10.1007","volume":"82","author":[{"given":"Miriam","family":"Bellver","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3055-339X","authenticated-orcid":false,"given":"Carles","family":"Ventura","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carina","family":"Silberer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ioannis","family":"Kazakos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jordi","family":"Torres","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xavier","family":"Giro-i-Nieto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,7,27]]},"reference":[{"key":"13413_CR1","unstructured":"Anayurt H, Ozyegin SA, Cetin U, Aktas U, Kalkan S (2019) Searching for ambiguous objects in videos using relational referring expressions. In: Proceedings of the british machine vision conference (BMVC)"},{"key":"13413_CR2","doi-asserted-by":"crossref","unstructured":"Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) End-to-end object detection with transformers. In: European conference on computer vision, pp 213\u2013229. Springer","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"13413_CR3","doi-asserted-by":"crossref","unstructured":"Carreira J, Zisserman A (2017) Quo vadis, action recognition? a new model and the kinetics dataset. In: proceedings of the IEEE conference on computer vision and pattern recognition, pp 6299\u20136308","DOI":"10.1109\/CVPR.2017.502"},{"key":"13413_CR4","doi-asserted-by":"crossref","unstructured":"Chen DJ, Jia S, Lo YC, Chen HT, Liu TL (2019) See-through-text grouping for referring image segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 7454\u20137463","DOI":"10.1109\/ICCV.2019.00755"},{"key":"13413_CR5","unstructured":"Chen LC, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587"},{"key":"13413_CR6","unstructured":"Chen X, Ma L, Chen J, Jie Z, Liu W, Luo J (2018) Real-time referring expression comprehension by single-stage grounding network. arXiv:1812.03426"},{"key":"13413_CR7","unstructured":"Chen YW, Tsai YH, Wang T, Lin YY, Yang MH (2019) Referring expression object segmentation with caption-aware consistency. In: British machine vision conference (BMVC)"},{"key":"13413_CR8","doi-asserted-by":"crossref","unstructured":"Cirik V, Berg-Kirkpatrick T, Morency LP (2018) Using syntax to ground referring expressions in natural images AAAI","DOI":"10.1609\/aaai.v32i1.12343"},{"issue":"4","key":"13413_CR9","doi-asserted-by":"publisher","first-page":"1047","DOI":"10.2307\/2529886","volume":"38","author":"M Davies","year":"1982","unstructured":"Davies M, Fleiss JL (1982) Measuring agreement for multinomial data. Biometrics 38(4):1047\u20131051","journal-title":"Biometrics"},{"key":"13413_CR10","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics, pp 4171\u20134186"},{"key":"13413_CR11","doi-asserted-by":"crossref","unstructured":"Feng Q, Ablavsky V, Bai Q, Li G, Sclaroff S (2020) Real-time visual object tracking with natural language description. In: The IEEE winter conference on applications of computer vision, pp 700\u2013709","DOI":"10.1109\/WACV45572.2020.9093425"},{"key":"13413_CR12","doi-asserted-by":"crossref","unstructured":"Gavrilyuk K, Ghodrati A, Li Z, Snoek CG (2018) Actor and action video segmentation from a sentence. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5958\u20135966","DOI":"10.1109\/CVPR.2018.00624"},{"key":"13413_CR13","unstructured":"Hamp B, Feldweg H (1997) GermaNet - a lexical-semantic net for German. In: Automatic information extraction and building of lexical semantic resources for NLP applications. https:\/\/www.aclweb.org\/anthology\/W97-0802. Accessed 20 July 2022"},{"key":"13413_CR14","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"13413_CR15","unstructured":"Herv\u00e1s R, Finlayson M (2010) The prevalence of descriptive referring expressions in news and narrative. In: Proceedings of the ACL 2010 conference short papers, pp 49\u201354. Association for Computational Linguistics, Uppsala, Sweden. https:\/\/www.aclweb.org\/anthology\/P10-2010. Accessed 20 July 2022"},{"key":"13413_CR16","doi-asserted-by":"crossref","unstructured":"Hu R, Rohrbach M, Darrell T (2016) Segmentation from natural language expressions. In: European conference on computer vision, pp 108\u2013124. Springer","DOI":"10.1007\/978-3-319-46448-0_7"},{"key":"13413_CR17","doi-asserted-by":"crossref","unstructured":"Hu Z, Feng G, Sun J, Zhang L, Lu H (2020) Bi-directional relationship inferring network for referring image segmentation. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 4424\u20134433","DOI":"10.1109\/CVPR42600.2020.00448"},{"key":"13413_CR18","doi-asserted-by":"crossref","unstructured":"Huang S, Hui T, Liu S, Li G, Wei Y, Han J, Liu L, Li B (2020) Referring image segmentation via cross-modal progressive comprehension. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 10488\u201310497","DOI":"10.1109\/CVPR42600.2020.01050"},{"key":"13413_CR19","doi-asserted-by":"crossref","unstructured":"Johnson J, Hariharan B, van der Maaten L, Fei-Fei L, Lawrence Zitnick C, Girshick R (2017) Clevr: a diagnostic dataset for compositional language and elementary visual reasoning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2901\u20132910","DOI":"10.1109\/CVPR.2017.215"},{"key":"13413_CR20","doi-asserted-by":"crossref","unstructured":"Kazemzadeh S, Ordonez V, Matten M, Berg T (2014) ReferItGame: Referring to objects in photographs of natural scenes. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 787\u2013798. https:\/\/www.aclweb.org\/anthology\/D14-1086. Accessed 20 July 2022","DOI":"10.3115\/v1\/D14-1086"},{"key":"13413_CR21","doi-asserted-by":"crossref","unstructured":"Khoreva A, Rohrbach A, Schiele B (2018) Video object segmentation with language referring expressions. In: Asian conference on computer vision, pp 123\u2013141. Springer","DOI":"10.1007\/978-3-030-20870-7_8"},{"key":"13413_CR22","unstructured":"Kr\u00e4henb\u00fchl P, Koltun V (2011) Efficient inference in fully connected crfs with gaussian edge potentials. In: Advances in neural information processing systems, pp 109\u2013117"},{"key":"13413_CR23","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/6393.001.0001","volume-title":"Speaking: From Intention To Articulation","author":"WJM Levelt","year":"1989","unstructured":"Levelt WJM (1989) Speaking: From Intention To Articulation. MIT Press, Cambridge, MA"},{"key":"13413_CR24","doi-asserted-by":"crossref","unstructured":"Li Z, Tao R, Gavves E, Snoek CG, Smeulders AW (2017) Tracking by natural language specification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6495\u20136503","DOI":"10.1109\/CVPR.2017.777"},{"key":"13413_CR25","doi-asserted-by":"crossref","unstructured":"Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Doll\u00e1r P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer vision \u2013 ECCV 2014, pp 740\u2013755. Springer International Publishing. Cham","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"13413_CR26","doi-asserted-by":"crossref","unstructured":"Liu C, Lin Z, Shen X, Yang J, Lu X, Yuille A (2017) Recurrent multimodal interaction for referring image segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 1271\u20131280","DOI":"10.1109\/ICCV.2017.143"},{"key":"13413_CR27","doi-asserted-by":"crossref","unstructured":"Liu D, Zhang H, Wu F, Zha ZJ (2019) Learning to assemble neural module tree networks for visual grounding. In: Proceedings of the IEEE international conference on computer vision, pp 4673\u20134682","DOI":"10.1109\/ICCV.2019.00477"},{"key":"13413_CR28","doi-asserted-by":"crossref","unstructured":"Liu J, Wang L, Yang M (2017) Referring expression generation and comprehension via attributes. In: IEEE International conference on computer vision, ICCV 2017, Venice, Italy, october 22-29, 2017, pp 4866\u20134874. IEEE Computer Society","DOI":"10.1109\/ICCV.2017.520"},{"key":"13413_CR29","doi-asserted-by":"crossref","unstructured":"Liu R, Liu C, Bai Y, Yuille AL (2019) Clevr-ref+: Diagnosing visual reasoning with referring expressions. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 4185\u20134194","DOI":"10.1109\/CVPR.2019.00431"},{"key":"13413_CR30","doi-asserted-by":"crossref","unstructured":"Mao J, Huang J, Toshev A, Camburu O, Yuille AL, Murphy K (2016) Generation and comprehension of unambiguous object descriptions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 11\u201320","DOI":"10.1109\/CVPR.2016.9"},{"key":"13413_CR31","doi-asserted-by":"crossref","unstructured":"Nagaraja VK, Morariu VI, Davis LS (2016) Modeling context between objects for referring expression understanding. In: European conference on computer vision (ECCV)","DOI":"10.1007\/978-3-319-46493-0_48"},{"key":"13413_CR32","doi-asserted-by":"crossref","unstructured":"Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532\u20131543","DOI":"10.3115\/v1\/D14-1162"},{"key":"13413_CR33","doi-asserted-by":"crossref","unstructured":"Perazzi F, Pont-Tuset J, McWilliams B, Van Gool L, Gross M, Sorkine-Hornung A (2016) A benchmark dataset and evaluation methodology for video object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 724\u2013732","DOI":"10.1109\/CVPR.2016.85"},{"key":"13413_CR34","unstructured":"Pont-Tuset J, Perazzi F, Caelles S, Arbel\u00e1ez P, Sorkine-Hornung A, Van Gool L (2017) The 2017 davis challenge on video object segmentation. arXiv:1704.00675"},{"key":"13413_CR35","doi-asserted-by":"crossref","unstructured":"Qiu X, Sun T, Xu Y, Shao Y, Dai N, Huang X (2020) Pre-trained models for natural language processing: A survey. Sci China Technol Sci, pp 1\u201326","DOI":"10.1007\/s11431-020-1647-3"},{"key":"13413_CR36","doi-asserted-by":"crossref","unstructured":"Reiter E, Dale R (1992) A fast algorithm for the generation of referring expressions. In: COLING 1992 volume 1: The 15th international conference on computational linguistics. https:\/\/www.aclweb.org\/anthology\/C92-1038. Accessed 20 July 2022","DOI":"10.3115\/992066.992105"},{"key":"13413_CR37","doi-asserted-by":"publisher","first-page":"153","DOI":"10.3389\/fpsyg.2016.00153","volume":"7","author":"P Rubio-Fern\u00e1ndez","year":"2016","unstructured":"Rubio-Fern\u00e1ndez P (2016) How redundant are redundant color adjectives? an efficiency-based analysis of color overspecification. Front Psychol 7:153","journal-title":"Front Psychol"},{"key":"13413_CR38","doi-asserted-by":"crossref","unstructured":"Seo S, Lee JY, Han B (2020) Urvos: Unified referring video object segmentation network with a large-scale benchmark. In: Proceedings of the european conference on computer vision (ECCV)","DOI":"10.1007\/978-3-030-58555-6_13"},{"key":"13413_CR39","unstructured":"Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations (ICLR)"},{"key":"13413_CR40","doi-asserted-by":"crossref","unstructured":"Wang H, Deng C, Yan J, Tao D (2019) Asymmetric cross-guided attention network for actor and action video segmentation from natural language query. In: Proceedings of the IEEE international conference on computer vision, pp 3939\u20133948","DOI":"10.1109\/ICCV.2019.00404"},{"key":"13413_CR41","doi-asserted-by":"crossref","unstructured":"Wang P, Wu Q, Cao J, Shen C, Gao L, Van Den Hengel A (2018) Neighbourhood watch: Referring expression comprehension via language-guided graph attention networks. 2019 IEEE\/CVF Conf Comput Vis Pattern Recognit (CVPR), pp 1960\u20131968","DOI":"10.1109\/CVPR.2019.00206"},{"key":"13413_CR42","unstructured":"Xie E, Wang W, Yu Z, Anandkumar A, Alvarez JM, Luo P (2021) Segformer: Simple and efficient design for semantic segmentation with transformers. Adv Neural Inf Process Syst, vol 34"},{"key":"13413_CR43","doi-asserted-by":"crossref","unstructured":"Xu C, Hsieh SH, Xiong C, Corso JJ (2015) Can humans fly? action understanding with multiple classes of actors. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2264\u20132273","DOI":"10.1109\/CVPR.2015.7298839"},{"key":"13413_CR44","doi-asserted-by":"crossref","unstructured":"Xu N, Yang L, Fan Y, Yang J, Yue D, Liang Y, Price B, Cohen S, Huang T (2018) Youtube-vos: Sequence-to-sequence video object segmentation. In: Proceedings of the european conference on computer vision (ECCV), pp 585\u2013601","DOI":"10.1007\/978-3-030-01228-1_36"},{"key":"13413_CR45","doi-asserted-by":"crossref","unstructured":"Yang S, Li G, Yu Y (2019) Dynamic graph attention for referring expression comprehension. 2019 IEEE\/CVF Int Conf Comput Vis (ICCV), pp 4643\u20134652","DOI":"10.1109\/ICCV.2019.00474"},{"key":"13413_CR46","doi-asserted-by":"crossref","unstructured":"Ye L, Rochan M, Liu Z, Wang Y (2019) Cross-modal self-attention network for referring image segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 10502\u201310511","DOI":"10.1109\/CVPR.2019.01075"},{"key":"13413_CR47","unstructured":"Yi K, Gan C, Li Y, Kohli P, Wu J, Torralba A, Tenenbaum JB (2019) Clevrer: Collision events for video representation and reasoning. In: International conference on learning representations"},{"key":"13413_CR48","doi-asserted-by":"crossref","unstructured":"Yu L, Lin Z, Shen X, Yang J, Lu X, Bansal M, Berg TL (2018) Mattnet: Modular attention network for referring expression comprehension. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1307\u20131315","DOI":"10.1109\/CVPR.2018.00142"},{"key":"13413_CR49","doi-asserted-by":"crossref","unstructured":"Yu L, Poirson P, Yang S, Berg AC, Berg TL (2016) Modeling context in referring expressions. In: European conference on computer vision, pp 69\u201385. Springer","DOI":"10.1007\/978-3-319-46475-6_5"},{"key":"13413_CR50","doi-asserted-by":"crossref","unstructured":"Yu L, Tan H, Bansal M, Berg TL (2017) A joint speaker-listener-reinforcer model for referring expressions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7282\u20137290","DOI":"10.1109\/CVPR.2017.375"},{"key":"13413_CR51","doi-asserted-by":"crossref","unstructured":"Zhang C, Li W, Ouyang W, Wang Q, Kim WS, Hong S (2019) Referring expression comprehension with semantic visual relationship and word mapping. In: Proceedings of the 27th ACM International Conference on Multimedia, MM \u201919, pp 1258\u20131266. Association for Computing Machinery, Nice, France","DOI":"10.1145\/3343031.3351063"},{"key":"13413_CR52","doi-asserted-by":"publisher","unstructured":"Zhu Y, Kiros R, Zemel R, Salakhutdinov R, Urtasun R, Torralba A, Fidler S (2015) Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In: Proceedings of the 2015 IEEE international conference on computer vision (ICCV), ICCV \u201915, pp 19\u201327. IEEE Computer Society, USA. https:\/\/doi.org\/10.1109\/ICCV.2015.11","DOI":"10.1109\/ICCV.2015.11"}],"container-title":["Multimedia Tools and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-022-13413-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11042-022-13413-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-022-13413-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,30]],"date-time":"2024-09-30T00:46:14Z","timestamp":1727657174000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11042-022-13413-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,27]]},"references-count":52,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,1]]}},"alternative-id":["13413"],"URL":"https:\/\/doi.org\/10.1007\/s11042-022-13413-x","relation":{},"ISSN":["1380-7501","1573-7721"],"issn-type":[{"value":"1380-7501","type":"print"},{"value":"1573-7721","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,27]]},"assertion":[{"value":"4 December 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 March 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 July 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 July 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}