{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T17:25:18Z","timestamp":1791566718286,"version":"4.3.4"},"reference-count":45,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2021,11,22]],"date-time":"2021-11-22T00:00:00Z","timestamp":1637539200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2018YFB1402600"],"award-info":[{"award-number":["2018YFB1402600"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62072423"],"award-info":[{"award-number":["62072423"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2022,7,31]]},"abstract":"<jats:p>Person search has long been treated as a crucial and challenging task to support deeper insight in personalized summarization and personality discovery. Traditional methods, e.g., person re-identification and face recognition techniques, which profile video characters based on visual information, are often limited by relatively fixed poses or small variation of viewpoints and suffer from more realistic scenes with high motion complexity (e.g., movies). At the same time, long videos such as movies often have logical story lines and are composed of continuously developmental plots. In this situation, different persons usually meet on a specific occasion, in which informative social cues are performed. We notice that these social cues could semantically profile their personality and benefit person search task in two aspects. First, persons with certain relationships usually co-occur in short intervals; in case one of them is easier to be identified, the social relation cues extracted from their co-occurrences could further benefit the identification for the harder ones. Second, social relations could reveal the association between certain scenes and characters (e.g., classmate relationship may only exist among students), which could narrow down candidates into certain persons with a specific relationship. In this way, high-level social relation cues could improve the effectiveness of person search. Along this line, in this article, we propose a social context-aware framework, which fuses visual and social contexts to profile persons in more semantic perspectives and better deal with person search task in complex scenarios. Specifically, we first segment videos into several independent scene units and abstract out social contexts within these scene units. Then, we construct inner-personal links through a graph formulation operation for each scene unit, in which both visual cues and relation cues are considered. Finally, we perform a relation-aware label propagation to identify characters\u2019 occurrences, combining low-level semantic cues (i.e., visual cues) and high-level semantic cues (i.e., relation cues) to further enhance the accuracy. Experiments on real-world datasets validate that our solution outperforms several competitive baselines.<\/jats:p>","DOI":"10.1145\/3480967","type":"journal-article","created":{"date-parts":[[2021,11,22]],"date-time":"2021-11-22T17:41:55Z","timestamp":1637602915000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Social Context-aware Person Search in Videos via Multi-modal Cues"],"prefix":"10.1145","volume":"40","author":[{"given":"Dan","family":"Li","sequence":"first","affiliation":[{"name":"University of Science and Technology of China, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tong","family":"Xu","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peilun","family":"Zhou","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weidong","family":"He","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanbin","family":"Hao","sequence":"additional","affiliation":[{"name":"City University of Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi","family":"Zheng","sequence":"additional","affiliation":[{"name":"Huawei Technologies, Hangzhou, Zhejiang, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Enhong","family":"Chen","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,11,22]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00530-010-0182-0"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2016.7477590"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.126.2.187"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00644"},{"key":"e_1_3_3_6_2","doi-asserted-by":"crossref","unstructured":"Qiong Cao Li Shen Weidi Xie Omkar M. Parkhi and Andrew Zisserman. 2017. VGGFace2: A dataset for recognising faces across pose and age. https:\/\/ieeexplore.ieee.org\/abstract\/document\/8373813.","DOI":"10.1109\/FG.2018.00020"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00225"},{"key":"e_1_3_3_8_2","doi-asserted-by":"crossref","unstructured":"Di Chen Shanshan Zhang Wanli Ouyang Jian Yang and Ying Tai. 2018. Person search via a mask-guided two-stream CNN model. https:\/\/openaccess.thecvf.com\/content_ECCV_2018\/html\/Di_Chen_Person_Search_via_ECCV_2018_paper.html.","DOI":"10.1007\/978-3-030-01234-2_45"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.149"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00482"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2015.04.005"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2014.2359646"},{"key":"e_1_3_3_14_2","unstructured":"Mark Everingham Luc Van Gool Christopher K. I. Williams John Winn and Andrew Zisserman. 2007. The PASCAL visual object classes challenge 2007 (VOC2007) results. http:\/\/citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.478.6547&rep=rep1&type=pdf."},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240508.3240541"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46487-9_6"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01261-8_26"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR.2014.61"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2015.7301352"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2019.2946477"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2015.2440174"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.27"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298832"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.5555\/3016100.3016322"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.126.6.844"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00720"},{"key":"e_1_3_3_29_2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).","journal-title":"arXiv preprint arXiv:1409.1556"},{"key":"e_1_3_3_30_2","volume-title":"Proceedings of the International Conference on Machine learning Workshop","author":"Srivastava Nitish","year":"2012","unstructured":"Nitish Srivastava and Ruslan Salakhutdinov. 2012. Learning representations for multimodal data with deep belief nets. In Proceedings of the International Conference on Machine learning Workshop."},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.5555\/2999325.2999383"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.54"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2016.7477702"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/MMUL.2011.22"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2007.190672"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00677"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.360"},{"key":"e_1_3_3_38_2","unstructured":"Xiaojin Zhu and Zoubin Ghahramani. 2002. Learning from labeled and unlabeled data with label propagation. http:\/\/citeseerx.ist.psu.edu\/viewdoc\/summary?doi=10.1.1.13.8280."},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.597"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2016.2603342"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299113"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.474"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00731"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.531"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2018.2873599"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2014.2315595"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3480967","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3480967","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:23Z","timestamp":1750191503000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3480967"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,22]]},"references-count":45,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,7,31]]}},"alternative-id":["10.1145\/3480967"],"URL":"https:\/\/doi.org\/10.1145\/3480967","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,22]]},"assertion":[{"value":"2020-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-11-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}