{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,14]],"date-time":"2026-05-14T14:57:35Z","timestamp":1778770655322,"version":"3.51.4"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2022,9,22]],"date-time":"2022-09-22T00:00:00Z","timestamp":1663804800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"crossref","award":["N00014-17-1-2175"],"award-info":[{"award-number":["N00014-17-1-2175"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2022,12,31]]},"abstract":"<jats:p>The ability to anticipate human actions is critical to many cyber-physical systems, such as robots and autonomous vehicles. Computer vision and sensing algorithms to date have focused on extracting and predicting visual features that are explicit in the scene, such as color, appearance, actions, positions, and velocities, using video and physical measurements, such as object depth and motion. Human actions, however, are intrinsically influenced and motivated by many implicit factors such as context, human roles and interactions, past experience, and inner goals or intentions. For example, in a sport team, the team strategy, player role, and dynamic circumstances driven by the behavior of the opponents, all influence the actions of each player. This article proposes a holistic framework for incorporating visual features, as well as hidden information, such as social roles, and domain knowledge. The approach, relying on a novel dynamic Markov random field (DMRF) model, infers the instantaneous team strategy and, subsequently, the players\u2019 roles that are temporally evolving throughout the game. The results from the DMRF inference stage are then integrated with instantaneous visual features, such as individual actions and position, in order to perform holistic action anticipation using a multi-layer perceptron (MLP). The approach is demonstrated on the team sport of volleyball, by first training the DMRF and MLP offline with past videos, and, then, by applying them to new volleyball videos online. These results show that the method is able to infer the players\u2019 roles with an average accuracy of 86.99%, and anticipate future actions over a sequence of up to 46 frames with an average accuracy of 80.50%. Additionally, the method predicts the onset and duration of each action achieving a mean relative error of 14.57% and 15.67%, respectively.<\/jats:p>","DOI":"10.1145\/3531230","type":"journal-article","created":{"date-parts":[[2022,5,28]],"date-time":"2022-05-28T11:26:41Z","timestamp":1653737201000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["A Holistic Approach for Role Inference and Action Anticipation in Human Teams"],"prefix":"10.1145","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7366-8907","authenticated-orcid":false,"given":"Junyi","family":"Dong","sequence":"first","affiliation":[{"name":"Cornell University, Ithaca, New York"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3349-6180","authenticated-orcid":false,"given":"Qingze","family":"Huo","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, New York"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7652-6311","authenticated-orcid":false,"given":"Silvia","family":"Ferrari","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, New York"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,9,22]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1020281327116"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2762355"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00808"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.365"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00626"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW50498.2020.00519"},{"key":"e_1_3_1_8_2","volume-title":"Pattern Recognition and Machine Learning","author":"Bishop Christopher M.","year":"2006","unstructured":"Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning. springer."},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2010.09.017"},{"key":"e_1_3_1_10_2","first-page":"21","volume-title":"Proceedings of the Asian Conference on Computer Vision","author":"Chakraborty Anirban","year":"2014","unstructured":"Anirban Chakraborty and Amit K. Roy-Chowdhury. 2014. Context-aware activity forecasting. In Proceedings of the Asian Conference on Computer Vision. Springer, Singapore, 21\u201336."},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-60298-4_321"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2017.2771231"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/TBDATA.2021.3063048"},{"key":"e_1_3_1_14_2","volume-title":"Proceedings of the European Conference of Computer Vision","author":"Cheshire Evan","year":"2013","unstructured":"Evan Cheshire, Cibele Halasz, and Jose Krause Perin. 2013. Player tracking and analysis of basketball plays. In Proceedings of the European Conference of Computer Vision."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jeconom.2009.08.003"},{"key":"e_1_3_1_16_2","volume-title":"The Nature of Explanation","author":"Craik Kenneth James Williams","year":"1952","unstructured":"Kenneth James Williams Craik. 1952. The Nature of Explanation. Vol. 445. CUP Archive."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2003.1236770"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.23919\/ACC45564.2020.9147689"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447686"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1117\/12.526813"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2012.6247805"},{"key":"e_1_3_1_22_2","volume-title":"Information-Driven Planning and Control","author":"Ferrari Silvia","year":"2021","unstructured":"Silvia Ferrari and Thomas A. Wettergren. 2021. Information-Driven Planning and Control. MIT Press."},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00635"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00566"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.1984.4767596"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.17323\/978-5-7598-2337-7"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5540048"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CRV.2011.12"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.3390\/mti4020033"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.322"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10584-0_32"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_44"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.217"},{"key":"e_1_3_1_34_2","first-page":"287","volume-title":"Proceedings of the Robot Soccer World Cup","author":"Iocchi Luca","year":"2006","unstructured":"Luca Iocchi. 2006. Robust color segmentation through adaptive color distribution transformation. In Proceedings of the Robot Soccer World Cup. Springer, 287\u2013295."},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/11815921_1"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01016"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.5555\/1577069.1755843"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.5555\/1795555"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10578-9_45"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.2297321"},{"key":"e_1_3_1_41_2","unstructured":"Ralph Linton. 1936. The study of man: An introduction. D. Appleton-Century company incorporated."},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.616"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1063\/1.2770961"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2019.107125"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2019.00354"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1561\/0600000033"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.320"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40279-017-0786-z"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40279-019-01104-x"},{"key":"e_1_3_1_50_2","first-page":"1","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Rodriguez Cristian","year":"2018","unstructured":"Cristian Rodriguez, Basura Fernando, and Hongdong Li. 2018. Action anticipation by predicting future dynamic images. In Proceedings of the European Conference on Computer Vision. Munich, 1\u201310."},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.39"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206544"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01249-6_19"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.453"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2020.107511"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2018.2797266"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2914577"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2018.00227"},{"key":"e_1_3_1_59_2","volume-title":"Cengage Advantage Books: Introduction to Sociology","author":"Tischler Henry L.","year":"2013","unstructured":"Henry L. Tischler. 2013. Cengage Advantage Books: Introduction to Sociology. Cengage Learning."},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/1899412.1899419"},{"key":"e_1_3_1_61_2","article-title":"VLFeat: An open and portable library of computer vision algorithms","author":"Vedaldi A.","year":"2008","unstructured":"A. Vedaldi and B. Fulkerson. 2008. VLFeat: An open and portable library of computer vision algorithms. Retrieved from http:\/\/www.vlfeat.org\/.","journal-title":"Retrieved from http:\/\/www.vlfeat.org\/"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.18"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46478-7_51"},{"issue":"6","key":"e_1_3_1_64_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3232230","article-title":"Random-forest-inspired neural networks","volume":"9","author":"Wang Suhang","year":"2018","unstructured":"Suhang Wang, Charu Aggarwal, and Huan Liu. 2018. Random-forest-inspired neural networks. ACM Transactions on Intelligent Systems and Technology 9, 6 (2018), 1\u201325.","journal-title":"ACM Transactions on Intelligent Systems and Technology"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.psychsport.2019.02.001"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/2890104"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_18"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240508.3240572"},{"issue":"3","key":"e_1_3_1_69_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2168752.2168757","article-title":"Robust visual tracking using an effective appearance model based on sparse coding","volume":"3","author":"Zhang Shengping","year":"2012","unstructured":"Shengping Zhang, Hongxun Yao, Xin Sun, and Shaohui Liu. 2012. Robust visual tracking using an effective appearance model based on sparse coding. ACM Transactions on Intelligent Systems and Technology 3, 3 (2012), 1\u201318.","journal-title":"ACM Transactions on Intelligent Systems and Technology"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/2629483"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3531230","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3531230","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3531230","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:31:31Z","timestamp":1750188691000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3531230"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,22]]},"references-count":69,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,12,31]]}},"alternative-id":["10.1145\/3531230"],"URL":"https:\/\/doi.org\/10.1145\/3531230","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,22]]},"assertion":[{"value":"2021-12-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-04-11","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-09-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}