{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,18]],"date-time":"2026-01-18T08:41:40Z","timestamp":1768725700843,"version":"3.49.0"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"EICS","license":[{"start":{"date-parts":[[2024,6,17]],"date-time":"2024-06-17T00:00:00Z","timestamp":1718582400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006374","name":"Fonds National de la Recherche Luxembourg","doi-asserted-by":"publisher","award":["C21\/IS\/15883550"],"award-info":[{"award-number":["C21\/IS\/15883550"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2024,6,17]]},"abstract":"<jats:p>One of the main benefits of large interactive surfaces (e.g. wall-sized displays) lies in their support for collocated collaboration by facilitating simultaneous interactions with the displays and high awareness of other group members' actions. In the context of remote collaboration, this awareness information needs to be acquired through digital means such as video feeds, which typically offer very limited information on non-verbal communication aspects, including on workspace awareness. We describe a new approach we have implemented to tackle that challenge through a multimodal pipeline that deals with tracking, attributing, transmitting, and visualising non-verbal information through what we refer to as workspace awareness cues, across wall-sized displays placed at distant locations. Our approach relies on commodity depth cameras combined with screen configuration information to generate deictic cues such as pointing targets and gaze direction. It also leverages recent artificial intelligence breakthroughs to attribute such cues to identified individuals and augment them with additional gestural interactions. In the present paper, we expand on the details and rationale behind our approach, describe its technical implementation, validate its novelty with regards to the existing literature, and report on early but promising results from an evaluation we conducted based on a mixed-presence decision-making scenario across two distant wall-sized displays.<\/jats:p>","DOI":"10.1145\/3664634","type":"journal-article","created":{"date-parts":[[2024,6,17]],"date-time":"2024-06-17T18:25:37Z","timestamp":1718648737000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Supporting Mixed-Presence Awareness across Wall-Sized Displays Using a Tracking Pipeline based on Depth Cameras"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2841-6708","authenticated-orcid":false,"given":"Adrien","family":"Coppens","sequence":"first","affiliation":[{"name":"Luxembourg Institute of Science and Technology, Esch-sur-Alzette, Luxembourg"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1198-5701","authenticated-orcid":false,"given":"Johannes","family":"Hermen","sequence":"additional","affiliation":[{"name":"Luxembourg Institute of Science and Technology, Esch-sur-Alzette, Luxembourg"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8645-5326","authenticated-orcid":false,"given":"Lou","family":"Schwartz","sequence":"additional","affiliation":[{"name":"Luxembourg Institute of Science and Technology, Esch-sur-Alzette, Luxembourg"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-5732-2385","authenticated-orcid":false,"given":"Christian","family":"Moll","sequence":"additional","affiliation":[{"name":"Luxembourg Institute of Science and Technology, Esch-sur-Alzette, Luxembourg"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0198-3729","authenticated-orcid":false,"given":"Val\u00e9rie","family":"Maquil","sequence":"additional","affiliation":[{"name":"Luxembourg Institute of Science and Technology, Esch-sur-Alzette, Luxembourg"}]}],"member":"320","published-online":{"date-parts":[[2024,6,17]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702448"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025604"},{"key":"e_1_2_2_3_1","volume-title":"Anne Loomis Thompson, and Eric Horvitz","author":"Bohus Dan","year":"2021","unstructured":"Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Mihai Jalobeanu, Patrick Sweeney, Anne Loomis Thompson, and Eric Horvitz. 2021. Platform for Situated Intelligence. arXiv:2103.15975 [cs.AI]"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/800250.807503"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1049\/iet-cvi.2016.0178"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2019.2929257"},{"key":"e_1_2_2_7_1","unstructured":"Debaleena Chattopadhyay and Davide Bolchini. 2013. Laid-Back Touchless Collaboration around Wall-size Displays: Visual Feedback and Affordances. (2013)."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2015.7177390"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-39862-4_18"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-70517-8_5"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580877"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2929.2006.02418.x"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2557500.2557541"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.2012.6180869"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/2788890.2788914"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40840-3_4"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.2005484"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240822"},{"key":"e_1_2_2_19_1","volume-title":"Design Patterns: Abstraction and Reuse of Object-Oriented Design. In ECOOP' 93 --- Object-Oriented Programming, Oscar M","author":"Gamma Erich","year":"1993","unstructured":"Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. 1993. Design Patterns: Abstraction and Reuse of Object-Oriented Design. In ECOOP' 93 --- Object-Oriented Programming, Oscar M. Nierstrasz (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 406--431."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441796"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1023\/a:1021271517844"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2307798.2307804"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","unstructured":"Pieter Hintjens Martin Hurton and Ian Barber. 2020. ZeroMQ Message Transport Protocol. RFC 23. https:\/\/doi.org\/10.17487\/RFC9405","DOI":"10.17487\/RFC9405"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/142750.142977"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2576099"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1080\/00405849909543834"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/1839214.1839261"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2307798.2307811"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2018.2874647"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/iV.2015.70"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2166966.2166983"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1880071.1880107"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/CW.2010.17"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2865235"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3610661.3617149"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14539"},{"key":"e_1_2_2_37_1","unstructured":"Zicheng Liu. 2019. 3D Skeletal Tracking on Azure Kinect. (2019). https:\/\/cvme.github.io\/ CVPR2019 Workshop on 3D Computer Vision in Medical Environments."},{"key":"e_1_2_2_38_1","volume-title":"Juhyun Lee, Wan-Teh Chang, Wei Hua, Manfred Georg, and Matthias Grundmann.","author":"Lugaresi Camillo","year":"2019","unstructured":"Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, Wan-Teh Chang, Wei Hua, Manfred Georg, and Matthias Grundmann. 2019. MediaPipe: A Framework for Building Perception Pipelines. arXiv:1906.08172 [cs.DC]"},{"key":"e_1_2_2_39_1","volume-title":"International Conference on Artificial Reality and Telexistence. 4--9.","author":"Maimone Andrew","year":"2011","unstructured":"Andrew Maimone and Henry Fuchs. 2011. A First Look at a Telepresence System with Room-Sized Real-Time 3D Capture and Life-Sized Tracked Display Wall. In International Conference on Artificial Reality and Telexistence. 4--9."},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544549.3585830"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/0925-2312(91)90023-5"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766448"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1721355115"},{"key":"e_1_2_2_44_1","volume-title":"Proceedings of WACE","volume":"9","author":"Renambot Luc","year":"2004","unstructured":"Luc Renambot, Arun Rao, Rajvikram Singh, Byungil Jeong, Naveen Krishnaprasad, Venkatram Vishwanath, Vaidya Chandrasekhar, Nicholas Schwarz, Allan Spale, Charles Zhang, et al. 2004. Sage: the scalable adaptive graphics environment. In Proceedings of WACE, Vol. 9."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-73111-5_81"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/cvpr.2015.7298682"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.494"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-07788-8_19"},{"key":"e_1_2_2_49_1","unstructured":"Shigeki Takahashi. 2020. hand-gesture-recognition-using-mediapipe. https:\/\/github.com\/Kazuhito00\/hand-gesture-recognition-using-mediapipe"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/108844.108932"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/253671.253708"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531073.3534483"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/642611.642702"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2909132.2909258"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.10.081"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","unstructured":"Moritz Wiechers Alexander Nolte Michael Ksoll Thomas Herrmann and Andrea Kienle. 2013. User Tracking for Collaboration on Interactive Wall-Sized Displays.. In Mensch & Computer. 191--200. https:\/\/doi.org\/10.1524\/9783486781229","DOI":"10.1524\/9783486781229"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2971485.2971503"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/1958824.1958839"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2838739.2838819"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2494091.2494148"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3664634","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3664634","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T23:21:28Z","timestamp":1755904888000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3664634"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,17]]},"references-count":60,"journal-issue":{"issue":"EICS","published-print":{"date-parts":[[2024,6,17]]}},"alternative-id":["10.1145\/3664634"],"URL":"https:\/\/doi.org\/10.1145\/3664634","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,17]]},"assertion":[{"value":"2024-02-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}