{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T17:49:17Z","timestamp":1776102557038,"version":"3.50.1"},"reference-count":48,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,1,27]],"date-time":"2025-01-27T00:00:00Z","timestamp":1737936000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The European Union\u2019s Horizon 2020 within the framework SESAR 2020 research and innovation program","award":["101114838"],"award-info":[{"award-number":["101114838"]}]},{"name":"The European Union\u2019s Horizon 2020 within the framework SESAR 2020 research and innovation program","award":["101114765"],"award-info":[{"award-number":["101114765"]}]},{"name":"Trustworthy intelligent system for remote digital tower, TRUSTY","award":["101114838"],"award-info":[{"award-number":["101114838"]}]},{"name":"Trustworthy intelligent system for remote digital tower, TRUSTY","award":["101114765"],"award-info":[{"award-number":["101114765"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>This research explores the design and evaluation of a webcam-based presentation tool that enables presenters to directly interact with web content via free-hand gestures. Our approach consists of overlaying the webcam video feed on top of web browser content to enable live presentations of any webpage. To support interactive presentations, we designed free-hand gesture interactions with the webpage to enable pointing, clicking, panning, and zooming interactions. We propose three alternatives to enable free-hand clicking: dwell time, modal key control, and a pinching interaction technique. We conducted an exploratory user study of these alternative designs to gather insights on the usability of such systems from a presenter point of view, with a focus on understanding the impact of the three techniques on flow interruptions. The results indicate that the system we propose can be used to deliver presentations effectively and that natural gestures do not disturb the flow of the presentation.<\/jats:p>","DOI":"10.3390\/mti9020010","type":"journal-article","created":{"date-parts":[[2025,1,27]],"date-time":"2025-01-27T04:59:10Z","timestamp":1737953950000},"page":"10","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["The Presenter in the Browser: Design and Evaluation of Human Interactive Overlays with Web Content"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9732-4874","authenticated-orcid":false,"given":"Maxime","family":"Cordeil","sequence":"first","affiliation":[{"name":"School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane, QLD 4072, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0032-2953","authenticated-orcid":false,"given":"Anais","family":"Servais","sequence":"additional","affiliation":[{"name":"GIGA Research\u2014CRC Human Imaging, Li\u00e8ge University, 4000 Li\u00e8ge, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-1516-9623","authenticated-orcid":false,"given":"Guillaume","family":"Truong","sequence":"additional","affiliation":[{"name":"F\u00e9d\u00e9ration ENAC ISAE-SUPAERO ONERA, 31055 Toulouse, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9076-9571","authenticated-orcid":false,"given":"Tim","family":"Dwyer","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Monash University, Melbourne, VIC 3800, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9356-0976","authenticated-orcid":false,"given":"Dhaval","family":"Vyas","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane, QLD 4072, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4318-6717","authenticated-orcid":false,"given":"Christophe","family":"Hurter","sequence":"additional","affiliation":[{"name":"F\u00e9d\u00e9ration ENAC ISAE-SUPAERO ONERA, 31055 Toulouse, France"},{"name":"IPAL, International Research Lab (IRL2955) CNRS-A*STAR-NUS, Singapore 138632, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,1,27]]},"reference":[{"key":"ref_1","unstructured":"Zhang, F., Bazarevsky, V., Vakunov, A., Tkachenka, A., Sung, G., Chang, C.L., and Grundmann, M. (2020). Mediapipe hands: On-device real-time hand tracking. arXiv."},{"key":"ref_2","unstructured":"Hall, B.D., Bartram, L., and Brehmer, M. (November, January 29). Augmented Chironomia for Presenting Data to Remote Audiences. Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (UIST \u201922), Bend, OR, USA."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Davis, J.U., Asente, P., and Yang, X.D. (2023, January 10\u201314). Multimodal Direct Manipulation in Video Conferencing: Challenges and Opportunities. Proceedings of the 2023 ACM Designing Interactive Systems Conference (DIS \u201923), Pittsburgh, PA, USA.","DOI":"10.1145\/3563657.3596099"},{"key":"ref_4","unstructured":"Liao, J., Karim, A., Jadon, S.S., Kazi, R.H., and Suzuki, R. (November, January 29). RealityTalk: Real-Time Speech-Driven Augmented Presentation for AR Live Storytelling. Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (UIST \u201922), Bend, OR, USA."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Fourney, A., Terry, M., and Mann, R. (2010, January 6\u201310). Gesturing in the wild: Understanding the effects and implications of gesture-based interaction for dynamic presentations. Proceedings of the 24th BCS Interaction Specialist Group Conference (BCS \u201910), Dundee, UK.","DOI":"10.14236\/ewic\/HCI2010.29"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2416","DOI":"10.1109\/TVCG.2013.191","article-title":"SketchStory: Telling more engaging stories with data through freeform sketching","volume":"19","author":"Lee","year":"2013","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Brosz, J., Nacenta, M.A., Pusch, R., Carpendale, S., and Hurter, C. (2013, January 8\u201311). Transmogrification: Causal Manipulation of Visualizations. Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST \u201913), St. Andrews, UK.","DOI":"10.1145\/2501988.2502046"},{"key":"ref_8","unstructured":"Perlin, K., He, Z., and Rosenberg, K. (2018). Chalktalk: A Visualization and Communication Language\u2013As a Tool in the Domain of Computer Science Education. arXiv."},{"key":"ref_9","unstructured":"Liu, X.B., Kirilyuk, V., Yuan, X., Chi, P., Olwal, A., Chen, X.A., and Du, R. (November, January 29). Experiencing Augmented Communication with Real-time Visuals using Large Language Models in Visual Captions. Proceedings of the Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST), San Francisco, CA, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Saquib, N., Kazi, R.H., Wei, L.Y., and Li, W. (2019, January 4\u20139). Interactive Body-Driven Graphics for Augmented Video Performance. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI \u201919), Glasgow, UK.","DOI":"10.1145\/3290605.3300852"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Riche, N.H., Hurter, C., Diakopoulos, N., and Carpendale, S. (2018). Data-Driven Storytelling, CRC Press.","DOI":"10.1201\/9781315281575"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1145\/253671.253708","article-title":"Post-WIMP user interfaces","volume":"40","year":"1997","journal-title":"Commun. ACM"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1080\/07370024.1990.9667153","article-title":"A semantic analysis of the design space of input devices","volume":"5","author":"Mackinlay","year":"1990","journal-title":"Hum.\u2013Comput. Interact."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/174630.174631","article-title":"Integrality and Separability of Input Devices","volume":"1","author":"Jacob","year":"1994","journal-title":"ACM Trans. Comput.-Hum. Interact."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wolf, D., Gugenheimer, J., Combosch, M., and Rukzio, E. (2020, January 25\u201330). Understanding the Heisenberg Effect of Spatial Interaction: A Selection Induced Error for Spatially Tracked Input Devices. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI \u201920), Honolulu, HI, USA.","DOI":"10.1145\/3313831.3376876"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Vogel, D., and Balakrishnan, R. (2005, January 23\u201326). Distant freehand pointing and clicking on very large, high resolution displays. Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, Seattle, WA, USA.","DOI":"10.1145\/1095034.1095041"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bragdon, A., DeLine, R., Hinckley, K., and Morris, M.R. (2011, January 13\u201316). Code space: Touch + air gesture hybrid interactions for supporting developer meetings. Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS \u201911), New York, NY, USA.","DOI":"10.1145\/2076354.2076393"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s11042-010-0698-5","article-title":"Point & Click Mediated Interactions for Large Home Entertainment Displays","volume":"59","author":"Vatavu","year":"2012","journal-title":"Multimed. Tools Appl."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Pfeuffer, K., Mayer, B., Mardanbegi, D., and Gellersen, H. (2017, January 16\u201317). Gaze + pinch interaction in virtual reality. Proceedings of the 5th Symposium on Spatial User Interaction (SUI \u201917), Brighton, UK.","DOI":"10.1145\/3131277.3132180"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2687","DOI":"10.1007\/s11042-013-1501-1","article-title":"Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware","volume":"74","author":"Yeo","year":"2015","journal-title":"Multimed. Tools Appl."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s42979-020-00223-x","article-title":"An efficient human computer interaction through hand gesture using deep convolutional neural network","volume":"1","author":"Islam","year":"2020","journal-title":"SN Comput. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Chunduru, V., Roy, M., and Chittawadigi, R.G. (October, January 30). Hand tracking in 3d space using mediapipe and pnp method for intuitive control of virtual globe. Proceedings of the 2021 IEEE 9th Region 10 Humanitarian Technology Conference (R10-HTC), Bengaluru, India.","DOI":"10.1109\/R10-HTC53172.2021.9641587"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"van de Camp, F., Schick, A., and Stiefelhagen, R. (2013, January 21\u201326). How to click in mid-air. Proceedings of the Distributed, Ambient, and Pervasive Interactions: First International Conference, DAPI 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA. Proceedings 1.","DOI":"10.1007\/978-3-642-39351-8_9"},{"key":"ref_24","unstructured":"Hansen, J.P., Johansen, A.S., Hansen, D.W., Ito, K., and Mashino, S. (2003, January 22\u201327). Command without a click: Dwell time typing by mouse and gaze selections. Proceedings of the 10th International Conference on Human-Computer Interaction, Crete, Greece."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Yoo, S., Parker, C., Kay, J., and Tomitsch, M. (2015, January 7\u201310). To dwell or not to dwell: An evaluation of mid-air gestures for large information displays. Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction, Parkville, VIC, Australia.","DOI":"10.1145\/2838739.2838819"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1145\/3567718","article-title":"Push, Tap, Dwell, and Pinch: Evaluation of Four Mid-Air Selection Methods Augmented with Ultrasonic Haptic Feedback","volume":"6","author":"Dube","year":"2022","journal-title":"Proc. ACM Hum.-Comput. Interact."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Gupta, A., Pietrzak, T., Yau, C., Roussel, N., and Balakrishnan, R. (2017, January 17\u201320). Summon and Select: Rapid Interaction with Interface Controls in Mid-Air. Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS \u201917), Brighton, UK.","DOI":"10.1145\/3132272.3134120"},{"key":"ref_28","unstructured":"Csikszentmihalyi, M. (1990). Flow: The Psychology of Optimal Experience, Harper & Row."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Hurter, C., Girouard, A., Riche, N., and Plaisant, C. (2011, January 7\u201312). Active Progress Bars: Facilitating the Switch to Temporary Activities. Proceedings of the CHI \u201911 Extended Abstracts on Human Factors in Computing Systems (CHI EA \u201911), Vancouver, BC, Canada.","DOI":"10.1145\/1979742.1979883"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"495","DOI":"10.3758\/PBR.15.3.495","article-title":"Visible embodiment: Gestures as simulated action","volume":"15","author":"Hostetter","year":"2008","journal-title":"Psychon. Bull. Rev."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Czerwinski, M., Horvitz, E., and Wilhite, S. (2004, January 24\u201329). A diary study of task switching and interruptions. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vienna, Austria.","DOI":"10.1145\/985692.985715"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Borst, J.P., Taatgen, N.A., and van Rijn, H. (2015, January 18\u201323). What makes interruptions disruptive? A process-model account of the effects of the problem state bottleneck on task interruption and resumption. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Republic of Korea.","DOI":"10.1145\/2702123.2702156"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1080\/01449290310001644859","article-title":"Long-term working memory and interrupting messages in human\u2013computer interaction","volume":"23","author":"Oulasvirta","year":"2004","journal-title":"Behav. Inf. Technol."},{"key":"ref_34","unstructured":"Speier, C., Valacich, J., and Vessey, I. (1997, January 14\u201317). The effects of task interruption and information presentation on individual decision making. Proceedings of the International Conference on Information Systems (ICIS), Atlanta, GA, USA."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1109\/76.718513","article-title":"Summarization of videotaped presentations: Automatic analysis of motion and gesture","volume":"8","author":"Ju","year":"1998","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Matsumoto, D., Frank, M.G., and Hwang, H.S. (2012). Nonverbal Communication: Science and Applications, Sage Publications.","DOI":"10.4135\/9781452244037"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hosseini, M., Ihmels, T., Chen, Z., Koelle, M., M\u00fcller, H., and Boll, S. (2023, January 23\u201328). Towards a Consensus Gesture Set: A Survey of Mid-Air Gestures in HCI for Maximized Agreement Across Domains. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI \u201923), Hamburg, Germany.","DOI":"10.1145\/3544548.3581420"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Shi, Y., Taib, R., and Lichman, S. (2006, January 5\u20138). GestureCam: A Smart Camera for Gesture Recognition and Gesture-Controlled Web Navigation. Proceedings of the 2006 9th International Conference on Control, Automation, Robotics and Vision, Singapore.","DOI":"10.1109\/ICARCV.2006.345267"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"012018","DOI":"10.1088\/1757-899X\/528\/1\/012018","article-title":"The mental workload analysis of staff in study program of private educational organization","volume":"528","author":"Prabaswari","year":"2019","journal-title":"IOP Conf. Ser. Mater. Sci. Eng."},{"key":"ref_40","unstructured":"Card, S.K., Newell, A., and Moran, T.P. (1983). The Psychology of Human-Computer Interaction, L. Erlbaum Associates Inc."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Harrison, C., Tan, D., and Morris, D. (2010, January 10\u201315). Skinput: Appropriating the body as an input surface. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Atlanta, GA, USA.","DOI":"10.1145\/1753326.1753394"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Ballendat, T., Marquardt, N., and Greenberg, S. (2010, January 3\u20136). Proxemic Interaction: Designing for a Proximity and Orientation-Aware Environment. Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS \u201910), New York, NY, USA.","DOI":"10.1145\/1936652.1936676"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Li, H., Wang, Y., and Qu, H. (2024, January 11\u201316). Where are we so far? Understanding data storytelling tools from the perspective of human-ai collaboration. Proceedings of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3613904.3642726"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Hoque, E., and Islam, M.S. (2024). Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directions. Comput. Graph. Forum, e15266. online version of record.","DOI":"10.1111\/cgf.15266"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Shi, Y.Z., Li, H., Ruan, L., and Qu, H. (2024). Constraint representation towards precise data-driven storytelling. arXiv.","DOI":"10.1109\/GEN4DS63889.2024.00006"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Cao, Y., Kazi, R.H., Wei, L.Y., Aneja, D., and Xia, H. (2024, January 11\u201316). Elastica: Adaptive Live Augmented Presentations with Elastic Mappings Across Modalities. Proceedings of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3613904.3642725"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1145\/3698131","article-title":"VisConductor: Affect-Varying Widgets for Animated Data Storytelling in Gesture-Aware Augmented Video Presentation","volume":"8","author":"Brehmer","year":"2024","journal-title":"Proc. ACM Hum.-Comput. Interact."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Tong, W., Shigyo, K., Yuan, L.P., Fan, M., Pong, T.C., Qu, H., and Xia, M. (IEEE Trans. Vis. Comput. Graph., 2024). VisTellAR: Embedding Data Visualization to Short-form Videos Using Mobile Augmented Reality, IEEE Trans. Vis. Comput. Graph., early access.","DOI":"10.1109\/TVCG.2024.3372104"}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/9\/2\/10\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T10:37:05Z","timestamp":1759919825000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/9\/2\/10"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,27]]},"references-count":48,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["mti9020010"],"URL":"https:\/\/doi.org\/10.3390\/mti9020010","relation":{},"ISSN":["2414-4088"],"issn-type":[{"value":"2414-4088","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,27]]}}}