{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,31]],"date-time":"2026-01-31T09:29:45Z","timestamp":1769851785109,"version":"3.49.0"},"reference-count":35,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2022,7,26]],"date-time":"2022-07-26T00:00:00Z","timestamp":1658793600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006360","name":"German Federal Ministry for Economic Affairs and Energy","doi-asserted-by":"publisher","award":["ZF4132603SS9"],"award-info":[{"award-number":["ZF4132603SS9"]}],"id":[{"id":"10.13039\/501100006360","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>The German manufacturing industry has been carrying out new developments towards the next industrial revolution, focusing on smart manufacturing environments. Our work emphasizes human-centered control rooms in the context of production plants. Increased automation does not have to come with less human control. Therefore, we report on multimodal collaborative interaction techniques to augment industrial control rooms. In particular, we include mobile workers who use the control room while being in the production hall using tablets or specifically mixed reality glasses. Collaborative annotation dashboards support discussions and a shared understanding among analysts. Manufacturing-related data can be integrated into business analytics environments so that holistic analyses can be performed. Multimodal interaction techniques can support effective interaction with the control room based on the users\u2019 preferences. Immersive experience through mixed reality-based three-dimensional visualizations and interaction possibilities support users in obtaining a clear understanding of the underlying data.<\/jats:p>","DOI":"10.3390\/fi14080224","type":"journal-article","created":{"date-parts":[[2022,7,26]],"date-time":"2022-07-26T22:03:42Z","timestamp":1658873022000},"page":"224","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Augmenting Industrial Control Rooms with Multimodal Collaborative Interaction Techniques"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0937-3551","authenticated-orcid":false,"given":"Jessica","family":"Rubart","sequence":"first","affiliation":[{"name":"Department of Environmental Engineering and Applied Computer Science, Ostwestfalen-Lippe University of Applied Sciences and Arts, 37671 H\u00f6xter, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0581-0662","authenticated-orcid":false,"given":"Valentin","family":"Grimm","sequence":"additional","affiliation":[{"name":"Department of Environmental Engineering and Applied Computer Science, Ostwestfalen-Lippe University of Applied Sciences and Arts, 37671 H\u00f6xter, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8670-7204","authenticated-orcid":false,"given":"Jonas","family":"Potthast","sequence":"additional","affiliation":[{"name":"Department of Environmental Engineering and Applied Computer Science, Ostwestfalen-Lippe University of Applied Sciences and Arts, 37671 H\u00f6xter, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kagermann, H., Helbig, J., Hellinger, A., and Wahlster, W. (2013). Recommendations for Implementing the Strategic Initiative INDUSTRIE 4.0: Securing the Future of German Manufacturing Industry, Forschungsunion. Final Report of the Industrie 4.0 Working Group.","DOI":"10.3390\/sci4030026"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"M\u00fcller, J., Schwarz, T., Butscher, S., and Reiterer, H. (2014, January 27\u201330). Back to tangibility: A post-WIMP perspective on control room design. Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, Como, Italy.","DOI":"10.1145\/2598153.2598161"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1049\/iet-its:20060038","article-title":"Multimodal user interface for traffic incident management in control room","volume":"1","author":"Choi","year":"2007","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1082","DOI":"10.4103\/jfmpc.jfmpc_1826_20","article-title":"Experience of setting up of Control room for COVID-19 at NCDC, New Delhi","volume":"10","author":"Kaushal","year":"2021","journal-title":"J. Fam. Med. Prim. Care"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Koskinen, H.M.K., Laarni, J.O., and Honkamaa, P.M. (2008, January 5\u201310). Hands-on the process control: Users preferences and associations on hand movements. Proceedings of the CHI\u201908 Extended Abstracts on Human Factors in Computing Systems, Florence, Italy.","DOI":"10.1145\/1358628.1358808"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Fagerl\u00f6nn, J., Hammarberg, K., Lindberg, S., Sirkka, A., and Larsson, S. (2017, January 23\u201326). Designing a multimodal warning display for an industrial control room. Proceedings of the 12th International Audio Mostly Conference on Augmented and Participatory Sound and Music Experiences, London, UK.","DOI":"10.1145\/3123514.3123516"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1080\/10447318.2020.1741118","article-title":"Human-centered artificial intelligence: Reliable, safe & trustworthy","volume":"36","author":"Shneiderman","year":"2020","journal-title":"Int. J. Hum. Comput. Interact."},{"key":"ref_8","unstructured":"Bourguet, M.L. (2003). Designing and Prototyping Multimodal Commands, IOS Press."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Rubart, J. (2018, January 9\u201310). Multimodal Interaction with Hypermedia Structures. Proceedings of the 1st Workshop on Human Factors in Hypertext, Baltimore, MD, USA.","DOI":"10.1145\/3215611.3215613"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Hesenius, M., and Gruhn, V. (2013, January 9\u201313). MVIC\u2013An MVC Extension for Interactive, Multimodal Applications. Proceedings of the European Conference on Software Architecture, Paris, France.","DOI":"10.1007\/978-3-642-39031-9_30"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Rubart, J., Haake, J.M., Tietze, D.A., and Wang, W. (2001, January 14\u201318). Organizing shared enterprise workspaces using component-based cooperative hypermedia. Proceedings of the 12th ACM Conference on Hypertext and Hypermedia, \u00c5rhus, Denmark.","DOI":"10.1145\/504216.504240"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Berthold, H., R\u00f6sch, P., Z\u00f6ller, S., Wortmann, F., Carenini, A., Campbell, S., Bisson, P., and Strohmaier, F. (2010, January 22\u201326). An architecture for ad-hoc and collaborative business intelligence. Proceedings of the 2010 EDBT\/ICDT Workshops, Lausanne, Switzerland.","DOI":"10.1145\/1754239.1754254"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1142\/S1793351X13500013","article-title":"Semantic computing and business intelligence","volume":"7","author":"Kim","year":"2013","journal-title":"Int. J. Semant. Comput."},{"key":"ref_14","unstructured":"Datar, P. (2016). Introducing the SAP Digital Boardroom, SAP Press."},{"key":"ref_15","unstructured":"Magerkurth, C., Memisoglu, M., Engelke, T., and Streitz, N. (2004, January 17\u201319). Towards the next generation of tabletop gaming experiences. Proceedings of the Graphics interface, London, ON, Canada."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Rubart, J. (2014, January 16\u201319). A cooperative multitouch scrum task board for synchronous face-to-face collaboration. Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, Dresden, Germany.","DOI":"10.1145\/2669485.2669551"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Key, A., Howe, B., Perry, D., and Aragon, C. (2012, January 20\u201324). Vizdeck: Self-organizing dashboards for visual analytics. Proceedings of the 2012 ACM Sigmod International Conference on Management of Data, Scottsdale, AZ, USA.","DOI":"10.1145\/2213836.2213931"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Domova, V. (2020). Designing Visualization and Interaction for Industrial Control Rooms of the Future, Link\u00f6ping University Electronic Press.","DOI":"10.3384\/diss.diva-165625"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Rubart, J., Lietzau, B., S\u00f6ehlke, P., Alex, B., Becker, S., and Wienb\u00f6eker, T. (February, January 30). Semantic navigation and discussion in a digital boardroom. Proceedings of the 2017 IEEE 11th International Conference on Semantic Computing (ICSC), San Diego, CA, USA.","DOI":"10.1109\/ICSC.2017.39"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Rubart, J., Lietzau, B., and S\u00f6hlke, P. (2020, January 3\u20135). Analyzing manufacturing data in a digital control room making use of semantic annotations. Proceedings of the 2020 IEEE 14th International Conference on Semantic Computing (ICSC), San Diego, CA, USA.","DOI":"10.1109\/ICSC.2020.00084"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Shneiderman, B. (2003). The eyes have it: A task by data type taxonomy for information visualizations. The Craft of Information Visualization, Elsevier.","DOI":"10.1016\/B978-155860915-0\/50046-9"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Draho\u0161, P., Ku\u010dera, E., Haffner, O., and Klimo, I. (February, January 31). Trends in industrial communication and OPC UA. Proceedings of the 2018 Cybernetics & Informatics (K&I), Makytou, Slovakia.","DOI":"10.1109\/CYBERI.2018.8337560"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Millais, P., Jones, S.L., and Kelly, R. (2018, January 21\u201326). Exploring data in virtual reality: Comparisons with 2d data visualizations. Proceedings of the Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/3170427.3188537"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, X., Besan\u00e7on, L., Rousseau, D., Sereno, M., Ammi, M., and Isenberg, T. (2020, January 25\u201330). Towards an understanding of augmented reality extensions for existing 3D data analysis tools. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3313831.3376657"},{"key":"ref_25","first-page":"237","article-title":"User Interface Design f\u00fcr Augmented Reality-basierte Assistenzsysteme","volume":"12","author":"Stern","year":"2020","journal-title":"RSG"},{"key":"ref_26","unstructured":"Blok\u0161a, J. (2017). Design Guidelines for User Interface for Augmented Reality. [Master\u2019s Thesis, Masaryk University]."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2413","DOI":"10.1109\/TVCG.2016.2593778","article-title":"Do you see what I see? The effect of gaze tracking on task space remote collaboration","volume":"22","author":"Gupta","year":"2016","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1007\/s12193-020-00335-x","article-title":"The combination of visual communication cues in mixed reality remote collaboration","volume":"14","author":"Kim","year":"2020","journal-title":"J. Multimodal User Interfaces"},{"key":"ref_29","unstructured":"(2022, June 30). Rory Brown. Available online: https:\/\/web.archive.org\/web\/20220725104915\/https:\/\/vr-compare.com\/compare?h1=EkSDYv0cW&h2=98wLZvTJQ."},{"key":"ref_30","unstructured":"(2022, June 30). Microsoft. Available online: https:\/\/docs.microsoft.com\/en-us\/power-bi\/consumer\/mobile\/mobile-hololens2-app."},{"key":"ref_31","unstructured":"(2022, June 30). SmartFactoryOWL. Available online: https:\/\/www.smartfactory-owl.de."},{"key":"ref_32","unstructured":"Otto, J., Schriegel, S., and Niggemann, O. (2022, June 30). Eine Taxonomie f\u00fcr Plug-and-Produce 2015. Available online: https:\/\/www.researchgate.net\/profile\/Oliver-Niggemann\/publication\/272175027_Eine_Taxonomie_fur_Plug-and-Produce\/links\/55b89e9d08aed621de05f083\/Eine-Taxonomie-fuer-Plug-and-Produce.pdf."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Rubart, J. (2007, January 6\u20139). Architecting structure-aware applications. Proceedings of the Eighteenth Conference on Hypertext and Hypermedia, Eindhoven, The Netherlands.","DOI":"10.1145\/1286240.1286296"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Vovk, A., Wild, F., Guest, W., and Kuula, T. (2018, January 21\u201326). Simulator sickness in augmented reality training using the Microsoft HoloLens. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/3173574.3173783"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Potthast, J., Grimm, V., and Rubart, J. (2022, January 4\u20137). Immersive experience of multidimensional data using mixed reality based scatterplots. Proceedings of the Mensch und Computer, Darmstadt, Germany.","DOI":"10.1145\/3543758.3547515"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/8\/224\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:56:50Z","timestamp":1760140610000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/8\/224"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,26]]},"references-count":35,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["fi14080224"],"URL":"https:\/\/doi.org\/10.3390\/fi14080224","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,26]]}}}