{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T07:52:31Z","timestamp":1762069951096,"version":"build-2065373602"},"reference-count":56,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,10,14]],"date-time":"2022-10-14T00:00:00Z","timestamp":1665705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Funds through FCT","award":["UIDB\/00127\/2020"],"award-info":[{"award-number":["UIDB\/00127\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Augmented Reality (AR) solutions are emerging in multiple scenarios of application as Industry 4.0 takes shape. In particular, for remote collaboration, flexible mechanisms such as authoring tools can be used to generate instructions and assist human operators as they experience increased complexity in their daily tasks. In addition to the traditional handicap of ensuring instructions can be intuitively created without having to understand complicated AR concepts, another relevant topic is the fact that the quality of said instructions is not properly analyzed prior to the tools being evaluated. This means that the characteristics of the visual content are not adequately assessed beforehand. Hence, it is essential to be aware of the cognitive workload associated with AR instructions to assert if they can be easily understood and accepted before being deployed in real-world scenarios. To address this, we focused on AR during sessions of remote guidance. Based on a participatory process with domain experts from the industry sector, a prototype for creating AR-based instructions was developed, and a user study with two parts was conducted: (1) first, a set of step-by-step instructions was produced, and their visual characteristics were evaluated by 129 participants based on a set of relevant dimensions; (2) afterward, these instructions were used by nine participants to understand if they could be used to assist on-site collaborators during real-life remote maintenance tasks. The results suggest that the AR instructions offer low visual complexity and considerable visual impact, clarity, and directed focus, thus improving situational understanding and promoting task resolution.<\/jats:p>","DOI":"10.3390\/mti6100092","type":"journal-article","created":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T03:43:58Z","timestamp":1665978238000},"page":"92","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Are the Instructions Clear? Evaluating the Visual Characteristics of Augmented Reality Content for Remote Guidance"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4454-710X","authenticated-orcid":false,"given":"Bernardo","family":"Marques","sequence":"first","affiliation":[{"name":"DigiMedia, DeCA, University of Aveiro, 3810-193 Aveiro, Portugal"},{"name":"IEETA, DETI, University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"given":"Carlos","family":"Ferreira","sequence":"additional","affiliation":[{"name":"IEETA, DEGEIT, University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9858-8249","authenticated-orcid":false,"given":"Samuel","family":"Silva","sequence":"additional","affiliation":[{"name":"IEETA, DETI, University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"given":"Andreia","family":"Santos","sequence":"additional","affiliation":[{"name":"GeoBioTec, DGeo, University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3754-2749","authenticated-orcid":false,"given":"Paulo","family":"Dias","sequence":"additional","affiliation":[{"name":"IEETA, DETI, University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"given":"Beatriz Sousa","family":"Santos","sequence":"additional","affiliation":[{"name":"IEETA, DETI, University of Aveiro, 3810-193 Aveiro, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"103229","DOI":"10.1016\/j.compind.2020.103229","article-title":"A systematic design method of adaptive augmented reality work instruction for complex industrial operations","volume":"119","author":"Geng","year":"2020","journal-title":"Comput. Ind."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.cie.2017.09.016","article-title":"Smart operators in industry 4.0: A human-centered approach to enhance operators\u2019 capabilities and competencies within the new smart factory context","volume":"113","author":"Longo","year":"2017","journal-title":"Comput. Ind. Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"284","DOI":"10.1080\/24725854.2018.1493244","article-title":"Augmented reality technology in the manufacturing industry: A review of the last decade","volume":"51","author":"Bottani","year":"2019","journal-title":"IISE Trans."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.ijhcs.2019.05.011","article-title":"Revisiting Collaboration through Mixed Reality: The Evolution of Groupware","volume":"131","author":"Ens","year":"2019","journal-title":"Int. J. Hum.-Comput. Stud."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1007\/s12193-020-00335-x","article-title":"The combination of visual communication cues in mixed reality remote collaboration","volume":"14","author":"Kim","year":"2020","journal-title":"J. Multimodal User Interfaces"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1007\/s00170-021-07049-8","article-title":"Using augmented reality for industrial quality assurance: A shop floor user study","volume":"115","author":"Alves","year":"2021","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3147","DOI":"10.1007\/s00170-021-06871-4","article-title":"Improving AR-powered remote assistance: A new approach aimed to foster operator\u2019s autonomy and optimize the use of skilled resources","volume":"114","author":"Calandra","year":"2021","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1007\/s00170-017-0846-4","article-title":"Evaluating the effectiveness of spatial augmented reality in smart manufacturing: A solution for manual working stations","volume":"94","author":"Uva","year":"2018","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4095","DOI":"10.1007\/s00170-019-03527-2","article-title":"A framework for augmented reality guidance in industry","volume":"102","author":"Zubizarreta","year":"2019","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.rcim.2019.02.003","article-title":"Augmented reality in support of Industry 4.0\u2014Implementation challenges and success factors","volume":"58","author":"Masood","year":"2019","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1511","DOI":"10.1007\/s12008-020-00716-2","article-title":"Competencies for Industry 4.0","volume":"14","author":"Escobar","year":"2020","journal-title":"Int. J. Interact. Des. Manuf."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"875","DOI":"10.1007\/s00170-019-04254-4","article-title":"An augmented reality tool to detect and annotate design variations in an Industry 4.0 approach","volume":"105","author":"Bruno","year":"2019","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Boboc, R.G., G\u00eerbacia, F., and Butil\u0103, E.V. (2020). The Application of Augmented Reality in the Automotive Industry: A Systematic Literature Review. Appl. Sci., 10.","DOI":"10.3390\/app10124259"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1769","DOI":"10.1007\/s00170-021-08449-6","article-title":"Minimal AR: Visual asset optimization for the authoring of augmented reality work instructions in manufacturing","volume":"119","author":"Laviola","year":"2021","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.compind.2018.04.021","article-title":"Augmented reality via expert demonstration authoring (AREDA)","volume":"105","author":"Bhattacharya","year":"2019","journal-title":"Comput. Ind."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"103208","DOI":"10.1016\/j.compind.2020.103208","article-title":"Developing augmented reality capabilities for industry 4.0 small enterprises: Lessons learnt from a content authoring case study","volume":"117","author":"Sinclair","year":"2020","journal-title":"Comput. Ind."},{"key":"ref_17","unstructured":"Winer, E.H. (2018). Authoring Augmented Reality Work Instructions by Expert Demonstration, Iowa State University. Technical Report."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Gimeno, J., Morillo, P., Ordu\u00f1a, J.M., and Fern\u00e1ndez, M. (2013). An easy-to-use AR authoring tool for industrial applications. Computer Vision, Imaging and Computer Graphics. Theory and Application, Springer.","DOI":"10.1007\/978-3-642-38241-3_2"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"798032","DOI":"10.3389\/frvir.2022.798032","article-title":"A Review on Augmented Reality Authoring Toolkits for Education","volume":"3","author":"Dengel","year":"2022","journal-title":"Front. Virtual Real."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"B\u00e9gout, P., Duval, T., Kubicki, S., Charbonnier, B., and Bricard, E. (2020, January 7\u201310). WAAT: A Workstation AR Authoring Tool for Industry 4.0. Proceedings of the International Conference on Augmented Reality, Virtual Reality and Computer Graphics, Lecce, Italy.","DOI":"10.1007\/978-3-030-58468-9_22"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Roberto, R.A., Lima, J.P., Mota, R.C., and Teichrieb, V. (2016, January 17\u201322). Authoring tools for augmented reality: An analysis and classification of content design tools. Proceedings of the International Conference of Design, User Experience, and Usability, Toronto, ON, Canada.","DOI":"10.1007\/978-3-319-40406-6_22"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/j.procs.2013.11.023","article-title":"Authoring Software for Augmented Reality Applications for the Use of Maintenance and Training Process","volume":"25","author":"Ramirez","year":"2013","journal-title":"Procedia Comput. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1319","DOI":"10.1016\/j.promfg.2017.07.260","article-title":"The Challenge of Introducing AR in Industry\u2014Results of a Participative Process Involving Maintenance Engineers","volume":"11","author":"Quint","year":"2017","journal-title":"Procedia Manuf."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.compind.2018.08.007","article-title":"A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications","volume":"103","author":"Erkoyuncu","year":"2018","journal-title":"Comput. Ind."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1130","DOI":"10.1016\/j.procir.2018.03.061","article-title":"General Requirements for Industrial Augmented Reality Applications","volume":"72","author":"Quandt","year":"2018","journal-title":"Procedia CIRP"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"103112","DOI":"10.1016\/j.compind.2019.07.002","article-title":"Adopting augmented reality in the age of industrial digitalisation","volume":"115","author":"Masood","year":"2020","journal-title":"Comput. Ind."},{"key":"ref_27","unstructured":"Kim, S., Billinghurst, M., Lee, C., and Lee, G. (2018, January 13\u201316). Using Freeze Frame and Visual Notifications in an Annotation Drawing Interface for Remote Collaboration. Proceedings of the Transactions on Internet & Information Systems, San Francisco, CA, USA."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Johnson, S., Gibson, M., and Mutlu, B. (2015, January 14\u201318). Handheld or Handsfree? Remote Collaboration via Lightweight Head-Mounted Displays and Handheld Devices. Proceedings of the ACM Conference on Computer Supported Cooperative Work & Social Computing, Vancouver, BC, Canada.","DOI":"10.1145\/2675133.2675176"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"5","DOI":"10.3389\/frobt.2019.00005","article-title":"The Effects of Sharing Awareness Cues in Collaborative Mixed Reality","volume":"6","author":"Piumsomboon","year":"2019","journal-title":"Front. Robot. AI"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2947","DOI":"10.1109\/TVCG.2018.2868591","article-title":"Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008\u20132017)","volume":"24","author":"Kim","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"106159","DOI":"10.1016\/j.cie.2019.106159","article-title":"A survey of industrial augmented reality","volume":"139","author":"Mariano","year":"2020","journal-title":"Comput. Ind. Eng."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Marques, B., Silva, S., Alves, J., Araujo, T., Dias, P., and Santos, B.S. (IEEE Trans. Vis. Comput. Graph., 2021). A Conceptual Model and Taxonomy for Collaborative Augmented Reality, IEEE Trans. Vis. Comput. Graph., early access.","DOI":"10.1109\/TVCG.2021.3101545"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.procir.2020.01.137","article-title":"Classification of industrial Augmented Reality use cases","volume":"91","author":"Dumitrescu","year":"2020","journal-title":"Procedia CIRP"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.promfg.2018.01.021","article-title":"Augmented Reality in Maintenance: An information-centred design framework","volume":"19","author":"Erkoyuncu","year":"2018","journal-title":"Procedia Manuf."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.chb.2018.04.054","article-title":"Augmented reality tools for industrial applications: What are potential key performance indicators and who benefits?","volume":"87","author":"Jetter","year":"2018","journal-title":"Comput. Hum. Behav."},{"key":"ref_36","unstructured":"Marques, B., Silva, S., Dias, P., and Santos, B.S. (2022, January 12\u201316). Remote Collaboration using Augmented Reality: Development and Evaluation. Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Christchurch, New Zealand."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.rcim.2017.06.002","article-title":"A systematic review of augmented reality applications in maintenance","volume":"49","author":"Palmarini","year":"2018","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"106195","DOI":"10.1016\/j.cie.2019.106195","article-title":"Augmented reality in support of intelligent manufacturing\u2014A systematic literature review","volume":"140","author":"Egger","year":"2020","journal-title":"Comput. Ind. Eng."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1016\/j.cag.2021.10.009","article-title":"A vision for contextualized evaluation of remote collaboration supported by AR","volume":"102","author":"Marques","year":"2021","journal-title":"Comput. Graph."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1007\/s12008-021-00798-6","article-title":"Remote Collaboration in Maintenance Contexts using Augmented Reality: Insights from a Participatory Process","volume":"16","author":"Marques","year":"2022","journal-title":"Int. J. Interact. Des. Manuf."},{"key":"ref_41","unstructured":"Marques, B., Silva, S., Rocha, A., Dias, P., and Santos, B.S. (April, January 27). Remote Asynchronous Collaboration in Maintenance scenarios using Augmented Reality and Annotations. Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Lisbon, Portugal."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Madeira, T., Marques, B., Alves, J., Dias, P., and Santos, B.S. (2021, January 23\u201325). Exploring Annotations and Hand Tracking in Augmented Reality for Remote Collaboration. Proceedings of the Human Systems Engineering and Design III, Pula, Croatia.","DOI":"10.1007\/978-3-030-58282-1_14"},{"key":"ref_43","unstructured":"Marques, B., Ferreira, C., Silva, S., Santos, A., Dias, P., and Sousa Santos, B. (2021, January 4\u20135). Exploring Remote Augmented Reality as a 2D Authoring Tool for Creation of Guiding Instructions. Proceedings of the International Conference on Graphics and Interaction, ICGI 2021, Porto, Portugal."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Bresciani, S., Blackwell, A.F., and Eppler, M. (2008, January 7\u201310). A Collaborative Dimensions Framework: Understanding the Mediating Role of Conceptual Visualizations in Collaborative Knowledge Work. Proceedings of the Hawaii International Conference on System Sciences, Waikoloa, HI, USA.","DOI":"10.1109\/HICSS.2008.7"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Bresciani, S., and Eppler, M.J. (2009, January 15\u201317). Beyond Knowledge Visualization Usability: Toward a Better Understanding of Business Diagram Adoption. Proceedings of the International Conference Information Visualisation, Barcelona, Spain.","DOI":"10.1109\/IV.2009.13"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1073","DOI":"10.1109\/TVCG.2009.188","article-title":"The Benefits of Synchronous Collaborative Information Visualization: Evidence from an Experimental Evaluation","volume":"15","author":"Bresciani","year":"2009","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Eppler, M.J. (2011, January 13\u201315). What is an Effective Knowledge Visualization? Insights from a Review of Seminal Concepts. Proceedings of the 2011 15th International Conference on Information Visualisation, London, UK.","DOI":"10.1109\/IV.2011.13"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/j.ijhcs.2012.02.006","article-title":"Collaborative knowledge building with shared video representations","volume":"71","author":"Barthel","year":"2013","journal-title":"Int. J. Hum.-Comput. Stud."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Ribeiro, F.C., de Souza, J.M., and de Paula, M.M.V. (2015, January 6\u20138). Use of information visualization techniques in a collaborative context. Proceedings of the IEEE International Conference on Computer Supported Cooperative Work in Design (CSCWD), Calabria, Italy.","DOI":"10.1109\/CSCWD.2015.7230937"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Kernbach, S., and Svetina Nabergoj, A. (2018, January 10\u201313). Visual Design Thinking: Understanding the Role of Knowledge Visualization in the Design Thinking Process. Proceedings of the International Conference Information Visualisation (IV), Fisciano, Italy.","DOI":"10.1109\/iV.2018.00068"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.procir.2017.03.154","article-title":"Augmented Reality Application to Support Remote Maintenance as a Service in the Robotics Industry","volume":"63","author":"Mourtzis","year":"2017","journal-title":"Procedia CIRP"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1007\/s12008-013-0199-7","article-title":"A context-aware augmented reality system to assist the maintenance operators","volume":"8","author":"Zhu","year":"2014","journal-title":"Int. J. Interact. Des. Manuf."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1016\/j.compind.2013.11.004","article-title":"Augmented reality on large screen for interactive maintenance instructions","volume":"65","author":"Fiorentino","year":"2014","journal-title":"Comput. Ind."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Nielsen, J., and Landauer, T.K. (1993, January 24\u201329). A Mathematical Model of the Finding of Usability Problems. Proceedings of the Proceedings of the INTERACT\u201993 and CHI\u201993 Conference on Human Factors in Computing Systems, Amsterdam, The Netherlands.","DOI":"10.1145\/169059.169166"},{"key":"ref_55","unstructured":"Tullis, T., and Albert, W. (2013). Measuring the User Experience, Second Edition: Collecting, Analyzing, and Presenting Usability Metrics, Morgan Kaufmann Publishers Inc."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"619","DOI":"10.1016\/j.cag.2021.08.006","article-title":"A critical analysis on remote collaboration mediated by augmented reality: Making a case for improved characterization and evaluation of the collaborative process","volume":"102","author":"Marques","year":"2022","journal-title":"Comput. Graph."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/6\/10\/92\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:53:54Z","timestamp":1760144034000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/6\/10\/92"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,14]]},"references-count":56,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["mti6100092"],"URL":"https:\/\/doi.org\/10.3390\/mti6100092","relation":{},"ISSN":["2414-4088"],"issn-type":[{"type":"electronic","value":"2414-4088"}],"subject":[],"published":{"date-parts":[[2022,10,14]]}}}