{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T15:31:05Z","timestamp":1773329465355,"version":"3.50.1"},"reference-count":38,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,9,6]],"date-time":"2022-09-06T00:00:00Z","timestamp":1662422400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"PTC Inc","award":["2022409"],"award-info":[{"award-number":["2022409"]}]},{"name":"PTC Inc","award":["86600233"],"award-info":[{"award-number":["86600233"]}]},{"name":"Technion Autonomous Systems Program","award":["2022409"],"award-info":[{"award-number":["2022409"]}]},{"name":"Technion Autonomous Systems Program","award":["86600233"],"award-info":[{"award-number":["86600233"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Learning through augmented reality (AR) and virtual reality (VR) experiences has become a valuable approach in modern robotics education. This study evaluated this approach and investigated how 99 first-year industrial engineering students explored robot systems through such online experiences while staying at home. The objective was to examine learning in the AR\/VR environment and evaluate its contribution to understanding the robot systems and to fostering integrative thinking. During the AR experiences that we developed using Vuforia Studio, the students learned about TurtleBot2 and RACECAR MN robots while disassembling and modifying their models and by obtaining information about their components. In the VR experience with the RacecarSim simulator, the students explored sensor-based robot navigation. Quizzes were used to assess understanding of robot systems, and a post-workshop questionnaire evaluated the workshop\u2019s contribution to learning about the robots and to training integrative thinking skills. The data indicate that the students gained understanding of the robot systems, appreciated the contribution of the augmented and virtual reality apps, and widely used integrative thinking throughout the practice. Our study shows that AR apps and virtual simulators can be effectively used for experiential learning about robot systems in online courses. However, these experiences cannot replace practice with real robots.<\/jats:p>","DOI":"10.3390\/robotics11050090","type":"journal-article","created":{"date-parts":[[2022,9,8]],"date-time":"2022-09-08T04:18:32Z","timestamp":1662610712000},"page":"90","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Augmented and Virtual Reality Experiences for Learning Robotics and Training Integrative Thinking Skills"],"prefix":"10.3390","volume":"11","author":[{"given":"Igor","family":"Verner","sequence":"first","affiliation":[{"name":"Faculty of Education in Science and Technology, Technion-Israel Institute of Technology, Haifa 3200003, Israel"}]},{"given":"Dan","family":"Cuperman","sequence":"additional","affiliation":[{"name":"Faculty of Education in Science and Technology, Technion-Israel Institute of Technology, Haifa 3200003, Israel"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2206-9530","authenticated-orcid":false,"given":"Huberth","family":"Perez-Villalobos","sequence":"additional","affiliation":[{"name":"Faculty of Education in Science and Technology, Technion-Israel Institute of Technology, Haifa 3200003, Israel"}]},{"given":"Alex","family":"Polishuk","sequence":"additional","affiliation":[{"name":"Faculty of Education in Science and Technology, Technion-Israel Institute of Technology, Haifa 3200003, Israel"}]},{"given":"Sergei","family":"Gamer","sequence":"additional","affiliation":[{"name":"Faculty of Industrial Engineering and Management, Technion-Israel Institute of Technology, Haifa 3200003, Israel"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1007\/s10798-020-09590-1","article-title":"Using systems thinking to understand the evolving role of technology in the design process","volume":"32","author":"Huber","year":"2022","journal-title":"Int. J. Technol. Des. Educ."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1007\/s11633-009-0319-9","article-title":"Virtual reality: A state-of-the-art survey","volume":"6","author":"Zhou","year":"2009","journal-title":"Int. J. Autom. Comput."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1659","DOI":"10.1007\/s10798-021-09669-3","article-title":"Using virtual reality to enhance food technology education","volume":"32","author":"Gorman","year":"2022","journal-title":"Int. J. Technol. Des. Educ."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Hoenig, W., Milanes, C., Scaria, L., Phan, T., Bolas, M., and Ayanian, N. (October, January 28). Mixed reality for robotics. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7354138"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Song, P., Yu, H., and Winkler, S. (2008, January 8\u20139). Vision-based 3D finger interactions for mixed reality games with physics simulation. Proceedings of the ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry, Singapore.","DOI":"10.1145\/1477862.1477871"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"165","DOI":"10.14689\/ejer.2018.74.9","article-title":"The effect of augmented reality applications in the learning process: A meta-analysis study","volume":"18","author":"Ozdemir","year":"2018","journal-title":"Eurasian J. Educ. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"106316","DOI":"10.1016\/j.chb.2020.106316","article-title":"Effects of augmented reality on learning and cognitive load in university physics laboratory courses","volume":"108","author":"Thees","year":"2020","journal-title":"Comput. Hum. Behav."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1111\/bjet.12900","article-title":"The use of augmented reality to foster conceptual knowledge acquisition in STEM laboratory courses\u2014Theoretical background and empirical results","volume":"51","author":"Altmeyer","year":"2020","journal-title":"Br. J. Educ. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1361","DOI":"10.1002\/cae.22156","article-title":"Evaluating the impact of the augmented reality learning environment on electronics laboratory skills of engineering students","volume":"27","author":"Singh","year":"2019","journal-title":"Comput. Appl. Eng. Educ."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"964","DOI":"10.1080\/10494820.2018.1552873","article-title":"The effectiveness of using augmented reality (AR) on assembling and exploring educational mobile robot in pedagogical virtual machine","volume":"28","author":"AlNajdi","year":"2020","journal-title":"Interact. Learn. Environ."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1007\/s10956-011-9345-9","article-title":"A pilot study of the effectiveness of augmented reality to enhance the use of remote labs in electrical engineering education","volume":"21","author":"Borrero","year":"2012","journal-title":"J. Sci. Educ. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Verner, I., Cuperman, D., and Polishuk, A. (2022, January 7\u201311). Inservice teachers explore RACECAR MN in physical and augmented environments. Proceedings of the 2022 17th Annual System of Systems Engineering Conference (SOSE), Rochester, NY, USA.","DOI":"10.1109\/SOSE55472.2022.9812639"},{"key":"ref_13","unstructured":"(2022, August 28). TurtleBot2. Open-Source Robot Development Kit. Available online: https:\/\/www.turtlebot.com\/turtlebot2\/."},{"key":"ref_14","unstructured":"(2022, August 28). MITLL RACECAR-MN. Available online: https:\/\/mitll-racecar-mn.readthedocs.io\/en\/latest\/."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1037\/edu0000606","article-title":"Benefits of interactive graphic organizers in online learning: Evidence for generative learning theory","volume":"113","author":"Wang","year":"2021","journal-title":"J. Educ. Psychol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1207\/S15327809JLS1204_1","article-title":"Designing for knowledge integration: The impact of instructional time","volume":"12","author":"Clark","year":"2003","journal-title":"J. Learn. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1080\/00221309.1962.9711837","article-title":"A subsumption theory of meaningful verbal learning and retention","volume":"66","author":"Ausubel","year":"1962","journal-title":"J. Gen. Psychol."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"De Weck, O.L., Roos, D., and Magee, C.L. (2011). Engineering Systems: Meeting Human Needs in a Complex Technological World, MIT Press.","DOI":"10.7551\/mitpress\/8799.001.0001"},{"key":"ref_19","unstructured":"Shen, Z., and Zhu, Y. (2011, January 26\u201329). Complex engineering system learning through study of engineering failure cases using 3D animations. Proceedings of the ASEE 2011 Annual Conference and Exposition, Vancouver, BC, Canada."},{"key":"ref_20","first-page":"44","article-title":"Towards an interpretive measurement framework to assess the levels of integrated and integrative thinking within organizations","volume":"5","author":"Sewchurran","year":"2015","journal-title":"Risk Gov. Control Financ. Mark. Inst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1177\/0959354310388344","article-title":"Integrative thinking is the key: An evaluation of current research into the development of adult thinking","volume":"21","author":"Kallio","year":"2011","journal-title":"Theory Psychol."},{"key":"ref_22","unstructured":"Martin, R.L. (2009). The Opposable Mind: How Successful Leaders Win through Integrative Thinking, Harvard Business School Publishing."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Tynj\u00e4l\u00e4, P., Kallio, E.K., and Heikkinen, H.L. (2020). Professional expertise, integrative thinking, wisdom, and phronesis. Development of Adult Thinking, Routledge. [1st ed.].","DOI":"10.4324\/9781315187464-10"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Qadir, J., Yau, K.L.A., Imran, M.A., and Al-Fuqaha, A. (2022, January 21\u201324). Engineering education, moving into 2020s: Essential competencies for effective 21st century electrical & computer engineers. Proceedings of the 2020 IEEE Frontiers in Education Conference (FIE), Uppsala, Sweden.","DOI":"10.35542\/osf.io\/gptse"},{"key":"ref_25","unstructured":"Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., and Ng, A.Y. (2009, January 12\u201313). ROS: An open-source robot operating system. Proceedings of the ICRA Workshop on Open-Source Software, Kobe, Japan."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Pasek, Z.J. (2017, January 4\u20137). Helping engineers develop and exercise creative muscles. Proceedings of the Canadian Engineering Education Association Conference (CEEA), Toronto, ON, Canada.","DOI":"10.24908\/pceea.v0i0.10203"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"536","DOI":"10.1080\/09645292.2015.1117580","article-title":"An exploratory study of creativity, personality, and schooling achievement","volume":"24","author":"Berlin","year":"2016","journal-title":"Educ. Econ."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Malik, A., and Setiawan, A. (2015, January 14). The development of higher order thinking laboratory to improve transferable skills of students. Proceedings of the 2015 International Conference on Innovation in Engineering and Vocational Education, Bandung, Indonesia.","DOI":"10.2991\/icieve-15.2016.9"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Asok, D., Abirami, A.M., Angeline, N., and Lavanya, R. (2016, January 9\u201310). Active learning environment for achieving higher-order thinking skills in engineering education. Proceedings of the 2016 IEEE 4th International Conference on MOOCs, Innovation and Technology in Education (MITE), Innovation, and Technology in Education (MITE), Madurai, India.","DOI":"10.1109\/MITE.2016.020"},{"key":"ref_30","unstructured":"Rawat, K.S., and Massiha, G.H. (May, January 26). A hands-on laboratory-based approach to undergraduate robotics education. Proceedings of the IEEE International Conference on Robotics and Automation, New Orleans, LA, USA."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cuperman, D., Verner, I., Perez, H., Gamer, S., and Polishuk, A. (2021, January 15\u201318). Fostering integrative thinking through an online AR-based robot system analysis. Proceedings of the World Engineering Education Forum, Madrid, Spain.","DOI":"10.1109\/WEEF\/GEDC53299.2021.9894732"},{"key":"ref_32","unstructured":"Ortega, P.E., Lagoudas, M.Z., and Froyd, J.E. (2017, January 24\u201328). Overview and comparison of assessment tools for integrative thinking. Proceedings of the 2017 ASEE Annual Conference & Exposition, Columbus, OH, USA."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1037\/0021-9010.78.2.311","article-title":"Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation","volume":"78","author":"Kraiger","year":"1993","journal-title":"Int. J. Appl. Psychol."},{"key":"ref_34","first-page":"1376","article-title":"Robotics laboratory classes for spatial training of novice engineering students","volume":"31","author":"Verner","year":"2015","journal-title":"Int. J. Eng. Educ."},{"key":"ref_35","first-page":"1691","article-title":"Exploring affordances of robot manipulators in an introductory engineering course","volume":"36","author":"Verner","year":"2020","journal-title":"Int. J. Eng. Educ."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Chen, S., Fishberg, A., Shimelis, E., Grimm, J., van Broekhoven, S., Shin, R., and Karaman, S. (2020, January 1). A Hands-on Middle-School Robotics Software Program at MIT. Proceedings of the IEEE Integrated STEM Education Conference, Princeton, NJ, USA.","DOI":"10.1109\/ISEC49744.2020.9280694"},{"key":"ref_37","unstructured":"Goodrick, D. (2014). Comparative Case Studies, United Nations Children\u2019s Fund (UNICEF). Methodological Briefs: Impact Evaluation 9."},{"key":"ref_38","first-page":"63","article-title":"Educational robotics: Open questions and new challenges","volume":"6","author":"Alimisis","year":"2013","journal-title":"Themes Sci. Technol. Educ."}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/11\/5\/90\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:24:05Z","timestamp":1760142245000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/11\/5\/90"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,6]]},"references-count":38,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["robotics11050090"],"URL":"https:\/\/doi.org\/10.3390\/robotics11050090","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,6]]}}}