{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T06:45:19Z","timestamp":1778827519139,"version":"3.51.4"},"reference-count":42,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2023,5,29]],"date-time":"2023-05-29T00:00:00Z","timestamp":1685318400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science Foundation","award":["1742242"],"award-info":[{"award-number":["1742242"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>To better prepare future generations, knowledge about computers and programming are one of the many skills that are part of almost all Science, Technology, Engineering, and Mathematic programs; however, teaching and learning programming is a complex task that is generally considered difficult by students and teachers alike. One approach to engage and inspire students from a variety of backgrounds is the use of educational robots. Unfortunately, previous research presents mixed results on the effectiveness of educational robots on student learning. One possibility for this lack of clarity may be because students have a wide variety of styles of learning. It is possible that the use of kinesthetic feedback, in addition to the normally used visual feedback, may improve learning with educational robots by providing a richer, multi-modal experience that may appeal to a larger number of students with different learning styles. It is also possible, however, that the addition of kinesthetic feedback, and how it may interfere with the visual feedback, may decrease a student\u2019s ability to interpret the program commands being executed by a robot, which is critical for program debugging. In this work, we investigated whether human participants were able to accurately determine a sequence of program commands performed by a robot when both kinesthetic and visual feedback were being used together. Command recall and end point location determination were compared to the typically used visual-only method, as well as a narrative description. Results from 10 sighted participants indicated that individuals were able to accurately determine a sequence of movement commands and their magnitude when using combined kinesthetic + visual feedback. Participants\u2019 recall accuracy of program commands was actually better with kinesthetic + visual feedback than just visual feedback. Although the recall accuracy was even better with the narrative description, this was primarily due to participants confusing an absolute rotation command with a relative rotation command with the kinesthetic + visual feedback. Participants\u2019 zone location accuracy of the end point after a command was executed was significantly better for both the kinesthetic + visual feedback and narrative methods compared to the visual-only method. Together, these results suggest that the use of both kinesthetic + visual feedback improves an individual\u2019s ability to interpret program commands, rather than decreases it.<\/jats:p>","DOI":"10.3390\/s23115159","type":"journal-article","created":{"date-parts":[[2023,5,29]],"date-time":"2023-05-29T03:39:36Z","timestamp":1685331576000},"page":"5159","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Kinesthetic Feedback for Understanding Program Execution"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9222-8867","authenticated-orcid":false,"given":"Satinder","family":"Gill","sequence":"first","affiliation":[{"name":"Department of Biomedical Engineering, Virginia Commonwealth University, Richmond, VA 23219, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bryson J.","family":"Goolsby","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, Virginia Commonwealth University, Richmond, VA 23219, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dianne T. V.","family":"Pawluk","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, Virginia Commonwealth University, Richmond, VA 23219, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,29]]},"reference":[{"key":"ref_1","unstructured":"(2023, April 10). United States Census Bureau, Available online: https:\/\/www.census.gov\/quickfacts\/fact\/table\/US\/PST045221."},{"key":"ref_2","unstructured":"(2022, December 28). U.S. Bureau of Labor Statistics, Available online: https:\/\/www.bls.gov\/opub\/btn\/volume-10\/why-computer-occupations-are-behind-strong-stem-employment-growth.htm."},{"key":"ref_3","unstructured":"Grover, S. (2011, January 8\u201312). Robotics and Engineering for Middle and High School Students to Develop Computational Thinking. Proceedings of the Annual Meeting of the American Educational Research Association, New Orleans, LA, USA."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Henderson, P.B., Cortina, T.J., Hazzan, O., and Wing, J.M. (2007, January 7\u201311). Computational Thinking. Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education, Covington, KY, USA.","DOI":"10.1145\/1227310.1227378"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1007\/s10956-015-9581-5","article-title":"Defining Computational Thinking for Mathematics and Science Classrooms","volume":"25","author":"Weintrop","year":"2016","journal-title":"J. Sci. Educ. Technol."},{"key":"ref_6","unstructured":"Blakemore, L. (2017). Does Teaching Computer Programming within Key Stage 1 of the Primary Curriculum Enhance Children\u2019s Problem Solving Skills, University of Sheffield."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/j.compedu.2012.11.016","article-title":"Problem Solving by 5\u20136 Years Old Kindergarten Children in a Computer Programming Environment: A Case Study","volume":"63","author":"Fessakis","year":"2013","journal-title":"Comput. Educ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1145\/1595496.1562957","article-title":"Influence of the Familiarization with \u201cScratch\u201d on Future Teachers\u2019 Opinions and Attitudes about Programming and ICT in Education","volume":"41","author":"Fesakis","year":"2009","journal-title":"ACM SIGCSE Bull."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1080\/14759390500200213","article-title":"A Case for Ubiquitous, Integrated Computing in Teacher Education","volume":"14","author":"Kay","year":"2005","journal-title":"Technol. Pedagog. Educ."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1145\/2038876.2038883","article-title":"The Nature of CS in K--12 Curricula: The Roots of Confusion","volume":"2","author":"Armoni","year":"2011","journal-title":"ACM Inroads"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1513593.1513597","article-title":"STREAM: A First Programming Process","volume":"9","author":"Caspersen","year":"2009","journal-title":"ACM Trans. Comput. Educ."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.iheduc.2015.06.003","article-title":"The Effect of Locus of Control on Learners\u2019 Sense of Community and Academic Success in the Context of Online Learning Communities","volume":"27","author":"Alper","year":"2015","journal-title":"Internet High. Educ."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"McCracken, M., Almstrum, V., Diaz, D., Guzdial, M., Hagan, D., Kolikat, Y.B.-D., Laxer, C., Thomas, L., Utting, I., and Wilusz, T. (2001, January 24\u201330). A Multi-National, Multi-Institutional Study of Assessment of Programming Skills of First-Year CS Students. Proceedings of the Working Group Reports from ITiCSE on Innovation and Technology in Computer Science Education, Kent, UK.","DOI":"10.1145\/572133.572137"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1145\/1083431.1083474","article-title":"Why the High Attrition Rate for Computer Science Students: Some Thoughts and Observations","volume":"37","author":"Beaubouef","year":"2005","journal-title":"ACM SIGCSE Bull."},{"key":"ref_15","unstructured":"Kau\u010di\u010d, B., and Asi\u010d, T. (2011, January 23\u201327). Improving Introductory Programming with Scratch?. Proceedings of the 2011 Proceedings of the 34th International Convention MIPRO, Opatija, Croatia."},{"key":"ref_16","unstructured":"Ozoran, D., Cagiltay, N.E., and Topalli, D. (November, January 31). Using Scratch in Introduction to Programming Course for Engi-Neering Students. Proceedings of the 2nd International Engineering Education Conference (IEEC2012), Antalya, Turkey."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.compedu.2018.01.011","article-title":"Improving Programming Skills in Engineering Education through Problem-Based Game Projects with Scratch","volume":"120","author":"Topali","year":"2018","journal-title":"Comput. Educ."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Garner, S. (2009, January 15\u201317). Learning to Program from Scratch. Proceedings of the 2009 Ninth IEEE International Conference on Advanced Learning Technologies, Riga, Latvia.","DOI":"10.1109\/ICALT.2009.50"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Maloney, J.H., Peppler, K., Kafai, Y., Resnick, M., and Rusk, N. (2008, January 12\u201315). Programming by Choice: Urban Youth Learning Programming with Scratch. Proceedings of the SIGCSE \u201908: Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education, Portland, OR, USA.","DOI":"10.1145\/1352135.1352260"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Doherty, L., and Kumar, V. (2009, January 4\u20136). Teaching Programming Through Games. Proceedings of the 2009 International Workshop on Technology for Education, Banglore, India.","DOI":"10.1109\/T4E.2009.5314120"},{"key":"ref_21","first-page":"45","article-title":"Pair Programming as a Modern Method of Teaching Computer Science","volume":"3","author":"Rugelj","year":"2008","journal-title":"Int. J. Emerg. Technol. Learn."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2","DOI":"10.7771\/2157-9288.1223","article-title":"A Systematic Review of Studies on Educational Robotics","volume":"9","author":"Anwar","year":"2019","journal-title":"J. Pre-College Eng. Educ. Res."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Eguchi, A. (2022, January 27\u201329). Understanding Machine Learning Through AI-Powered Educational Robotics\u2014Pilot Study with Undergraduate Students. Proceedings of the International Conference on Robotics in Education (RiE), Bratislava, Slovakia.","DOI":"10.1007\/978-3-031-12848-6_5"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Papadakis, S., Vaiopoulou, J., Sifaki, E., Stamovlasis, D., and Kalogiannakis, M. (2021). Attitudes towards the Use of Educational Robotics: Exploring Pre-Service and In-Service Early Childhood Teacher Profiles. Educ. Sci., 11.","DOI":"10.3390\/educsci11050204"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"978","DOI":"10.1016\/j.compedu.2011.10.006","article-title":"Exploring the Educational Potential of Robotics in Schools: A Systematic Review","volume":"58","author":"Benitti","year":"2012","journal-title":"Comput. Educ."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1080\/15391523.2007.10782481","article-title":"Robotics as Means to Increase Achievement Scores in an Informal Learning Environment","volume":"39","author":"Barker","year":"2007","journal-title":"J. Res. Technol. Educ."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Nugent, G., Barker, B., Grandgenett, N., and Adamchuk, V. (2009, January 18\u201321). The Use of Digital Manipulatives in K-12: Robotics, GPS\/GIS and Programming. Proceedings of the 2009 39th IEEE Frontiers in Education Conference, San Antonio, TX, USA.","DOI":"10.1109\/FIE.2009.5350828"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1002\/tea.20238","article-title":"Robotics and Science Literacy: Thinking Skills, Science Process Skills and Systems Understanding","volume":"45","author":"Sullivan","year":"2008","journal-title":"J. Res. Sci. Teach."},{"key":"ref_29","first-page":"437","article-title":"Robotics as an Educational Tool: Impact of Lego Mindstorms","volume":"7","author":"Afari","year":"2017","journal-title":"Int. J. Inf. Educ. Technol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1007\/s10798-007-9043-3","article-title":"Robotics Projects and Learning Concepts in Science, Technology and Problem Solving","volume":"19","author":"Barak","year":"2009","journal-title":"Int. J. Technol. Des. Educ."},{"key":"ref_31","first-page":"182","article-title":"The Effect of LEGO Training on Pupils\u2019 School Performance in Mathematics, Problem Solving Ability and Attitude: Swedish Data","volume":"9","author":"Hussain","year":"2006","journal-title":"J. Educ. Technol. Soc."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1080\/15391523.2007.10782505","article-title":"Acquisition of Physics Content Knowledge and Scientific Inquiry Skills in a Robotics Summer Camp","volume":"40","author":"Williams","year":"2007","journal-title":"J. Res. Technol. Educ."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1177\/0013164409344507","article-title":"Attempted Validation of the Scores of the VARK: Learning Styles Inventory With Multitrait\u2013Multimethod Confirmatory Factor Analysis Models","volume":"70","author":"Leite","year":"2010","journal-title":"Educ. Psychol. Meas."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1007\/s10648-015-9315-5","article-title":"Giving Learning a Helping Hand: Finger Tracing of Temperature Graphs on an IPad","volume":"27","author":"Agostinhi","year":"2015","journal-title":"Educ. Psychol. Rev."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.learninstruc.2014.10.002","article-title":"Getting the Point: Tracing Worked Examples Enhances Learning","volume":"35","author":"Hu","year":"2015","journal-title":"Learn. Instr."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1016\/j.cogdev.2004.05.003","article-title":"The Visuo-Haptic and Haptic Exploration of Letters Increases the Kindergarten-Children\u2019s Understanding of the Alphabetic Principle","volume":"19","author":"Bara","year":"2004","journal-title":"Cogn. Dev."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"459","DOI":"10.2466\/pms.1981.53.2.459","article-title":"Vision and Kinesthesis in Spatial Short-Term Memory","volume":"53","author":"Podbros","year":"1981","journal-title":"Percept. Mot. Skills"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kurniawan, O., Lee, N.T.S., Datta, S.S., and Leong, P.K. (2018, January 4\u20137). Effectiveness of Physical Robot Versus Robot Simulator in Teaching Introductory Programming. Proceedings of the 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), Wollongong, Australia.","DOI":"10.1109\/TALE.2018.8615190"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1007\/s10648-020-09551-z","article-title":"Does Touching Real Objects Affect Learning?","volume":"33","author":"Novak","year":"2020","journal-title":"Educ. Psychol. Rev."},{"key":"ref_40","unstructured":"(2022, December 31). Pololu Robotics and Electronics. Available online: https:\/\/www.pololu.com\/product\/4976."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"648","DOI":"10.3389\/fnhum.2015.00648","article-title":"Are All Spatial Reference Frames Egocentric? Reinterpreting Evidence for Allocentric, Object-Centered, or World-Centered Reference Frames","volume":"9","author":"Filimon","year":"2015","journal-title":"Front. Hum. Neurosci."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Gill, S., and Pawluk, D.T.V. (2022). Design of a \u201cCobot Tactile Display\u201d for Accessing Virtual Diagrams by Blind and Visually Impaired Users. Sensors, 22.","DOI":"10.3390\/s22124468"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/11\/5159\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:44:06Z","timestamp":1760125446000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/11\/5159"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,29]]},"references-count":42,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["s23115159"],"URL":"https:\/\/doi.org\/10.3390\/s23115159","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,29]]}}}