{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T07:03:33Z","timestamp":1780556613685,"version":"3.54.1"},"reference-count":36,"publisher":"ASME International","issue":"9","license":[{"start":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T00:00:00Z","timestamp":1753401600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.asme.org\/publications-submissions\/publishing-information\/legal-policies"}],"funder":[{"DOI":"10.13039\/501100003398","name":"Shanxi Scholarship Council of China","doi-asserted-by":"publisher","award":["Grant 2023-071"],"award-info":[{"award-number":["Grant 2023-071"]}],"id":[{"id":"10.13039\/501100003398","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>With the wide application of mobile robots, traditional robot programming faces a series of challenges due to its complex task settings and user interactions. In the context of Industry 5.0, we are committed to researching a spatial programming method based on AR (augmented reality) technology to realize human\u2013robot collaboration. In this article, the AR-based spatial perception method is studied, and the virtual coordinate system in AR is established through the virtual reality model fusion perception method. By analyzing the relationship between each virtual coordinate system, the robot virtual space operation is planned, and then the physical world robot operation is controlled. A spatial multimodal AR interaction interface covering gesture, speech, EEG signals, and gaze control is designed to realize different functions for each modal interaction method. In the laboratory environment, the feasibility and effectiveness of the system are tested and verified by taking the coal mining machine interceptor tooth repair task as an example and selecting multiple coal mining machine interceptor tooth repairs for flexible operation tasks in the testing process. The results of the study show that the spatial programming method based on AR technology can significantly improve the efficiency of task execution of operator-controlled mobile robots and enhance the human\u2013robot interaction experience.<\/jats:p>","DOI":"10.1115\/1.4069117","type":"journal-article","created":{"date-parts":[[2025,7,9]],"date-time":"2025-07-09T18:22:02Z","timestamp":1752085322000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":1,"title":["An Augmented Reality Programming Approach to Incorporate Spatial Multimodal Interaction"],"prefix":"10.1115","volume":"25","author":[{"given":"Jingquan","family":"Liu","sequence":"first","affiliation":[{"name":"Taiyuan University of Technology College of Mechanical Engineering, , \u00a0 , ;","place":["Taiyuan, China, 030024"]},{"name":"Taiyuan University of Technology Shanxi Key Laboratory of Fully Mechanized Coal Mining Equipment, , \u00a0 ,","place":["Taiyuan, China, 030024"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuewen","family":"Wang","sequence":"additional","affiliation":[{"name":"Taiyuan University of Technology College of Mechanical Engineering, , \u00a0 , ;","place":["Taiyuan, China, 030024"]},{"name":"Taiyuan University of Technology Shanxi Key Laboratory of Fully Mechanized Coal Mining Equipment, , \u00a0 ,","place":["Taiyuan, China, 030024"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiacheng","family":"Xie","sequence":"additional","affiliation":[{"name":"Taiyuan University of Technology College of Mechanical Engineering, , \u00a0 , ;","place":["Taiyuan, China, 030024"]},{"name":"Taiyuan University of Technology Shanxi Key Laboratory of Fully Mechanized Coal Mining Equipment, , \u00a0 ,","place":["Taiyuan, China, 030024"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuguang","family":"Liu","sequence":"additional","affiliation":[{"name":"Taiyuan University of Technology College of Mechanical Engineering, , \u00a0 , ;","place":["Taiyuan, China, 030024"]},{"name":"Taiyuan University of Technology Shanxi Key Laboratory of Fully Mechanized Coal Mining Equipment, , \u00a0 ,","place":["Taiyuan, China, 030024"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lang","family":"Qin","sequence":"additional","affiliation":[{"name":"Taiyuan University of Technology College of Mechanical Engineering, , \u00a0 , ;","place":["Taiyuan, China, 030024"]},{"name":"Taiyuan University of Technology Shanxi Key Laboratory of Fully Mechanized Coal Mining Equipment, , \u00a0 ,","place":["Taiyuan, China, 030024"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weidong","family":"Shen","sequence":"additional","affiliation":[{"name":"Taiyuan University of Technology College of Mechanical Engineering, , \u00a0 , ;","place":["Taiyuan, China, 030024"]},{"name":"Taiyuan University of Technology Shanxi Key Laboratory of Fully Mechanized Coal Mining Equipment, , \u00a0 ,","place":["Taiyuan, China, 030024"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"33","published-online":{"date-parts":[[2025,7,25]]},"reference":[{"key":"2025072509143954700_CIT0001","doi-asserted-by":"publisher","first-page":"494","DOI":"10.1109\/TASE.2024.3360476","article-title":"Human-Cyber-Physical System for Industry 5.0: A Review From a Human-Centric Perspective","volume":"22","author":"Lou","year":"2024","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"2025072509143954700_CIT0002","doi-asserted-by":"publisher","first-page":"101701","DOI":"10.1016\/j.aei.2022.101701","article-title":"A Framework and Method for Human\u2013Robot Cooperative Safe Control Based on Digital Twin","volume":"53","author":"Li","year":"2022","journal-title":"Adv. Eng. Inform."},{"key":"2025072509143954700_CIT0003","doi-asserted-by":"publisher","first-page":"102411","DOI":"10.1016\/j.aei.2024.102411","article-title":"FedHIP: Federated Learning for Privacy-Preserving Human Intention Prediction in Human\u2013Robot Collaborative Assembly Tasks","volume":"60","author":"Cai","year":"2024","journal-title":"Adv. Eng. Inform."},{"key":"2025072509143954700_CIT0004","doi-asserted-by":"publisher","first-page":"107541","DOI":"10.1016\/j.compag.2022.107541","article-title":"Human\u2013Robot Collaboration Systems in Agricultural Tasks: A Review and Roadmap","volume":"204","author":"Adamides","year":"2023","journal-title":"Comput. Electron. Agric."},{"issue":"3","key":"2025072509143954700_CIT0005","doi-asserted-by":"publisher","first-page":"313","DOI":"10.3390\/mi15030313","article-title":"Continuum Robots and Magnetic Soft Robots: From Models to Interdisciplinary Challenges for Medical Applications","volume":"15","author":"Wang","year":"2024","journal-title":"Micromachines"},{"issue":"3","key":"2025072509143954700_CIT0006","doi-asserted-by":"publisher","first-page":"172","DOI":"10.3390\/machines13030172","article-title":"Layout Optimization of Multi-robot Manufacturing Processing Systems: Applications in Directed Energy Deposition-Arc Additive Manufacturing and Jig-Less Welding","volume":"13","author":"Terzakis","year":"2025","journal-title":"Machines"},{"key":"2025072509143954700_CIT0007","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1016\/j.promfg.2021.10.018","article-title":"A Method for Cutting Force Estimation Through Joint Current Signals in Robotic Machining","volume":"55","author":"Stavropoulos","year":"2021","journal-title":"Procedia Manuf."},{"issue":"8","key":"2025072509143954700_CIT0008","doi-asserted-by":"publisher","first-page":"916","DOI":"10.1080\/0951192X.2015.1130251","article-title":"Human\u2013Robot Interaction Review and Challenges on Task Planning and Programming","volume":"29","author":"Tsarouchi","year":"2016","journal-title":"Int. J. Comput. Integr. Manuf."},{"issue":"2","key":"2025072509143954700_CIT0009","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1016\/j.rcim.2011.08.004","article-title":"Recent Progress on Programming Methods for Industrial Robots","volume":"28","author":"Pan","year":"2012","journal-title":"Rob. Comput. Integr. Manuf."},{"issue":"2","key":"2025072509143954700_CIT0010","doi-asserted-by":"publisher","first-page":"823","DOI":"10.1109\/LRA.2019.2893018","article-title":"A Variational Approach to Minimum-Jerk Trajectories for Psychological Safety in Collaborative Assembly Stations","volume":"4","author":"Rojas","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"2025072509143954700_CIT0011","doi-asserted-by":"publisher","first-page":"102638","DOI":"10.1016\/j.rcim.2023.102638","article-title":"An Augmented Reality-Assisted Interaction Approach Using Deep Reinforcement Learning and Cloud-Edge Orchestration for User-Friendly Robot Teaching","volume":"85","author":"Liu","year":"2024","journal-title":"Rob. Comput. Integr. Manuf."},{"key":"2025072509143954700_CIT0012","doi-asserted-by":"publisher","first-page":"102770","DOI":"10.1016\/j.rcim.2024.102770","article-title":"Augmented Reality Spatial Programming Paradigm Applied to End-User Robot Programming","volume":"89","author":"Kapinus","year":"2024","journal-title":"Rob. Comput. Integr. Manuf."},{"issue":"5","key":"2025072509143954700_CIT0013","doi-asserted-by":"publisher","first-page":"254","DOI":"10.5772\/55640","article-title":"Spatial Programming for Industrial Robots Through Task Demonstration","volume":"10","author":"Lambrecht","year":"2013","journal-title":"Int. J. Adv. Rob. Syst."},{"key":"2025072509143954700_CIT0014","doi-asserted-by":"crossref","first-page":"119","DOI":"10.4337\/9781800889972.00015","volume-title":"Handbook of Artificial Intelligence at Work","author":"Stavropoulos","year":"2024"},{"key":"2025072509143954700_CIT0015","doi-asserted-by":"publisher","first-page":"108050","DOI":"10.1016\/j.cie.2022.108050","article-title":"Framework for a Closed-Loop Cooperative Human Cyber-physical System for the Mining Industry Driven by VR and AR: MHCPS","volume":"168","author":"Xie","year":"2022","journal-title":"Comput. Ind. Eng."},{"key":"2025072509143954700_CIT0016","doi-asserted-by":"publisher","first-page":"949","DOI":"10.1016\/j.jmsy.2024.05.001","article-title":"A New XR-Based Human\u2013Robot Collaboration Assembly System Based on Industrial Metaverse","volume":"74","author":"Xie","year":"2024","journal-title":"J. Manuf. Syst."},{"key":"2025072509143954700_CIT0017","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1016\/j.jmsy.2024.11.007","article-title":"A Multivariate Fusion Collision Detection Method for Dynamic Operations in HRC Systems","volume":"78","author":"Fang","year":"2025","journal-title":"J. Manuf. Syst."},{"issue":"6","key":"2025072509143954700_CIT0018","doi-asserted-by":"publisher","first-page":"061011","DOI":"10.1115\/1.4051054","article-title":"Automated Procedure Reconfiguration Framework for Augmented Reality-Guided Maintenance Applications","volume":"21","author":"Young","year":"2021","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"4","key":"2025072509143954700_CIT0019","doi-asserted-by":"publisher","first-page":"JAMDSM0050","DOI":"10.1299\/jamdsm.2023jamdsm0050","article-title":"Strategic Navigation for Robotic Wheelchair Based on Geometrical Information of Augmented Reality Markers","volume":"17","author":"Maeda","year":"2023","journal-title":"J. Adv. Mech. Des. Syst. Manuf."},{"key":"2025072509143954700_CIT0020","doi-asserted-by":"publisher","first-page":"102321","DOI":"10.1016\/j.rcim.2022.102321","article-title":"AR-Assisted Digital Twin-Enabled Robot Collaborative Manufacturing System With Human-in-the-Loop","volume":"76","author":"Li","year":"2022","journal-title":"Rob. Comput. Integr. Manuf."},{"issue":"1","key":"2025072509143954700_CIT0021","doi-asserted-by":"publisher","first-page":"011004","DOI":"10.1115\/1.4031981","article-title":"Object Tracking With a Range Camera for Augmented Reality Assembly Assistance","volume":"16","author":"Radkowski","year":"2016","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"key":"2025072509143954700_CIT0022","first-page":"1","article-title":"A MR-Assisted and Scene Perception System for Human\u2013Robot Collaborative Disassembly of Power Batteries","author":"Duan","year":"2023"},{"key":"2025072509143954700_CIT0023","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1016\/j.jmsy.2024.03.015","article-title":"Unleashing Mixed-Reality Capability in Deep Reinforcement Learning-Based Robot Motion Generation Towards Safe Human\u2013Robot Collaboration","volume":"74","author":"Li","year":"2024","journal-title":"J. Manuf. Syst."},{"key":"2025072509143954700_CIT0024","first-page":"2796","article-title":"Mixed Reality as Communication Medium for Human\u2013Robot Collaboration","author":"Macci\u00f2","year":"2022"},{"issue":"3","key":"2025072509143954700_CIT0025","doi-asserted-by":"publisher","first-page":"031001","DOI":"10.1115\/1.4062543","article-title":"Assistive Sensory Feedback for Trajectory Tracking in Augmented Reality","volume":"24","author":"Wang","year":"2024","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"key":"2025072509143954700_CIT0026","doi-asserted-by":"crossref","first-page":"103012","DOI":"10.1016\/j.rcim.2025.103012","article-title":"Towards Safe Motion Planning for Industrial Human\u2013Robot Interaction: A Co-evolution Approach Based on Human Digital Twin and Mixed Reality","volume":"95","author":"Feng","year":"2025","journal-title":"Rob. Comput. Integr. Manuf."},{"key":"2025072509143954700_CIT0027","doi-asserted-by":"publisher","first-page":"17305","DOI":"10.1109\/ACCESS.2023.3245833","article-title":"Multimodal Multi-user Mixed Reality Human\u2013Robot Interface for Remote Operations in Hazardous Environments","volume":"11","author":"Szczurek","year":"2023","journal-title":"IEEE Access"},{"issue":"5","key":"2025072509143954700_CIT0028","doi-asserted-by":"publisher","first-page":"1985","DOI":"10.1007\/s10845-023-02096-2","article-title":"HAR2bot: A Human-Centered Augmented Reality Robot Programming Method With the Awareness of Cognitive Load","volume":"35","author":"Yang","year":"2024","journal-title":"J. Intell. Manuf."},{"issue":"9","key":"2025072509143954700_CIT0029","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00170-021-07937-z","article-title":"A Closed-Loop Brain-Computer Interface With Augmented Reality Feedback for Industrial Human\u2013Robot Collaboration","volume":"124","author":"Ji","year":"2023","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"2025072509143954700_CIT0030","doi-asserted-by":"publisher","first-page":"104664","DOI":"10.1016\/j.bspc.2023.104664","article-title":"A Portable SSVEP-BCI System for Rehabilitation Exoskeleton in Augmented Reality Environment","volume":"83","author":"Wang","year":"2023","journal-title":"Biomed. Signal Process. Control"},{"issue":"3","key":"2025072509143954700_CIT0031","doi-asserted-by":"publisher","first-page":"483","DOI":"10.7507\/1001-5515.202011039","article-title":"Robotic Arm Control System Based on Augmented Reality Brain-Computer Interface and Computer Vision","volume":"38","author":"Chen","year":"2021","journal-title":"J. Biomed. Eng."},{"key":"2025072509143954700_CIT0032","doi-asserted-by":"publisher","first-page":"544","DOI":"10.1109\/TNSRE.2022.3228124","article-title":"Brain-Controlled, AR-Based Home Automation System Using SSVEP-Based Brain\u2013Computer Interface and EOG-Based Eye Tracker: A Feasibility Study for the Elderly End User","volume":"31","author":"Park","year":"2023","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"2025072509143954700_CIT0033","doi-asserted-by":"publisher","first-page":"3448","DOI":"10.1109\/TNSRE.2023.3308778","article-title":"SSVEP-Based Brain\u2013Computer Interface Controlled Robotic Platform With Velocity Modulation","volume":"31","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"2025072509143954700_CIT0034","doi-asserted-by":"publisher","first-page":"5990","DOI":"10.1109\/ACCESS.2019.2963442","article-title":"SSVEP Stimulus Layout Effect on Accuracy of Brain\u2013Computer Interfaces in Augmented Reality Glasses","volume":"8","author":"Zhao","year":"2020","journal-title":"IEEE Access"},{"issue":"2","key":"2025072509143954700_CIT0035","doi-asserted-by":"publisher","first-page":"322","DOI":"10.26599\/TST.2021.9010085","article-title":"Study on Robot Grasping System of SSVEP-BCI Based on Augmented Reality Stimulus","volume":"28","author":"Zhang","year":"2023","journal-title":"Tsinghua Sci. Technol."},{"issue":"3","key":"2025072509143954700_CIT0036","doi-asserted-by":"publisher","first-page":"1608","DOI":"10.1109\/TVCG.2018.2873737","article-title":"Towards BCI-Based Interfaces for Augmented Reality: Feasibility, Design and Evaluation","volume":"26","author":"Si-Mohammed","year":"2020","journal-title":"IEEE Trans. Visual Comput. Graph."}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4069117\/7519308\/jcise-24-1636.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4069117\/7519308\/jcise-24-1636.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T13:14:43Z","timestamp":1753449283000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/25\/9\/091008\/1219487\/An-Augmented-Reality-Programming-Approach-to"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,25]]},"references-count":36,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2025,9,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4069117","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"value":"1530-9827","type":"print"},{"value":"1944-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,25]]},"article-number":"091008"}}