{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,18]],"date-time":"2026-02-18T23:41:34Z","timestamp":1771458094088,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,8,12]],"date-time":"2021-08-12T00:00:00Z","timestamp":1628726400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>The increasing digitalization and advancement in information communication technologies has greatly changed how humans interact with digital information. Nowadays, it is not sufficient to only display relevant data in production activities, as the enormous amount of data generated from smart devices can overwhelm operators without being fully utilized. Operators often require extensive knowledge of the machines in use to make informed decisions during processes such as maintenance and production. To enable novice operators to access such knowledge, it is important to reinvent the way of interacting with digitally enhanced smart devices. In this research, a mobile augmented reality remote monitoring system is proposed to help operators with low knowledge and experience level comprehend digital twin data of a device and interact with the device. It analyses both historic logs as well as real-time data through a cloud server and enriches 2D data with 3D models and animations in the 3D physical space. A cloud-based machine learning algorithm is applied to transform learned knowledge into live presentations on a mobile device for users to interact with. A scaled-down case study is conducted using a tower crane model to demonstrate the potential benefits as well as implications when the system is deployed in industrial environments. This user study verifies that the proposed solution yields consistent measurable improvements for novice users in human-device interaction that is statistically significant.<\/jats:p>","DOI":"10.3390\/computers10080099","type":"journal-article","created":{"date-parts":[[2021,8,12]],"date-time":"2021-08-12T10:54:41Z","timestamp":1628765681000},"page":"99","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["An Integrated Mobile Augmented Reality Digital Twin Monitoring System"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7194-5883","authenticated-orcid":false,"given":"F.","family":"He","sequence":"first","affiliation":[{"name":"NUS Graduate School for Integrative Sciences and Engineering, National University of Singapore, Singapore 119077, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9569-3350","authenticated-orcid":false,"given":"S. K.","family":"Ong","sequence":"additional","affiliation":[{"name":"Mechanical Engineering Department, National University of Singapore, 9 Engineering Drive 1, Singapore 117576, Singapore"}]},{"given":"A. Y. C.","family":"Nee","sequence":"additional","affiliation":[{"name":"Mechanical Engineering Department, National University of Singapore, 9 Engineering Drive 1, Singapore 117576, Singapore"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1093\/iwc\/iws024","article-title":"Exploring Interpretations of Data from the Internet of Things in the Home","volume":"25","author":"Brown","year":"2013","journal-title":"Interact. Comput."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1007\/s40684-016-0015-5","article-title":"Smart manufacturing: Past research, present findings, and future directions","volume":"3","author":"Kang","year":"2016","journal-title":"Int. J. Precis. Eng. Manuf. Green Technol."},{"key":"ref_3","unstructured":"Grieves, M. (2021, June 25). Digital Twin: Manufacturing Excellence through Virtual Factory Replication. Available online: http:\/\/www.themanufacturinginstitute.org\/~\/media\/E323C4D8F75A470E8C96D7A07F0A14FB\/DI_2018_Deloitte_MFI_skills_gap_FoW_study."},{"key":"ref_4","first-page":"46","article-title":"Embedded interaction: Interacting with the internet of things","volume":"2","author":"Kranz","year":"2009","journal-title":"IEEE Internet Comput."},{"key":"ref_5","unstructured":"Heun, V., Kasahara, S., and Maes, P. (May, January 27). Smarter objects: Using AR technology to program physical objects and their interactions. Proceedings of the CHI\u201913 Extended Abstracts on Human Factors in Computing Systems, Paris, France."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1145\/2584670","article-title":"User interfaces for smart things\u2014A generative approach with semantic interaction descriptions","volume":"21","author":"Mayer","year":"2014","journal-title":"ACM Trans. Comput. Hum. Interact."},{"key":"ref_7","unstructured":"Isaias, P., and Blashki, K. (2014). Augmented Reality Interfaces for Smart Objects in Ubiquitous Computing Environments. Human-Computer Interfaces and Interactivity: Emergent Research and Applications, Information Science Reference."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3563","DOI":"10.1007\/s00170-017-0233-1","article-title":"Digital twin-driven product design, manufacturing and service with big data","volume":"94","author":"Tao","year":"2018","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1149","DOI":"10.1007\/s00170-018-1617-6","article-title":"Digital twin-based smart production management and control framework for the complex product assembly shop-floor","volume":"96","author":"Zhuang","year":"2018","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1109\/MC.2013.31","article-title":"The future of human-in-the-loop cyber-physical systems","volume":"1","author":"Schirner","year":"2013","journal-title":"Computer"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/j.ijhcs.2017.12.002","article-title":"From smart objects to smart experiences: An end-user development approach","volume":"114","author":"Ardito","year":"2018","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Li, W., Nee, A.Y.C., and Ong, S.K. (2017). A state-of-the-art review of augmented reality in engineering analysis and simulation. Multimodal Technol. Interact., 1.","DOI":"10.3390\/mti1030017"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.procs.2013.11.004","article-title":"Augmented reality aided manufacturing","volume":"25","author":"Barna","year":"2013","journal-title":"Procedia Comput. Sci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1080\/10447318.2014.994194","article-title":"Augmented reality-based manual assembly support with visual features for different degrees of difficulty","volume":"31","author":"Radkowski","year":"2015","journal-title":"Int. J. Hum. Comput. Interact."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"011004","DOI":"10.1115\/1.4031981","article-title":"Object tracking with a range camera for augmented reality assembly assistance","volume":"16","author":"Radkowski","year":"2016","journal-title":"J. Comput. Inf. Sci. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s40436-015-0131-4","article-title":"A comprehensive survey of augmented reality assembly research","volume":"4","author":"Wang","year":"2016","journal-title":"Adv. Manuf."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1016\/j.cirp.2012.05.010","article-title":"Augmented reality applications in design and manufacturing","volume":"61","author":"Nee","year":"2012","journal-title":"CIRP Ann."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.20870\/IJVR.2010.9.2.2767","article-title":"A survey of augmented reality technologies, applications and limitations","volume":"9","author":"Poelman","year":"2010","journal-title":"Int. J. Virtual Real."},{"key":"ref_19","first-page":"3","article-title":"Designing interactions with multilevel auditory displays in mobile audio-augmented reality","volume":"23","author":"Aylett","year":"2016","journal-title":"ACM Trans. Comput. Hum. Interact."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1016\/j.compind.2013.11.004","article-title":"Augmented reality on large screen for interactive maintenance instructions","volume":"65","author":"Fiorentino","year":"2014","journal-title":"Comput. Ind."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/MMUL.2017.3051517","article-title":"Augmented reality in reality","volume":"24","author":"Ling","year":"2017","journal-title":"IEEE MultiMedia"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Li, P., Qin, T., Hu, B., Zhu, F., and Shen, S. (2017, January 9\u201313). Monocular visual-inertial state estimation for mobile augmented reality. Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Nantes, France.","DOI":"10.1109\/ISMAR.2017.18"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","article-title":"ORB-SLAM: A versatile and accurate monocular SLAM system","volume":"31","author":"Montiel","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"He, F., Ong, S.K., and Nee, A.Y.C. (2019). A Mobile Solution for Augmenting a Manufacturing Environment with User-Generated Annotations. Information, 10.","DOI":"10.3390\/info10020060"},{"key":"ref_25","first-page":"579","article-title":"3D Selection Techniques for Mobile Augmented Reality Head-Mounted Displays","volume":"29","author":"Takashima","year":"2016","journal-title":"Interact. Comput."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tao, F., Zhang, M., and Nee, A.Y.C. (2019). Digital Twin Driven Smart Manufacturing, Academic Press.","DOI":"10.1016\/B978-0-12-817630-6.00010-2"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Schroeder, G., Steinmetz, C., Pereira, C.E., Muller, I., Garcia, N., Espindola, D., and Rodrigues, R. (2016, January 19\u201321). Visualising the digital twin using web services and augmented reality. Proceedings of the 2016 IEEE 14th International Conference on Industrial Informatics, Poitiers, France.","DOI":"10.1109\/INDIN.2016.7819217"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1016\/j.promfg.2018.10.070","article-title":"Towards improving the future of manufacturing through digital twin and augmented reality technologies","volume":"17","author":"Rabah","year":"2018","journal-title":"Procedia Manuf."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"898","DOI":"10.1016\/j.procir.2019.03.223","article-title":"Visualisation of the Digital Twin data in manufacturing by using Augmented Reality","volume":"81","author":"Zhu","year":"2019","journal-title":"Procedia CIRP"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3491","DOI":"10.1007\/s00170-021-06838-5","article-title":"Machining process-oriented monitoring method based on digital twin via augmented reality","volume":"113","author":"Liu","year":"2021","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1109\/TII.2012.2209668","article-title":"Computer communication within industrial distributed environment\u2014A survey","volume":"9","author":"Gaj","year":"2013","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Varsaluoma, J., V\u00e4\u00e4t\u00e4j\u00e4, H., Heimonen, T., Tiitinen, K., Hakulinen, J., Turunen, M., and Nieminen, H. (2017, January 11\u201314). Usage Data Analytics for Human-Machine Interactions with Flexible Manufacturing Systems: Opportunities and Challenges. Proceedings of the 2017 21st International Conference Information Visualisation (IV), London, UK.","DOI":"10.1109\/iV.2017.38"},{"key":"ref_33","unstructured":"Salomon, D. (2004). Data Compression: The Complete Reference, Springer."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.ijinfomgt.2014.10.007","article-title":"Beyond the hype: Big data concepts, methods, and analytics","volume":"35","author":"Gandomi","year":"2015","journal-title":"Int. J. Inf. Manag."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.cirp.2017.04.038","article-title":"Towards a Digital Twin for real-time geometry assurance in individualized production","volume":"66","author":"Carlson","year":"2017","journal-title":"CIRP Ann."},{"key":"ref_36","unstructured":"Linowes, J., and Babilinski, K. (2021, June 25). Augmented Reality for Developers: Build Practical Augmented Reality Applications with Unity, ARCore, ARKit, and Vuforia. Available online: https:\/\/www.packtpub.com\/web-development\/augmented-reality-developers."},{"key":"ref_37","first-page":"747","article-title":"Similarity-based classification: Concepts and algorithms","volume":"10","author":"Chen","year":"2009","journal-title":"J. Mach. Learn. Res."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.ijhcs.2005.06.002","article-title":"Current practice in measuring usability: Challenges to usability studies and research","volume":"64","year":"2006","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Sauro, J., and Lewis, J.R. (2016). Quantifying the User Experience: Practical Statistics for User Research, Morgan Kaufmann.","DOI":"10.1016\/B978-0-12-802308-2.00002-3"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Nielsen, J. (1994). Usability Engineering, Morgan Kaufmann.","DOI":"10.1016\/B978-0-08-052029-2.50009-7"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.advengsoft.2015.04.014","article-title":"Real-time finite element structural analysis in augmented reality","volume":"87","author":"Huang","year":"2015","journal-title":"Adv. Eng. Softw."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/10\/8\/99\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:45:00Z","timestamp":1760165100000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/10\/8\/99"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,12]]},"references-count":41,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["computers10080099"],"URL":"https:\/\/doi.org\/10.3390\/computers10080099","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,12]]}}}