{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T04:18:39Z","timestamp":1770956319195,"version":"3.50.1"},"reference-count":51,"publisher":"ASME International","issue":"1","content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2018,3,1]]},"abstract":"<jats:p>An important part of the engineering design process is prototyping, where designers build and test their designs. This process is typically iterative, time consuming, and manual in nature. For a given task, there are multiple objects that can be used, each with different time units associated with accomplishing the task. Current methods for reducing time spent during the prototyping process have focused primarily on optimizing designer to designer interactions, as opposed to designer to tool interactions. Advancements in commercially available sensing systems (e.g., the Kinect) and machine learning algorithms have opened the pathway toward real-time observation of designer's behavior in engineering workspaces during prototype construction. Toward this end, this work hypothesizes that an object O being used for task i is distinguishable from object O being used for task j, where i is the correct task and j is the incorrect task. The contributions of this work are: (i) the ability to recognize these objects in a free roaming engineering workshop environment and (ii) the ability to distinguish between the correct and incorrect use of objects used during a prototyping task. By distinguishing the difference between correct and incorrect uses, incorrect behavior (which often results in wasted time and materials) can be detected and quickly corrected. The method presented in this work learns as designers use objects, and infers the proper way to use them during prototyping. In order to demonstrate the effectiveness of the proposed method, a case study is presented in which participants in an engineering design workshop are asked to perform correct and incorrect tasks with a tool. The participants' movements are analyzed by an unsupervised clustering algorithm to determine if there is a statistical difference between tasks being performed correctly and incorrectly. Clusters which are a plurality incorrect are found to be significantly distinct for each node considered by the method, each with p \u226a 0.001.<\/jats:p>","DOI":"10.1115\/1.4037434","type":"journal-article","created":{"date-parts":[[2017,7,29]],"date-time":"2017-07-29T07:31:09Z","timestamp":1501313469000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":13,"title":["An Unsupervised Machine Learning Approach to Assessing Designer Performance During Physical Prototyping"],"prefix":"10.1115","volume":"18","author":[{"given":"Matthew L.","family":"Dering","sequence":"first","affiliation":[{"name":"Computer Science and Engineering, Pennsylvania State University, University Park, PA 16801 e-mail:"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Conrad S.","family":"Tucker","sequence":"additional","affiliation":[{"name":"Industrial Engineering, Pennsylvania State University, University Park, PA 16801 e-mail:"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Soundar","family":"Kumara","sequence":"additional","affiliation":[{"name":"Industrial Engineering, Pennsylvania State University, University Park, PA 16801 e-mail:"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"33","published-online":{"date-parts":[[2017,11,13]]},"reference":[{"issue":"7","key":"2019100602071243600_bib1","doi-asserted-by":"publisher","first-page":"071402","DOI":"10.1115\/1.4030049","article-title":"Automated Discovery of Lead Users and Latent Product Features by Mining Large Scale Social Media Networks","volume":"137","year":"2015","journal-title":"ASME J. Mech. Des."},{"issue":"3","key":"2019100602071243600_bib2","doi-asserted-by":"publisher","first-page":"031003","DOI":"10.1115\/1.4029562","article-title":"Quantifying Product Favorability and Extracting Notable Product Features Using Large Scale Social Media Data","volume":"15","year":"2015","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"4","key":"2019100602071243600_bib3","doi-asserted-by":"publisher","first-page":"514","DOI":"10.1115\/1.1897408","article-title":"Enhancing Discrete Choice Demand Modeling for Decision-Based Design","volume":"127","year":"2005","journal-title":"ASME J. Mech. Des."},{"issue":"7","key":"2019100602071243600_bib4","doi-asserted-by":"publisher","first-page":"071008","DOI":"10.1115\/1.3149845","article-title":"Optimal Experimental Design of Human Appraisals for Modeling Consumer Preferences in Engineering Design","volume":"131","year":"2009","journal-title":"ASME J. Mech. Des."},{"issue":"15","key":"2019100602071243600_bib5","doi-asserted-by":"publisher","first-page":"2955","DOI":"10.1080\/00207540410001691929","article-title":"Data-Mining-Based Methodology for the Design of Product Families","volume":"42","year":"2004","journal-title":"Int. J. Prod. Res."},{"issue":"1","key":"2019100602071243600_bib6","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1016\/j.arcontrol.2007.03.003","article-title":"Data Mining in Design of Products and Production Systems","volume":"31","year":"2007","journal-title":"Annu. Rev. Control"},{"issue":"11","key":"2019100602071243600_bib7","doi-asserted-by":"publisher","first-page":"111008","DOI":"10.1115\/1.4004987","article-title":"Trend Mining for Predictive Product Design","volume":"133","year":"2011","journal-title":"ASME J. Mech. Des."},{"issue":"1","key":"2019100602071243600_bib8","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1080\/0954482032000101731","article-title":"Product Modularity: Measures and Design Methods","volume":"15","year":"2004","journal-title":"J. Eng. Des."},{"issue":"1","key":"2019100602071243600_bib9","doi-asserted-by":"publisher","first-page":"011007","DOI":"10.1115\/1.3086032","article-title":"A Component Taxonomy as a Framework for Computational Design Synthesis","volume":"9","year":"2009","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"6","key":"2019100602071243600_bib10","doi-asserted-by":"publisher","first-page":"061002","DOI":"10.1115\/1.3116253","article-title":"A Fuzzy Method for Propagating Functional Architecture Constraints to Physical Architecture","volume":"131","year":"2009","journal-title":"ASME J. Mech. Des."},{"issue":"4","key":"2019100602071243600_bib11","doi-asserted-by":"publisher","first-page":"370","DOI":"10.1115\/1.2429697","article-title":"Review of Metamodeling Techniques in Support of Engineering Design Optimization","volume":"129","year":"2007","journal-title":"ASME J. Mech. Des."},{"issue":"4","key":"2019100602071243600_bib12","doi-asserted-by":"publisher","first-page":"945","DOI":"10.1115\/1.2204974","article-title":"Understanding the Effects of Model Uncertainty in Robust Design With Computer Experiments","volume":"128","year":"2006","journal-title":"ASME J. Mech. Des."},{"issue":"6","key":"2019100602071243600_bib13","doi-asserted-by":"publisher","first-page":"443","DOI":"10.1080\/09544828.2011.615302","article-title":"A User Study of Interpretability of Engineering Design Representations","volume":"23","year":"2012","journal-title":"J. Eng. Des."},{"issue":"1","key":"2019100602071243600_bib14","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1016\/j.destud.2011.06.005","article-title":"The Psychological Experience of Prototyping","volume":"33","year":"2012","journal-title":"Des. Stud."},{"issue":"6","key":"2019100602071243600_bib15","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1016\/j.destud.2005.04.005","article-title":"A Study of Prototypes, Design Activity, and Design Outcome","volume":"26","year":"2005","journal-title":"Des. Stud."},{"key":"2019100602071243600_bib16","first-page":"367","article-title":"What Do Prototypes Prototype","volume-title":"Handbook of Human-Computer Interaction"},{"issue":"4","key":"2019100602071243600_bib17","doi-asserted-by":"publisher","first-page":"491","DOI":"10.1115\/1.2826709","article-title":"Accelerating Product Development by the Exchange of Preliminary Product Design Information","volume":"117","year":"1995","journal-title":"ASME J. Mech. Des."},{"key":"2019100602071243600_bib18","doi-asserted-by":"publisher","first-page":"56","DOI":"10.3141\/2081-06","article-title":"Ultrawideband for Automated Real-Time Three-Dimensional Location Sensing for Workforce, Equipment, and Material Positioning and Tracking","volume":"2081","year":"2008","journal-title":"Transp. Res. Rec."},{"issue":"2","key":"2019100602071243600_bib19","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1145\/1375761.1375762","article-title":"The Anatomy of Prototypes: Prototypes as Filters, Prototypes as Manifestations of Design Ideas","volume":"15","year":"2008","journal-title":"ACM Trans. Comput.-Hum. Interact."},{"key":"2019100602071243600_bib20","doi-asserted-by":"crossref","unstructured":"Dalal, N., and Triggs, B., 2005, \u201cHistograms of Oriented Gradients for Human Detection,\u201d IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, June 20\u201325, pp. 886\u2013893.10.1109\/CVPR.2005.177","DOI":"10.1109\/CVPR.2005.177"},{"key":"2019100602071243600_bib21","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G., 2011, \u201cORB: An Efficient Alternative to SIFT or SURF,\u201d IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain, Nov. 6\u201313, pp. 2564\u20132571.10.1109\/ICCV.2011.6126544","DOI":"10.1109\/ICCV.2011.6126544"},{"issue":"5","key":"2019100602071243600_bib22","doi-asserted-by":"publisher","first-page":"368","DOI":"10.1111\/j.1467-8667.2010.00690.x","article-title":"Automated Object Identification Using Optical Video Cameras on Construction Sites","volume":"26","year":"2011","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"2019100602071243600_bib23","doi-asserted-by":"crossref","unstructured":"Bo, L., Ren, X., and Fox, D., 2011, \u201cDepth Kernel Descriptors for Object Recognition,\u201d IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, CA, Sept. 25\u201330, pp. 821\u2013826.10.1109\/IROS.2011.6095119","DOI":"10.1109\/IROS.2011.6095119"},{"key":"2019100602071243600_bib24","doi-asserted-by":"crossref","unstructured":"Ren, X., Bo, L., and Fox, D., 2012, \u201cRGB-(D) Scene Labeling: Features and Algorithms,\u201d IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, June 16\u201321, pp. 2759\u20132766.10.1109\/CVPR.2012.6247999","DOI":"10.1109\/CVPR.2012.6247999"},{"issue":"3","key":"2019100602071243600_bib25","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","year":"2015","journal-title":"Int. J. Comput. Vision"},{"key":"2019100602071243600_bib26","first-page":"1097","article-title":"ImageNet Classification With Deep Convolutional Neural Networks","year":"2012","journal-title":"NIPS"},{"key":"2019100602071243600_bib27","article-title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","volume-title":"arXiv:1409.1556","year":"2014"},{"key":"2019100602071243600_bib28","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1007\/978-1-4614-3185-5_59","article-title":"Augmented Reality Teaching and Learning","volume-title":"Handbook of Research on Educational Communications and Technology","year":"2014"},{"issue":"3","key":"2019100602071243600_bib29","doi-asserted-by":"publisher","first-page":"777","DOI":"10.1016\/j.cad.2012.08.001","article-title":"Engineering Design Using Game-Enhanced CAD: The Potential to Augment the User Experience With Game Elements","volume":"45","year":"2013","journal-title":"Comput.-Aided Des."},{"issue":"11\u201312","key":"2019100602071243600_bib30","doi-asserted-by":"publisher","first-page":"768","DOI":"10.1007\/s00170-003-1604-3","article-title":"A Solution to Integrate Computer-Aided Design (CAD) and Virtual Reality (VR) Databases in Design and Manufacturing Processes","volume":"22","year":"2003","journal-title":"Int. J. Adv. Manuf. Technol."},{"issue":"5","key":"2019100602071243600_bib31","doi-asserted-by":"publisher","first-page":"445","DOI":"10.1016\/j.cad.2008.10.014","article-title":"VR\u2013CAD Integration: Multimodal Immersive Interaction and Advanced Haptic Paradigms for Implicit Edition of CAD Models","volume":"42","year":"2010","journal-title":"Comput.-Aided Des."},{"issue":"3","key":"2019100602071243600_bib32","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1017\/S0890060409000250","article-title":"Analyzing Opportunities for Using Interactive Augmented Prototyping in Design Practice","volume":"23","year":"2009","journal-title":"J. Artif. Intell. Eng. Des. Anal. Manuf."},{"issue":"2","key":"2019100602071243600_bib33","doi-asserted-by":"publisher","first-page":"024503","DOI":"10.1115\/1.4006431","article-title":"Augmented Technical Drawings: A Novel Technique for Natural Interactive Visualization of Computer-Aided Design Models","volume":"12","year":"2012","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"4","key":"2019100602071243600_bib34","doi-asserted-by":"publisher","first-page":"041007","DOI":"10.1115\/1.4028588","article-title":"The Influence of Interaction Technology on the Learning of Assembly Tasks Using Virtual Reality","volume":"14","year":"2014","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"3","key":"2019100602071243600_bib35","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1007\/s001380100064","article-title":"Motion Segmentation and Pose Recognition With Motion History Gradients","volume":"13","year":"2002","journal-title":"Mach. Vision Appl."},{"key":"2019100602071243600_bib36","unstructured":"Ribeiro, P. C., Santos-Victor, J., and Lisboa, P., 2005, \u201cHuman Activity Recognition From Video: Modeling, Feature Selection and Classification Architecture,\u201d International Workshop on Human Activity Recognition and Modelling (HAREM), Oxford, UK, Sept. 9, pp. 61\u201378.https:\/\/www.researchgate.net\/publication\/237448747_Human_Activity_Recognition_from_Video_modeling_feature_selection_and_classification_architecture"},{"key":"2019100602071243600_bib37","doi-asserted-by":"crossref","unstructured":"Li, W., Zhang, Z., and Liu, Z., 2010, \u201cAction Recognition Based on a Bag of 3D Points,\u201d IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), San Francisco, CA, June 13\u201318, pp. 9\u201314.10.1109\/CVPRW.2010.5543273","DOI":"10.1109\/CVPRW.2010.5543273"},{"issue":"1","key":"2019100602071243600_bib38","doi-asserted-by":"publisher","first-page":"011006","DOI":"10.1115\/1.4025810","article-title":"Toward Safe Human Robot Collaboration by Using Multiple Kinects Based Real-Time Human Tracking","volume":"14","year":"2014","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"key":"2019100602071243600_bib39","doi-asserted-by":"publisher","DOI":"10.1115\/DETC2014-34770","article-title":"Quantifying Emotional States Based on Body Language Data Using Non Invasive Sensors","volume-title":"ASME","year":"2014"},{"key":"2019100602071243600_bib40","unstructured":"Tucker, C., and Kumara, S., 2015, \u201cAn Automated Object-Task Mining Model for Providing Students With Real Time Performance Feedback,\u201d American Society for Engineering Education Annual Conference and Exposition (ASEE), Seattle, WA, June 14\u201317.https:\/\/www.google.co.in\/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0ahUKEwi3i6ig4-fVAhWR2YMKHeNwA1EQFggwMAE&url=https%3A%2F%2Fwww.asee.org%2Fpublic%2Fconferences%2F56%2Fpapers%2F13126%2Fdownload&usg=AFQjCNEmkbMFMpGjZUWtqYfK-2gnAoQtfw"},{"issue":"3","key":"2019100602071243600_bib41","doi-asserted-by":"publisher","first-page":"438","DOI":"10.1109\/TSMCB.2005.846654","article-title":"Visual Learning by Imitation With Motor Representations","volume":"35","year":"2005","journal-title":"IEEE Trans. Syst. Man Cybern., Part B"},{"key":"2019100602071243600_bib42","article-title":"Learning Object Arrangements in 3D Scenes Using Human Context","volume-title":"arXiv:1206.6462","year":"2012"},{"issue":"8","key":"2019100602071243600_bib43","doi-asserted-by":"publisher","first-page":"951","DOI":"10.1177\/0278364913478446","article-title":"Learning Human Activities and Object Affordances From RGB-D Videos","volume":"32","year":"2013","journal-title":"Int. J. Rob. Res."},{"key":"2019100602071243600_bib44","unstructured":"Yu, G., Liu, Z., and Yuan, J., 2014, \u201cDiscriminative Orderlet Mining for Real-Time Recognition of Human-Object Interaction,\u201d Asian Conference on Computer Vision (ACCV), Singapore, Nov. 1\u20135, pp. 50\u201365.https:\/\/eeeweba.ntu.edu.sg\/computervision\/Research%20Papers\/2014\/Discriminative%20Orderlet%20Mining%20For%20Real-time%20Recognition%20of%20Human-Object%20Interaction.pdf"},{"issue":"4","key":"2019100602071243600_bib45","doi-asserted-by":"publisher","first-page":"992","DOI":"10.1016\/j.compedu.2009.10.002","article-title":"The Impact of Online Video Lecture Recordings and Automated Feedback on Student Performance","volume":"54","year":"2010","journal-title":"Comput. Educ."},{"issue":"2","key":"2019100602071243600_bib46","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1109\/TE.2004.825220","article-title":"An Automated Feedback System for Computer Organization Projects","volume":"47","year":"2004","journal-title":"IEEE Trans. Educ."},{"issue":"4","key":"2019100602071243600_bib47","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1002\/j.2168-9830.2010.tb01072.x","article-title":"Students' Conceptions of Tutor and Automated Feedback in Professional Writing","volume":"99","year":"2010","journal-title":"J. Eng. Educ."},{"key":"2019100602071243600_bib48","doi-asserted-by":"crossref","unstructured":"Patel, R. A., Hartzler, A., Pratt, W., Back, A., Czerwinski, M., and Roseway, A., 2013, \u201cVisual Feedback on Nonverbal Communication: A Design Exploration With Healthcare Professionals,\u201d Seventh International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), Venice, Italy, May 5\u20138, pp. 105\u2013112.10.4108\/icst.pervasivehealth.2013.252024","DOI":"10.4108\/pervasivehealth.2013.252024"},{"key":"2019100602071243600_bib49","doi-asserted-by":"publisher","DOI":"10.1115\/DETC2006-99723","article-title":"The Reincarnation of the Engineering \u201cShop\u201d","volume-title":"ASME","year":"2006"},{"key":"2019100602071243600_bib50","doi-asserted-by":"crossref","unstructured":"Le, Q.V . , 2013, \u201cBuilding High-Level Features Using Large Scale Unsupervised Learning,\u201d IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, BC, Canada, May 26\u201331, pp. 8595\u20138598.10.1109\/ICASSP.2013.6639343","DOI":"10.1109\/ICASSP.2013.6639343"},{"issue":"253","key":"2019100602071243600_bib51","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1080\/01621459.1951.10500769","article-title":"The Kolmogorov-Smirnov Test for Goodness of Fit","volume":"46","year":"1951","journal-title":"J. Am. Stat. Assoc."}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4037434\/6101486\/jcise_018_01_011002.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4037434\/6101486\/jcise_018_01_011002.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,24]],"date-time":"2025-06-24T18:45:37Z","timestamp":1750790737000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/doi\/10.1115\/1.4037434\/366462\/An-Unsupervised-Machine-Learning-Approach-to"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,11,13]]},"references-count":51,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2018,3,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4037434","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"value":"1530-9827","type":"print"},{"value":"1944-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,11,13]]},"article-number":"011002"}}