{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T05:48:34Z","timestamp":1762321714870,"version":"3.41.2"},"reference-count":55,"publisher":"ASME International","issue":"4","content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2018,12,1]]},"abstract":"<jats:p>We introduce an intuitive gesture-based interaction technique for creating and manipulating simple three-dimensional (3D) shapes. Specifically, the developed interface utilizes low-cost depth camera to capture user's hand gesture as the input, maps different gestures to system commands and generates 3D models from midair 3D sketches (as opposed to traditional two-dimensional (2D) sketches). Our primary contribution is in the development of an intuitive gesture-based interface that enables novice users to rapidly construct conceptual 3D models. Our development extends current works by proposing both design and technical solutions to the challenges of the gestural modeling interface for conceptual 3D shapes. The preliminary user study results suggest that the developed framework is intuitive to use and able to create a variety of 3D conceptual models.<\/jats:p>","DOI":"10.1115\/1.4040982","type":"journal-article","created":{"date-parts":[[2018,8,9]],"date-time":"2018-08-09T18:37:00Z","timestamp":1533839820000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":16,"title":["Conceptual Three-Dimensional Modeling Using Intuitive Gesture-Based Midair Three-Dimensional Sketching Technique"],"prefix":"10.1115","volume":"18","author":[{"given":"Jinmiao","family":"Huang","sequence":"first","affiliation":[{"name":"ABB Inc., Bloomfield, CT 06002 e-mail:"}]},{"given":"Rahul","family":"Rai","sequence":"additional","affiliation":[{"name":"Manufacturing and Design (MAD) Lab, Department of Mechanical and Aerospace Engineering, University at Buffalo (UB)-SUNY, Buffalo, NY 14260 e-mail:"}]}],"member":"33","published-online":{"date-parts":[[2018,9,5]]},"reference":[{"issue":"3","key":"2019100314082186800_bib1","doi-asserted-by":"publisher","first-page":"136","DOI":"10.1016\/j.cad.2008.06.007","article-title":"Impact of Cad Tools on Creative Problem Solving in Engineering Design","volume":"41","year":"2009","journal-title":"Comput.-Aided Des."},{"key":"2019100314082186800_bib2","doi-asserted-by":"crossref","unstructured":"Sharma, A., Madhvanath, S., Shekhawat, A., and Billinghurst, M., 2011, \u201cMozart: A Multimodal Interface for Conceptual 3D Modeling,\u201d 13th International Conference on Multimodal Interfaces (ICMI), Alicante, Spain, Nov. 14\u201318, pp. 307\u2013310.https:\/\/www.researchgate.net\/publication\/221052364_MozArt_A_Multimodal_Interface_for_Conceptual_3D_Modeling","DOI":"10.1145\/2070481.2070538"},{"key":"2019100314082186800_bib3","doi-asserted-by":"crossref","unstructured":"Rubine, D., 1991, \u201cSpecifying Gestures by Example,\u201d 18th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '91), Providence, RI, Apr. 27\u201330, pp. 329\u2013337.","DOI":"10.1145\/122718.122753"},{"issue":"3","key":"2019100314082186800_bib4","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1002\/scj.4690230304","article-title":"A Hand Gesture Recognition Method and Its Application","volume":"23","year":"1992","journal-title":"Syst. Comput. Jpn."},{"issue":"3","key":"2019100314082186800_bib5","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1016\/j.ijhcs.2012.07.004","article-title":"Finger-Based Multitouch Interface for Performing 3D CAD Operations","volume":"71","year":"2013","journal-title":"Int. J. Human-Comput. Stud."},{"key":"2019100314082186800_bib6","doi-asserted-by":"crossref","unstructured":"Seth, A., Su, H.-J., and Vance, J. M., 2006, \u201cSharp: A System for Haptic Assembly and Realistic Prototyping,\u201d ASME Paper No. DETC2006-99476. 10.1115\/DETC2006-99476","DOI":"10.1115\/DETC2006-99476"},{"article-title":"Using Hand Motions in Conceptual Shape Design","volume-title":"Product Engineering","year":"2008","key":"2019100314082186800_bib7"},{"key":"2019100314082186800_bib8","doi-asserted-by":"crossref","unstructured":"Segen, J., and Kumar, S., 1998, \u201cGesture Vr: Vision-Based 3D Hand Interace for Spatial Interaction,\u201d Sixth ACM International Conference on Multimedia, Bristol, UK, Sept. 13\u201316, pp. 455\u2013464.","DOI":"10.1145\/290747.290822"},{"key":"2019100314082186800_bib9","doi-asserted-by":"crossref","unstructured":"Wang, R., Paris, S., and Popovi\u0107, J., 2011, \u201c6D Hands: Markerless Hand-Tracking for Computer Aided Design,\u201d 24th Annual ACM Symposium on User Interface Software and Technology, Santa Barbara, CA, Oct. 16\u201319, pp. 549\u2013558.https:\/\/homes.cs.washington.edu\/~jovan\/papers\/wang-2011-6dh.pdf","DOI":"10.1145\/2047196.2047269"},{"key":"2019100314082186800_bib10","doi-asserted-by":"crossref","unstructured":"Bai, H., Gao, L., El-Sana, J., and Billinghurst, M., \u201cFree-Hand Interaction for Handheld Augmented Reality Using an RGB-Depth Camera,\u201d SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, Hong Kong, Nov. 19\u201322, Paper No. 22.","DOI":"10.1145\/2543651.2543667"},{"key":"2019100314082186800_bib11","doi-asserted-by":"crossref","unstructured":"Stannus, S., Fu, W.-T., and Lucieer, A., 2014, \u201cNatural 7DOF Input for 3D Navigation,\u201d 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design, Sydney, Australia, Dec. 2\u20135, pp. 216\u2013219.","DOI":"10.1145\/2686612.2686646"},{"key":"2019100314082186800_bib12","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1016\/j.cad.2013.08.039","article-title":"Gafinc: Gaze and Finger Control Interface for 3D Model Manipulation in Cad Application","volume":"46","year":"2014","journal-title":"Comput.-Aided Des."},{"issue":"2","key":"2019100314082186800_bib13","doi-asserted-by":"publisher","first-page":"021008","DOI":"10.1115\/1.4023588","article-title":"Handy-Potter: Rapid Exploration of Rotationally Symmetric Shapes Through Natural Hand Motions","volume":"13","year":"2013","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"2","key":"2019100314082186800_bib14","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1007\/s11042-013-1382-3","article-title":"Virtual Pottery: A Virtual 3D Audiovisual Interface Using Natural Hand Motions","volume":"73","year":"2014","journal-title":"Multimedia Tools Appl."},{"key":"2019100314082186800_bib15","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.cad.2015.06.006","article-title":"A Gesture-Free Geometric Approach for Mid-Air Expression of Design Intent in 3D Virtual Pottery","volume":"69","year":"2015","journal-title":"Comput.-Aided Des."},{"issue":"6","key":"2019100314082186800_bib16","doi-asserted-by":"publisher","first-page":"799","DOI":"10.1080\/16864360.2016.1168223","article-title":"Mixed Reality to Design Lower Limb Prosthesis","volume":"13","year":"2016","journal-title":"Comput.-Aided Des. Appl."},{"issue":"10","key":"2019100314082186800_bib17","doi-asserted-by":"publisher","first-page":"1020","DOI":"10.1016\/j.cad.2011.05.009","article-title":"Conceptual Design and Modification of Freeform Surfaces Using Dual Shape Representations in Augmented Reality Environments","volume":"44","year":"2012","journal-title":"Comput.-Aided Des."},{"key":"2019100314082186800_bib18","first-page":"1","article-title":"Gesture-Based System for Next Generation Natural and Intuitive Interfaces","volume":"15","year":"2018","journal-title":"Ai Edam"},{"issue":"1","key":"2019100314082186800_bib19","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1016\/j.cag.2008.09.013","article-title":"Sketch-Based Modeling: A Survey","volume":"33","year":"2009","journal-title":"Comput. Graph."},{"key":"2019100314082186800_bib20","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1145\/1281500.1281532","article-title":"Teddy: A Sketching Interface for 3D Freeform Design","volume-title":"ACM SIGGRAPH 2007 Courses","year":"2007"},{"issue":"3","key":"2019100314082186800_bib21","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1145\/1276377.1276429","article-title":"Fibermesh: Designing Freeform Surfaces With 3D Curves","volume":"26","year":"2007","journal-title":"ACM Trans. Graph. (TOG)"},{"issue":"3","key":"2019100314082186800_bib22","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1111\/1467-8659.t01-1-00709","article-title":"Free-Form Sketching With Variational Implicit Surfaces","volume":"21","year":"2002","journal-title":"Comput. Graph. Forum"},{"issue":"1","key":"2019100314082186800_bib23","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1111\/j.1467-8659.2004.00006.x","article-title":"Prototype Modeling From Sketched Silhouettes Based on Convolution Surfaces","volume":"23","year":"2004","journal-title":"Comput. Graph. Forum"},{"key":"2019100314082186800_bib24","doi-asserted-by":"crossref","unstructured":"Alexe, A., Gaildrat, V., and Barthe, L., 2004, \u201cInteractive Modelling From Sketches Using Spherical Implicit Functions,\u201d Third International Conference on Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa, Stellenbosch, South Africa, Nov. 3\u20135, pp. 25\u201334.","DOI":"10.1145\/1029949.1029953"},{"key":"2019100314082186800_bib25","doi-asserted-by":"crossref","unstructured":"Schmidt, R., Wyvill, B., Sousa, M. C., and Jorge, J. A., 2006, \u201cShapeshop: Sketch-Based Solid Modeling With Blobtrees,\u201d ACM SIGGRAPH 2006 Courses, ACM, New York, p. 14","DOI":"10.1145\/1185657.1185775"},{"issue":"9\u201312","key":"2019100314082186800_bib26","doi-asserted-by":"publisher","first-page":"2669","DOI":"10.1007\/s00170-013-4883-3","article-title":"Integration of Sketch-Based Conceptual Design and Commercial Cad Systems for Manufacturing","volume":"68","year":"2013","journal-title":"Int. J. Adv. Manuf. Technol."},{"first-page":"19","article-title":"Sketch: An Interface for Sketching 3D Scenes","year":"2007","key":"2019100314082186800_bib27"},{"volume-title":"Proceedings of the 10th International Conference on Computer Aided Architectural Design Research in Asia","key":"2019100314082186800_bib28","article-title":"Sketchboard: The Simple 3D Modeling From Architectural Sketch Recognition"},{"issue":"14","key":"2019100314082186800_bib29","doi-asserted-by":"publisher","first-page":"851","DOI":"10.1016\/S0010-4485(00)00078-6","article-title":"From On-Line Sketching to 2D and 3D Geometry: A System Based on Fuzzy Knowledge","volume":"32","year":"2000","journal-title":"Comput.-Aided Des."},{"issue":"4","key":"2019100314082186800_bib30","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1145\/2010324.1964930","article-title":"Probabilistic Reasoning for Assembly-Based 3D Modeling","volume":"30","year":"2011","journal-title":"ACM Trans. Graph. (TOG)"},{"issue":"4","key":"2019100314082186800_bib31","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185551","article-title":"A Probabilistic Model for Component-Based Shape Synthesis","volume":"31","year":"2012","journal-title":"ACM Trans. Graph."},{"key":"2019100314082186800_bib32","doi-asserted-by":"crossref","unstructured":"Chaudhuri, S., Kalogerakis, E., Giguere, S., and Funkhouser, T., 2013, \u201cAttribit: Content Creation With Semantic Attributes,\u201d 26th Annual ACM Symposium on User Interface Software and Technology, St. Andrews, UK, Oct. 8\u201311, pp. 193\u2013202.","DOI":"10.1145\/2501988.2502008"},{"key":"2019100314082186800_bib33","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1016\/j.cad.2015.10.002","article-title":"Assembly-Based Conceptual 3D Modeling With Unlabeled Components Using Probabilistic Factor Graph","volume":"74","year":"2016","journal-title":"Comput.-Aided Des."},{"issue":"6","key":"2019100314082186800_bib34","doi-asserted-by":"publisher","first-page":"226","DOI":"10.1145\/3130800.3130821","article-title":"Complement Me: Weakly-Supervised Component Suggestions for 3D Modeling","volume":"36","year":"2017","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"2019100314082186800_bib35","doi-asserted-by":"crossref","unstructured":"Krupka, E., Karmon, K., Bloom, N., Freedman, D., Gurvich, I., Hurvitz, A., Leichter, I., Smolin, Y., Tzairi, Y., Vinnikov, A., and Bar-Hillel, A., 2017, \u201cToward Realistic Hands Gesture Interface: Keeping It Simple for Developers and Machines,\u201d CHI Conference on Human Factors in Computing Systems, Denver, CO, May 6\u201311, pp. 1887\u20131898.","DOI":"10.1145\/3025453.3025508"},{"issue":"5","key":"2019100314082186800_bib36","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1080\/16864360.2016.1273584","article-title":"A Virtual Environment to Emulate Tailor's Work","volume":"14","year":"2017","journal-title":"Comput.-Aided Des. Appl."},{"article-title":"Gestures Mixed Reality","year":"2018","author":"Microsoft","key":"2019100314082186800_bib37"},{"key":"2019100314082186800_bib38","volume-title":"CGAL User and Reference Manual","author":"The CGAL Project,","year":"2018","edition":"4.12 ed."},{"issue":"1\u20134","key":"2019100314082186800_bib39","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1007\/BF01553881","article-title":"Constrained Delaunay Triangulations","volume":"4","year":"1989","journal-title":"Algorithmica"},{"key":"2019100314082186800_bib40","doi-asserted-by":"crossref","unstructured":"Mapari, R. B., and Kharat, G., 2016, \u201cAmerican Static Signs Recognition Using Leap Motion Sensor,\u201d Second International Conference on Information and Communication Technology for Competitive Strategies, Udaipur, India, Mar. 4\u20135, p. 67.","DOI":"10.1145\/2905055.2905125"},{"key":"2019100314082186800_bib41","doi-asserted-by":"crossref","unstructured":"Marin, G., Dominio, F., and Zanuttigh, P., 2014, \u201cHand Gesture Recognition With Leap Motion and Kinect Devices,\u201d IEEE International Conference on Image Processing (ICIP), Paris, France, Oct. 27\u201330, pp. 1565\u20131569.10.1109\/ICIP.2014.7025313","DOI":"10.1109\/ICIP.2014.7025313"},{"key":"2019100314082186800_bib42","doi-asserted-by":"crossref","unstructured":"Zeiler, M. D., and Fergus, R., 2014, \u201cVisualizing and Understanding Convolutional Networks,\u201d European Conference on Computer Vision, Zurich, Switzerland, Sept. 6\u201312, pp. 818\u2013833.https:\/\/cs.nyu.edu\/~fergus\/papers\/zeilerECCV2014.pdf","DOI":"10.1007\/978-3-319-10590-1_53"},{"article-title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","volume-title":"1409:1556","year":"2014","key":"2019100314082186800_bib43"},{"issue":"3","key":"2019100314082186800_bib44","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","year":"2015","journal-title":"Int. J. Comput. Vision (IJCV)"},{"year":"2015","key":"2019100314082186800_bib45","article-title":"Keras"},{"key":"2019100314082186800_bib46","doi-asserted-by":"crossref","unstructured":"Dalal, N., and Triggs, B., 2005, \u201cHistograms of Oriented Gradients for Human Detection,\u201d IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, June 20\u201325, pp. 886\u2013893.10.1109\/CVPR.2005.177","DOI":"10.1109\/CVPR.2005.177"},{"issue":"4","key":"2019100314082186800_bib47","doi-asserted-by":"publisher","first-page":"518","DOI":"10.1016\/j.cag.2005.05.005","article-title":"Ladder, a Sketching Language for User Interface Developers","volume":"29","year":"2005","journal-title":"Comput. Graph."},{"issue":"4","key":"2019100314082186800_bib48","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1016\/j.cag.2005.05.004","article-title":"An Image-Based, Trainable Symbol Recognizer for Hand-Drawn Sketches","volume":"29","year":"2005","journal-title":"Comput. Graph."},{"key":"2019100314082186800_bib49","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1016\/B978-0-08-051584-7.50027-9","article-title":"Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition","volume-title":"Readings in Speech Recognition","year":"1990"},{"issue":"2","key":"2019100314082186800_bib50","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1109\/TIT.1967.1054010","article-title":"Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm","volume":"13","year":"1967","journal-title":"IEEE Transactions Inf. Theory"},{"key":"2019100314082186800_bib51","doi-asserted-by":"crossref","unstructured":"Tsang, S., Balakrishnan, R., Singh, K., and Ranjan, A., 2004, \u201cA Suggestive Interface for Image Guided 3D Sketching,\u201d SIGCHI Conference on Human Factors in Computing Systems, Vienna, Austria, Apr. 24\u201329, pp. 591\u2013598.","DOI":"10.1145\/985692.985767"},{"volume-title":"Brave Nui World: Designing Natural User Interfaces for Touch and Gesture","year":"2011","key":"2019100314082186800_bib52"},{"year":"2003","key":"2019100314082186800_bib53","article-title":"A Procedure for Developing Intuitive and Ergonomic Gesture Interfaces for HCI"},{"key":"2019100314082186800_bib54","doi-asserted-by":"crossref","unstructured":"Thakur, A., and Rai, R., 2015, \u201cUser Study of Hand Gestures for Gesture Based 3D CAD Modeling,\u201d ASME Paper No. DETC2015-46086. 10.1115\/DETC2015-46086","DOI":"10.1115\/DETC2015-46086"},{"volume-title":"Tactile Display for Virtual 3D Shape Rendering","year":"2017","key":"2019100314082186800_bib55"}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4040982\/6367164\/jcise_018_04_041014.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4040982\/6367164\/jcise_018_04_041014.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,6]],"date-time":"2025-07-06T08:22:16Z","timestamp":1751790136000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/doi\/10.1115\/1.4040982\/460058\/Conceptual-ThreeDimensional-Modeling-Using"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,5]]},"references-count":55,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,12,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4040982","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"type":"print","value":"1530-9827"},{"type":"electronic","value":"1944-7078"}],"subject":[],"published":{"date-parts":[[2018,9,5]]},"article-number":"041014"}}