{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,2]],"date-time":"2025-08-02T18:51:57Z","timestamp":1754160717777,"version":"3.41.2"},"reference-count":44,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2025,8]]},"abstract":"<jats:p>We introduce VR-Doh, an open-source, hands-on 3D modeling system that enables intuitive creation and manipulation of elastoplastic objects in Virtual Reality (VR). By customizing the Material Point Method (MPM) for real-time simulation of hand-induced large deformations and enhancing 3D Gaussian Splatting for seamless rendering, VR-Doh provides an interactive and immersive 3D modeling experience. Users can naturally sculpt, deform, and edit objects through both contact- and gesture-based hand-object interactions. To achieve real-time performance, our system incorporates localized simulation techniques, particle-level collision handling, and the decoupling of physical and appearance representations, ensuring smooth and responsive interactions. VR-Doh supports both object creation and editing, enabling diverse modeling tasks such as designing food items, characters, and interlocking structures, all resulting in simulation-ready assets. User studies with both novice and experienced participants highlight the system's intuitive design, immersive feedback, and creative potential. Compared to existing geometric modeling tools, VR-Doh offers enhanced accessibility and natural interaction, making it a powerful tool for creative exploration in VR.<\/jats:p>","DOI":"10.1145\/3731154","type":"journal-article","created":{"date-parts":[[2025,7,27]],"date-time":"2025-07-27T04:02:22Z","timestamp":1753588942000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["VR-Doh: Hands-on 3D Modeling in Virtual Reality"],"prefix":"10.1145","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-8947-9108","authenticated-orcid":false,"given":"Zhaofeng","family":"Luo","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"},{"name":"Peking University, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-8038-2692","authenticated-orcid":false,"given":"Zhitong","family":"Cui","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"},{"name":"Zhejiang University, Hangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8124-3300","authenticated-orcid":false,"given":"Shijian","family":"Luo","sequence":"additional","affiliation":[{"name":"Zhejiang University, Hangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7358-433X","authenticated-orcid":false,"given":"Mengyu","family":"Chu","sequence":"additional","affiliation":[{"name":"Peking University, Beijing, China"},{"name":"State Key Laboratory of General Artificial Intelligence, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9868-7311","authenticated-orcid":false,"given":"Minchen","family":"Li","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"}]}],"member":"320","published-online":{"date-parts":[[2025,7,26]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3332165.3347942"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/WHC49131.2021.9517171"},{"key":"e_1_2_2_3_1","volume-title":"Free-form Shape Modeling in XR: A Systematic Review. arXiv preprint arXiv:2401.00924","author":"Chatterjee Shounak","year":"2024","unstructured":"Shounak Chatterjee. 2024. Free-form Shape Modeling in XR: A Systematic Review. arXiv preprint arXiv:2401.00924 (2024)."},{"key":"e_1_2_2_4_1","volume-title":"PhyVR: Physics-based Multi-material and Free-hand Interaction in VR. In 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 454\u2013462","author":"Deng Hanchen","year":"2023","unstructured":"Hanchen Deng, Jin Li, Yang Gao, Xiaohui Liang, Hongyu Wu, and Aimin Hao. 2023. PhyVR: Physics-based Multi-material and Free-hand Interaction in VR. In 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 454\u2013462."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.gmod.2004.06.008"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.2018.8446595"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459757"},{"key":"e_1_2_2_8_1","doi-asserted-by":"crossref","unstructured":"Noura Faraj Jean-Marc Thiery and Tamy Boubekeur. 2013. Progressive medial axis filtration. In SIGGRAPH Asia 2013 Technical Briefs. 1\u20134.","DOI":"10.1145\/2542355.2542359"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/127719.122747"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.2018.8448284"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201293"},{"key":"e_1_2_2_12_1","first-page":"1","article-title":"Taichi: a language for high-performance computation on spatially sparse data structures","volume":"38","author":"Hu Yuanming","year":"2019","unstructured":"Yuanming Hu, Tzu-Mao Li, Luke Anderson, Jonathan Ragan-Kelley, and Fr\u00e9do Durand. 2019. Taichi: a language for high-performance computation on spatially sparse data structures. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1\u201316.","journal-title":"ACM Transactions on Graphics (TOG)"},{"volume-title":"A soft hand model for physically-based manipulation of virtual objects. In 2011 IEEE virtual reality conference","author":"Jacobs Jan","key":"e_1_2_2_13_1","unstructured":"Jan Jacobs and Bernd Froehlich. 2011. A soft hand model for physically-based manipulation of virtual objects. In 2011 IEEE virtual reality conference. IEEE, 11\u201318."},{"key":"e_1_2_2_14_1","volume-title":"DAPI 2014, Held as Part of HCI Interational 2014, Heraklion, Crete, Greece, June 22\u201327, 2014. Proceedings 2. Springer, 130\u2013141","author":"Jang A","year":"2014","unstructured":"Sung-A Jang, Hyung-il Kim, Woontack Woo, and Graham Wakefield. 2014. Airsculpt: A wearable augmented reality 3d sculpting system. In Distributed, Ambient, and Pervasive Interactions: Second International Conference, DAPI 2014, Held as Part of HCI Interational 2014, Heraklion, Crete, Greece, June 22\u201327, 2014. Proceedings 2. Springer, 130\u2013141."},{"key":"e_1_2_2_15_1","doi-asserted-by":"crossref","unstructured":"Chenfanfu Jiang Craig Schroeder Joseph Teran Alexey Stomakhin and Andrew Selle. 2016. The material point method for simulating continuum materials. In Acm siggraph 2016 courses. 1\u201352.","DOI":"10.1145\/2897826.2927348"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3641519.3657448"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DUI.2008.4476591"},{"key":"e_1_2_2_18_1","volume-title":"Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology 3, 3","author":"Kennedy Robert S","year":"1993","unstructured":"Robert S Kennedy, Norman E Lane, Kevin S Berbaum, and Michael G Lilienthal. 1993. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology 3, 3 (1993), 203\u2013220."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3592433"},{"key":"e_1_2_2_20_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs\/1412.6980 (2014). https:\/\/api.semanticscholar.org\/CorpusID:6628106"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/HUMANOIDS.2015.7363441"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459753"},{"key":"e_1_2_2_23_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2753755","article-title":"Q-mat: Computing medial axis transform by quadratic error minimization","volume":"35","author":"Li Pan","year":"2015","unstructured":"Pan Li, Bin Wang, Feng Sun, Xiaohu Guo, Caiming Zhang, and Wenping Wang. 2015. Q-mat: Computing medial axis transform by quadratic error minimization. ACM Transactions on Graphics (TOG) 35, 1 (2015), 1\u201316.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_24_1","volume-title":"Off-the-Shelf Integrated Solution. In 2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR). IEEE, 743\u2013753","author":"Lougiakis Christos","year":"2024","unstructured":"Christos Lougiakis, Jorge Juan Gonz\u00e5lez, Giorgos Ganias, Akrivi Katifori, Ioannis-Panagiotis, and Maria Roussou. 2024. Comparing Physics-based Hand Interaction in Virtual Reality: Custom Soft Body Simulation vs. Off-the-Shelf Integrated Solution. In 2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR). IEEE, 743\u2013753."},{"key":"e_1_2_2_25_1","volume-title":"Measuring usability with the use questionnaire12. Usability interface 8, 2","author":"Lund Arnold M","year":"2001","unstructured":"Arnold M Lund. 2001. Measuring usability with the use questionnaire12. Usability interface 8, 2 (2001), 3\u20136."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3635636.3656188"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DUI.2010.5444720"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/364338.364395"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450337.3483475"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1275808.1276429"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3379337.3415884"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/SMI.2008.4548000"},{"key":"e_1_2_2_33_1","volume-title":"Hafnia Hands: A Multi-Skin Hand Texture Resource for Virtual Reality Research. Frontiers in Virtual Reality 3","author":"Pohl Henning","year":"2022","unstructured":"Henning Pohl and Aske Mottelson. 2022. Hafnia Hands: A Multi-Skin Hand Texture Resource for Virtual Reality Research. Frontiers in Virtual Reality 3 (2022)."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478513.3480511"},{"key":"e_1_2_2_35_1","volume-title":"SurfaceBrush: from virtual reality drawings to manifold surfaces. arXiv preprint arXiv:1904.12297","author":"Rosales Enrique","year":"2019","unstructured":"Enrique Rosales, Jafet Rodriguez, and Alla Sheffer. 2019. SurfaceBrush: from virtual reality drawings to manifold surfaces. arXiv preprint arXiv:1904.12297 (2019)."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/365024.365114"},{"key":"e_1_2_2_37_1","volume-title":"Rodmesh: Two-handed 3D surface modeling in virtual reality. In Vision, modeling and visualization. 1\u201310.","author":"Schulz HJ","year":"2019","unstructured":"HJ Schulz, M Teschner, and M Wimmer. 2019. Rodmesh: Two-handed 3D surface modeling in virtual reality. In Vision, modeling and visualization. 1\u201310."},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1174429.1174467"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417768"},{"key":"e_1_2_2_40_1","volume-title":"Medial spheres for shape approximation","author":"Stolpner Svetlana","year":"2011","unstructured":"Svetlana Stolpner, Paul Kry, and Kaleem Siddiqi. 2011. Medial spheres for shape approximation. IEEE transactions on pattern analysis and machine intelligence 34, 6 (2011), 1234\u20131240."},{"key":"e_1_2_2_41_1","volume-title":"Medial meshes-a compact and accurate representation of medial axis transform","author":"Sun Feng","year":"2015","unstructured":"Feng Sun, Yi-King Choi, Yizhou Yu, and Wenping Wang. 2015. Medial meshes-a compact and accurate representation of medial axis transform. IEEE transactions on visualization and computer graphics 22, 3 (2015), 1278\u20131290."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.2018.8447555"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.00420"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445158"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3731154","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,27]],"date-time":"2025-07-27T04:12:54Z","timestamp":1753589574000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3731154"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,26]]},"references-count":44,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,8]]}},"alternative-id":["10.1145\/3731154"],"URL":"https:\/\/doi.org\/10.1145\/3731154","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"type":"print","value":"0730-0301"},{"type":"electronic","value":"1557-7368"}],"subject":[],"published":{"date-parts":[[2025,7,26]]},"assertion":[{"value":"2025-07-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}