{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T07:22:17Z","timestamp":1783063337701,"version":"3.54.6"},"reference-count":68,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T00:00:00Z","timestamp":1783036800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2026,7,3]]},"abstract":"<jats:p>Providing haptic feedback in virtual reality (VR) remains challenging, with active haptic systems often being bulky and expensive, while passive haptics typically require pre-fabricated objects or extensive real-world setup. We present ProXeek, an LLM-based multi-agent system that opportunistically leverages real-world objects as haptic proxies for VR experiences. During VR development, the designers could annotate the virtual interactables with emphasized haptic characteristics using our node-based editing interface. On the user side, end users capture environmental snapshots via commodity VR headset, and the system employs LLM-based multimodal reasoning and multi-objective optimization to match virtual objects with viable physical proxies. Our technical evaluation demonstrates that ProXeek effectively identifies viable proxies, confirming the effectiveness of the algorithmic constraint design in our matching pipeline. Our user study further reveals that the ProXeek-selected physical objects could deliver significantly higher haptic fidelity and enhance presence compared to 3D-printed replicas and visual-only interaction.<\/jats:p>","DOI":"10.1145\/3811349","type":"journal-article","created":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T07:05:51Z","timestamp":1783062351000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["ProXeek: Seeking and Leveraging Real-World Objects and Environments as Haptic Proxies for Virtual Reality through Multimodal Reasoning"],"prefix":"10.1145","volume":"45","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6194-3094","authenticated-orcid":false,"given":"Haichen","family":"Gao","sequence":"first","affiliation":[{"name":"City University of Hong Kong, School of Creative Media, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-0599-3664","authenticated-orcid":false,"given":"Tianrui","family":"Hu","sequence":"additional","affiliation":[{"name":"City University of Hong Kong, School of Creative Media, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0939-4177","authenticated-orcid":false,"given":"Zeshui","family":"Li","sequence":"additional","affiliation":[{"name":"City University of Hong Kong, School of Creative Media, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6740-4921","authenticated-orcid":false,"given":"Kening","family":"Zhu","sequence":"additional","affiliation":[{"name":"City University of Hong Kong, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,7,3]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173724"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581514"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858226"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR46266.2020.00044"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173663"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126594.3126667"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.2019.8798074"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2807442.2807463"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.01599"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126594.3126599"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642292"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376470"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544549.3585871"},{"key":"e_1_2_2_14_1","volume-title":"PantographHaptics: A Technique for Large-Surface Passive Haptic Interactions using Pantograph Mechanisms","author":"Friedel Marcus KE","year":"2025","unstructured":"Marcus KE Friedel, Zachary McKendrick, Ehud Sharlin, and Ryo Suzuki. 2025. PantographHaptics: A Technique for Large-Surface Passive Haptic Interactions using Pantograph Mechanisms. IEEE Transactions on Visualization and Computer Graphics (2025)."},{"key":"e_1_2_2_15_1","doi-asserted-by":"crossref","unstructured":"Haichen Gao Shaoyu Cai Yuhong Wu and Kening Zhu. 2024. ThermOuch: a wearable thermo-haptic device for inducing pain sensation in virtual reality through thermal grill illusion. In SIGGRAPH asia 2024 conference papers. 1\u201312.","DOI":"10.1145\/3680528.3687620"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3472749.3474782"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR50242.2020.00062"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173225.3173241"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300577"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3526113.3545650"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858134"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3139131.3143416"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/522258.836098"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE60146.2024.00066"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3654777.3676389"},{"key":"e_1_2_2_26_1","volume-title":"Passive haptics significantly enhances virtual environments","author":"Insko Brent Edward","unstructured":"Brent Edward Insko. 2001. Passive haptics significantly enhances virtual environments. The University of North Carolina at Chapel Hill."},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3586183.3606793"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530113"},{"key":"e_1_2_2_29_1","volume-title":"Scene2Hap: Combining LLMs and Physical Modeling for Automatically Generating Vibrotactile Signals for Full VR Scenes. arXiv preprint arXiv:2504.19611","author":"Jingu Arata","year":"2025","unstructured":"Arata Jingu, Easa AliAbbasi, Paul Strohmeier, and J\u00fcrgen Steimle. 2025. Scene2Hap: Combining LLMs and Physical Modeling for Automatically Generating Vibrotactile Signals for Full VR Scenes. arXiv preprint arXiv:2504.19611 (2025)."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.4324\/9780203771976"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2022.3205181"},{"key":"e_1_2_2_32_1","volume-title":"Hand movements: A window into haptic object recognition. Cognitive psychology 19, 3","author":"Lederman Susan J","year":"1987","unstructured":"Susan J Lederman and Roberta L Klatzky. 1987. Hand movements: A window into haptic object recognition. Cognitive psychology 19, 3 (1987), 342\u2013368."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445285"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3698061.3734410"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713441"},{"key":"e_1_2_2_36_1","volume-title":"Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1\u201313","author":"Cheng Ta Ying","year":"2020","unstructured":"Chuan-en Lin, Ta Ying Cheng, and Xiaojuan Ma. 2020. Architect: Building interactive virtual experiences from physical affordances by bringing human-in-the-loop. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1\u201313."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.1999.756952"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3721238.3730694"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445777"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/505008.505026"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3714006"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242587.3242648"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501953"},{"key":"e_1_2_2_44_1","volume-title":"Text-Driven Generative Framework for Multimodal Visual and Haptic Texture Synthesis. In 2025 IEEE World Haptics Conference (WHC). IEEE, 140\u2013146","author":"Naeem Myrah","year":"2025","unstructured":"Myrah Naeem, Mudassir Ibrahim Awan, and Seokhee Jeon. 2025. Text-Driven Generative Framework for Multimodal Visual and Haptic Texture Synthesis. In 2025 IEEE World Haptics Conference (WHC). IEEE, 140\u2013146."},{"key":"e_1_2_2_45_1","doi-asserted-by":"crossref","unstructured":"Momoka Nakayama Risako Kawashima Shintaro Murakami Yuta Takeuchi Tatsuya Mori and Dai Takanashi. 2024. A Method for Generating Tactile Sensations from Textual Descriptions Using Generative AI. In SIGGRAPH Asia 2024 Posters. 1\u20132.","DOI":"10.1145\/3681756.3697870"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3678717.3691226"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2003.1159607"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3654777.3676348"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2016.23"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713834"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702389"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3332165.3347891"},{"key":"e_1_2_2_53_1","volume-title":"Oasis: Procedurally generated social virtual spaces from 3d scanned real spaces","author":"Sra Misha","year":"2017","unstructured":"Misha Sra, Sergio Garrido-Jurado, and Pattie Maes. 2017. Oasis: Procedurally generated social virtual spaces from 3d scanned real spaces. IEEE transactions on visualization and computer graphics 24, 12 (2017), 3174\u20133187."},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376303"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3379337.3415828"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925883"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713609"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3472749.3474821"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3526113.3545682"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3472749.3474800"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3332165.3347958"},{"key":"e_1_2_2_62_1","volume-title":"Haptic repurposing with GenAI. arXiv preprint arXiv:2406.07228","author":"Wang Haoyu","year":"2024","unstructured":"Haoyu Wang. 2024. Haptic repurposing with GenAI. arXiv preprint arXiv:2406.07228 (2024)."},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173660"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1162\/105474698565686"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.01525"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580769"},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300923"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2019.07.003"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T07:09:26Z","timestamp":1783062566000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3811349"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,3]]},"references-count":68,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,7,3]]}},"alternative-id":["10.1145\/3811349"],"URL":"https:\/\/doi.org\/10.1145\/3811349","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,3]]},"assertion":[{"value":"2026-01-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-27","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-07-03","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}