{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T06:02:05Z","timestamp":1784268125033,"version":"3.55.0"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T00:00:00Z","timestamp":1573171200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2019,12,31]]},"abstract":"<jats:p>\n            We propose\n            <jats:italic>Neural State Machine<\/jats:italic>\n            , a novel data-driven framework to guide characters to achieve goal-driven actions with precise scene interactions. Even a seemingly simple task such as sitting on a chair is notoriously hard to model with supervised learning. This difficulty is because such a task involves complex planning with periodic and non-periodic motions reacting to the scene geometry to precisely position and orient the character. Our proposed deep auto-regressive framework enables modeling of multi-modal scene interaction behaviors purely from data. Given high-level instructions such as the goal location and the action to be launched there, our system computes a series of movements and transitions to reach the goal in the desired state. To allow characters to adapt to a wide range of geometry such as different shapes of furniture and obstacles, we incorporate an efficient data augmentation scheme to randomly switch the 3D geometry while maintaining the context of the original motion. To increase the precision to reach the goal during runtime, we introduce a control scheme that combines egocentric inference and goal-centric inference. We demonstrate the versatility of our model with various scene interaction tasks such as sitting on a chair, avoiding obstacles, opening and entering through a door, and picking and carrying objects generated in real-time just from a single model.\n          <\/jats:p>","DOI":"10.1145\/3355089.3356505","type":"journal-article","created":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T20:27:58Z","timestamp":1573244878000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":260,"title":["Neural state machine for character-scene interactions"],"prefix":"10.1145","volume":"38","author":[{"given":"Sebastian","family":"Starke","sequence":"first","affiliation":[{"name":"University of Edinburgh, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"He","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Edinburgh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taku","family":"Komura","sequence":"additional","affiliation":[{"name":"University of Edinburgh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jun","family":"Saito","sequence":"additional","affiliation":[{"name":"Adobe Research"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,11,8]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925893"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2485895.2485905"},{"key":"e_1_2_2_3_1","volume-title":"An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271","author":"Bai Shaojie","year":"2018","unstructured":"Shaojie Bai, J Zico Kolter, and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018). http:\/\/arxiv.org\/abs\/1803.01271"},{"key":"e_1_2_2_4_1","volume-title":"Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012","author":"Chang Angel X","year":"2015","unstructured":"Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. 2015. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012 (2015). http:\/\/arxiv.org\/abs\/1512.03012"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2011.01889.x"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/636886.636889"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275048"},{"key":"e_1_2_2_8_1","volume-title":"Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs). CoRR abs\/1511.07289","author":"Clevert Djork-Arn\u00e9","year":"2015","unstructured":"Djork-Arn\u00e9 Clevert, Thomas Unterthiner, and Sepp Hochreiter. 2015. Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs). CoRR abs\/1511.07289 (2015). arXiv:1511.07289 http:\/\/arxiv.org\/abs\/1511.07289"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/2919332.2919834"},{"key":"e_1_2_2_10_1","volume-title":"Pal","author":"Harvey F\u00e9lix G.","year":"2018","unstructured":"F\u00e9lix G. Harvey and Christopher J. Pal. 2018. Recurrent Transition Networks for Character Locomotion. CoRR abs\/1810.02363 (2018). arXiv:1810.02363 http:\/\/arxiv.org\/abs\/1810.02363"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","unstructured":"Nicolas Heess Srinivasan Sriram Jay Lemmon Josh Merel Greg Wayne Yuval Tassa Tom Erez Ziyu Wang Ali Eslami Martin Riedmiller et al. 2017. Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286 (2017). 10.1145\/1833349.1778865","DOI":"10.1145\/1833349.1778865"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073663"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925975"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1991.3.1.79"},{"key":"e_1_2_2_15_1","volume-title":"Proc. SCA. 117--126","author":"Kim Manmyung","year":"2012","unstructured":"Manmyung Kim, Youngseok Hwang, Kyunglyul Hyun, and Jehee Lee. 2012. Tiling motion patches. In Proc. SCA. 117--126. http:\/\/dl.acm.org\/citation.cfm?id=2422356.2422375"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566607"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275071"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141972"},{"key":"e_1_2_2_19_1","volume-title":"Auto-conditioned recurrent networks for extended complex human motion synthesis. arXiv preprint arXiv:1707.05363","author":"Li Zimo","year":"2017","unstructured":"Zimo Li, Yi Zhou, Shuangjiu Xiao, Chong He, Zeng Huang, and Hao Li. 2017. Auto-conditioned recurrent networks for extended complex human motion synthesis. arXiv preprint arXiv:1707.05363 (2017). http:\/\/arxiv.org\/abs\/1707.05363"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1833349.1778865"},{"key":"e_1_2_2_21_1","volume-title":"Proc. SCA. 145--154","author":"Lo Wan-Yen","year":"2012","unstructured":"Wan-Yen Lo, Claude Knaus, and Matthias Zwicker. 2012. Learning motion controllers with adaptive depth perception. In Proc. SCA. 145--154. http:\/\/dl.acm.org\/citation.cfm?id=2422356.2422378"},{"key":"e_1_2_2_22_1","volume-title":"Proc. SCA. 29--38","author":"Lo Wan-Yen","year":"2008","unstructured":"Wan-Yen Lo and Matthias Zwicker. 2008. Real-time planning for parameterized human motion. In Proc. SCA. 29--38. http:\/\/dl.acm.org\/citation.cfm?id=1632592.1632598"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366172"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185539"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073313"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201311"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925881"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073602"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275014"},{"key":"e_1_2_2_30_1","volume-title":"Proc. NIPS. 5099--5108","author":"Qi Charles Ruizhongtai","year":"2017","unstructured":"Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017. Point-Net++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. In Proc. NIPS. 5099--5108. http:\/\/papers.nips.cc\/paper\/7095-pointnet-deep-hierarchical-feature-learning-on-point-sets-in-a-metric-space.pdf"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/38.708559"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8659.00516"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276510"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1457515.1409067"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00901"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.1167"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661271"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130833"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185537"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201397"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201366"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508412"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3355089.3356505","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3355089.3356505","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:44:41Z","timestamp":1750203881000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3355089.3356505"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,8]]},"references-count":42,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2019,12,31]]}},"alternative-id":["10.1145\/3355089.3356505"],"URL":"https:\/\/doi.org\/10.1145\/3355089.3356505","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,8]]},"assertion":[{"value":"2019-11-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}