{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T23:04:04Z","timestamp":1767999844759,"version":"3.49.0"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,7,30]],"date-time":"2018-07-30T00:00:00Z","timestamp":1532908800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,8,31]]},"abstract":"<jats:p>We present a learning-based method to control a coupled 2D system involving both fluid and rigid bodies. Our approach is used to modify the fluid\/rigid simulator's behavior by applying control forces only at the simulation domain boundaries. The rest of the domain, corresponding to the interior, is governed by the Navier-Stokes equation for fluids and Newton-Euler's equation for the rigid bodies. We represent our controller using a general neural-net, which is trained using deep reinforcement learning. Our formulation decomposes a control task into two stages: a precomputation training stage and an online generation stage. We utilize various fluid properties, e.g., the liquid's velocity field or the smoke's density field, to enhance the controller's performance. We set up our evaluation benchmark by letting controller drive fluid jets move on the domain boundary and allowing them to shoot fluids towards a rigid body to accomplish a set of challenging 2D tasks such as keeping a rigid body balanced, playing a two-player ping-pong game, and driving a rigid body to sequentially hit specified points on the wall. In practice, our approach can generate physically plausible animations.<\/jats:p>","DOI":"10.1145\/3197517.3201334","type":"journal-article","created":{"date-parts":[[2018,7,31]],"date-time":"2018-07-31T15:56:23Z","timestamp":1533052583000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":44,"title":["Fluid directed rigid body control using deep reinforcement learning"],"prefix":"10.1145","volume":"37","author":[{"given":"Pingchuan","family":"Ma","sequence":"first","affiliation":[{"name":"Nankai University, China"}]},{"given":"Yunsheng","family":"Tian","sequence":"additional","affiliation":[{"name":"Nankai University, China"}]},{"given":"Zherong","family":"Pan","sequence":"additional","affiliation":[{"name":"University of North Carolina at Chapel Hill"}]},{"given":"Bo","family":"Ren","sequence":"additional","affiliation":[{"name":"Nankai University, China"}]},{"given":"Dinesh","family":"Manocha","sequence":"additional","affiliation":[{"name":"University of Maryland at College Park"}]}],"member":"320","published-online":{"date-parts":[[2018,7,30]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185558"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276502"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.5555\/1272690.1272719"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2008.107"},{"key":"e_1_2_2_5_1","volume-title":"Pre-computed Liquid Spaces with Generative Neural Networks. arXiv to appear (Apr","author":"Bonev Boris","year":"2017","unstructured":"Boris Bonev , Lukas Prantl , and Nils Thuerey . 2017. Pre-computed Liquid Spaces with Generative Neural Networks. arXiv to appear (Apr 2017 ), 14. Boris Bonev, Lukas Prantl, and Nils Thuerey. 2017. Pre-computed Liquid Spaces with Generative Neural Networks. arXiv to appear (Apr 2017), 14."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1964921.1964977"},{"key":"e_1_2_2_7_1","volume-title":"Hyperparameter Search in Machine Learning. arXiv preprint arXiv:1502.02127","author":"Claesen Marc","year":"2015","unstructured":"Marc Claesen and Bart De Moor . 2015. Hyperparameter Search in Machine Learning. arXiv preprint arXiv:1502.02127 ( 2015 ). Marc Claesen and Bart De Moor. 2015. Hyperparameter Search in Machine Learning. arXiv preprint arXiv:1502.02127 (2015)."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566645"},{"key":"e_1_2_2_9_1","volume-title":"Non-Linear Control for Underactuated Mechanical Systems","author":"Fantoni Isabelle","unstructured":"Isabelle Fantoni and Rogelio Lozano . 2001. Non-Linear Control for Underactuated Mechanical Systems . Springer-Verlag New York, Inc. , Secaucus, NJ, USA . Isabelle Fantoni and Rogelio Lozano. 2001. Non-Linear Control for Underactuated Mechanical Systems. Springer-Verlag New York, Inc., Secaucus, NJ, USA."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015743"},{"key":"e_1_2_2_11_1","first-page":"6","article-title":"Some Guidelines and Guarantees for Common Random Numbers. Manage","volume":"38","author":"Glasserman Paul","year":"1992","unstructured":"Paul Glasserman and David D. Yao . 1992 . Some Guidelines and Guarantees for Common Random Numbers. Manage . Sci. 38 , 6 (June 1992), 884--908. Paul Glasserman and David D. Yao. 1992. Some Guidelines and Guarantees for Common Random Numbers. Manage. Sci. 38, 6 (June 1992), 884--908.","journal-title":"Sci."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1162\/106365601750190398"},{"key":"e_1_2_2_13_1","volume-title":"Advances in Neural Information Processing Systems 14 (NIPS","author":"Kakade Sham","year":"2001","unstructured":"Sham Kakade . 2001. A Natural Policy Gradient . In Advances in Neural Information Processing Systems 14 (NIPS 2001 ), Thomas G. Dietterich, Suzanna Becker , and Zoubin Ghahramani (Eds.). MIT Press , 1531--1538. Sham Kakade. 2001. A Natural Policy Gradient. In Advances in Neural Information Processing Systems 14 (NIPS 2001), Thomas G. Dietterich, Suzanna Becker, and Zoubin Ghahramani (Eds.). MIT Press, 1531--1538."},{"key":"e_1_2_2_14_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba . 2014 . Adam : A Method for Stochastic Optimization. CoRR abs\/1412.6980 (2014). Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs\/1412.6980 (2014)."},{"key":"e_1_2_2_15_1","volume-title":"Proceedings of the 25th International Conference on Neural Information Processing Systems -","volume":"1","author":"Krizhevsky Alex","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E. Hinton . 2012. ImageNet Classification with Deep Convolutional Neural Networks . In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 (NIPS'12). Curran Associates Inc., USA, 1097--1105. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 (NIPS'12). Curran Associates Inc., USA, 1097--1105."},{"key":"e_1_2_2_16_1","volume-title":"A Second-Order Well-Balanced Positivity Preserving Central-Upwind Scheme for the Saint-Venant System. Commun. Math. Sci. 5, 1 (03","author":"Kurganov Alexander","year":"2007","unstructured":"Alexander Kurganov and Guergana Petrova . 2007. A Second-Order Well-Balanced Positivity Preserving Central-Upwind Scheme for the Saint-Venant System. Commun. Math. Sci. 5, 1 (03 2007 ), 133--160. https:\/\/projecteuclid.org:443\/euclid.cms\/1175797625 Alexander Kurganov and Guergana Petrova. 2007. A Second-Order Well-Balanced Positivity Preserving Central-Upwind Scheme for the Saint-Venant System. Commun. Math. Sci. 5, 1 (03 2007), 133--160. https:\/\/projecteuclid.org:443\/euclid.cms\/1175797625"},{"key":"e_1_2_2_17_1","volume-title":"Layton and Michiel van de Panne","author":"Anita","year":"2002","unstructured":"Anita T. Layton and Michiel van de Panne . 2002 . A numerically efficient and stable algorithm for animating water waves. The Visual Computer 18, 1 (01 Feb 2002), 41--53. Anita T. Layton and Michiel van de Panne. 2002. A numerically efficient and stable algorithm for animating water waves. The Visual Computer 18, 1 (01 Feb 2002), 41--53."},{"key":"e_1_2_2_18_1","first-page":"1","article-title":"End-to-end Training of Deep Visuomotor Policies","volume":"17","author":"Levine Sergey","year":"2016","unstructured":"Sergey Levine , Chelsea Finn , Trevor Darrell , and Pieter Abbeel . 2016 . End-to-end Training of Deep Visuomotor Policies . J. Mach. Learn. Res. 17 , 1 (Jan. 2016), 1334--1373. Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. 2016. End-to-end Training of Deep Visuomotor Policies. J. Mach. Learn. Res. 17, 1 (Jan. 2016), 1334--1373.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3083723"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2008.37"},{"key":"e_1_2_2_21_1","volume-title":"Advances in Neural Information Processing Systems 30 (NIPS","author":"Lowe Ryan","year":"2017","unstructured":"Ryan Lowe , Yi Wu , Aviv Tamar , Jean Harb , Pieter Abbeel , and Igor Mordatch . 2017. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments . In Advances in Neural Information Processing Systems 30 (NIPS 2017 ). 6382--6393. Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. 2017. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. In Advances in Neural Information Processing Systems 30 (NIPS 2017). 6382--6393."},{"key":"e_1_2_2_22_1","volume-title":"Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction","author":"Masci Jonathan","unstructured":"Jonathan Masci , Ueli Meier , Dan Cire\u015fan , and J\u00fcrgen Schmidhuber . 2011. Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction . Springer Berlin Heidelberg , Berlin, Heidelberg , 52--59. Jonathan Masci, Ueli Meier, Dan Cire\u015fan, and J\u00fcrgen Schmidhuber. 2011. Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction. Springer Berlin Heidelberg, Berlin, Heidelberg, 52--59."},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015744"},{"key":"e_1_2_2_24_1","volume-title":"Proceedings of The 33rd International Conference on Machine Learning (Proceedings of Machine Learning Research), Maria Fiorina Balcan and Kilian Q. Weinberger (Eds.)","volume":"48","author":"Mnih Volodymyr","year":"2016","unstructured":"Volodymyr Mnih , Adria Puigdomenech Badia , Mehdi Mirza , Alex Graves , Timothy Lillicrap , Tim Harley , David Silver , and Koray Kavukcuoglu . 2016 . Asynchronous Methods for Deep Reinforcement Learning . In Proceedings of The 33rd International Conference on Machine Learning (Proceedings of Machine Learning Research), Maria Fiorina Balcan and Kilian Q. Weinberger (Eds.) , Vol. 48 . PMLR, New York, New York, USA , 1928--1937. Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous Methods for Deep Reinforcement Learning. In Proceedings of The 33rd International Conference on Machine Learning (Proceedings of Machine Learning Research), Maria Fiorina Balcan and Kilian Q. Weinberger (Eds.), Vol. 48. PMLR, New York, New York, USA, 1928--1937."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531344"},{"key":"e_1_2_2_26_1","unstructured":"J. Nocedal and S. J. Wright. 2006. Numerical Optimization (2nd ed.). Springer New York.  J. Nocedal and S. J. Wright. 2006. Numerical Optimization (2nd ed.). Springer New York."},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508429"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3016963"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073602"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/944020.944025"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344880"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601126"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1399504.1360645"},{"key":"e_1_2_2_34_1","volume-title":"Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research), Francis Bach and David Blei (Eds.)","volume":"37","author":"Schulman John","year":"2015","unstructured":"John Schulman , Sergey Levine , Pieter Abbeel , Michael Jordan , and Philipp Moritz . 2015 a. Trust Region Policy Optimization . In Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research), Francis Bach and David Blei (Eds.) , Vol. 37 . PMLR, Lille, France , 1889--1897. John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. 2015a. Trust Region Policy Optimization. In Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research), Francis Bach and David Blei (Eds.), Vol. 37. PMLR, Lille, France, 1889--1897."},{"key":"e_1_2_2_35_1","unstructured":"John Schulman Philipp Moritz Sergey Levine Michael Jordan and Pieter Abbeel. 2015b. High-Dimensional Continuous Control Using Generalized Advantage Estimation. (2015). arXiv:arXiv:1506.02438  John Schulman Philipp Moritz Sergey Levine Michael Jordan and Pieter Abbeel. 2015b. High-Dimensional Continuous Control Using Generalized Advantage Estimation. (2015). arXiv:arXiv:1506.02438"},{"key":"e_1_2_2_36_1","unstructured":"John Schulman Filip Wolski Prafulla Dhariwal Alec Radford and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. (2017). arXiv:arXiv:1707.06347  John Schulman Filip Wolski Prafulla Dhariwal Alec Radford and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. (2017). arXiv:arXiv:1707.06347"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073368.1073401"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2956233"},{"key":"e_1_2_2_39_1","unstructured":"J. Tompson K. Schlachter P. Sprechmann and K. Perlin. 2016. Accelerating Eulerian Fluid Simulation With Convolutional Networks. ArXiv e-prints (July 2016). arXiv:cs.CV\/1607.03597  J. Tompson K. Schlachter P. Sprechmann and K. Perlin. 2016. Accelerating Eulerian Fluid Simulation With Convolutional Networks. ArXiv e-prints (July 2016). arXiv:cs.CV\/1607.03597"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882337"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276395"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360624"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185521"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130833"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1275808.1276509"},{"key":"e_1_2_2_46_1","doi-asserted-by":"crossref","unstructured":"Matthew D. Zeiler Dilip Krishnan Graham W. Taylor and Rob Fergus. 2010. Deconvolutional networks. 2528--2535.  Matthew D. Zeiler Dilip Krishnan Graham W. Taylor and Rob Fergus. 2010. Deconvolutional networks. 2528--2535.","DOI":"10.1109\/CVPR.2010.5539957"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461999"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073298"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201334","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3197517.3201334","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:39:44Z","timestamp":1750210784000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201334"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,30]]},"references-count":48,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,8,31]]}},"alternative-id":["10.1145\/3197517.3201334"],"URL":"https:\/\/doi.org\/10.1145\/3197517.3201334","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,30]]},"assertion":[{"value":"2018-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}