{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T20:04:34Z","timestamp":1779912274923,"version":"3.53.1"},"reference-count":59,"publisher":"SAGE Publications","issue":"10-11","license":[{"start":{"date-parts":[[2024,10,3]],"date-time":"2024-10-03T00:00:00Z","timestamp":1727913600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100003399","name":"Science and Technology Commission of Shanghai Municipality","doi-asserted-by":"publisher","award":["2021SHZDZX0102"],"award-info":[{"award-number":["2021SHZDZX0102"]}],"id":[{"id":"10.13039\/501100003399","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100018625","name":"Science and Technology Innovation Plan Of Shanghai Science and Technology Commission","doi-asserted-by":"publisher","award":["21511101200"],"award-info":[{"award-number":["21511101200"]}],"id":[{"id":"10.13039\/501100018625","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Shanghai Qi Zhi Institute"},{"name":"National University of Singapore startup grant"},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2021ZD0110700"],"award-info":[{"award-number":["2021ZD0110700"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2025,9]]},"abstract":"<jats:p>\n                    Model-based reinforcement learning is recognized with the potential to be significantly more sample efficient than model-free reinforcement learning. How an accurate\n                    <jats:italic toggle=\"yes\">model<\/jats:italic>\n                    can be developed automatically and efficiently from raw sensory inputs (such as images), especially for complex environments and tasks, is a challenging problem that hinders the broad application of model-based reinforcement learning in the real world. In this work, we propose a sensing-aware model-based reinforcement learning system called SAM-RL. Leveraging the differentiable physics-based simulation and rendering, SAM-RL automatically updates the model by comparing rendered images with real raw images and produces the policy efficiently. With the sensing-aware learning pipeline, SAM-RL allows a robot to select an informative viewpoint to monitor the task process. We apply our framework to real world experiments for accomplishing three manipulation tasks: robotic assembly, tool manipulation, and deformable object manipulation. We demonstrate the effectiveness of SAM-RL via extensive experiments. Videos are available on our project webpage.\n                  <\/jats:p>","DOI":"10.1177\/02783649241284653","type":"journal-article","created":{"date-parts":[[2024,10,4]],"date-time":"2024-10-04T12:43:11Z","timestamp":1728045791000},"page":"1767-1783","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":3,"title":["SAM-RL: Sensing-aware model-based reinforcement learning via differentiable physics-based simulation and rendering"],"prefix":"10.1177","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4328-0420","authenticated-orcid":false,"given":"Jun","family":"Lv","sequence":"first","affiliation":[{"name":"Shanghai Jiao Tong University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-3041-9816","authenticated-orcid":false,"given":"Yunhai","family":"Feng","sequence":"additional","affiliation":[{"name":"University of California San Diego"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cheng","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Irvine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuang","family":"Zhao","sequence":"additional","affiliation":[{"name":"University of Irvine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3412-5755","authenticated-orcid":false,"given":"Lin","family":"Shao","sequence":"additional","affiliation":[{"name":"National University of Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cewu","family":"Lu","sequence":"additional","affiliation":[{"name":"Shanghai Jiao Tong University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2024,10,3]]},"reference":[{"key":"e_1_3_3_2_1","unstructured":"Akkaya I Andrychowicz M Chociej M et al. (2019) Solving rubik\u2019s cube with a robot hand. arXiv preprint arXiv:1910.07113."},{"key":"e_1_3_3_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417833"},{"key":"e_1_3_3_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487281"},{"key":"e_1_3_3_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2014.2328779"},{"key":"e_1_3_3_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2014.6943031"},{"key":"e_1_3_3_7_1","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2022.XVIII.016"},{"key":"e_1_3_3_8_1","doi-asserted-by":"crossref","unstructured":"Collander C Beksi WJ Huber M (2021) Learning the next best view for 3d point clouds via topological features in 2021 IEEE international conference on robotics and automation (ICRA) Xi'an China 30 June 2021.","DOI":"10.1109\/ICRA48506.2021.9561389"},{"key":"e_1_3_3_9_1","volume-title":"\u2018Pybullet, a Python Module for Physics Simulation for Games, Robotics and Machine Learning\u2019","author":"Coumans E","year":"2016","unstructured":"Coumans E, Bai Y (2016\u20132021) \u2018Pybullet, a Python Module for Physics Simulation for Games, Robotics and Machine Learning\u2019, https:\/\/pybullet.org."},{"issue":"4","key":"e_1_3_3_10_1","article-title":"Bundlefusion: real-time globally consistent 3d reconstruction using on-the-fly surface re-integration","volume":"36","author":"Dai A","year":"2017","unstructured":"Dai A, Nie\u00dfner M, Zoll\u00f6fer M, et al. (2017) Bundlefusion: real-time globally consistent 3d reconstruction using on-the-fly surface re-integration. ACM Transactions on Graphics 2017 (TOG) 36(4).","journal-title":"ACM Transactions on Graphics 2017 (TOG)"},{"key":"e_1_3_3_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3490168"},{"key":"e_1_3_3_12_1","doi-asserted-by":"crossref","unstructured":"Fan H Su H Guibas LJ (2017) A point set generation network for 3d object reconstruction from a single image. Proceedings of the IEEE conference on computer vision and pattern recognition Honolulu HI USA 21 July 2017.","DOI":"10.1109\/CVPR.2017.264"},{"key":"e_1_3_3_13_1","doi-asserted-by":"crossref","unstructured":"Foissotte T Stasse O Escande A et al. (2009) A two-steps next-best-view algorithm for autonomous 3d object modeling by a humanoid robot. 2009 IEEE international conference on robotics and automation Kobe Japan 12 May 2009.","DOI":"10.1109\/ROBOT.2009.5152350"},{"key":"e_1_3_3_14_1","unstructured":"Fujimoto S Hoof H Meger D (2018) Addressing function approximation error in actor-critic methods. International conference on machine learning. PMLR Stockholm Sweden 10 July 2018."},{"key":"e_1_3_3_15_1","unstructured":"Haarnoja T Zhou A Abbeel P et al. (2018) Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. International conference on machine learning. PMLR Stockholm Sweden 10 July 2018."},{"key":"e_1_3_3_16_1","volume-title":"\u2018Dream to Control: Learning Behaviors by Latent Imagination\u2019","author":"Hafner D","year":"2019","unstructured":"Hafner D, Lillicrap T, Ba J, et al. (2019) \u2018Dream to Control: Learning Behaviors by Latent Imagination\u2019, arXiv preprint arXiv:1912.01603."},{"key":"e_1_3_3_17_1","unstructured":"Hafner D Lillicrap T Fischer I et al. (2019) Learning latent dynamics for planning from pixels. International conference on machine learning. PMLR Long Beach California 9 June 2019."},{"key":"e_1_3_3_18_1","volume-title":"Mastering Atari With Discrete World Models","author":"Hafner D","year":"2020","unstructured":"Hafner D, Lillicrap T, Norouzi M, et al. (2020) Mastering Atari With Discrete World Models. arXiv preprint arXiv:2010.02193."},{"key":"e_1_3_3_19_1","doi-asserted-by":"crossref","unstructured":"Han Y Zhan IH Zhao W et al. (2022) A double branch next-best-view network and novel robot system for active object reconstruction. 2022 international conference on robotics and automation (ICRA) Philadelphia Pennsylvania USA 23\u201327 May 2022.","DOI":"10.1109\/ICRA46639.2022.9811769"},{"key":"e_1_3_3_20_1","doi-asserted-by":"crossref","unstructured":"He K Gkioxari G Doll\u00e1r P et al. (2017) Mask r-cnn. 2017 IEEE international conference on computer vision. ICCV) Venice Italy 22 October 2017.","DOI":"10.1109\/ICCV.2017.322"},{"key":"e_1_3_3_21_1","article-title":"Dojo: a differentiable simulator for robotics","author":"Howell TA","year":"2022","unstructured":"Howell TA, Le Cleac\u2019h S, Kolter JZ, et al. (2022) Dojo: a differentiable simulator for robotics. Robotics: Science and Systems 2022. under review. arXiv:2203.00806.","journal-title":"Robotics: Science and Systems 2022. under review"},{"key":"e_1_3_3_22_1","article-title":"Difftaichi: differentiable programming for physical simulation","author":"Hu Y","year":"2020","unstructured":"Hu Y, Anderson L, Li T-M, et al. (2020) Difftaichi: differentiable programming for physical simulation. ICLR. arXiv:1910.00935.","journal-title":"ICLR"},{"key":"e_1_3_3_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2015.7353947"},{"key":"e_1_3_3_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047270"},{"key":"e_1_3_3_25_1","unstructured":"Jatavallabhula KM Macklin M Golemo F et al. (2021) gradsim: differentiable simulation for system identification and visuomotor control. In: International conference on learning representations (ICLR) Vienna Austria 4 May 2021."},{"key":"e_1_3_3_26_1","volume-title":"Differentiable Rendering: A Survey","author":"Kato H","year":"2020","unstructured":"Kato H, Beker D, Morariu M, et al. (2020) Differentiable Rendering: A Survey. arXiv preprint arXiv:2006.12057."},{"key":"e_1_3_3_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3059619"},{"key":"e_1_3_3_28_1","doi-asserted-by":"crossref","unstructured":"Krainin M Curless B Fox D (2011) Autonomous generation of complete 3d object models using next best view manipulation planning. 2011 IEEE international conference on robotics and automation Shanghai China 9 May 2011.","DOI":"10.1109\/ICRA.2011.5980429"},{"key":"e_1_3_3_29_1","doi-asserted-by":"crossref","unstructured":"Kriegel S Bodenm\u00fcller T Suppa M et al. (2011) A surface-based next-best-view approach for automated 3d model completion of unknown objects. 2011 IEEE international conference on robotics and automation Shanghai China 9 May 2011.","DOI":"10.1109\/ICRA.2011.5979947"},{"key":"e_1_3_3_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11554-013-0386-6"},{"key":"e_1_3_3_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2014.12.004"},{"key":"e_1_3_3_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793485"},{"key":"e_1_3_3_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275109"},{"key":"e_1_3_3_34_1","volume-title":"Diffcloth: differentiable cloth simulation with dry frictional contact","author":"Li Y","year":"2022","unstructured":"Li Y, Du T, Wu K, et al. (2022) Diffcloth: differentiable cloth simulation with dry frictional contact. New York: ACM Trans. Graph."},{"key":"e_1_3_3_35_1","article-title":"Differentiable cloth simulation for inverse problems","volume":"32","author":"Liang J","year":"2019","unstructured":"Liang J, Lin M, Koltun V (2019) Differentiable cloth simulation for inverse problems. Advances in Neural Information Processing Systems 32.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00780"},{"key":"e_1_3_3_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793506"},{"key":"e_1_3_3_38_1","volume-title":"A Survey on Model-Based Reinforcement Learning","author":"Luo F-M","year":"2022","unstructured":"Luo F-M, Xu T, Lai H, et al. (2022) A Survey on Model-Based Reinforcement Learning. arXiv preprint arXiv:2206.09328."},{"key":"e_1_3_3_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9811859"},{"key":"e_1_3_3_40_1","unstructured":"Ma P Du T Tenenbaum JB et al. (2021) Risp: rendering-invariant state predictor with differentiable simulation and rendering for cross-domain parameter estimation. International conference on learning representations Vienna Austria 4 May 2021."},{"key":"e_1_3_3_41_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_3_3_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-66823-5_33"},{"key":"e_1_3_3_43_1","volume-title":"\u2018Scalable Differentiable Physics for Learning and Control\u2019","author":"Qiao Y-L","year":"2020","unstructured":"Qiao Y-L, Liang J, Koltun V, et al. (2020) \u2018Scalable Differentiable Physics for Learning and Control\u2019, arXiv preprint arXiv:2007.02168."},{"key":"e_1_3_3_44_1","first-page":"17123","article-title":"Differentiable simulation of soft multi-body systems","volume":"34","author":"Qiao Y","year":"2021","unstructured":"Qiao Y, Liang J, Koltun V, et al. (2021) Differentiable simulation of soft multi-body systems. Advances in Neural Information Processing Systems 34: 17123\u201317135.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_45_1","first-page":"627","volume-title":"Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics\u2019, Vol. 15 of Proceedings of Machine Learning Research, PMLR, Fort Lauderdale","author":"Ross S","year":"2011","unstructured":"Ross S, Gordon G, Bagnell D (2011) A reduction of imitation learning and structured prediction to no-regret online learning. In: Gordon G, Dunson D, Dud\u00edk M (eds) Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics\u2019, Vol. 15 of Proceedings of Machine Learning Research, PMLR, Fort Lauderdale. Piscataway: IEEE, 627\u2013635."},{"key":"e_1_3_3_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9197134"},{"key":"e_1_3_3_47_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature16961"},{"key":"e_1_3_3_48_1","volume-title":"Residual Policy Learning","author":"Silver T","year":"2018","unstructured":"Silver T, Allen K, Tenenbaum J, et al. (2018) Residual Policy Learning. arXiv preprint arXiv:1812.06298."},{"key":"e_1_3_3_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9561173"},{"key":"e_1_3_3_50_1","volume-title":"Diffcloud: Real-to-Sim From Point Clouds With Differentiable Simulation and Rendering of Deformable Objects","author":"Sundaresan P","year":"2022","unstructured":"Sundaresan P, Antonova R, Bohg J (2022) Diffcloud: Real-to-Sim From Point Clouds With Differentiable Simulation and Rendering of Deformable Objects. arXiv preprint arXiv:2204.03139."},{"key":"e_1_3_3_51_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2003.09.004"},{"key":"e_1_3_3_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8460696"},{"key":"e_1_3_3_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIE.2015.2508929"},{"key":"e_1_3_3_54_1","volume-title":"\u2018Benchmarking Model-Based Reinforcement Learning\u2019","author":"Wang T","year":"2019","unstructured":"Wang T, Bao X, Clavera I, et al. (2019) \u2018Benchmarking Model-Based Reinforcement Learning\u2019, arXiv preprint arXiv:1907.02057."},{"key":"e_1_3_3_55_1","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2021.XVII.034"},{"issue":"4","key":"e_1_3_3_56_1","article-title":"3d shapenets for 2.5 d object recognition and next-best-view prediction","volume":"2","author":"Wu Z","year":"2014","unstructured":"Wu Z, Song S, Khosla A, et al. (2014) 3d shapenets for 2.5 d object recognition and next-best-view prediction. arXiv preprint arXiv:1406.5670 2(4).","journal-title":"arXiv preprint arXiv:1406.5670"},{"key":"e_1_3_3_57_1","volume-title":"Compare Contact Model-Based Control and Contact Model-Free Learning: A Survey of Robotic Peg-in-Hole Assembly Strategies","author":"Xu J","year":"2019","unstructured":"Xu J, Hou Z, Liu Z, et al. (2019) Compare Contact Model-Based Control and Contact Model-Free Learning: A Survey of Robotic Peg-in-Hole Assembly Strategies. arXiv preprint arXiv:1904.05240."},{"key":"e_1_3_3_58_1","first-page":"11784","volume-title":"International Conference on Machine Learning\u2019. PMLR","author":"Yang M","year":"2021","unstructured":"Yang M, Nachum O (2021) Representation matters: offline pretraining for sequential decision making. International Conference on Machine Learning\u2019. PMLR. Piscataway: IEEE, 11784\u201311794."},{"key":"e_1_3_3_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392383"},{"key":"e_1_3_3_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3388769.3407454"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649241284653","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/02783649241284653","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649241284653","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:17:23Z","timestamp":1777457843000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/02783649241284653"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,3]]},"references-count":59,"journal-issue":{"issue":"10-11","published-print":{"date-parts":[[2025,9]]}},"alternative-id":["10.1177\/02783649241284653"],"URL":"https:\/\/doi.org\/10.1177\/02783649241284653","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,3]]}}}