{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T02:42:51Z","timestamp":1772851371229,"version":"3.50.1"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2021,5,1]],"date-time":"2021-05-01T00:00:00Z","timestamp":1619827200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,9]],"date-time":"2021-06-09T00:00:00Z","timestamp":1623196800000},"content-version":"vor","delay-in-days":39,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Max Planck Institute for Intelligent Systems"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Auton Robot"],"published-print":{"date-parts":[[2021,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In many robotic applications, it is crucial to maintain a belief about the state of a system, which serves as input for planning and decision making and provides feedback during task execution. Bayesian Filtering algorithms address this state estimation problem, but they require models of process dynamics and sensory observations and the respective noise characteristics of these models. Recently, multiple works have demonstrated that these models can be learned by end-to-end training through differentiable versions of recursive filtering algorithms. In this work, we investigate the advantages of <jats:italic>differentiable filters<\/jats:italic> (DFs) over both unstructured learning approaches and manually-tuned filtering algorithms, and provide practical guidance to researchers interested in applying such differentiable filters. For this, we implement DFs with four different underlying filtering algorithms and compare them in extensive experiments. Specifically, we (i) evaluate different implementation choices and training approaches, (ii) investigate how well complex models of uncertainty can be learned in DFs, (iii) evaluate the effect of end-to-end training through DFs and (iv) compare the DFs among each other and to unstructured LSTM models.<\/jats:p>","DOI":"10.1007\/s10514-021-09990-9","type":"journal-article","created":{"date-parts":[[2021,6,9]],"date-time":"2021-06-09T15:06:30Z","timestamp":1623251190000},"page":"561-578","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":36,"title":["How to train your differentiable filter"],"prefix":"10.1007","volume":"45","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5793-1875","authenticated-orcid":false,"given":"Alina","family":"Kloss","sequence":"first","affiliation":[]},{"given":"Georg","family":"Martius","sequence":"additional","affiliation":[]},{"given":"Jeannette","family":"Bohg","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,6,9]]},"reference":[{"key":"9990_CR1","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Man\u00e9, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Vi\u00e9gas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., & Zheng, X. (2015) TensorFlow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org"},{"key":"9990_CR2","unstructured":"Amos, B., Jimenez, I., Sacks, J., Boots, B., & Kolter, J.Z. (2018) Differentiable mpc for end-to-end planning and control. In Advances in neural information processing systems, Curran Associates, Inc., pp. 8289\u20138300"},{"key":"9990_CR3","unstructured":"Archer, E., Park, I.M., Buesing, L., Cunningham, J., & Paninski, L. (2015) Black box variational inference for state space models. arXiv preprint arXiv:1511.07367."},{"key":"9990_CR4","doi-asserted-by":"publisher","unstructured":"Bauza, M., & Rodriguez, A. (2017) A probabilistic data-driven model for planar pushing. In IEEE international conference on robotics and automation (pp. 3008\u20133015). https:\/\/doi.org\/10.1109\/ICRA.2017.7989345.","DOI":"10.1109\/ICRA.2017.7989345"},{"issue":"4","key":"9990_CR5","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1016\/j.jprocont.2011.01.001","volume":"21","author":"VA Bavdekar","year":"2011","unstructured":"Bavdekar, V. A., Deshpande, A. P., & Patwardhan, S. C. (2011). Identification of process and measurement noise covariance for state and parameter estimation using extended kalman filter. Journal of Process Control, 21(4), 585\u2013601. https:\/\/doi.org\/10.1016\/j.jprocont.2011.01.001.","journal-title":"Journal of Process Control"},{"key":"9990_CR6","unstructured":"Donti, P., Amos, B., & Kolter, J.Z. (2017). Task-based end-to-end model learning in stochastic optimization. In Advances in neural information processing systems, Curran Associates, Inc., pp. 5484\u20135494."},{"key":"9990_CR7","unstructured":"Farquhar, G., Rocktaeschel, T., Igl, M., & Whiteson, S. (2018). TreeQN and ATreec: Differentiable tree planning for deep reinforcement learning. In International conference on learning representations."},{"key":"9990_CR8","unstructured":"Fraccaro, M., Kamronn, S., Paquet, U., & Winther, O. (2017) A disentangled recognition and nonlinear dynamics model for unsupervised learning. In Advances in neural information processing systems (pp. 3601\u20133610)."},{"key":"9990_CR9","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., & Urtasun, R. (2012) Are we ready for autonomous driving? The kitti vision benchmark suite. In Conference on computer vision and pattern recognition.","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"9990_CR10","unstructured":"Girin, L., Leglaive, S., Bie, X., Diard, J., Hueber, T., & Alameda-Pineda, X. (2020) Dynamical variational autoencoders: A comprehensive review. arXiv preprint arXiv:2008.12595."},{"key":"9990_CR11","unstructured":"Guez, A., Weber, T., Antonoglou, I., Simonyan, K., Vinyals, O., Wierstra, D., Munos, R., & Silver, D. (2018) Learning to search with mctsnets. In International conference on machine learning, PMLR (Vol. 80, pp. 1817\u20131826)."},{"key":"9990_CR12","unstructured":"Haarnoja, T., Ajay, A., Levine, S., Abbeel, P. (2016) Backprop KF: Learning discriminative deterministic state estimators. In Advances in neural information processing systems (pp. 4376\u20134384)."},{"issue":"8","key":"9990_CR13","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735\u20131780.","journal-title":"Neural Computation"},{"key":"9990_CR14","unstructured":"Holl, P., Thuerey, N., & Koltun, V. (2020). Learning to control pdes with differentiable physics. In International conference on learning representations."},{"key":"9990_CR15","unstructured":"Jonschkowski, R., & Brock, O. (2016). End-to-end learnable histogram filters. In Workshop on deep learning for action and interaction at NIPS."},{"key":"9990_CR16","doi-asserted-by":"crossref","unstructured":"Jonschkowski, R., Rastogi, D., & Brock, O. (2018). Differentiable particle filters: End-to-end learning with algorithmic priors. In Robotics: science and systems, Pittsburgh, USA.","DOI":"10.15607\/RSS.2018.XIV.001"},{"key":"9990_CR17","unstructured":"Karkus, P., Hsu, D., & Lee, W. S. (2017). QMDP-Net: Deep learning for planning under partial observability. In Advances in neural information processing systems (pp. 4694\u20134704)."},{"key":"9990_CR18","unstructured":"Karkus, P., Hsu, D., & Lee, W.S. (2018) Particle filter networks with application to visual localization. In Conference on robot learning (pp 169\u2013178)."},{"key":"9990_CR19","doi-asserted-by":"crossref","unstructured":"Karkus, P., Ma, X., Hsu, D., Kaelbling, L.P., Lee, W.S., & Lozano-P\u00e9rez, T. (2019). Differentiable algorithm networks for composable robot learning. In Robotics: Science and systems.","DOI":"10.15607\/RSS.2019.XV.039"},{"key":"9990_CR20","unstructured":"Karl, M., Soelch, M., Bayer, J., & van\u00a0der Smagt, P. (2017) Deep variational bayes filters: Unsupervised learning of state space models from raw data. In International conference on learning representations."},{"key":"9990_CR21","doi-asserted-by":"crossref","unstructured":"Kersting, K., Plagemann, C., Pfaff, P., & Burgard, W. (2007) Most likely heteroscedastic gaussian process regression. In International conference on machine learning, ACM (pp. 393\u2013400).","DOI":"10.1145\/1273496.1273546"},{"key":"9990_CR22","unstructured":"Kingma, D.P., & Ba, J. (2015) Adam: A method for stochastic optimization. In Bengio, Y., & LeCun, Y. (eds.) International conference on learning representations."},{"key":"9990_CR23","doi-asserted-by":"publisher","unstructured":"Kloss, A., Schaal, S., & Bohg, J. (2020). Combining learned and analytical models for predicting action effects from sensory data. The International Journal of Robotics Research. https:\/\/doi.org\/10.1177\/0278364920954896.","DOI":"10.1177\/0278364920954896"},{"key":"9990_CR24","unstructured":"Krishnan, R.G., Shalit, U., & Sontag, D. (2016) Structured inference networks for nonlinear state space models. arXiv preprint arXiv:1609.09869."},{"key":"9990_CR25","unstructured":"Le, T.A., Igl, M., Rainforth, T., Jin, T., & Wood, F. (2018) Auto-encoding sequential monte carlo. In International conference on learning representations, https:\/\/openreview.net\/forum?id=BJ8c3f-0b."},{"key":"9990_CR26","doi-asserted-by":"publisher","unstructured":"Lynch, K.M., Maekawa, H., & Tanie, K. (1992). Manipulation and active sensing by pushing using tactile feedback. In IEEE international conference intelligent robots and systems (Vol. 1, pp. 416\u2013421). https:\/\/doi.org\/10.1109\/IROS.1992.587370.","DOI":"10.1109\/IROS.1992.587370"},{"key":"9990_CR27","doi-asserted-by":"publisher","first-page":"5101","DOI":"10.1609\/aaai.v34i04.5952","volume":"34","author":"X Ma","year":"2020","unstructured":"Ma, X., Karkus, P., Hsu, D., & Lee, W. S. (2020). Particle filter recurrent neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 5101\u20135108.","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"9990_CR28","unstructured":"Maddison, C.J., Lawson, D., Tucker, G., Heess, N., Norouzi, M., Mnih, A., Doucet, A., & Teh, Y.W. (2017) Filtering variational objectives. In Proceedings of the 31st international conference on neural information processing systems (pp. 6576\u20136586)."},{"key":"9990_CR29","unstructured":"Murphy, K.P. (1998) Switching kalman filters."},{"key":"9990_CR30","unstructured":"Naesseth, C., Linderman, S., Ranganath, R., & Blei, D. (2018) Variational sequential monte carlo. In International conference on artificial intelligence and statistics, PMLR (pp. 968\u2013977)."},{"key":"9990_CR31","unstructured":"Oh, J., Singh, S., & Lee, H. (2017) Value prediction network. In Advances in neural information processing systems, Curran Associates, Inc. (pp. 6118\u20136128)."},{"key":"9990_CR32","unstructured":"Okada, M., Rigazio, L., & Aoshima, T. (2017) Path integral networks: End-to-end differentiable optimal control. arXiv preprint arXiv:1706.09597."},{"key":"9990_CR33","unstructured":"Pereira, M., Fan, D. D., An, G. N., & Theodorou, E. (2018) Mpc-inspired neural network policies for sequential decision making. arXiv preprint arXiv:1802.05803."},{"key":"9990_CR34","doi-asserted-by":"crossref","unstructured":"Pont\u00f3n, B., Schaal, S., & Righetti, L. (2020) On the effects of measurement uncertainty in optimal control of contact interactions. In Algorithmic foundations of robotics XII, Springer (pp. 784\u2013799).","DOI":"10.1007\/978-3-030-43089-4_50"},{"key":"9990_CR35","doi-asserted-by":"crossref","unstructured":"Tamar, A., Wu, Y., Thomas, G., Levine, S., & Abbeel, P. (2016) Value iteration networks. In Advances in neural information processing systems (pp. 2154\u20132162).","DOI":"10.24963\/ijcai.2017\/700"},{"issue":"5","key":"9990_CR36","doi-asserted-by":"publisher","first-page":"1084","DOI":"10.1162\/0899766053491887","volume":"17","author":"E Todorov","year":"2005","unstructured":"Todorov, E. (2005). Stochastic optimal control and estimation methods adapted to the noise characteristics of the sensorimotor system. Neural Computation, 17(5), 1084\u20131108.","journal-title":"Neural Computation"},{"issue":"2","key":"9990_CR37","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1002\/aic.690460209","volume":"46","author":"J Valappil","year":"2000","unstructured":"Valappil, J., & Georgakis, C. (2000). Systematic estimation of state noise statistics for extended kalman filters. AIChE Journal, 46(2), 292\u2013308.","journal-title":"AIChE Journal"},{"key":"9990_CR38","unstructured":"Watter, M., Springenberg, J., Boedecker, J., & Riedmiller, M. (2015) Embed to control: A locally linear latent dynamics model for control from raw images. In Advances in neural information processing systems (pp. 2746\u20132754)."},{"key":"9990_CR39","doi-asserted-by":"publisher","unstructured":"Yu, K.T., Bauza, M., Fazeli, N., & Rodriguez, A. (2016) More than a million ways to be pushed. A high-fidelity experimental dataset of planar pushing. In IEEE international conference on intelligent robots and systems (pp. 30\u201337). https:\/\/doi.org\/10.1109\/IROS.2016.7758091, data available from http:\/\/web.mit.edu\/mcube\/\/push-dataset.","DOI":"10.1109\/IROS.2016.7758091"},{"key":"9990_CR40","unstructured":"Zhu, M., Murphy, K., & Jonschkowski, R. (2020) Towards differentiable resampling. arXiv preprint arXiv:2004.11938."}],"container-title":["Autonomous Robots"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10514-021-09990-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10514-021-09990-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10514-021-09990-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,2]],"date-time":"2021-07-02T10:53:17Z","timestamp":1625223197000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10514-021-09990-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5]]},"references-count":40,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,5]]}},"alternative-id":["9990"],"URL":"https:\/\/doi.org\/10.1007\/s10514-021-09990-9","relation":{},"ISSN":["0929-5593","1573-7527"],"issn-type":[{"value":"0929-5593","type":"print"},{"value":"1573-7527","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5]]},"assertion":[{"value":"15 December 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 May 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 June 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}