{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T02:44:21Z","timestamp":1777085061598,"version":"3.51.4"},"reference-count":67,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2021,3,14]],"date-time":"2021-03-14T00:00:00Z","timestamp":1615680000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,14]],"date-time":"2021-03-14T00:00:00Z","timestamp":1615680000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Alma Mater Studiorum - Universit\u00e0 di Bologna"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2021,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In real-world applications, inferring the intentions of expert agents (e.g., human operators) can be fundamental to understand how possibly conflicting objectives are managed, helping to interpret the demonstrated behavior. In this paper, we discuss how inverse reinforcement learning (IRL) can be employed to retrieve the reward function implicitly optimized by expert agents acting in real applications. Scaling IRL to real-world cases has proved challenging as typically only a fixed dataset of demonstrations is available and further interactions with the environment are not allowed. For this reason, we resort to a class of truly batch model-free IRL algorithms and we present three application scenarios: (1) the high-level decision-making problem in the highway driving scenario, and (2) inferring the user preferences in a social network (Twitter), and (3) the management of the water release in the Como Lake. For each of these scenarios, we provide formalization, experiments and a discussion to interpret the obtained results.<\/jats:p>","DOI":"10.1007\/s10994-020-05939-8","type":"journal-article","created":{"date-parts":[[2021,3,14]],"date-time":"2021-03-14T18:02:31Z","timestamp":1615744951000},"page":"2541-2576","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Dealing with multiple experts and non-stationarity in inverse reinforcement learning: an application to real-life problems"],"prefix":"10.1007","volume":"110","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4227-0741","authenticated-orcid":false,"given":"Amarildo","family":"Likmeta","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alberto Maria","family":"Metelli","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Giorgia","family":"Ramponi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrea","family":"Tirinzoni","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matteo","family":"Giuliani","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marcello","family":"Restelli","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,3,14]]},"reference":[{"issue":"13","key":"5939_CR1","doi-asserted-by":"publisher","first-page":"1608","DOI":"10.1177\/0278364910371999","volume":"29","author":"P Abbeel","year":"2010","unstructured":"Abbeel, P., Coates, A., & Ng, A. Y. (2010). Autonomous helicopter aerobatics through apprenticeship learning. The International Journal of Robotics Research, 29(13), 1608\u20131639.","journal-title":"The International Journal of Robotics Research"},{"key":"5939_CR2","doi-asserted-by":"crossref","unstructured":"Abbeel, P., & Ng, A.\u00a0Y. (2004). Apprenticeship learning via inverse reinforcement learning. In Proceedings of the Twenty-first International Conference on Machine Learning, ICML \u201904, p. 1, New York, NY, USA. ACM.","DOI":"10.1145\/1015330.1015430"},{"key":"5939_CR3","doi-asserted-by":"crossref","unstructured":"Almingol, J., & Montesano, L. (2015). Learning multiple behaviours using hierarchical clustering of rewards. In 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4608\u20134613.","DOI":"10.1109\/IROS.2015.7354033"},{"issue":"2","key":"5939_CR4","doi-asserted-by":"publisher","first-page":"339","DOI":"10.1007\/s10115-016-0987-z","volume":"51","author":"S Aminikhanghahi","year":"2017","unstructured":"Aminikhanghahi, S., & Cook, D. J. (2017). A survey of methods for time series change point detection. Knowledge and Information Systems, 51(2), 339\u2013367.","journal-title":"Knowledge and Information Systems"},{"issue":"5","key":"5939_CR5","doi-asserted-by":"publisher","first-page":"469","DOI":"10.1016\/j.robot.2008.10.024","volume":"57","author":"BD Argall","year":"2009","unstructured":"Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5), 469\u2013483.","journal-title":"Robotics and Autonomous Systems"},{"key":"5939_CR6","unstructured":"Babes, M., Marivate, V., Subramanian, K., & Littman, M.\u00a0L. (2011). Apprenticeship learning about multiple intentions. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 897\u2013904."},{"key":"5939_CR7","unstructured":"Barratt, S. (2018). A matrix gaussian distribution. arXiv preprint arXiv:1804.11010."},{"key":"5939_CR8","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1613\/jair.806","volume":"15","author":"J Baxter","year":"2001","unstructured":"Baxter, J., & Bartlett, P. L. (2001). Infinite-horizon policy-gradient estimation. Journal of Artificial Intelligence Research, 15, 319\u2013350.","journal-title":"Journal of Artificial Intelligence Research"},{"issue":"1","key":"5939_CR9","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1090\/qam\/102435","volume":"16","author":"R Bellman","year":"1958","unstructured":"Bellman, R. (1958). On a routing problem. Quarterly of applied mathematics, 16(1), 87\u201390.","journal-title":"Quarterly of applied mathematics"},{"key":"5939_CR10","unstructured":"Bogert, K., & Doshi, P. (2014). Multi-robot inverse reinforcement learning under occlusion with interactions. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems, pp. 173\u2013180. Citeseer."},{"key":"5939_CR11","unstructured":"Boularias, A., Kober, J., & Peters, J. (2011). Relative entropy inverse reinforcement learning. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 182\u2013189."},{"issue":"8","key":"5939_CR12","doi-asserted-by":"publisher","first-page":"1271","DOI":"10.1080\/14697688.2019.1571683","volume":"19","author":"H Buehler","year":"2019","unstructured":"Buehler, H., Gonon, L., Teichmann, J., & Wood, B. (2019). Deep hedging. Quantitative Finance, 19(8), 1271\u20131291.","journal-title":"Quantitative Finance"},{"key":"5939_CR13","volume-title":"Statistical inference","author":"G Casella","year":"2002","unstructured":"Casella, G., & Berger, R. L. (2002). Statistical inference (Vol. 2). CA: Duxbury Pacific Grove."},{"key":"5939_CR14","doi-asserted-by":"crossref","unstructured":"Codevilla, F., Miiller, M., L\u00f3pez, A., Koltun, V., & Dosovitskiy, A. (2018). End-to-end driving via conditional imitation learning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1\u20139. IEEE.","DOI":"10.1109\/ICRA.2018.8460487"},{"key":"5939_CR15","unstructured":"Das, S., & Lavoie, A. (2014). The effects of feedback on human behavior in social media: An inverse reinforcement learning model. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems, pp. 653\u2013660. International Foundation for Autonomous Agents and Multiagent Systems."},{"issue":"12","key":"5939_CR16","first-page":"1","volume":"2","author":"MP Deisenroth","year":"2013","unstructured":"Deisenroth, M. P., Neumann, G., Peters, J., et al. (2013). A survey on policy search for robotics. Foundations and Trends in Robotics, 2(12), 1\u2013142.","journal-title":"Foundations and Trends in Robotics"},{"key":"5939_CR17","doi-asserted-by":"crossref","unstructured":"Dempster, M. A.\u00a0H., & Romahi, Y.\u00a0S. (2002). Intraday fx trading: An evolutionary reinforcement learning approach. In International Conference on Intelligent Data Engineering and Automated Learning, pp. 347\u2013358. Springer.","DOI":"10.1007\/3-540-45675-9_52"},{"key":"5939_CR18","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017). Carla: An open urban driving simulator. arXiv preprint arXiv:1711.03938."},{"issue":"5","key":"5939_CR19","doi-asserted-by":"publisher","first-page":"388","DOI":"10.1177\/1059712313491614","volume":"21","author":"P Englert","year":"2013","unstructured":"Englert, P., Paraschos, A., Deisenroth, M. P., & Peters, J. (2013). Probabilistic model-based imitation learning. Adaptive Behavior, 21(5), 388\u2013403.","journal-title":"Adaptive Behavior"},{"key":"5939_CR20","unstructured":"Finn, C., Levine, S., & Abbeel, P. (2016). Guided cost learning: Deep inverse optimal control via policy optimization. In Proceedings of the 33rd International Conference on International Conference on Machine Learning\u2014Volume 48, ICML\u201916, pp. 49\u201358. JMLR.org."},{"key":"5939_CR21","unstructured":"Finn, C., Yu, T., Zhang, T., Abbeel, P., & Levine, S. (2017). One-shot visual imitation learning via meta-learning. arXiv preprint arXiv:1709.04905."},{"key":"5939_CR22","doi-asserted-by":"crossref","unstructured":"Giuliani, M., Li, Y., Castelletti, A., & Gandolfi, C. (2016). A coupled human-natural systems analysis of irrigated agriculture under changing climate. Water Resources Research.","DOI":"10.1002\/2016WR019363"},{"key":"5939_CR23","doi-asserted-by":"publisher","first-page":"9133","DOI":"10.1029\/2019WR025035","volume":"55","author":"M Giuliani","year":"2019","unstructured":"Giuliani, M., Zaniolo, M., Castelletti, A., Davoli, G., & Block, P. (2019). Detecting the state of the climate system via artificial intelligence to improve seasonal forecasts and inform reservoir operations. Water Resources Research, 55, 9133\u20139147.","journal-title":"Water Resources Research"},{"issue":"2","key":"5939_CR24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3054912","volume":"50","author":"A Hussein","year":"2017","unstructured":"Hussein, A., Gaber, M. M., Elyan, E., & Jayne, C. (2017). Imitation learning: A survey of learning methods. ACM Computing Surveys (CSUR), 50(2), 1\u201335.","journal-title":"ACM Computing Surveys (CSUR)"},{"key":"5939_CR25","doi-asserted-by":"crossref","unstructured":"Kiran, B.\u00a0R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A.\u00a0A., Yogamani, S., & P\u00e9rez, P. (2020). Deep reinforcement learning for autonomous driving: A survey. arXiv preprint arXiv:2002.00444.","DOI":"10.1109\/TITS.2021.3054625"},{"key":"5939_CR26","first-page":"1007","volume":"25","author":"E Klein","year":"2012","unstructured":"Klein, E., Geist, M., Piot, B., & Pietquin, O. (2012). Inverse reinforcement learning through structured classification. Advances in Neural Information Processing Systems, 25, 1007\u20131015.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"5939_CR27","doi-asserted-by":"crossref","unstructured":"Klein, E., Piot, B., Geist, M., & Pietquin, O. (2013). A cascaded supervised learning approach to inverse reinforcement learning. In Proceedings of the 2013th European Conference on Machine Learning and Knowledge Discovery in Databases\u2014Volume Part I, ECMLPKDD\u201913, pp. 1\u201316. Springer, Berlin, Heidelberg.","DOI":"10.1007\/978-3-642-40988-2_1"},{"issue":"11","key":"5939_CR28","doi-asserted-by":"publisher","first-page":"1238","DOI":"10.1177\/0278364913495721","volume":"32","author":"J Kober","year":"2013","unstructured":"Kober, J., Bagnell, J. A., & Peters, J. (2013). Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32(11), 1238\u20131274.","journal-title":"The International Journal of Robotics Research"},{"key":"5939_CR29","first-page":"849","volume":"21","author":"J Kober","year":"2009","unstructured":"Kober, J., & Peters, J. R. (2009). Policy search for motor primitives in robotics. Advances in Neural Information Processing Systems, 21, 849\u2013856.","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"3&4","key":"5939_CR30","first-page":"128","volume":"5","author":"D Krajzewicz","year":"2012","unstructured":"Krajzewicz, D., Erdmann, J., Behrisch, M., & Bieker, L. (2012). Recent development and applications of SUMO\u2014Simulation of Urban Mobility. International Journal on Advances in Systems and Measurements, 5(3&4), 128\u2013138.","journal-title":"International Journal on Advances in Systems and Measurements"},{"key":"5939_CR31","doi-asserted-by":"crossref","unstructured":"Kuderer, M., Gulati, S., & Burgard, W. (2015). Learning driving styles for autonomous vehicles from demonstration. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 2641\u20132646. IEEE.","DOI":"10.1109\/ICRA.2015.7139555"},{"issue":"2","key":"5939_CR32","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1016\/S0047-259X(03)00096-4","volume":"88","author":"O Ledoit","year":"2004","unstructured":"Ledoit, O., & Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. Journal of Multivariate Analysis, 88(2), 365\u2013411.","journal-title":"Journal of Multivariate Analysis"},{"issue":"1","key":"5939_CR33","first-page":"1334","volume":"17","author":"S Levine","year":"2016","unstructured":"Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2016). End-to-end training of deep visuomotor policies. The Journal of Machine Learning Research, 17(1), 1334\u20131373.","journal-title":"The Journal of Machine Learning Research"},{"key":"5939_CR34","doi-asserted-by":"publisher","first-page":"103568","DOI":"10.1016\/j.robot.2020.103568","volume":"131","author":"A Likmeta","year":"2020","unstructured":"Likmeta, A., Metelli, A. M., Tirinzoni, A., Giol, R., Restelli, M., & Romano, D. (2020). Combining reinforcement learning with rule-based controllers for transparent and general decision-making in autonomous driving. Robotics and Autonomous Systems, 131, 103568.","journal-title":"Robotics and Autonomous Systems"},{"key":"5939_CR35","unstructured":"Mason, E. (2018). Beyond full rationality: modeling tradeoff dynamics in multi-objective water management. PhD thesis, Politecnico di Milano, Italy."},{"issue":"4","key":"5939_CR36","doi-asserted-by":"publisher","first-page":"3162","DOI":"10.1002\/2017WR021431","volume":"54","author":"E Mason","year":"2018","unstructured":"Mason, E., Giuliani, M., Castelletti, A., & Amigoni, F. (2018). Identifying and modelling dynamic preference evolution in multipurpose water resources systems. Water Resources Research, 54(4), 3162\u20133175.","journal-title":"Water Resources Research"},{"key":"5939_CR37","unstructured":"Metelli, A.\u00a0M., Pirotta, M., & Restelli, M. (2017). Compatible reward inverse reinforcement learning. In Guyon, I., Luxburg, U.\u00a0V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems 30, pp. 2050\u20132059. Curran Associates, Inc."},{"key":"5939_CR38","doi-asserted-by":"crossref","unstructured":"Nevmyvaka, Y., Feng, Y., & Kearns, M. (2006). Reinforcement learning for optimized trade execution. In Proceedings of the 23rd International Conference on Machine Learning, ICML \u201906, pp. 673\u2013680, New York, NY, USA. Association for Computing Machinery.","DOI":"10.1145\/1143844.1143929"},{"key":"5939_CR39","unstructured":"Ng, A.\u00a0Y., & Russell, S.\u00a0J. (2000a). Algorithms for inverse reinforcement learning. In Proceedings of the Seventeenth International Conference on Machine Learning, ICML \u201900, pp. 663\u2013670, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc."},{"key":"5939_CR40","unstructured":"Ng, A.\u00a0Y., & Russell, S.\u00a0J. (2000b). Algorithms for inverse reinforcement learning. In ICML, pp. 663\u2013670. Morgan Kaufmann."},{"key":"5939_CR41","volume-title":"Numerical optimization","author":"J Nocedal","year":"2006","unstructured":"Nocedal, J., & Wright, S. (2006). Numerical optimization. Berlin: Springer."},{"issue":"1\u20132","key":"5939_CR42","first-page":"1","volume":"7","author":"T Osa","year":"2018","unstructured":"Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., Peters, J., et al. (2018). An algorithmic perspective on imitation learning. Foundations and Trends\u00ae in Robotics, 7(1\u20132), 1\u2013179.","journal-title":"Foundations and Trends\u00ae in Robotics"},{"key":"5939_CR43","doi-asserted-by":"crossref","unstructured":"Pan, Y., Cheng, C.-A., Saigol, K., Lee, K., Yan, X., Theodorou, E., & Boots, B. (2017). Agile autonomous driving using end-to-end deep imitation learning. arXiv preprint arXiv:1709.07174.","DOI":"10.15607\/RSS.2018.XIV.056"},{"key":"5939_CR44","unstructured":"Papini, M., Pirotta, M., & Restelli, M. (2019). Smoothing policies and safe policy gradients. arXiv preprint arXiv:1905.03231."},{"issue":"4","key":"5939_CR45","doi-asserted-by":"publisher","first-page":"682","DOI":"10.1016\/j.neunet.2008.02.003","volume":"21","author":"J Peters","year":"2008","unstructured":"Peters, J., & Schaal, S. (2008). Reinforcement learning of motor skills with policy gradients. Neural Networks, 21(4), 682\u2013697.","journal-title":"Neural Networks"},{"issue":"3","key":"5939_CR46","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1007\/s11257-018-9207-8","volume":"28","author":"G Piao","year":"2018","unstructured":"Piao, G., & Breslin, J. G. (2018). Inferring user interests in microblogging social networks: A survey. User Modeling and User-Adapted Interaction, 28(3), 277\u2013329.","journal-title":"User Modeling and User-Adapted Interaction"},{"key":"5939_CR47","unstructured":"Pirotta, M. (2016). Reinforcement learning: from theory to algorithms. PhD thesis, Politecnico di Milano, Italy."},{"key":"5939_CR48","doi-asserted-by":"crossref","unstructured":"Pirotta, M., & Restelli, M. (2016). Inverse reinforcement learning through policy gradient minimization. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI\u201916, pp. 1993\u20131999. AAAI Press.","DOI":"10.1609\/aaai.v30i1.10313"},{"key":"5939_CR49","doi-asserted-by":"publisher","DOI":"10.1002\/9780470316887","volume-title":"Markov Decision Processes: Discrete Stochastic Dynamic Programming","author":"ML Puterman","year":"1994","unstructured":"Puterman, M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York, NY, USA: Wiley."},{"key":"5939_CR50","unstructured":"Ramponi, G., Likmeta, A., Metelli, A.\u00a0M., Tirinzoni, A., & Restelli, M. (2020). Truly batch model-free inverse reinforcement learning about multiple intentions. In The 23nd International Conference on Artificial Intelligence and Statistics."},{"key":"5939_CR51","doi-asserted-by":"crossref","unstructured":"Ratliff, N.\u00a0D., Bagnell, J.\u00a0A., & Zinkevich, M.\u00a0A. (2006). Maximum margin planning. In Proceedings of the 23rd International Conference on Machine Learning, ICML \u201906, pp. 729\u2013736, New York, NY, USA. ACM.","DOI":"10.1145\/1143844.1143936"},{"key":"5939_CR52","doi-asserted-by":"crossref","unstructured":"Ross, S., Melik-Barkhudarov, N., Shankar, K.\u00a0S., Wendel, A., Dey, D., Bagnell, J.\u00a0A., & Hebert, M. (2013). Learning monocular reactive uav control in cluttered natural environments. In 2013 IEEE international conference on robotics and automation, pp. 1765\u20131772. IEEE.","DOI":"10.1109\/ICRA.2013.6630809"},{"issue":"1","key":"5939_CR53","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1007\/s13278-019-0551-4","volume":"9","author":"AM Sadri","year":"2019","unstructured":"Sadri, A. M., Hasan, S., & Ukkusuri, S. V. (2019). Joint inference of user community and interest patterns in social interaction networks. Social Network Analysis and Mining, 9(1), 11.","journal-title":"Social Network Analysis and Mining"},{"key":"5939_CR54","doi-asserted-by":"crossref","unstructured":"Saravia, E., Wu, S.-C., & Chen, Y.-S. (2017). A dynamic influence keyword model for identifying implicit user interests on social networks. In Proceedings of the 2017 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining 2017, pp. 1160\u20131163.","DOI":"10.1145\/3110025.3120987"},{"issue":"Sep","key":"5939_CR55","first-page":"1265","volume":"6","author":"G Shani","year":"2005","unstructured":"Shani, G., Heckerman, D., & Brafman, R. I. (2005). An mdp-based recommender system. Journal of Machine Learning Research, 6(Sep), 1265\u20131295.","journal-title":"Journal of Machine Learning Research"},{"key":"5939_CR56","doi-asserted-by":"crossref","unstructured":"Silver, D., Bagnell, J.\u00a0A., & Stentz, A. (2013). Learning autonomous driving styles and maneuvers from expert demonstration. In Experimental Robotics, pp. 371\u2013386. Springer.","DOI":"10.1007\/978-3-319-00065-7_26"},{"key":"5939_CR57","unstructured":"Song, X., Nie, L., Zhang, L., Liu, M., & Chua, T.-S. (2015). Interest inference via structure-constrained multi-source multi-task learning. Twenty-Fourth International Joint Conference on Artificial. (Intelligence)."},{"key":"5939_CR58","volume-title":"Reinforcement learning: An introduction","author":"RS Sutton","year":"2018","unstructured":"Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. Adaptive computation and machine learning: MIT Press, second edition."},{"key":"5939_CR59","unstructured":"Sutton, R.\u00a0S., McAllester, D.\u00a0A., Singh, S.\u00a0P., & Mansour, Y. (2000). Policy gradient methods for reinforcement learning with function approximation. In Solla, S., Leen, T., and M\u00fcller, K., editors, Advances in Neural Information Processing Systems 12, pp. 1057\u20131063. MIT Press."},{"key":"5939_CR60","doi-asserted-by":"crossref","unstructured":"Tateo, D., Pirotta, M., Restelli, M., & Bonarini, A. (2017). Gradient-based minimization for multi-expert inverse reinforcement learning. In 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1\u20138. IEEE.","DOI":"10.1109\/SSCI.2017.8280919"},{"key":"5939_CR61","doi-asserted-by":"publisher","first-page":"107299","DOI":"10.1016\/j.sigpro.2019.107299","volume":"167","author":"C Truong","year":"2020","unstructured":"Truong, C., Oudre, L., & Vayatis, N. (2020). Selective review of offline change point detection methods. Signal Processing, 167, 107299.","journal-title":"Signal Processing"},{"key":"5939_CR62","first-page":"1757","volume":"31","author":"R Warlop","year":"2018","unstructured":"Warlop, R., Lazaric, A., & Mary, J. (2018). Fighting boredom in recommender systems with linear reinforcement learning. Advances in Neural Information Processing Systems, 31, 1757\u20131768.","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"3\u20134","key":"5939_CR63","first-page":"229","volume":"8","author":"RJ Williams","year":"1992","unstructured":"Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3\u20134), 229\u2013256.","journal-title":"Machine Learning"},{"issue":"10","key":"5939_CR64","doi-asserted-by":"publisher","first-page":"1073","DOI":"10.1177\/0278364917722396","volume":"36","author":"M Wulfmeier","year":"2017","unstructured":"Wulfmeier, M., Rao, D., Wang, D. Z., Ondruska, P., & Posner, I. (2017). Large-scale cost function learning for path planning using deep inverse reinforcement learning. The International Journal of Robotics Research, 36(10), 1073\u20131087.","journal-title":"The International Journal of Robotics Research"},{"key":"5939_CR65","doi-asserted-by":"crossref","unstructured":"Xu, Z., Ru, L., Xiang, L., & Yang, Q. (2011). Discovering user interest on twitter with a modified author-topic model. In 2011 IEEE\/WIC\/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, vol.\u00a01, pp. 422\u2013429. IEEE.","DOI":"10.1109\/WI-IAT.2011.47"},{"key":"5939_CR66","doi-asserted-by":"crossref","unstructured":"Zhang, T., McCarthy, Z., Jow, O., Lee, D., Chen, X., Goldberg, K., & Abbeel, P. (2018). Deep imitation learning for complex manipulation tasks from virtual reality teleoperation. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1\u20138. IEEE.","DOI":"10.1109\/ICRA.2018.8461249"},{"key":"5939_CR67","unstructured":"Ziebart, B. D., Maas, A., Bagnell, J. A., & Dey, A. K. (2008). Maximum entropy inverse reinforcement learning. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (Vol. 3, pp. 1433\u20131438)."}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-020-05939-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-020-05939-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-020-05939-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T12:36:47Z","timestamp":1671626207000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-020-05939-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,14]]},"references-count":67,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2021,9]]}},"alternative-id":["5939"],"URL":"https:\/\/doi.org\/10.1007\/s10994-020-05939-8","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,14]]},"assertion":[{"value":"15 March 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 September 2020","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 December 2020","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 March 2021","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}