{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T15:25:46Z","timestamp":1775661946081,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":46,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,5,19]],"date-time":"2021-05-19T00:00:00Z","timestamp":1621382400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"DGA\/MRIS","award":["Validation of drones"],"award-info":[{"award-number":["Validation of drones"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,5,19]]},"DOI":"10.1145\/3447928.3456707","type":"proceedings-article","created":{"date-parts":[[2022,5,19]],"date-time":"2022-05-19T11:03:14Z","timestamp":1652958194000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["A few lessons learned in reinforcement learning for quadcopter attitude control"],"prefix":"10.1145","author":[{"given":"Nicola","family":"Bernini","sequence":"first","affiliation":[{"name":"Uber ATCP, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mikhail","family":"Bessa","sequence":"additional","affiliation":[{"name":"Uber ATCP, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"R\u00e9mi","family":"Delmas","sequence":"additional","affiliation":[{"name":"Uber ATCP, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arthur","family":"Gold","sequence":"additional","affiliation":[{"name":"Uber ATCP, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eric","family":"Goubault","sequence":"additional","affiliation":[{"name":"Ecole polytechnique, Palaiseau, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Romain","family":"Pennec","sequence":"additional","affiliation":[{"name":"Uber ATCP, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sylvie","family":"Putot","sequence":"additional","affiliation":[{"name":"Ecole polytechnique, Palaiseau, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fran\u00e7ois","family":"Sillion","sequence":"additional","affiliation":[{"name":"Uber ATCP, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,5,19]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3302504.3311817"},{"key":"e_1_3_2_1_2_1","series-title":"LNCS","volume-title":"CAV","author":"Akazaki Takumi","year":"2015","unstructured":"Takumi Akazaki and Ichiro Hasuo . Time robustness in MTL and expressivity in hybrid system falsification . In CAV , volume 9207 of LNCS . Springer , 2015 . Takumi Akazaki and Ichiro Hasuo. Time robustness in MTL and expressivity in hybrid system falsification. In CAV, volume 9207 of LNCS. Springer, 2015."},{"key":"e_1_3_2_1_3_1","volume-title":"Zhaodan Kong, Mac Schwager, and Calin Belta. Q-learning for robust satisfaction of signal temporal logic specifications","author":"Aksaray Derya","year":"2016","unstructured":"Derya Aksaray , Austin Jones , Zhaodan Kong, Mac Schwager, and Calin Belta. Q-learning for robust satisfaction of signal temporal logic specifications . In CDC. IEEE , 2016 . Derya Aksaray, Austin Jones, Zhaodan Kong, Mac Schwager, and Calin Belta. Q-learning for robust satisfaction of signal temporal logic specifications. In CDC. IEEE, 2016."},{"key":"e_1_3_2_1_4_1","volume-title":"Understanding deep neural networks with rectified linear units. CoRR, abs\/1611.01491","author":"Arora Raman","year":"2016","unstructured":"Raman Arora , Amitabh Basu , Poorya Mianjy , and Anirbit Mukherjee . Understanding deep neural networks with rectified linear units. CoRR, abs\/1611.01491 , 2016 . Raman Arora, Amitabh Basu, Poorya Mianjy, and Anirbit Mukherjee. Understanding deep neural networks with rectified linear units. CoRR, abs\/1611.01491, 2016."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-17465-1_5"},{"key":"e_1_3_2_1_6_1","volume-title":"ACRA","author":"Bangura Moses","year":"2012","unstructured":"Moses Bangura and Robert Mahony . Nonlinear dynamic modeling for high performance control of a quadrotor . In ACRA , 2012 . Moses Bangura and Robert Mahony. Nonlinear dynamic modeling for high performance control of a quadrotor. In ACRA, 2012."},{"key":"e_1_3_2_1_7_1","volume-title":"The explicit linear quadratic regulator for constrained systems. Automatica, 38(1)","author":"Bemporad Alberto","year":"2002","unstructured":"Alberto Bemporad , Manfred Morari , Vivek Dua , and Efstratios N. Pistikopoulos . The explicit linear quadratic regulator for constrained systems. Automatica, 38(1) , 2002 . Alberto Bemporad, Manfred Morari, Vivek Dua, and Efstratios N. Pistikopoulos. The explicit linear quadratic regulator for constrained systems. Automatica, 38(1), 2002."},{"key":"e_1_3_2_1_8_1","volume-title":"Reinforcement Learning and Optimal Control","author":"Bertsekas D.P.","year":"2019","unstructured":"D.P. Bertsekas . Reinforcement Learning and Optimal Control . Athena Scientific optimization and computation series. 2019 . D.P. Bertsekas. Reinforcement Learning and Optimal Control. Athena Scientific optimization and computation series. 2019."},{"key":"e_1_3_2_1_9_1","unstructured":"Bitcraze. https:\/\/store.bitcraze.io\/.  Bitcraze. https:\/\/store.bitcraze.io\/."},{"key":"e_1_3_2_1_10_1","volume-title":"Learning for quadcopter control","author":"Bjarre Lukas","year":"2019","unstructured":"Lukas Bjarre . Learning for quadcopter control , 2019 . Lukas Bjarre. Learning for quadcopter control, 2019."},{"key":"e_1_3_2_1_11_1","first-page":"236","article-title":"Extending signal temporal logic with signal-value freezing operator","author":"Brim Lubos","year":"2014","unstructured":"Lubos Brim , Petr Dluhos , David Safr\u00e1nek , and Tomas Vejpustek . STL : Extending signal temporal logic with signal-value freezing operator . Inf. Comput. , 236 , 2014 . Lubos Brim, Petr Dluhos, David Safr\u00e1nek, and Tomas Vejpustek. STL: Extending signal temporal logic with signal-value freezing operator. Inf. Comput., 236, 2014.","journal-title":"Inf. Comput."},{"key":"e_1_3_2_1_12_1","volume-title":"ICML","author":"Deisenroth M.","year":"2011","unstructured":"M. Deisenroth and C. Rasmussen . PILCO: A model-based and data-efficient approach to policy search . In ICML , 2011 . M. Deisenroth and C. Rasmussen. PILCO: A model-based and data-efficient approach to policy search. In ICML, 2011."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40787-1_27"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3302504.3311807"},{"key":"e_1_3_2_1_15_1","volume-title":"Robustness of temporal logic specifications for continuous-time signals. Theor. Comput. Sci., 410(42)","author":"Fainekos Georgios E.","year":"2009","unstructured":"Georgios E. Fainekos and George J. Pappas . Robustness of temporal logic specifications for continuous-time signals. Theor. Comput. Sci., 410(42) , 2009 . Georgios E. Fainekos and George J. Pappas. Robustness of temporal logic specifications for continuous-time signals. Theor. Comput. Sci., 410(42), 2009."},{"key":"e_1_3_2_1_16_1","volume-title":"ICRA","author":"Fei Fan","year":"2020","unstructured":"Fan Fei , Zhan Tu , and Xinyan Deng . Learn-to-recover : Retrofitting uavs with reinforcement learning-assisted flight control under cyberphysical attacks . In ICRA , 2020 . Fan Fei, Zhan Tu, and Xinyan Deng. Learn-to-recover: Retrofitting uavs with reinforcement learning-assisted flight control under cyberphysical attacks. In ICRA, 2020."},{"key":"e_1_3_2_1_17_1","volume-title":"Two-level lattice neural network architectures for control of nonlinear systems. CoRR, abs\/2004.09628","author":"Ferlez James","year":"2020","unstructured":"James Ferlez , Xiaowu Sun , and Yasser Shoukry . Two-level lattice neural network architectures for control of nonlinear systems. CoRR, abs\/2004.09628 , 2020 . James Ferlez, Xiaowu Sun, and Yasser Shoukry. Two-level lattice neural network architectures for control of nonlinear systems. CoRR, abs\/2004.09628, 2020."},{"key":"e_1_3_2_1_18_1","unstructured":"F\u00f6rster Julian.\n  System Identification of the Crazyflie 2.0 Nano Quadrocopter\n  . B.S. Thesis ETH Zurich 2015\n  .  F\u00f6rster Julian. System Identification of the Crazyflie 2.0 Nano Quadrocopter. B.S. Thesis ETH Zurich 2015."},{"key":"e_1_3_2_1_19_1","first-page":"80","article-title":"Addressing function approximation error in actor-critic methods","author":"Fujimoto S","year":"2018","unstructured":"S Fujimoto , H van Hoof , D Meger , Addressing function approximation error in actor-critic methods . Proceedings of Machine Learning Research , 80 , 2018 . S Fujimoto, H van Hoof, D Meger, et al. Addressing function approximation error in actor-critic methods. Proceedings of Machine Learning Research, 80, 2018.","journal-title":"Proceedings of Machine Learning Research"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i04.5814"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/LCSYS.2020.3001875"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3302504.3311794"},{"key":"e_1_3_2_1_23_1","volume-title":"Soft actor-critic algorithms and applications. CoRR, abs\/1812.05905","author":"Haarnoja Tuomas","year":"2018","unstructured":"Tuomas Haarnoja , Aurick Zhou , Kristian Hartikainen , George Tucker , Sehoon Ha , Jie Tan , Vikash Kumar , Henry Zhu , Abhishek Gupta , Pieter Abbeel , and Sergey Levine . Soft actor-critic algorithms and applications. CoRR, abs\/1812.05905 , 2018 . Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, and Sergey Levine. Soft actor-critic algorithms and applications. CoRR, abs\/1812.05905, 2018."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CDC40024.2019.9029429"},{"key":"e_1_3_2_1_25_1","volume-title":"Hiplot, interactive high-dimensionality plots. https:\/\/github.com\/facebookresearch\/hiplot","author":"Haziza D.","year":"2020","unstructured":"D. Haziza , J. Rapin , and G. Synnaeve . Hiplot, interactive high-dimensionality plots. https:\/\/github.com\/facebookresearch\/hiplot , 2020 . D. Haziza, J. Rapin, and G. Synnaeve. Hiplot, interactive high-dimensionality plots. https:\/\/github.com\/facebookresearch\/hiplot, 2020."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2017.2720851"},{"key":"e_1_3_2_1_27_1","volume-title":"Deep drone acrobatics. CoRR, abs\/2006.05768","author":"Kaufmann Elia","year":"2020","unstructured":"Elia Kaufmann , Antonio Loquercio , Ren\u00e9 Ranftl , Matthias M\u00fcller , Vladlen Koltun , and Davide Scaramuzza . Deep drone acrobatics. CoRR, abs\/2006.05768 , 2020 . Elia Kaufmann, Antonio Loquercio, Ren\u00e9 Ranftl, Matthias M\u00fcller, Vladlen Koltun, and Davide Scaramuzza. Deep drone acrobatics. CoRR, abs\/2006.05768, 2020."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301273"},{"key":"e_1_3_2_1_29_1","volume-title":"Developing a self-learning drone","author":"Koning Tim","year":"2020","unstructured":"Tim Koning . Developing a self-learning drone , 2020 . Tim Koning. Developing a self-learning drone, 2020."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2019.2930489"},{"key":"e_1_3_2_1_31_1","volume-title":"Temporal logic guided safe reinforcement learning using control barrier functions","author":"Li Xiao","year":"2019","unstructured":"Xiao Li and Calin Belta . Temporal logic guided safe reinforcement learning using control barrier functions . 2019 . Xiao Li and Calin Belta. Temporal logic guided safe reinforcement learning using control barrier functions. 2019."},{"key":"e_1_3_2_1_32_1","volume-title":"Continuous control with deep reinforcement learning. CoRR, abs\/1509.02971","author":"Lillicrap T.","year":"2016","unstructured":"T. Lillicrap , J. Hunt , A. Pritzel , N. Heess , T. Erez , Y. Tassa , D. Silver , and Daan Wierstra . Continuous control with deep reinforcement learning. CoRR, abs\/1509.02971 , 2016 . T. Lillicrap, J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. CoRR, abs\/1509.02971, 2016."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/LCSYS.2018.2853182"},{"key":"e_1_3_2_1_34_1","volume-title":"6th IFAC Conference on Nonlinear Model Predictive Control NMPC","author":"Lucia Sergio","year":"2018","unstructured":"Sergio Lucia and Benjamin Karg . A deep learning-based approach to robust nonlinear model predictive control. IFAC-PapersOnLine, 51(20), 2018 . 6th IFAC Conference on Nonlinear Model Predictive Control NMPC 2018 . Sergio Lucia and Benjamin Karg. A deep learning-based approach to robust nonlinear model predictive control. IFAC-PapersOnLine, 51(20), 2018. 6th IFAC Conference on Nonlinear Model Predictive Control NMPC 2018."},{"key":"e_1_3_2_1_35_1","volume-title":"Mobile Robotics and Autonomous Systems Laboratory","author":"Luis Carlos","year":"2016","unstructured":"Carlos Luis and J\u00e9r\u00f4me Le Ny . Design of a Trajectory Tracking Controller for a Nanoquadcopter. Technical report , Mobile Robotics and Autonomous Systems Laboratory , Polytechnique Montreal , 2016 . Carlos Luis and J\u00e9r\u00f4me Le Ny. Design of a Trajectory Tracking Controller for a Nanoquadcopter. Technical report, Mobile Robotics and Autonomous Systems Laboratory, Polytechnique Montreal, 2016."},{"key":"e_1_3_2_1_36_1","volume-title":"Cristian Ioan Vasile, and Calin Belta. Arithmetic-geometric mean robustness for control from signal temporal logic specifications","author":"Mehdipour Noushin","year":"2019","unstructured":"Noushin Mehdipour , Cristian Ioan Vasile, and Calin Belta. Arithmetic-geometric mean robustness for control from signal temporal logic specifications . In ACC. IEEE , 2019 . Noushin Mehdipour, Cristian Ioan Vasile, and Calin Belta. Arithmetic-geometric mean robustness for control from signal temporal logic specifications. In ACC. IEEE, 2019."},{"key":"e_1_3_2_1_37_1","volume-title":"Sim-to-(multi)-real: Transfer of low-level robust control policies to multiple quadrotors. CoRR, abs\/1903.04628","author":"Molchanov Artem","year":"2019","unstructured":"Artem Molchanov , Tao Chen , Wolfgang H\u00f6nig , James A. Preiss , Nora Ayanian , and Gaurav S. Sukhatme . Sim-to-(multi)-real: Transfer of low-level robust control policies to multiple quadrotors. CoRR, abs\/1903.04628 , 2019 . Artem Molchanov, Tao Chen, Wolfgang H\u00f6nig, James A. Preiss, Nora Ayanian, and Gaurav S. Sukhatme. Sim-to-(multi)-real: Transfer of low-level robust control policies to multiple quadrotors. CoRR, abs\/1903.04628, 2019."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2016.2630505"},{"key":"e_1_3_2_1_40_1","volume-title":"How to make Deep RL work in practice","author":"Rao Nirnai","year":"2020","unstructured":"Nirnai Rao , Elie Aljalbout , Axel Sauer , and Sami Haddadin . How to make Deep RL work in practice . 2020 . Nirnai Rao, Elie Aljalbout, Axel Sauer, and Sami Haddadin. How to make Deep RL work in practice. 2020."},{"key":"e_1_3_2_1_41_1","volume-title":"MIG. ACM","author":"Reda Daniele","year":"2020","unstructured":"Daniele Reda , Tianxin Tao , and Michiel van de Panne. Learning to locomote: Understanding how environment design matters for deep reinforcement learning . In MIG. ACM , 2020 . Daniele Reda, Tianxin Tao, and Michiel van de Panne. Learning to locomote: Understanding how environment design matters for deep reinforcement learning. In MIG. ACM, 2020."},{"key":"e_1_3_2_1_42_1","volume-title":"Proximal policy optimization algorithms. CoRR, abs\/1707.06347","author":"Schulman John","year":"2017","unstructured":"John Schulman , Filip Wolski , Prafulla Dhariwal , Alec Radford , and Oleg Klimov . Proximal policy optimization algorithms. CoRR, abs\/1707.06347 , 2017 . John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, abs\/1707.06347, 2017."},{"key":"e_1_3_2_1_43_1","first-page":"13823","article-title":"Reducing overestimation bias by increasing representation dissimilarity in ensemble based deep Q-learning. arXiv cs.","author":"Sheikh Hassam Ullah","year":"2006","unstructured":"Hassam Ullah Sheikh and Ladislau B\u00f6l\u00f6ni . Reducing overestimation bias by increasing representation dissimilarity in ensemble based deep Q-learning. arXiv cs. LG 2006 . 13823 , 2020. Hassam Ullah Sheikh and Ladislau B\u00f6l\u00f6ni. Reducing overestimation bias by increasing representation dissimilarity in ensemble based deep Q-learning. arXiv cs.LG 2006.13823, 2020.","journal-title":"LG"},{"key":"e_1_3_2_1_44_1","volume-title":"D2rl: Deep dense architectures in reinforcement learning","author":"Sinha Samarth","year":"2020","unstructured":"Samarth Sinha , Homanga Bharadhwaj , Aravind Srinivas , and Animesh Garg . D2rl: Deep dense architectures in reinforcement learning , 2020 . Samarth Sinha, Homanga Bharadhwaj, Aravind Srinivas, and Animesh Garg. D2rl: Deep dense architectures in reinforcement learning, 2020."},{"issue":"2","key":"e_1_3_2_1_45_1","article-title":"Reinforcement learning is direct adaptive optimal control","volume":"12","author":"Sutton R. S.","year":"1992","unstructured":"R. S. Sutton , A. G. Barto , and R. J. Williams . Reinforcement learning is direct adaptive optimal control . IEEE Control Systems Magazine , 12 ( 2 ), 1992 . R. S. Sutton, A. G. Barto, and R. J. Williams. Reinforcement learning is direct adaptive optimal control. IEEE Control Systems Magazine, 12(2), 1992.","journal-title":"IEEE Control Systems Magazine"},{"key":"e_1_3_2_1_46_1","volume-title":"ICML","volume":"119","author":"Wang Che","year":"2020","unstructured":"Che Wang , Yanqiu Wu , Quan Vuong , and Keith Ross . Striving for simplicity and performance in off-policy DRL: Output normalization and non-uniform sampling . In ICML , volume 119 , 2020 . Che Wang, Yanqiu Wu, Quan Vuong, and Keith Ross. Striving for simplicity and performance in off-policy DRL: Output normalization and non-uniform sampling. In ICML, volume 119, 2020."},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/LCSYS.2020.3001663"}],"event":{"name":"HSCC '21: 24th ACM International Conference on Hybrid Systems: Computation and Control","location":"Nashville Tennessee","acronym":"HSCC '21","sponsor":["SIGBED ACM Special Interest Group on Embedded Systems"]},"container-title":["Proceedings of the 24th International Conference on Hybrid Systems: Computation and Control"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447928.3456707","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447928.3456707","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:28:24Z","timestamp":1750195704000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447928.3456707"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,19]]},"references-count":46,"alternative-id":["10.1145\/3447928.3456707","10.1145\/3447928"],"URL":"https:\/\/doi.org\/10.1145\/3447928.3456707","relation":{},"subject":[],"published":{"date-parts":[[2021,5,19]]},"assertion":[{"value":"2021-05-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}