{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T10:08:25Z","timestamp":1777716505021,"version":"3.51.4"},"reference-count":35,"publisher":"SAGE Publications","issue":"10","license":[{"start":{"date-parts":[[2015,5,26]],"date-time":"2015-05-26T00:00:00Z","timestamp":1432598400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2015,9]]},"abstract":"<jats:p>This paper considers the problem of deploying a robot from a specification given as a temporal logic statement about some properties satisfied by the regions of a large, partitioned environment. We assume that the robot has noisy sensors and actuators and model its motion through the regions of the environment as a Markov decision process (MDP). The robot control problem becomes finding the control policy which maximizes the probability of satisfying the temporal logic task on the MDP. For a large environment, obtaining transition probabilities for each state\u2013action pair, as well as solving the necessary optimization problem for the optimal policy, are computationally intensive. To address these issues, we propose an approximate dynamic programming framework based on a least-squares temporal difference learning method of the actor\u2013critic type. This framework operates on sample paths of the robot and optimizes a randomized control policy with respect to a small set of parameters. The transition probabilities are obtained only when needed. Simulations confirm that convergence of the parameters translates to an approximately optimal policy.<\/jats:p>","DOI":"10.1177\/0278364915581505","type":"journal-article","created":{"date-parts":[[2015,5,27]],"date-time":"2015-05-27T09:44:44Z","timestamp":1432719884000},"page":"1329-1344","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":23,"title":["Temporal logic motion control using actor\u2013critic methods"],"prefix":"10.1177","volume":"34","author":[{"given":"Jing","family":"Wang","sequence":"first","affiliation":[{"name":"Division of System Engineering, Department of Mechanical Engineering, and Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xuchu","family":"Ding","sequence":"additional","affiliation":[{"name":"Embedded Systems and Networks Group, United Technologies Research Center, East Hartford, CT, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Morteza","family":"Lahijanian","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Rice University, Houston, TX, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ioannis Ch.","family":"Paschalidis","sequence":"additional","affiliation":[{"name":"Division of System Engineering, Department of Mechanical Engineering, and Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Calin A.","family":"Belta","sequence":"additional","affiliation":[{"name":"Division of System Engineering, Department of Mechanical Engineering, and Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2015,5,26]]},"reference":[{"key":"bibr1-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1007\/1-4020-8141-3_38"},{"key":"bibr2-0278364915581505","unstructured":"Baier C, Katoen JP (2008) Principles of Model Checking. Cambridge, MA: MIT Press, pp. 2620\u20132649."},{"key":"bibr3-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.1983.6313077"},{"key":"bibr4-0278364915581505","volume-title":"Neuro-Dynamic Programming","author":"Bertsekas D","year":"1996"},{"key":"bibr5-0278364915581505","volume-title":"Dynamic Programming and Optimal Control","author":"Bertsekas DP","year":"1995"},{"key":"bibr6-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2010.5509503"},{"key":"bibr7-0278364915581505","unstructured":"Boyan JA (1999) Least-squares temporal difference learning. In: ICML, pp. 49\u201356."},{"key":"bibr8-0278364915581505","volume-title":"Model Checking","author":"Clarke EM","year":"1999"},{"key":"bibr9-0278364915581505","volume-title":"Introduction to Algorithms","volume":"2","author":"Cormen TH","year":"2001"},{"key":"bibr10-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0032043"},{"key":"bibr11-0278364915581505","unstructured":"De Alfaro L (1997) Formal verification of probabilistic systems. PhD Thesis, Stanford University."},{"key":"bibr12-0278364915581505","first-page":"3515","volume-title":"18th IFAC world congress","author":"Ding XC","year":"2011"},{"key":"bibr13-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2012.6225290"},{"key":"bibr14-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2011.6160485"},{"key":"bibr15-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1002\/nav.21481"},{"key":"bibr16-0278364915581505","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2014.X.039"},{"key":"bibr17-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-36387-4"},{"key":"bibr18-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2009.5400278"},{"key":"bibr19-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1137\/S0363012901385691"},{"key":"bibr20-0278364915581505","unstructured":"Konda VR (2002) Actor\u2013critic algorithms. PhD Thesis, Massachusetts Institute of Technology."},{"key":"bibr21-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2007.363946"},{"key":"bibr22-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2011.2172150"},{"key":"bibr23-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2010.5509686"},{"key":"bibr24-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2013.2246095"},{"key":"bibr25-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2004.1428622"},{"key":"bibr26-0278364915581505","volume-title":"Twenty-eighth AAAI conference on artificial intelligence (AAAI 2014)","author":"Luna R","year":"2014"},{"key":"bibr27-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1002\/9780470316887"},{"key":"bibr28-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2004.1302413"},{"key":"bibr29-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1090\/crmm\/023"},{"key":"bibr30-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2014.7039527"},{"key":"bibr31-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/9780470544785"},{"key":"bibr32-0278364915581505","volume-title":"Reinforcement learning: An introduction","author":"Sutton RS","year":"1998"},{"key":"bibr33-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-48778-6_16"},{"key":"bibr34-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2012.6426174"},{"key":"bibr35-0278364915581505","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2009.5399536"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364915581505","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0278364915581505","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364915581505","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:18:37Z","timestamp":1777457917000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0278364915581505"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,5,26]]},"references-count":35,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2015,9]]}},"alternative-id":["10.1177\/0278364915581505"],"URL":"https:\/\/doi.org\/10.1177\/0278364915581505","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,5,26]]}}}