{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,25]],"date-time":"2025-05-25T04:03:05Z","timestamp":1748145785057,"version":"3.41.0"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[1999,7,1]],"date-time":"1999-07-01T00:00:00Z","timestamp":930787200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[1999,7,1]],"date-time":"1999-07-01T00:00:00Z","timestamp":930787200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Autonomous Robots"],"published-print":{"date-parts":[[1999,7]]},"DOI":"10.1023\/a:1008921914343","type":"journal-article","created":{"date-parts":[[2002,12,22]],"date-time":"2002-12-22T17:46:35Z","timestamp":1040579195000},"page":"77-88","source":"Crossref","is-referenced-by-count":17,"title":["Reinforcement Learning Soccer Teams with Incomplete World Models"],"prefix":"10.1007","volume":"7","author":[{"given":"Marco","family":"Wiering","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rafa\u0142","family":"Sa\u0142ustowicz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"J\u00fcrgen","family":"Schmidhuber","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","reference":[{"key":"232138_CR1","doi-asserted-by":"crossref","unstructured":"Albus, J.S. 1975. A new approach to manipulator control: The cerebellar model articulation controller (CMAC). Dynamic Systems, Measurement and Control, pp. 220-227.","DOI":"10.1115\/1.3426922"},{"key":"232138_CR2","first-page":"38","volume-title":"Machine Learning: Proceedings of the Twelfth International Conference","author":"S. Baluja","year":"1995","unstructured":"Baluja, S. and Caruana, R. 1995. Removing the genetics from the standard genetic algorithm. In Machine Learning: Proceedings of the Twelfth International Conference, A. Prieditis and S. Russell (Eds.), Morgan Kaufmann Publishers: San Francisco, CA, pp. 38-46."},{"key":"232138_CR3","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TSMC.1983.6313077","volume":"SMC-13","author":"A.G. Barto","year":"1983","unstructured":"Barto, A.G., Sutton, R.S., and Anderson, C.W. 1983. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13:834-846.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics"},{"key":"232138_CR4","doi-asserted-by":"crossref","unstructured":"Bellman, R. 1961. Adaptive Control Processes, Princeton University Press.","DOI":"10.1515\/9781400874668"},{"key":"232138_CR5","volume-title":"Neuro-Dynamic Programming","author":"D.P. Bertsekas","year":"1996","unstructured":"Bertsekas, D.P. and Tsitsiklis, J.N. 1996. Neuro-Dynamic Programming, Athena Scientific: Belmont, MA."},{"key":"232138_CR6","unstructured":"Chapman, D. and Kaelbling, L.P. 1991. Input generalization in delayed reinforcement learning. In Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI), Morgan Kaufman, Vol. 2, pp. 726-731."},{"key":"232138_CR7","first-page":"183","volume-title":"Proceedings of an International Conference on Genetic Algorithms and Their Applications","author":"N.L. Cramer","year":"1985","unstructured":"Cramer, N.L. 1985. A representation for the adaptive generation of simple sequential programs. In Proceedings of an International Conference on Genetic Algorithms and Their Applications, J.J. Grefenstette (Ed.), Lawrence Erlbaum Associates: Hillsdale, NJ, pp. 183-187."},{"key":"232138_CR8","unstructured":"Dickmanns, D., Schmidhuber, J., and Winklhofer, A. 1986. Der genetische Algorithmus: Eine Implementierung in Prolog. Fortgeschrittenenpraktikum, Institut f\u00fcr Informatik, Lehrstuhl Prof. Radig, Technische Universit\u00e4t M\u00fcnchen."},{"key":"232138_CR9","volume-title":"Adaptation in Natural and Artificial Systems","author":"J.H. Holland","year":"1975","unstructured":"Holland, J.H. 1975. Adaptation in Natural and Artificial Systems, University of Michigan Press: Ann Arbor."},{"key":"232138_CR10","doi-asserted-by":"crossref","unstructured":"Kaelbling, L. 1993. Learning in Embedded Systems, MIT Press.","DOI":"10.7551\/mitpress\/4168.001.0001"},{"key":"232138_CR11","volume-title":"Advances in Neural Information Processing Systems 12","author":"M. Kearns","year":"1999","unstructured":"Kearns, M. and Singh, S. 1999. Finite-sample convergence rates for Q-learning and indirect algorithms. In Advances in Neural Information Processing Systems 12, M. Kearns, S.A. Solla, and D. Cohn (Eds.), MIT Press: Cambridge, MA."},{"key":"232138_CR12","unstructured":"Koza, J.R. 1992. Genetic evolution and co-evolution of computer programs. In Artificial Life II, C.G. Langton, C. Taylor, J.D. Farmer, and S. Rasmussen (Eds.), Addison Wesley Publishing Company, pp. 313-324."},{"key":"232138_CR13","volume-title":"Reinforcement Learning for Robots Using Neural Networks","author":"L.-J. Lin","year":"1993","unstructured":"Lin, L.-J. 1993. Reinforcement Learning for Robots Using Neural Networks. Ph.D. Thesis, Carnegie Mellon University, Pittsburgh."},{"key":"232138_CR14","first-page":"103","volume":"13","author":"A. Moore","year":"1993","unstructured":"Moore, A. and Atkeson, C.G. 1993. Prioritized sweeping: Reinforcement learning with less data and less time. Machine Learning, 13:103-130.","journal-title":"Machine Learning"},{"key":"232138_CR15","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1162\/neco.1992.4.4.473","volume":"4","author":"S.J. Nowlan","year":"1992","unstructured":"Nowlan, S.J. and Hinton, G.E. 1992. Simplifying neural networks by soft weight sharing. Neural Computation, 4:173-193.","journal-title":"Neural Computation"},{"key":"232138_CR16","first-page":"283","volume":"22","author":"J. Peng","year":"1996","unstructured":"Peng, J. and Williams, R. 1996. Incremental multi-step Q-learning. Machine Learning, 22:283-290.","journal-title":"Machine Learning"},{"key":"232138_CR17","unstructured":"Rechenberg, I. 1971. Evolutions strategie\u2014Optimierung technischer Systeme nach Prinzipien der biologischen Evolution, Dissertation, Published in 1973 by Fromman-Holzboog."},{"key":"232138_CR18","series-title":"Technical Report","volume-title":"On-line Q-learning using connectionist sytems","author":"G.A. Rummery","year":"1994","unstructured":"Rummery, G.A. and Niranjan, M. 1994. On-line Q-learning using connectionist sytems. Technical Report CUED\/F-INFENG-TR 166, Cambridge University, UK."},{"issue":"2","key":"232138_CR19","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1162\/evco.1997.5.2.123","volume":"5","author":"R.P. Sa\u0142ustowicz","year":"1997","unstructured":"Sa\u0142ustowicz, R.P. and Schmidhuber, J. 1997. Probabilistic incremental program evolution. Evolutionary Computation, 5(2):123-141.","journal-title":"Evolutionary Computation"},{"key":"232138_CR20","first-page":"502","volume-title":"Proceedings of the Fourth International Conference on Neural Information Processing (ICONIP'97)","author":"R.P. Sa\u0142ustowicz","year":"1997","unstructured":"Sa\u0142ustowicz, R.P., Wiering, M.A., and Schmidhuber, J. 1997a. Evolving soccer strategies. In Proceedings of the Fourth International Conference on Neural Information Processing (ICONIP'97), Springer-Verlag: Singapore, pp. 502-506."},{"key":"232138_CR21","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"769","DOI":"10.1007\/BFb0020247","volume-title":"Proceedings of the Seventh International Conference on Artificial Neural Networks (ICANN'97)","author":"R.P. Sa\u0142ustowicz","year":"1997","unstructured":"Sa\u0142ustowicz, R.P., Wiering, M.A., and Schmidhuber, J. 1997b. On learning soccer strategies. In Proceedings of the Seventh International Conference on Artificial Neural Networks (ICANN'97), volume 1327 of Lecture Notes in Computer Science, W. Gerstner, A. Germond, M. Hasler, and J.-D. Nicoud (Eds.), Springer-Verlag: Berlin, Heidelberg, pp. 769-774."},{"issue":"2\/3","key":"232138_CR22","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1023\/A:1007570708568","volume":"33","author":"R.P. Sa\u0142ustowicz","year":"1998","unstructured":"Sa\u0142ustowicz, R.P., Wiering, M.A., and Schmidhuber, J. 1998. Learning team strategies: Soccer case studies. Machine Learning, 33(2\/3):263-282.","journal-title":"Machine Learning"},{"key":"232138_CR23","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1147\/rd.33.0210","volume":"3","author":"A.L. Samuel","year":"1959","unstructured":"Samuel, A.L. 1959. Some studies in machine learning using the game of checkers. IBM Journal on Research and Development, 3:210-229.","journal-title":"IBM Journal on Research and Development"},{"key":"232138_CR24","series-title":"Technical Report","volume-title":"Experiments with reinforcement learning in problems with continuous state and action spaces","author":"J.C. Santamaria","year":"1996","unstructured":"Santamaria, J.C., Sutton, R.S., and Ram, A. 1996. Experiments with reinforcement learning in problems with continuous state and action spaces. Technical Report CIONS 96-088, Georgia Institute of Technology, Atlanta."},{"key":"232138_CR25","unstructured":"Schmidhuber, J. 1995. On learning how to learn learning strategies. Technical Report FKI-198-94, Fakult\u00e4t f\u00fcr Informatik, Technische Universit\u00e4t M\u00fcnchen, Revised January 1995."},{"key":"232138_CR26","doi-asserted-by":"crossref","unstructured":"Schmidhuber, J., Zhao, J., and Schraudolph, N. 1997a. Reinforcement learning with self-modifying policies. In Learning to Learn, S. Thrun and L. Pratt (Eds.), Kluwer, pp. 293-309.","DOI":"10.1007\/978-1-4615-5529-2_12"},{"key":"232138_CR27","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1023\/A:1007383707642","volume":"28","author":"J. Schmidhuber","year":"1997","unstructured":"Schmidhuber, J., Zhao, J., and Wiering, M. 1997b. Shifting inductive bias with success-story algorithm, adaptive Levin search, and incremental self-improvement. Machine Learning, 28:105-130.","journal-title":"Machine Learning"},{"key":"232138_CR28","first-page":"123","volume":"22","author":"S.P. Singh","year":"1996","unstructured":"Singh, S.P. and Sutton, R.S. 1996. Reinforcement learning with replacing eligibility traces. Machine Learning, 22:123-158.","journal-title":"Machine Learning"},{"key":"232138_CR29","first-page":"9","volume":"3","author":"R.S. Sutton","year":"1988","unstructured":"Sutton, R.S. 1988. Learning to predict by the methods of temporal differences. Machine Learning, 3:9-44.","journal-title":"Machine Learning"},{"key":"232138_CR30","first-page":"1038","volume-title":"Advances in Neural Information Processing Systems 8","author":"R.S. Sutton","year":"1996","unstructured":"Sutton, R.S. 1996. Generalization in reinforcement learning: Successful examples using sparse coarse coding. In Advances in Neural Information Processing Systems 8, D.S. Touretzky, M.C. Mozer, and M.E. Hasselmo (Eds.), MIT Press: Cambridge, MA, pp. 1038-1045."},{"key":"232138_CR31","unstructured":"Sutton, R.S. and Barto, A.G. 1988. Reinforcement Learning: An Introduction, MIT Press\/Bradford Books."},{"key":"232138_CR32","doi-asserted-by":"crossref","unstructured":"Thrun, S., Fox, D., and Burgard, W. 1998. A probabilistic approach to concurrent mapping and localization for mobile robots. Machine Learning, (31):29-53. Also appeared in Autonomous Robots, 5:253\u2013271, 1998 as joint issue.","DOI":"10.1023\/A:1008806205438"},{"key":"232138_CR33","volume-title":"Learning from Delayed Rewards","author":"C.J.C.H. Watkins","year":"1989","unstructured":"Watkins, C.J.C.H. 1989. Learning from Delayed Rewards. Ph.D. Thesis, King's College, Cambridge, England."},{"key":"232138_CR34","first-page":"279","volume":"8","author":"C.J.C.H. Watkins","year":"1992","unstructured":"Watkins, C.J.C.H. and Dayan, P. 1992. Q-learning. Machine Learning, 8:279-292.","journal-title":"Machine Learning"},{"key":"232138_CR35","unstructured":"Wiering, M.A. 1999. Explorations in Efficient Reinforcement Learning. Ph.D. Thesis, University of Amsterdam\/IDSIA."},{"key":"232138_CR36","doi-asserted-by":"crossref","unstructured":"Wiering, M.A. and Schmidhuber, J. 1998a. Efficient model-based exploration. In Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior: From Animals to Animats 6, J.A. Meyer and S.W. Wilson (Eds.), MIT Press\/Bradford Books, pp. 223-228.","DOI":"10.7551\/mitpress\/3119.003.0034"},{"issue":"1","key":"232138_CR37","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1023\/A:1007562800292","volume":"33","author":"M.A. Wiering","year":"1998","unstructured":"Wiering, M.A. and Schmidhuber, J. 1998b. Fast online Q(\u03bb). Machine Learning, 33(1):105-116.","journal-title":"Machine Learning"}],"container-title":["Autonomous Robots"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1023\/A:1008921914343.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1023\/A:1008921914343\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1023\/A:1008921914343.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,24]],"date-time":"2025-05-24T07:10:07Z","timestamp":1748070607000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1023\/A:1008921914343"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1999,7]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[1999,7]]}},"alternative-id":["232138"],"URL":"https:\/\/doi.org\/10.1023\/a:1008921914343","relation":{},"ISSN":["0929-5593","1573-7527"],"issn-type":[{"type":"print","value":"0929-5593"},{"type":"electronic","value":"1573-7527"}],"subject":[],"published":{"date-parts":[[1999,7]]}}}