{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T15:39:00Z","timestamp":1782833940327,"version":"3.54.5"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[1988,8,1]],"date-time":"1988-08-01T00:00:00Z","timestamp":586396800000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[1988,8]]},"DOI":"10.1007\/bf00115009","type":"journal-article","created":{"date-parts":[[2004,10,31]],"date-time":"2004-10-31T18:55:14Z","timestamp":1099248914000},"page":"9-44","source":"Crossref","is-referenced-by-count":2222,"title":["Learning to predict by the methods of temporal differences"],"prefix":"10.1007","volume":"3","author":[{"given":"Richard S.","family":"Sutton","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","reference":[{"key":"CR1","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1207\/s15516709cog0901_7","volume":"9","author":"D. H. Ackley","year":"1985","unstructured":"Ackley, D. H., Hinton, G. H., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9, 147?169.","journal-title":"Cognitive Science"},{"key":"CR2","unstructured":"Anderson, C. W. (1986). Learning and problem solving with multilayer connectionist systems. Doctoral dissertation. Department of Computer and Information Science. University of Massachusetts, Amherst."},{"key":"CR3","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/B978-0-934613-41-5.50014-3","volume-title":"Proceedings of the Fourth International Workshop on Machine Learning","author":"C. W. Anderson","year":"1987","unstructured":"Anderson, C. W. (1987). Strategy learning with multilayer connectionist representations. Proceedings of the Fourth International Workshop on Machine Learning (pp. 103?114). Irvine. CA: Morgan Kaufmann."},{"key":"CR4","first-page":"229","volume":"4","author":"A. G. Barto","year":"1985","unstructured":"Barto, A. G. (1985). Learning by statistical cooperation of self-interested neuron-like computing elements. Human Neurobiology, 4, 229?256.","journal-title":"Human Neurobiology"},{"key":"CR5","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TSMC.1983.6313077","volume":"13","author":"A. G. Barto","year":"1983","unstructured":"Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983). Neuronlike elements that can solve difficult learning control problems. IEEE Transactions on Systems. Man, and Cybernetics, 13, 834?846.","journal-title":"IEEE Transactions on Systems. Man, and Cybernetics"},{"key":"CR6","unstructured":"Booker, L. B. (1982). Intelligent behavior as an adaptation to the task environment. Doctoral dissertation. Department of Computer and Communication Sciences, University of Michigan. Ann Arbor."},{"key":"CR7","volume-title":"Machine learning: A guide to current research","author":"J. Christensen","year":"1986","unstructured":"Christensen, J. (1986). Learning static evaluation functions by linear regression. In T. M. Mitchell, J. G. Carbonell, & R. S. Michalski (Eds.). Machine learning: A guide to current research. Boston: Kluwer Academic"},{"key":"CR8","first-page":"148","volume-title":"Proceedings of the Fifth National Conference on Artificial Intelligence","author":"J. Christensen","year":"1986","unstructured":"Christensen, J., & Korf, R. E. (1986). A unified theory of hemistic evaluation functions and its application to learning. Proceedings of the Fifth National Conference on Artificial Intelligence (pp. 148?152). Philadelphia, PA: Morgan Kaufmann."},{"key":"CR9","volume-title":"Dynamic programming: Models and applications","author":"E. V. Denardo","year":"1982","unstructured":"Denardo, E. V. (1982). Dynamic programming: Models and applications. Englewood Cliffs, NJ: Prentice-Hall."},{"key":"CR10","volume-title":"Machine learning: An artificial intelligence approach","author":"T. G. Dietterich","year":"1986","unstructured":"Dietterich, T. G., & Michalski, R. S. (1986). Learning to predict sequences. In R. S. Michalski, J. G. Carbonell, & T. M. Mitchell (Eds.). Machine learning: An artificial intelligence approach (Vol. 2). Los Altos, CA Morgan Kaufmann."},{"key":"CR11","volume-title":"Model neural networks and behavior","author":"A. Gelperin","year":"1985","unstructured":"Gelperin, A., Hopfield, J. J., Tank, D. W. (1985). The logic of Limax learning. In A. Selverston (Ed.), Model neural networks and behavior. New York: Plenum Press."},{"key":"CR12","unstructured":"Hampson, S. E. (1983). A neural model of adaptive behavior. Doctoral dissertation, Department of Information and Computer Science. University of California, Irvine."},{"key":"CR13","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1007\/BF00317987","volume":"56","author":"S. E. Hampson","year":"1987","unstructured":"Hampson, S. E., & Volper, D. J. (1987). Disjunctive models of boolean category learning. Biological Cybernetics, 56, 121?137.","journal-title":"Biological Cybernetics"},{"key":"CR14","volume-title":"Machine learning: An artificial intelligence approach","author":"J. H. Holland","year":"1986","unstructured":"Holland, J. H. (1986). Escaping brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems. In R. S. Michalski, J. G. Carbonell, & T. M. Mitchell (Eds.). Machine learning: An artificial intelligence approach (Vol. 2). Los Altos, CA: Morgan Kaufmann."},{"key":"CR15","doi-asserted-by":"crossref","first-page":"455","DOI":"10.3758\/BF03205056","volume":"15","author":"E. J. Kehoe","year":"1987","unstructured":"Kehoe, E. J., Schreurs, B. G., & Graham, P. (1987). Temporal primacy over-rides prior training in serial compound conditioning of the rabbit's nictitating membrane response. Animal Learning and Behavior, 15, 455?464.","journal-title":"Animal Learning and Behavior"},{"key":"CR16","volume-title":"Finite Markov chains","author":"J. G. Kemeny","year":"1976","unstructured":"Kemeny, J. G., & Snell, J. L. (1976). Finite Markov chains, New York: Springer-Verlag."},{"key":"CR17","series-title":"Technical Report 87-1139","volume-title":"A neuronal model of classical conditioning","author":"A. H. Klopf","year":"1987","unstructured":"Klopf, A. H. (1987). A neuronal model of classical conditioning (Technical Report 87?1139). OH: Wright-Patterson Air Force Base, Wright Aeronautical Laboratories."},{"key":"CR18","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/0166-4328(86)90092-6","volume":"21","author":"J. W. Moore","year":"1986","unstructured":"Moore, J. W., Desmond, J. E., Berthier, N. E., Blazis, D. E. J., Sutton, R. S., & Barto, A. G. (1986). Simulation of the classically conditioned nictitating membrane response by a neuron-like adaptive element: Response topography, neuronal firing and interstimulus intervals. Behavioral Brain Research, 21, 143?154.","journal-title":"Behavioral Brain Research"},{"key":"CR19","series-title":"Technical Report","doi-asserted-by":"crossref","DOI":"10.21236\/ADA164453","volume-title":"Learning internal representations by error propagation","author":"D. E. Rumelhart","year":"1985","unstructured":"Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985). Learning internal representations by error propagation (Technical Report No. 8506). La Jolla: University of California, San Diego, Institute for Cognitive Science. Also in D. E. Rumelhart & J. L. McClelland (Eds.). Paralled distributed processing: Explorations in the microstructure of cognition (Vol. 1). Cambridge, MA: MIT Press."},{"key":"CR20","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1147\/rd.33.0210","volume":"3","author":"A. L. Samuel","year":"1959","unstructured":"Samuel, A. L. (1959). Some studies in machine learning using the game of checkers. IBM Journal on Research and Development, 3, 210?229. Reprinted in E. A. Feigenbaum & J. Feldman (Eds.). Computers and though. New York: McGraw-Hill.","journal-title":"IBM Journal on Research and Development"},{"key":"CR21","unstructured":"Sutton, R. S. (1984). Temporal credit assignment in reinforcement learning Doctoral dissertation, Department of Computer and Information Science. University of Massachusetts. Amherst."},{"key":"CR22","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1037\/0033-295X.88.2.135","volume":"88","author":"R. S. Sutton","year":"1981","unstructured":"Sutton, R. S., & Barto, A. G. (1981a). Toward a modern theory of adaptive networks: Expectation and prediction. Psychological Review, 88, 135?171.","journal-title":"Psychological Review"},{"key":"CR23","first-page":"217","volume":"4","author":"R. S. Sutton","year":"1981","unstructured":"Sutton, R. S., & Barto, A. G. (1981b). An adaptive network that constructs and uses an internal model of its environment. Cognition and Brain Theory, 4, 217?246.","journal-title":"Cognition and Brain Theory"},{"key":"CR24","first-page":"355","volume-title":"Proceedings of the Ninth Annual Conference of the Cognitive Science Society","author":"R. S. Sutton","year":"1987","unstructured":"Sutton, R. S., & Barto, A. G. (1987). A temporal-difference model of classical conditioning. Proceedings of the Ninth Annual Conference of the Cognitive Science Society (pp. 355?378). Seattle, WA: Lawrence Erlbaum."},{"key":"CR25","first-page":"54","volume-title":"Proceedings of the Seventh Annual Conference of the Cognitive Science Society","author":"R. S. Sutton","year":"1985","unstructured":"Sutton, R. S., & Pinette, B. (1985). The learning of world models by connectionist networks. Proceedings of the Seventh Annual Conference of the Cognitive Science Society (pp. 54?64). Irvine, CA: Lawrence Erlbaum."},{"key":"CR26","volume-title":"Matrix iterative analysis","author":"R. S. Varga","year":"1962","unstructured":"Varga, R. S. (1962). Matrix iterative analysis. Englewood Cliffs, NJ: Prentice-Hall."},{"key":"CR27","doi-asserted-by":"crossref","unstructured":"Widrow B., & Hoff, M. E. (1960). Adaptive switching circuits, 1960 WESCON Convention Record, Part IV (pp. 96?104).","DOI":"10.21236\/AD0241531"},{"key":"CR28","volume-title":"Adaptive signal processing","author":"B. Widrow","year":"1985","unstructured":"Widrow, B., & Stearns, S. D. (1985). Adaptive signal processing. Englewood Cliffs, NJ: Prentice-Hall."},{"key":"CR29","series-title":"Technical Report","volume-title":"Reinforcement learning in connectionist networks: A mathematical analysis","author":"R. J. Williams","year":"1986","unstructured":"Williams, R. J. (1986). Reinforcement learning in connectionist networks: A mathematical analysis (Technical Report No. 8605). La Jolla: University of California. San Diego. Institute for Cognitive Science."},{"key":"CR30","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1016\/S0019-9958(77)90354-0","volume":"34","author":"I. H. Witten","year":"1977","unstructured":"Witten, I. H. (1977). An adaptive optimal controller for discrete-time Markov environments. Information and Control, 34, 286?295.","journal-title":"Information and Control"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/BF00115009.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/BF00115009\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/BF00115009","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,4,8]],"date-time":"2019-04-08T14:09:17Z","timestamp":1554732557000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/BF00115009"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1988,8]]},"references-count":30,"journal-issue":{"issue":"1","published-print":{"date-parts":[[1988,8]]}},"alternative-id":["BF00115009"],"URL":"https:\/\/doi.org\/10.1007\/bf00115009","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[1988,8]]}}}