{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,7]],"date-time":"2025-10-07T08:41:51Z","timestamp":1759826511643,"version":"3.38.0"},"reference-count":123,"publisher":"SAGE Publications","issue":"3","license":[{"start":{"date-parts":[[1993,1,1]],"date-time":"1993-01-01T00:00:00Z","timestamp":725846400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Adaptive Behavior"],"published-print":{"date-parts":[[1993,1]]},"abstract":"<jats:p> A computational model of nervous system function during classical and instrumental conditioning is proposed. The model assumes the form of a hierarchical network of control systems. Each control system is capable of learning and is referred to as an associative control process (ACP). Learning systems consisting of ACP networks, employing the drive-reinforcement learning mechanism (Klopf, 1988) and engaging in real-time, closed-loop, goal-seeking interactions with environments, are capable of being classically and instrumentally conditioned, as demonstrated by means of computer simulations. In multiple-T mazes, the systems learn to chain responses that avoid punishment and that lead eventually to reward. The temporal order in which the responses are learned and extinguished during instrumental conditioning is consistent with that observed in animal learning. Also consistent with animal learning experimental evidence, the ACP network model accounts for a wide range of classical conditioning phenomena. ACP networks, at their current stage of development, are intended to model sensorimotor, limbic, and hypothalamic nervous system function, suggesting a relationship between classical and instrumental conditioning that extends Mowrer's (1956, 1960a\/1973) two-factor theory of learning. In conjunction with consideration of limbic system and hypothalamic function, the role of emotion in natural intelligence is modeled and discussed. ACP networks constitute solutions to temporal and structural credit assignment problems, suggesting a theoretical approach for the synthesis of machine intelligence. <\/jats:p>","DOI":"10.1177\/105971239300100302","type":"journal-article","created":{"date-parts":[[2007,3,11]],"date-time":"2007-03-11T06:45:25Z","timestamp":1173595525000},"page":"263-319","source":"Crossref","is-referenced-by-count":25,"title":["A Hierarchical Network of Control Systems that Learn: Modeling Nervous System Function During Classical and Instrumental Conditioning"],"prefix":"10.1177","volume":"1","author":[{"given":"A. Harry","family":"Klopf","sequence":"first","affiliation":[{"name":"Wright Laboratory"}]},{"given":"Scott E.","family":"Weaver","sequence":"additional","affiliation":[{"name":"Wright Laboratory"}]},{"given":"James S.","family":"Morgan","sequence":"additional","affiliation":[{"name":"Wright Laboratory"}]}],"member":"179","published-online":{"date-parts":[[1993,1,1]]},"reference":[{"volume-title":"The metaphorical brain","year":"1972","author":"Arbib, M.","key":"atypb1"},{"volume-title":"Design for a brain","year":"1952","author":"Ashby, W.R.","key":"atypb2"},{"volume-title":"Proceedings of the American Institute of Aeronautics and Astronautics Conference on Guidance, Navigation, and Control","author":"Baird, L.C.","key":"atypb3"},{"volume-title":"An introduction to connectionist learning control systems","year":"1992","author":"Baker, W.L.","key":"atypb4"},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-84545-1_25"},{"volume-title":"Some learning tasks from a control perspective","year":"1991","author":"Barto, A.G.","key":"atypb6"},{"volume-title":"Goal seeking components for adaptive intelligence: An initial assessment (Technical Report Number 81-1070)","year":"1981","author":"Barto, A.G.","key":"atypb7"},{"key":"atypb8","first-page":"835","volume":"13","author":"Barto, A.G.","year":"1983","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics"},{"key":"atypb9","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/4943.001.0001"},{"volume-title":"Learning and sequential decision making","year":"1990","author":"Barto, A.G.","key":"atypb10"},{"volume-title":"Intelligence as adaptive behavior: An experiment in computational neuroethology","year":"1990","author":"Beer, R.D.","key":"atypb11"},{"key":"atypb12","doi-asserted-by":"publisher","DOI":"10.1037\/0735-7044.97.4.563"},{"volume-title":"A unified account of classical conditioning and operant training","year":"1972","author":"Bindra, D.","key":"atypb13"},{"key":"atypb14","unstructured":"Booker, L.B. (1982). Intelligent behavior as an adaptation to the task environment (Doctoral dissertation, University of Michigan , 1982). Dissertation Abstracts International, 43, 469B."},{"key":"atypb15","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.ne.13.030190.002355"},{"key":"atypb16","doi-asserted-by":"publisher","DOI":"10.1152\/physrev.1987.67.2.329"},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.ne.09.030186.002251"},{"key":"atypb18","volume-title":"Physiology of behavior","author":"Carlson, N.R.","year":"1986","edition":"3"},{"volume-title":"Principles of mental physiology, with their applications to the training and discipline of the mind and the study of its morbid conditions","year":"1874","author":"Carpenter, W.B.","key":"atypb19"},{"volume-title":"Computational neuroethology: A provisional manifesto (Cognitive Science Research Paper 162)","year":"1990","author":"Cliff, D.T.","key":"atypb20"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.3758\/BF03205241"},{"volume-title":"The biochemical basis of neuropharmacology","year":"1991","author":"Cooper, J.R.","key":"atypb22"},{"volume-title":"Temporally adaptive responses in neural models: The stimulus trace","year":"1990","author":"Desmond, J.E.","key":"atypb23"},{"volume-title":"Contemporary animal learning theory","year":"1980","author":"Dickinson, A.","key":"atypb24"},{"volume-title":"What computers still can't do: A critique of artificial reason","year":"1992","author":"Dreyfus, H.L.","key":"atypb25"},{"volume-title":"Animal learning and cognition","year":"1985","author":"Flaherty, C.F.","key":"atypb26"},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-558703-7.50019-X"},{"volume-title":"Fostering self-control: Comments of a counselor","year":"1989","author":"Ford, E.E.","key":"atypb28"},{"volume-title":"Goal directed behavior: The concept of action in psychology","year":"1985","author":"Frese, M.","key":"atypb29"},{"volume-title":"The organization of action: A new synthesis","year":"1980","author":"Gallistel, C.R.","key":"atypb30"},{"volume-title":"The perception of the visual world","year":"1950","author":"Gibson, J.J.","key":"atypb31"},{"volume-title":"The senses considered as perceptual systems","year":"1966","author":"Gibson, J.J.","key":"atypb32"},{"volume-title":"The ecological approach to visual perception","year":"1979","author":"Gibson, J.J.","key":"atypb33"},{"key":"atypb34","doi-asserted-by":"publisher","DOI":"10.1037\/h0024290"},{"issue":"3","key":"atypb35","doi-asserted-by":"crossref","first-page":"298","DOI":"10.3758\/BF03327321","volume":"16","author":"Gluck, M.A.","year":"1988","journal-title":"Psychobiology"},{"volume-title":"Elements of a two-process theory of learning","year":"1975","author":"Gray, J.A.","key":"atypb36"},{"volume-title":"The neurophysiology of anxiety: An enquiry into the functions of the septo-hippocampal system","year":"1982","author":"Gray, J.A.","key":"atypb37"},{"key":"atypb38","volume-title":"The psychology of fear and stress","author":"Gray, J.A.","year":"1987","edition":"2"},{"key":"atypb39","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.68.4.828"},{"key":"atypb40","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-009-7758-7"},{"volume-title":"The adaptive brain","year":"1987","author":"Grossberg, S.","key":"atypb41"},{"key":"atypb42","volume-title":"Learning and memory","author":"Hall, J.F.","year":"1989","edition":"2"},{"key":"atypb43","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4684-6770-3"},{"key":"atypb44","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-558703-7.50007-3"},{"key":"atypb45","doi-asserted-by":"publisher","DOI":"10.1037\/h0059929"},{"volume-title":"A perception-action perspective on the development of manual movements","year":"1990","author":"Hofsten, C. von","key":"atypb46"},{"volume-title":"Principles of behavior","year":"1943","author":"Hull, C.L.","key":"atypb47"},{"volume-title":"A behavior system: An introduction to behavior theory concerning the individual organism","year":"1952","author":"Hull, C.L.","key":"atypb48"},{"key":"atypb49","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.ne.12.030189.000505"},{"key":"atypb50","doi-asserted-by":"publisher","DOI":"10.1109\/ICSYSE.1991.161107"},{"volume-title":"Action-perception as a pattern formation process","year":"1990","author":"Kelso, J.A.S.","key":"atypb51"},{"key":"atypb52","doi-asserted-by":"publisher","DOI":"10.1080\/00222895.1991.10118359"},{"volume-title":"Brain function and adaptive systems\u2014A heterostatic theory (Report No. 133 [AFCRL-72-0164])","year":"1972","author":"Klopf, A.H.","key":"atypb53"},{"volume-title":"The hedonistic neuron: A theory of memory, learning, and intelligence","year":"1982","author":"Klopf, A.H.","key":"atypb54"},{"volume-title":"AIP Conference Proceedings 151: Neural networks for computing","author":"Klopf, A.H.","key":"atypb55"},{"issue":"2","key":"atypb56","doi-asserted-by":"crossref","first-page":"85","DOI":"10.3758\/BF03333113","volume":"16","author":"Klopf, A.H.","year":"1988","journal-title":"Psychobiology"},{"key":"atypb57","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-148955-7.50011-4"},{"volume-title":"The role of time in natural intelligence: Implications of classical and instrumental conditioning for neuronal and neural network modeling","year":"1990","author":"Klopf, A.H.","key":"atypb58"},{"volume-title":"AIP Conference Proceedings 151: Neural networks for computing","author":"Kosko, B.","key":"atypb59"},{"key":"atypb60","doi-asserted-by":"publisher","DOI":"10.1037\/0735-7044.100.4.443"},{"volume-title":"Introduction to neural and cognitive modeling","year":"1991","author":"Levine, D.S.","key":"atypb61"},{"volume-title":"Protein phosphorylation and neuronal modulation","year":"1987","author":"Levitan, I.B.","key":"atypb62"},{"key":"atypb63","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-4754-8"},{"volume-title":"The psychology of animal learning","year":"1974","author":"Mackintosh, N.J.","key":"atypb64"},{"volume-title":"Conditioning and associative learning","year":"1983","author":"Mackintosh, N.J.","key":"atypb65"},{"key":"atypb66","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3115.001.0001"},{"volume-title":"Direct perception","year":"1981","author":"Michaels, C.F.","key":"atypb67"},{"volume-title":"Learnable drives and rewards","year":"1951","author":"Miller, N.E.","key":"atypb68"},{"volume-title":"Nebraska Symposium on Motivation","author":"Miller, N.E.","key":"atypb69"},{"key":"atypb70","doi-asserted-by":"publisher","DOI":"10.1109\/JRPROC.1961.287775"},{"volume-title":"Classical eye-blink conditioning : Brain systems and implementation of a computational model","year":"1990","author":"Moore, J.W.","key":"atypb71"},{"key":"atypb72","doi-asserted-by":"publisher","DOI":"10.1016\/0166-4328(86)90092-6"},{"volume-title":"Proceedings of the International Joint Conference on Neural Networks","author":"Morgan, J.S.","key":"atypb73"},{"key":"atypb74","doi-asserted-by":"publisher","DOI":"10.1088\/0954-898X_1_4_004"},{"key":"atypb75","doi-asserted-by":"publisher","DOI":"10.1037\/h0040613"},{"key":"atypb76","doi-asserted-by":"publisher","DOI":"10.1037\/10802-000"},{"key":"atypb77","doi-asserted-by":"publisher","DOI":"10.1037\/10772-000"},{"key":"atypb78","doi-asserted-by":"publisher","DOI":"10.1037\/0097-7403.18.2.182"},{"volume-title":"Conditioned reflexes","year":"1927","author":"Pavlov, I.P.","key":"atypb79"},{"volume-title":"Emotion: Theory, research, and experience","year":"1986","author":"Plutchik, R.","key":"atypb80"},{"volume-title":"Behavior: The control of perception","year":"1973","author":"Powers, W.T.","key":"atypb81"},{"key":"atypb82","doi-asserted-by":"publisher","DOI":"10.1126\/science.179.4071.351"},{"key":"atypb83","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.85.5.417"},{"key":"atypb84","doi-asserted-by":"publisher","DOI":"10.2466\/pms.1960.11.1.71"},{"key":"atypb85","doi-asserted-by":"publisher","DOI":"10.2466\/pms.1960.11.3.309"},{"volume-title":"Ideo-motor action","year":"1987","author":"Prinz, W.","key":"atypb86"},{"volume-title":"Pavlovian second-order conditioning: Some implications for instrumental behavior","year":"1977","author":"Rescorla, R.A.","key":"atypb87"},{"key":"atypb88","doi-asserted-by":"publisher","DOI":"10.1037\/0003-066X.42.2.119"},{"key":"atypb89","doi-asserted-by":"publisher","DOI":"10.1037\/h0024475"},{"volume-title":"Introduction to comparative cognition","year":"1987","author":"Roitblat, H.L.","key":"atypb90"},{"volume-title":"A cognitive action theory of learning","year":"1988","author":"Roitblat, H.L.","key":"atypb91"},{"key":"atypb92","doi-asserted-by":"publisher","DOI":"10.1159\/000413540"},{"volume-title":"Hierarchical organization of motor programs","year":"1987","author":"Rosenbaum, D.A.","key":"atypb93"},{"volume-title":"Proceedings of the 43rd Annual Meeting of the Psychonomic Society","author":"Schmajuk, N.A.","key":"atypb94"},{"volume-title":"Nebraska Symposium on Motivation","author":"Schneirla, T.C.","key":"atypb95"},{"key":"atypb96","doi-asserted-by":"publisher","DOI":"10.1037\/11444-000"},{"key":"atypb97","first-page":"291","volume":"40","author":"Sperry, R.W.","year":"1952","journal-title":"American Scientist"},{"volume-title":"Reciprocal action of reward and punishment mechanisms","year":"1964","author":"Stein, L.","key":"atypb98"},{"volume-title":"Operant conditioning of individual neurons","year":"1988","author":"Stein, L.","key":"atypb99"},{"volume-title":"Temporal credit assignment in reinforcement learning","year":"1984","author":"Sutton, R.S.","key":"atypb100"},{"key":"atypb101","doi-asserted-by":"publisher","DOI":"10.1007\/BF00115009"},{"volume-title":"Proceedings of the International Workshop on the Simulation of Adaptive Behavior: From animals to animats","author":"Sutton, R.S.","key":"atypb102"},{"key":"atypb103","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.88.2.135"},{"volume-title":"Proceedings of the Ninth Annual Conference of the Cognitive Science Society","author":"Sutton, R.S.","key":"atypb104"},{"volume-title":"Time-derivative models of Pavlovian reinforcement","year":"1990","author":"Sutton, R.S.","key":"atypb105"},{"volume-title":"Proceedings of the 1991 American Control Conference","author":"Sutton, R.S.","key":"atypb106"},{"key":"atypb107","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1007\/BF00341933","volume":"55","author":"Tesauro, G.","year":"1986","journal-title":"Biological Cybernetics"},{"volume-title":"Behavior systems and learning: From misbehavior to general principles","year":"1989","author":"Timberlake, W.","key":"atypb108"},{"volume-title":"Preliminaries to a theory of action with reference to vision","year":"1977","author":"Turvey, M.T.","key":"atypb109"},{"volume-title":"Biological view of reinforcement","year":"1989","author":"Vaccarino, F.J.","key":"atypb110"},{"volume-title":"Evolution of primate behavior","year":"1970","author":"Washburn, S.L.","key":"atypb111"},{"volume-title":"Learning from delayed rewards","year":"1989","author":"Watkins, C.J.C.H.","key":"atypb112"},{"volume-title":"A neurophysiological model of emotional and intentional behavior","year":"1974","author":"Weil, J.L.","key":"atypb113"},{"key":"atypb114","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.1987.289329"},{"key":"atypb115","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(90)90088-3"},{"key":"atypb116","first-page":"75","volume":"1","author":"Werbos, P.J.","year":"1992","journal-title":"Nanobiology"},{"volume-title":"Cybernetics or control and communication in the animal and the machine","year":"1948","author":"Wiener, N.","key":"atypb117"},{"volume-title":"Proceedings of the Sixth Yale Workshop on Adaptive and Learning Systems","author":"Williams, R.J.","key":"atypb118"},{"key":"atypb119","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-1895-9_18"},{"volume-title":"The brain and reward","year":"1989","author":"Wise, R.A.","key":"atypb120"},{"key":"atypb121","doi-asserted-by":"publisher","DOI":"10.7312\/wood90908"},{"volume-title":"Nebraska Symposium on Motivation","author":"Young, P.T.","key":"atypb122"},{"key":"atypb123","doi-asserted-by":"publisher","DOI":"10.1016\/0006-8993(77)90541-8"}],"container-title":["Adaptive Behavior"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/105971239300100302","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/105971239300100302","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,3]],"date-time":"2025-03-03T22:32:25Z","timestamp":1741041145000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/105971239300100302"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1993,1]]},"references-count":123,"journal-issue":{"issue":"3","published-print":{"date-parts":[[1993,1]]}},"alternative-id":["10.1177\/105971239300100302"],"URL":"https:\/\/doi.org\/10.1177\/105971239300100302","relation":{},"ISSN":["1059-7123","1741-2633"],"issn-type":[{"type":"print","value":"1059-7123"},{"type":"electronic","value":"1741-2633"}],"subject":[],"published":{"date-parts":[[1993,1]]}}}