{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T04:04:07Z","timestamp":1777521847222,"version":"3.51.4"},"reference-count":90,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2007,3,1]],"date-time":"2007-03-01T00:00:00Z","timestamp":1172707200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Adaptive Behavior"],"published-print":{"date-parts":[[2007,3]]},"abstract":"<jats:p>This article describes a unifying framework for five highly influential but disparate theories of natural learning and behavioral action selection. These theories are normally considered independently, with their own experimental procedures and results. The framework presented builds on a structure of connection types, propagation rules and learning rules, which are used in combination to integrate results from each theory into a whole. These connection types and rules form the action-selection calculus. The calculus will be used to discuss the areas of genuine difference between the factor theories and to identify areas where there is overlap and where apparently disparate findings have a common source. The discussion is illustrated with exemplar experimental procedures. The article focuses on predictive or anticipatory properties inherent in these action-selection and learning theories, and uses the dynamic expectancy model and its computer implementation SRS\/E as a mechanism to conduct this discussion.<\/jats:p>","DOI":"10.1177\/1059712306076254","type":"journal-article","created":{"date-parts":[[2007,2,27]],"date-time":"2007-02-27T09:20:35Z","timestamp":1172568035000},"page":"73-97","source":"Crossref","is-referenced-by-count":9,"title":["An Action-Selection Calculus"],"prefix":"10.1177","volume":"15","author":[{"given":"Mark","family":"Witkowski","sequence":"first","affiliation":[{"name":"Department of Computing, Imperial College London,"}]}],"member":"179","published-online":{"date-parts":[[2007,3]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(94)00054-5"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0003-3472(76)80002-4"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3119.003.0053"},{"key":"e_1_2_2_4_1","volume-title":"The rat: A study in behavior","author":"Barnett, S. A.","year":"1975","unstructured":"Barnett, S. A. (1975). The rat: A study in behavior. Chicago: University of Chicago Press."},{"key":"e_1_2_2_5_1","first-page":"215","volume-title":"Models of information processing in the basal ganglia","author":"Barto, A. G.","year":"1995","unstructured":"Barto, A. G. (1995). Adaptive critics and the basal ganglia. In J. C. Houk, J. Davis, & D. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 215-232). Cambridge, MA: MIT Press."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/0166-4328(82)90001-8"},{"key":"e_1_2_2_7_1","volume-title":"Dynamic programming","author":"Bellman, R.","year":"1957","unstructured":"Bellman, R. (1957). Dynamic programming. Princeton, NJ: Princeton University Press."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.neuro.24.1.139"},{"key":"e_1_2_2_9_1","first-page":"453","volume-title":"Classical conditioning II: Current theory and research","author":"Bindra, D.","year":"1972","unstructured":"Bindra, D. (1972). A unified account of classical conditioning and operant training. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current theory and research (pp. 453-481). New York: Appleton-Century Crofts."},{"key":"e_1_2_2_10_1","volume-title":"Motivation","author":"Bindra, D.","year":"1966","unstructured":"Bindra, D., & Stewart, J. (Eds.). (1966). Motivation. Harmondsworth: Penguin Books."},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780198538493.001.0001"},{"key":"e_1_2_2_12_1","volume-title":"B. F. Skinner: A life","author":"Bjork, D. W.","year":"1993","unstructured":"Bjork, D. W. (1993). B. F. Skinner: A life. New York: Basic Books."},{"key":"e_1_2_2_13_1","volume-title":"Operant conditioning: An experimental analysis of behaviour","author":"Blackman, D.","year":"1974","unstructured":"Blackman, D. (1974). Operant conditioning: An experimental analysis of behaviour. London: Methuen & Co."},{"key":"e_1_2_2_14_1","volume-title":"Mind as machine: A history of cognitive science","author":"Boden, M.","year":"2006","unstructured":"Boden, M. (2006). Mind as machine: A history of cognitive science. Oxford: Oxford University Press."},{"key":"e_1_2_2_15_1","volume-title":"Theories of learning","author":"Bower, G. H.","year":"1981","unstructured":"Bower, G. H., & Hilgard, E. R. (1981). Theories of learning. Englewood Cliffs, NJ: Prentice Hall (citations used refer to the 5th edition)."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/JRA.1986.1087032"},{"key":"e_1_2_2_17_1","volume-title":"Intelligence without reason","author":"Brooks, R. A.","year":"1991","unstructured":"Brooks, R. A. (1991a). Intelligence without reason. MIT AI Laboratory, AI Memo No 1293"},{"key":"e_1_2_2_17_2","volume-title":"Computers and Thought, International Joint Conference on Artificial Intelligence (IJCAI)","unstructured":"(prepared for Computers and Thought, International Joint Conference on Artificial Intelligence (IJCAI), preprint, April 1991)."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(91)90053-M"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3120.003.0017"},{"key":"e_1_2_2_20_1","volume-title":"Modelling of natural action selection: Proceedings of an International Workshop","author":"Bryson, J. J.","year":"2005","unstructured":"Bryson, J. J., Prescott, T. J., & Seth, A. K. (Eds.). (2005). Modelling of natural action selection: Proceedings of an International Workshop. Edinburgh, UK. Brighton, UK: Society for the Study of Artificial Intelligence and the Simulation of Behavior (SSAISB)."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-45002-3_6"},{"key":"e_1_2_2_22_1","first-page":"3","volume-title":"The selection of behavior","author":"Catania, A. C.","year":"1988","unstructured":"Catania, A. C. (1988). The operant behaviorism of B. F. Skinner. In A. C. Catania & S. Harnad (Eds.), The selection of behavior (pp. 3-8). Cambridge: Cambridge University Press."},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2006.05.004"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(02)00963-7"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature03721"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1177\/105971239400300104"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSSC.1968.300136"},{"key":"e_1_2_2_28_1","volume-title":"The organization of behavior","author":"Hebb, D. O.","year":"1949","unstructured":"Hebb, D. O. (1949). The organization of behavior. New York: Wiley."},{"key":"e_1_2_2_29_1","volume-title":"An introduction to theories of learning","author":"Hergenhahn, B. R.","year":"2005","unstructured":"Hergenhahn, B. R., & Olson, M. H. (2005). An introduction to theories of learning (7th edition). Englewood Cliffs, NJ: Prentice-Hall.","edition":"7"},{"key":"e_1_2_2_30_1","volume-title":"Dynamic programming and markov processes","author":"Howard, R. A.","year":"1960","unstructured":"Howard, R. A. (1960). Dynamic programming and markov processes. Cambridge, MA: MIT Press."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.1962.sp006837"},{"key":"e_1_2_2_32_1","volume-title":"Principles of behavior","author":"Hull, C.","year":"1943","unstructured":"Hull, C. (1943). Principles of behavior. New York: Apple-Century Crofts."},{"key":"e_1_2_2_33_1","first-page":"741","volume-title":"Encyclopedia of cognitive science","author":"Jones, G.","year":"2003","unstructured":"Jones, G., & Ritter, F. E. (2003). Production systems and rule-based inference. In L. Nadel (Ed.), Encyclopedia of cognitive science (pp. 741-747). London: Nature Publishing Group."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0361-9230(97)00094-4"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.301"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1177\/105971230501300205"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(91)90054-N"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/5605.001.0001"},{"key":"e_1_2_2_39_1","volume-title":"Elements of machine learning","author":"Langley, P.","year":"1996","unstructured":"Langley, P. (1996). Elements of machine learning. San Francisco, CA: Morgan Kaufmann."},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.20-22-08474.2000"},{"key":"e_1_2_2_41_1","first-page":"221","volume-title":"Physiological mechanisms in animal behaviour, Symposium of the Society of Experimental Biology","author":"Lorenz, K. Z.","year":"1950","unstructured":"Lorenz, K. Z. (1950). The comparative method in studying innate behaviour patterns. In Physiological mechanisms in animal behaviour, Symposium of the Society of Experimental Biology (Vol. 4, pp. 221-268). London: Academic Press."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0057598"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1037\/10626-002"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3115.003.0033"},{"key":"e_1_2_2_45_1","volume-title":"Motivation and personality","author":"Maslow, A. H.","year":"1987","unstructured":"Maslow, A. H. (1987). R. Frager et al. (Eds.), Motivation and personality (3rd edition). New York: Harper and Row (first published 1954).","edition":"3"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.3.667"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00993104"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0040613"},{"key":"e_1_2_2_49_1","volume-title":"Problem-solving methods in artificial intelligence","author":"Nilsson, N. J.","year":"1971","unstructured":"Nilsson, N. J. (1971). Problem-solving methods in artificial intelligence. New York: McGraw-Hill."},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1159\/000063561"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80089-9"},{"key":"e_1_2_2_52_1","volume-title":"The hippocampus as a cognitive map","author":"O\u2019Keefe, J.","year":"1978","unstructured":"O\u2019Keefe, J., & Nadel, L. (1978). The hippocampus as a cognitive map. Oxford: Clarendon Press."},{"key":"e_1_2_2_53_1","volume-title":"Conditioned reflexes","author":"Pavlov, I. P.","year":"1927","unstructured":"Pavlov, I. P. (1927). Conditioned reflexes. Oxford: Oxford University Press (available as Dover Publications facsimile reprint, 1960)."},{"key":"e_1_2_2_54_1","volume-title":"The cerebral cortex of man","author":"Penfield, W.","year":"1950","unstructured":"Penfield, W., & Rasmussen, T. B. (1950). The cerebral cortex of man. New York: Macmillan."},{"key":"e_1_2_2_55_1","volume-title":"Mind in evolution: An east-west synthesis of learned behavior and cognition","author":"Razran, G.","year":"1971","unstructured":"Razran, G. (1971). Mind in evolution: An east-west synthesis of learned behavior and cognition. Boston: Houghton Mifflin."},{"key":"e_1_2_2_56_1","first-page":"64","volume-title":"Classical conditioning II: Current theory and research","author":"Rescorla, R. A.","year":"1972","unstructured":"Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current theory and research (pp. 64-99). New York: Appleton-Century Crofts."},{"key":"e_1_2_2_57_1","volume-title":"Artificial intelligence: A modern approach","author":"Russell, S.","year":"2003","unstructured":"Russell, S., & Norvig, P. (2003). Artificial intelligence: A modern approach. Englewood Cliffs, NJ: Prentice Hall (citations used refer to the 2nd edition).","edition":"2"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(97)00041-9"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3117.003.0027"},{"key":"e_1_2_2_60_1","first-page":"1039","volume-title":"Proceedings of the 10th International Joint Conference on Artificial Intelligence (IJCAI)","author":"Schoppers, M. J.","unstructured":"Schoppers, M. J. (1987). Universal plans for reactive robots in unpredictable environments. In Proceedings of the 10th International Joint Conference on Artificial Intelligence (IJCAI) (pp. 1039-1046). Milan, Italy."},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1152\/jn.1998.80.1.1"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00275079"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(00)81025-9"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/225298.225300"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0361-9230(03)00069-8"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1037\/0097-7403.1.1.56"},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3120.003.0031"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00115009"},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3115.003.0040"},{"key":"e_1_2_2_70_1","first-page":"355","volume-title":"Proceedings of the 9th Annual Conference of the Cognitive Science Society","author":"Sutton, R. S.","unstructured":"Sutton, R. S., & Barto, A. G. (1987). A temporal-difference model of classical conditioning. In Proceedings of the 9th Annual Conference of the Cognitive Science Society (pp. 355-378). Seattle, WA. Hillsdale, NJ: Lawrence Erlbaum."},{"key":"e_1_2_2_71_1","first-page":"497","volume-title":"Learning and computational neuroscience: Foundations of adaptive networks","author":"Sutton, R. S.","year":"1990","unstructured":"Sutton, R. S., & Barto, A. G. (1990). Time-derivative models of Pavlovian reinforcement. In M. Gabriel & J. Moore (Eds.), Learning and computational neuroscience: Foundations of adaptive networks (pp. 497-537). Cambridge, MA: MIT Press."},{"key":"e_1_2_2_72_1","volume-title":"Reinforcement learning: An introduction","author":"Sutton, R. S.","year":"1998","unstructured":"Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press."},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0055171"},{"issue":"8","key":"e_1_2_2_74_1","article-title":"Animal intelligence: An experimental study of the associative processes in animals","volume":"2","author":"Thorndike, E. L.","year":"1898","unstructured":"Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Review, Monograph Supplements, 2(8).","journal-title":"Psychological Review, Monograph Supplements"},{"key":"e_1_2_2_75_1","volume-title":"The study of instinct","author":"Tinbergen, N.","year":"1951","unstructured":"Tinbergen, N. (1951). The study of instinct. Oxford: Clarendon Press."},{"key":"e_1_2_2_76_1","volume-title":"Purposive behavior in animals and men","author":"Tolman, E. C.","year":"1932","unstructured":"Tolman, E. C. (1932). Purposive behavior in animals and men. New York: Century Co."},{"key":"e_1_2_2_77_1","first-page":"257","article-title":"Introduction and removal of reward, and maze performance in rats","volume":"4","author":"Tolman, E. C.","year":"1930","unstructured":"Tolman, E. C., & Honzik, C. H. (1930). Introduction and removal of reward, and maze performance in rats. University of California Publications in Psychology, 4, 257-275","journal-title":"University of California Publications in Psychology"},{"key":"e_1_2_2_77_2","unstructured":"(cited Bower & Hilgard 1981 pp. 335-338)."},{"key":"e_1_2_2_78_1","first-page":"421","volume-title":"Artificial life: SFI studies in sciences of complexity","author":"Travers, M.","year":"1989","unstructured":"Travers, M. (1989). Animal construction kits. In C. Langton (Ed.), Artificial life: SFI studies in sciences of complexity (pp. 421-442). Reading, MA: Addison-Wesley."},{"key":"e_1_2_2_79_1","unstructured":"Tyrrell T. (1993). Computational mechanisms for action selection. Ph.D. thesis. University of Edinburgh."},{"key":"e_1_2_2_80_1","first-page":"387","article-title":"An evaluation of Maes\u2019s bottom-up mechanism for behavior selection","volume":"1","author":"Tyrrell, T.","year":"1994","unstructured":"Tyrrell, T. (1994). An evaluation of Maes\u2019s bottom-up mechanism for behavior selection. Adaptive Behavior, 1, 387-420.","journal-title":"Adaptive Behavior"},{"key":"e_1_2_2_81_1","doi-asserted-by":"publisher","DOI":"10.4324\/9780203465028"},{"key":"e_1_2_2_82_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.brainresbull.2004.01.005"},{"key":"e_1_2_2_83_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992698"},{"key":"e_1_2_2_84_1","doi-asserted-by":"publisher","DOI":"10.1002\/cne.901660205"},{"key":"e_1_2_2_85_1","unstructured":"Witkowski M. (1997). Schemes for learning and behaviour: A new expectancy model. Ph.D. Thesis University of London."},{"key":"e_1_2_2_86_1","first-page":"73","volume-title":"Proceedings of the 6th International Symposium on Intelligent Robotic Systems","author":"Witkowski, M.","unstructured":"Witkowski, M. (1998). Dynamic expectancy: An approach to behaviour shaping using a new method of reinforcement learning. In Proceedings of the 6th International Symposium on Intelligent Robotic Systems (pp. 73-81). Edinburgh, UK."},{"key":"e_1_2_2_87_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3120.003.0020"},{"key":"e_1_2_2_88_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-45002-3_5"}],"container-title":["Adaptive Behavior"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1059712306076254","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1059712306076254","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T16:15:34Z","timestamp":1777392934000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1059712306076254"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2007,3]]},"references-count":90,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2007,3]]}},"alternative-id":["10.1177\/1059712306076254"],"URL":"https:\/\/doi.org\/10.1177\/1059712306076254","relation":{},"ISSN":["1059-7123","1741-2633"],"issn-type":[{"value":"1059-7123","type":"print"},{"value":"1741-2633","type":"electronic"}],"subject":[],"published":{"date-parts":[[2007,3]]}}}