{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T04:04:26Z","timestamp":1777521866722,"version":"3.51.4"},"reference-count":61,"publisher":"SAGE Publications","issue":"4","license":[{"start":{"date-parts":[[2007,12,1]],"date-time":"2007-12-01T00:00:00Z","timestamp":1196467200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Adaptive Behavior"],"published-print":{"date-parts":[[2007,12]]},"abstract":"<jats:p>Emotion plays an important role in thinking. In this article we study affective control of the amount of simulated anticipatory behavior in adaptive agents using a computational model. Our approach is based on model-based reinforcement learning (RL) and inspired by the simulation hypothesis (Cotterill, 2001; Hesslow, 2002). The simulation hypothesis states that thinking is internal simulation of behavior using the same sensory-motor systems as those used for overt behavior. Here, we study the adaptiveness of an artificial agent, when action-selection bias is induced by an affect-controlled amount of simulated anticipatory behavior . To this end, we introduce an affect-controlled simulation-selection mechanism that uses the predictions of the agent's RL model to select anticipatory behaviors for simulation. Based on eXperiments with adaptive agents in two nondeterministic partially observable grid-worlds we conclude that (1) internal simulation has an adaptive benefit and (2) affective control can reduce the amount of simulation needed for this benefit. This is specifically the case if the following relation holds: positive affect decreases the amount of simulation towards simulating the best potential neXt action, while negative affect increases the amount of simulation towards simulating all potential neXt actions. In essence we use artificial affect to control mental eXploration versus eXploitations. Thus, agents \u201cfeeling positive\u201d can think ahead in a narrow sense and free up working memory resources, while agents \u201cfeeling negative\u201d must think ahead in a broad sense and maXimize usage of working memory. Our results are consistent with several psychological findings on the relation between affect and learning, and contribute to answering the question of when positive versus negative affect is useful during adaptation.<\/jats:p>","DOI":"10.1177\/1059712307084686","type":"journal-article","created":{"date-parts":[[2007,10,30]],"date-time":"2007-10-30T13:03:36Z","timestamp":1193749416000},"page":"397-422","source":"Crossref","is-referenced-by-count":25,"title":["Affect, Anticipation, and Adaptation: Affect-Controlled Selection of Anticipatory Simulation in Artificial Adaptive Agents"],"prefix":"10.1177","volume":"15","author":[{"given":"Joost","family":"Broekens","sequence":"first","affiliation":[{"name":"Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Walter A.","family":"Kosters","sequence":"additional","affiliation":[{"name":"Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fons J.","family":"Verbeek","sequence":"additional","affiliation":[{"name":"Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, The Netherlands,"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2007,12,1]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.106.3.529"},{"key":"atypb2","volume-title":"Proceedings of the 8th International Conference on Simulation of Adaptive Behavior","author":"Avila-Garcia, O."},{"key":"atypb3","volume-title":"Proceedings of the AISB'06 Symposium on Architecture of Brain and Mind","author":"Aylett, R."},{"key":"atypb4","doi-asserted-by":"publisher","DOI":"10.1016\/S1364-6613(00)01538-2"},{"key":"atypb5","volume-title":"Proceedings of the AISB'04 Symposium on Emotion, Cognition and A fective Computing","author":"Belavkin, R.V."},{"key":"atypb6","first-page":"106","volume":"52","author":"Berridge, K.C.","year":"2003","journal-title":"Cognition"},{"key":"atypb7","doi-asserted-by":"publisher","DOI":"10.1080\/095281398146824"},{"key":"atypb8","volume-title":"Proceedings of the AISB'06 Symposium on Biologically Inspired Robotics (Biro-net)","author":"Blanchard, A.J."},{"key":"atypb9","volume-title":"Proceedings of the 4th International Workshop on Artificial Intelligence in Economics and Management","author":"Botelho, L.M."},{"key":"atypb10","volume-title":"Proceedings of the CogSci'05 Conference","author":"Broekens, J."},{"key":"atypb11","volume-title":"Proceedings of the Third International Conference on Machine Learning and Applications","author":"Broekens, J."},{"key":"atypb12","volume-title":"IWINAC 2007, Part I, LNCS 4527","author":"Broekens, J."},{"key":"atypb13","volume-title":"Proceedings of the Workshop on Modeling Natural Action Selection","author":"Broekens, J."},{"key":"atypb14","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-45002-3_6"},{"key":"atypb15","volume-title":"Designing emotions for activity selection. Department of Computer Science Technical Report DAIMI PB 545","author":"Ca\u00f1amero, D.","year":"2000"},{"key":"atypb16","volume-title":"Feeling is believing: Some affective influences on belief","author":"Clore, G.L.","year":"2000"},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(02)00973-X"},{"key":"atypb18","volume-title":"Proceedings of the Workshop on Modeling Natural Action Selection","author":"Cos-Aguilera, I."},{"key":"atypb19","first-page":"1","volume":"64","author":"Cotterill, R.M.J.","year":"2001","journal-title":"Neurobiology"},{"key":"atypb20","doi-asserted-by":"publisher","DOI":"10.1080\/1358165042000283101"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.89.2.129"},{"key":"atypb22","volume-title":"Descartes' error: Emotion, reason, and the human brain","author":"Damasio, A.R.","year":"1994"},{"key":"atypb23","first-page":"89","volume":"42","author":"Davidson, R.J.","year":"2000","journal-title":"Cognition"},{"key":"atypb24","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(02)00963-7"},{"key":"atypb25","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1332574100"},{"key":"atypb26","first-page":"231","volume":"15","author":"Demiris, Y.","year":"2003","journal-title":"Science"},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00044-8"},{"key":"atypb28","doi-asserted-by":"publisher","DOI":"10.1037\/0278-7393.30.2.343"},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511659904.005"},{"key":"atypb30","doi-asserted-by":"publisher","DOI":"10.1038\/nature04587"},{"key":"atypb31","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511659904"},{"key":"atypb32","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511659904"},{"key":"atypb33","first-page":"385","volume":"4","author":"Gadanho, S.C.","year":"2003","journal-title":"Journal of Machine Learning Research"},{"key":"atypb34","volume-title":"Modularity & the psychoevolutionary theory of emotion. Mind and Cognition: An Anthology","author":"Griffith, P.E.","year":"1999"},{"key":"atypb35","doi-asserted-by":"publisher","DOI":"10.1016\/S1364-6613(02)01913-7"},{"key":"atypb36","volume-title":"Proceedings of the 8th International Conference on the Simulation of Adaptive Behavior","author":"Hoffmann, H."},{"key":"atypb37","volume-title":"IWINAC 2007, Part II, LNCS 4528","author":"Hogewoning, E."},{"key":"atypb38","doi-asserted-by":"publisher","DOI":"10.1613\/jair.301"},{"key":"atypb39","volume-title":"Proceedings of the AISB'05 Symposium on Agents that Want and Like","author":"Lahnstein, M."},{"key":"atypb40","volume-title":"Proceedings of the Twelfth International Conference on Machine Learning","author":"McCallum, A."},{"key":"atypb41","volume-title":"Proceedings of the AISB'06 Symposium on Nature Inspired Systems","author":"McMahon, A."},{"key":"atypb42","doi-asserted-by":"publisher","DOI":"10.1038\/nature03015"},{"key":"atypb43","first-page":"51","volume":"6","author":"Nason, S.","year":"2005","journal-title":"Systems Research"},{"key":"atypb44","doi-asserted-by":"publisher","DOI":"10.1037\/1528-3542.5.3.309"},{"key":"atypb45","doi-asserted-by":"publisher","DOI":"10.1023\/B:BTTJ.0000047603.37042.33"},{"key":"atypb46","doi-asserted-by":"publisher","DOI":"10.1017\/S0140525X00002429"},{"key":"atypb47","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8624.00040"},{"key":"atypb48","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.110.1.145"},{"key":"atypb49","volume-title":"Proceedings of the AISB'06 Symposium on Integrative Approaches to Machine Consciousness","author":"Salichs, M.A."},{"key":"atypb50","volume-title":"Appraisal considered as a process of multilevel sequential checking","author":"Scherer, K.R.","year":"2001"},{"key":"atypb51","doi-asserted-by":"publisher","DOI":"10.1126\/science.275.5306.1593"},{"key":"atypb52","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00228-9"},{"key":"atypb53","doi-asserted-by":"publisher","DOI":"10.1016\/j.concog.2005.11.005"},{"key":"atypb54","volume-title":"Proceedings of the Seventh International Conference on Machine Learning","author":"Sutton, R.S."},{"key":"atypb55","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton, R.S.","year":"1998"},{"key":"atypb56","volume-title":"Computational mechanisms for action selection. Ph.D. Thesis","author":"Tyrell, T.","year":"1993"},{"key":"atypb57","first-page":"121","volume":"123","author":"van Dartel, M.F.","year":"2005","journal-title":"Proceedings of the 5th International Workshop on Epigenetic Robotics. Lund University Cognitive Studies"},{"key":"atypb58","volume-title":"Proceedings of the 16th Belgium-Netherlands Conference on Artificial Intelligence","author":"van Dartel, M."},{"key":"atypb59","doi-asserted-by":"publisher","DOI":"10.1037\/1528-3542.5.4.456"},{"key":"atypb60","volume-title":"SAB'98 Work-shop on Grounding Emotions in Adaptive Systems, 5th International Conference of the Society for Adaptive Behavior","author":"Velasquez, J.D."},{"key":"atypb61","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2004.12.005"}],"container-title":["Adaptive Behavior"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1059712307084686","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1059712307084686","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T16:15:37Z","timestamp":1777392937000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1059712307084686"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2007,12]]},"references-count":61,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2007,12]]}},"alternative-id":["10.1177\/1059712307084686"],"URL":"https:\/\/doi.org\/10.1177\/1059712307084686","relation":{},"ISSN":["1059-7123","1741-2633"],"issn-type":[{"value":"1059-7123","type":"print"},{"value":"1741-2633","type":"electronic"}],"subject":[],"published":{"date-parts":[[2007,12]]}}}