{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T11:30:04Z","timestamp":1791977404387,"version":"4.4.0"},"reference-count":69,"publisher":"Public Library of Science (PLoS)","issue":"2","license":[{"start":{"date-parts":[[2026,2,6]],"date-time":"2026-02-06T00:00:00Z","timestamp":1770336000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["KO 3510\/5-2"],"award-info":[{"award-number":["KO 3510\/5-2"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["RU 1743\/7-2"],"award-info":[{"award-number":["RU 1743\/7-2"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>\n                    Having to control multiple tasks in parallel poses challenges for humans and artificial agents alike. In artificial intelligence, specific forms of reinforcement learning (RL), most notably hierarchical and model-based RL, have shown promising results in scenarios where tasks or skills need to be switched adaptively. However, RL agents still encounter difficulties when faced with serial multitasking that involves switching control between continuously running subtasks, such as changing the radio station while driving in traffic. Inspired by human cognitive processes, we hypothesize that maintaining a sense of control is a key mechanism facilitating such task-switching decisions. We propose a mathematical formulation of a situational sense of control that consists of two components: an evaluative indicator of the predictability of action outcomes and a predictive indicator of a need for control in individual subtasks. We integrate this model of a sense of control into a hierarchical RL agent and evaluate its performance in a\n                    <jats:italic>Collect Asteroids<\/jats:italic>\n                    game environment, in which one must alternate between navigating two spaceships to collect as many asteroids as possible. Comparing RL agents with and without a sense of control, as well as with human participants, shows that equipping RL agents with a sense of control results in significant performance improvements. Our findings indicate that agents equipped with a sense of control prioritize more complex tasks, exhibit increased switching behavior, and make switches at strategically optimal times, leading to superior overall performance. The incorporation of cognitive mechanisms, inspired by human behavior, into RL agents thus appears to yield considerable enhancements in performance when acting in complex and dynamic environments.\n                  <\/jats:p>","DOI":"10.1371\/journal.pone.0342305","type":"journal-article","created":{"date-parts":[[2026,2,6]],"date-time":"2026-02-06T18:37:56Z","timestamp":1770403076000},"page":"e0342305","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":1,"title":["Giving AI agents a sense of control facilitates reinforcement learning in multitasking scenarios"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9918-4609","authenticated-orcid":true,"given":"Annika","family":"\u00d6sterdiekhoff","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nils Wendel","family":"Heinrich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nele","family":"Russwinkel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefan","family":"Kopp","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,2,6]]},"reference":[{"issue":"8","key":"pone.0342305.ref001","doi-asserted-by":"crossref","first-page":"2749","DOI":"10.3390\/app10082749","article-title":"A survey on theories and applications for self-driving cars based on deep learning methods","volume":"10","author":"J Ni","year":"2020","journal-title":"Applied Sciences."},{"key":"pone.0342305.ref002","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.cogr.2023.04.001","article-title":"Artificial intelligence, machine learning and deep learning in advanced robotics, a review","volume":"3","author":"M Soori","year":"2023","journal-title":"Cognitive Robotics."},{"issue":"10","key":"pone.0342305.ref003","doi-asserted-by":"crossref","first-page":"596","DOI":"10.3390\/info15100596","article-title":"Artificial intelligence-enabled intelligent assistant for personalized and adaptive learning in higher education","volume":"15","author":"R Sajja","year":"2024","journal-title":"Information."},{"key":"pone.0342305.ref004","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"2018"},{"issue":"7587","key":"pone.0342305.ref005","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"D Silver","year":"2016","journal-title":"Nature."},{"issue":"7676","key":"pone.0342305.ref006","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of Go without human knowledge","volume":"550","author":"D Silver","year":"2017","journal-title":"Nature."},{"issue":"6419","key":"pone.0342305.ref007","doi-asserted-by":"crossref","first-page":"1140","DOI":"10.1126\/science.aar6404","article-title":"A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play","volume":"362","author":"D Silver","year":"2018","journal-title":"Science."},{"key":"pone.0342305.ref008","article-title":"Playing atari with deep reinforcement learning","author":"V Mnih","year":"2013","journal-title":"arXiv preprint"},{"issue":"7540","key":"pone.0342305.ref009","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"V Mnih","year":"2015","journal-title":"Nature."},{"key":"pone.0342305.ref010","first-page":"14","volume-title":"Lecture Notes in Computer Science","author":"M McKenzie","year":"2017"},{"issue":"3","key":"pone.0342305.ref011","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1016\/S1364-6613(03)00028-7","article-title":"Task switching","volume":"7","author":"S Monsell","year":"2003","journal-title":"Trends Cogn Sci."},{"issue":"4","key":"pone.0342305.ref012","doi-asserted-by":"crossref","first-page":"763","DOI":"10.1037\/0096-1523.27.4.763","article-title":"Executive control of cognitive processes in task switching","volume":"27","author":"JS Rubinstein","year":"2001","journal-title":"J Exp Psychol Hum Percept Perform."},{"issue":"4","key":"pone.0342305.ref013","doi-asserted-by":"crossref","first-page":"1441","DOI":"10.1016\/j.chb.2013.01.040","article-title":"Self-interruptions in discretionary multitasking","volume":"29","author":"RF Adler","year":"2013","journal-title":"Computers in Human Behavior."},{"key":"pone.0342305.ref014","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1016\/j.cortex.2018.06.009","article-title":"Definition: dual-tasking and multitasking","volume":"106","author":"SE MacPherson","year":"2018","journal-title":"Cortex."},{"issue":"9","key":"pone.0342305.ref015","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1111\/j.0956-7976.2004.00728.x","article-title":"The cost of a voluntary task switch","volume":"15","author":"CM Arrington","year":"2004","journal-title":"Psychol Sci."},{"issue":"2","key":"pone.0342305.ref016","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1016\/j.jarmac.2016.04.003","article-title":"The role of time on task in multi-task management.","volume":"5","author":"RS Gutzwiller","year":"2016","journal-title":"Journal of Applied Research in Memory and Cognition."},{"issue":"3","key":"pone.0342305.ref017","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1037\/0096-3445.136.3.370","article-title":"Discretionary task interleaving: heuristics for time allocation in cognitive foraging","volume":"136","author":"SJ Payne","year":"2007","journal-title":"J Exp Psychol Gen."},{"key":"pone.0342305.ref018","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.ijhcs.2015.01.002","article-title":"Discrete task switching in overload: a meta-analyses and a model","volume":"79","author":"CD Wickens","year":"2015","journal-title":"International Journal of Human-Computer Studies."},{"issue":"4","key":"pone.0342305.ref019","doi-asserted-by":"crossref","first-page":"728","DOI":"10.1177\/0018720813504216","article-title":"Choice in multitasking: how delays in the primary task turn a rational into an irrational multitasker","volume":"56","author":"I Katidioti","year":"2014","journal-title":"Hum Factors."},{"issue":"1655","key":"pone.0342305.ref020","doi-asserted-by":"crossref","first-page":"20130480","DOI":"10.1098\/rstb.2013.0480","article-title":"Model-based hierarchical reinforcement learning and human action control","volume":"369","author":"M Botvinick","year":"2014","journal-title":"Philos Trans R Soc Lond B Biol Sci."},{"issue":"2","key":"pone.0342305.ref021","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1007\/s10846-017-0468-y","article-title":"Survey of model-based reinforcement learning: applications on robotics","volume":"86","author":"AS Polydoros","year":"2017","journal-title":"J Intell Robot Syst."},{"issue":"5","key":"pone.0342305.ref022","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3453160","article-title":"Hierarchical reinforcement learning","volume":"54","author":"S Pateria","year":"2021","journal-title":"ACM Comput Surv."},{"issue":"4","key":"pone.0342305.ref023","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1023\/A:1025696116075","article-title":"Recent advances in hierarchical reinforcement learning","volume":"13","author":"AG Barto","year":"2003","journal-title":"Discrete Event Dynamic Systems."},{"key":"pone.0342305.ref024","unstructured":"Kulkarni TD, Narasimhan K, Saeedi A, Tenenbaum J. Hierarchical deep reinforcement learning: integrating temporal abstraction and intrinsic motivation. In: Advances in neural information processing systems. 2016. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2016\/file\/f442d33fa06832082290ad8544a8da27-Paper.pdf"},{"key":"pone.0342305.ref025","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/S0004-3702(99)00052-1","article-title":"Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning","volume":"112","author":"RS Sutton","year":"1999","journal-title":"Artificial Intelligence."},{"key":"pone.0342305.ref026","unstructured":"Nachum O, Gu SS, Lee H, Levine S. Data-efficient hierarchical reinforcement learning. In: Advances in Neural Information Processing Systems. 2018. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2018\/file\/e6384711491713d29bc63fc5eeb5ba4f-Paper.pdf"},{"key":"pone.0342305.ref027","unstructured":"Levy A, Konidaris G, Platt R, Saenko K. Learning multi-level hierarchies with hindsight. In: Proceedings of the 7th International Conference on Learning Representations. 2019. https:\/\/doi.org\/10.48550\/arXiv.1712.00948"},{"key":"pone.0342305.ref028","doi-asserted-by":"crossref","unstructured":"Lingler A, Talypova D, Jokinen JPP, Oulasvirta A, Wintersberger P. Supporting task switching with reinforcement learning. In: Proceedings of the CHI Conference on Human Factors in Computing Systems. 2024. p. 1\u201318. https:\/\/doi.org\/10.1145\/3613904.3642063","DOI":"10.1145\/3613904.3642063"},{"issue":"1","key":"pone.0342305.ref029","article-title":"An efficient approach to model-based hierarchical reinforcement learning","volume":"31","author":"Z Li","year":"2017","journal-title":"AAAI."},{"issue":"8059","key":"pone.0342305.ref030","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1038\/s41586-025-08744-2","article-title":"Mastering diverse control tasks through world models","volume":"640","author":"D Hafner","year":"2025","journal-title":"Nature."},{"issue":"1","key":"pone.0342305.ref031","first-page":"1","article-title":"The sense of control and the sense of agency","volume":"13","author":"E Pacherie","year":"2007","journal-title":"Psyche."},{"issue":"1","key":"pone.0342305.ref032","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/j.concog.2007.03.010","article-title":"Beyond the comparator model: a multifactorial two-step account of agency","volume":"17","author":"M Synofzik","year":"2008","journal-title":"Conscious Cogn."},{"issue":"4","key":"pone.0342305.ref033","article-title":"The sense of agency during continuous action: performance is more important than action-feedback association","volume":"10","author":"W Wen","year":"2015","journal-title":"PLoS One."},{"key":"pone.0342305.ref034","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1007\/978-3-642-45190-4_16","volume-title":"The cognitive neuroscience of metacognition","author":"J Metcalfe","year":"2014"},{"issue":"7","key":"pone.0342305.ref035","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1037\/0003-066X.54.7.480","article-title":"Apparent mental causation. Sources of the experience of will","volume":"54","author":"DM Wegner","year":"1999","journal-title":"Am Psychol."},{"key":"pone.0342305.ref036","doi-asserted-by":"crossref","first-page":"103824","DOI":"10.1016\/j.actpsy.2022.103824","article-title":"Cognitive load decreases the sense of agency during continuous action","volume":"233","author":"JA Dewey","year":"2023","journal-title":"Acta Psychol (Amst)."},{"issue":"1","key":"pone.0342305.ref037","doi-asserted-by":"crossref","first-page":"222","DOI":"10.3758\/s13423-018-1512-z","article-title":"Monitoring and control in multitasking","volume":"26","author":"S Schuch","year":"2019","journal-title":"Psychon Bull Rev."},{"key":"pone.0342305.ref038","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.cogsys.2021.11.005","article-title":"Towards autonomous artificial agents with an active self: modeling sense of control in situated action","volume":"72","author":"S Kahl","year":"2022","journal-title":"Cognitive Systems Research."},{"issue":"3","key":"pone.0342305.ref039","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1016\/0005-1098(89)90002-2","article-title":"Model predictive control: theory and practice\u2014a survey","volume":"25","author":"CE Garc\u00eda","year":"1989","journal-title":"Automatica."},{"issue":"5","key":"pone.0342305.ref040","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1002\/wcs.142","article-title":"Predictive coding","volume":"2","author":"Y Huang","year":"2011","journal-title":"Wiley Interdiscip Rev Cogn Sci."},{"issue":"2","key":"pone.0342305.ref041","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1037\/0096-3445.124.2.207","article-title":"Costs of a predictible switch between simple cognitive tasks.","volume":"124","author":"RD Rogers","year":"1995","journal-title":"Journal of Experimental Psychology: General."},{"issue":"2","key":"pone.0342305.ref042","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.ijhcs.2011.10.003","article-title":"Juggling on a high wire: multitasking effects on performance","volume":"70","author":"RF Adler","year":"2012","journal-title":"International Journal of Human-Computer Studies."},{"issue":"3","key":"pone.0342305.ref043","doi-asserted-by":"crossref","first-page":"284","DOI":"10.1007\/s42113-020-00093-9","article-title":"Hierarchical reinforcement learning explains task interleaving behavior","volume":"4","author":"C Gebhardt","year":"2020","journal-title":"Comput Brain Behav."},{"key":"pone.0342305.ref044","volume-title":"Markov decision processes: discrete stochastic dynamic programming","author":"ML Puterman","year":"2014"},{"key":"pone.0342305.ref045","unstructured":"\u00d6sterdiekhoff A, Heinrich NW, Russwinkel N, Kopp S. Sense of control in dynamic multitasking and its impact on voluntary task-switching behavior. In: Proceedings of the Annual Meeting of the Cognitive Science Society, 2024. https:\/\/escholarship.org\/uc\/item\/43g8p1s3"},{"issue":"7","key":"pone.0342305.ref046","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1038\/2870","article-title":"Central cancellation of self-produced tickle sensation","volume":"1","author":"SJ Blakemore","year":"1998","journal-title":"Nat Neurosci."},{"issue":"1404","key":"pone.0342305.ref047","doi-asserted-by":"crossref","first-page":"1771","DOI":"10.1098\/rstb.2000.0734","article-title":"Abnormalities in the awareness and control of action","volume":"355","author":"CD Frith","year":"2000","journal-title":"Philos Trans R Soc Lond B Biol Sci."},{"issue":"2","key":"pone.0342305.ref048","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1016\/j.concog.2008.03.004","article-title":"The \u201csense of agency\u201d and its underlying cognitive and neural mechanisms","volume":"17","author":"N David","year":"2008","journal-title":"Conscious Cogn."},{"key":"pone.0342305.ref049","doi-asserted-by":"crossref","first-page":"1272","DOI":"10.3389\/fpsyg.2016.01272","article-title":"What is the sense of agency and why does it matter?","volume":"7","author":"JW Moore","year":"2016","journal-title":"Front Psychol."},{"issue":"1","key":"pone.0342305.ref050","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/j.concog.2011.08.010","article-title":"Sense of agency in health and disease: a review of cue integration approaches","volume":"21","author":"JW Moore","year":"2012","journal-title":"Conscious Cogn."},{"issue":"1","key":"pone.0342305.ref051","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1038\/4580","article-title":"Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects","volume":"2","author":"RP Rao","year":"1999","journal-title":"Nat Neurosci."},{"key":"pone.0342305.ref052","doi-asserted-by":"crossref","DOI":"10.1017\/9781009089517","volume-title":"Data-driven science and engineering: machine learning, dynamical systems, and control","author":"SL Brunton","year":"2022"},{"key":"pone.0342305.ref053","doi-asserted-by":"crossref","first-page":"104074","DOI":"10.1016\/j.cognition.2019.104074","article-title":"Prediction error and regularity detection underlie two dissociable mechanisms for computing the sense of agency","volume":"195","author":"W Wen","year":"2020","journal-title":"Cognition."},{"issue":"6","key":"pone.0342305.ref054","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/S1364-6613(02)01907-1","article-title":"Abnormalities in the awareness of action","volume":"6","author":"SJ Blakemore","year":"2002","journal-title":"Trends Cogn Sci."},{"key":"pone.0342305.ref055","doi-asserted-by":"crossref","first-page":"20","DOI":"10.17148\/IARJSET.2015.2305","article-title":"Normalization: a preprocessing stage","author":"SGK Patro","year":"2015","journal-title":"International Advanced Research Journal in Science, Engineering and Technology."},{"issue":"3","key":"pone.0342305.ref056","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1007\/s10994-021-05946-3","article-title":"Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods","volume":"110","author":"E H\u00fcllermeier","year":"2021","journal-title":"Mach Learn."},{"key":"pone.0342305.ref057","volume-title":"Python reference manual","author":"G Van Rossum","year":"1995"},{"key":"pone.0342305.ref058","article-title":"Gymnasium: a standard interface for reinforcement learning environments","author":"M Towers","year":"2024","journal-title":"arXiv preprint"},{"key":"pone.0342305.ref059","article-title":"Openai gym","author":"G Brockman","year":"2016","journal-title":"arXiv preprint"},{"key":"pone.0342305.ref060","first-page":"1","article-title":"Stable-baselines3: reliable reinforcement learning implementations","author":"A Raffin","year":"2021","journal-title":"Journal of Machine Learning Research."},{"key":"pone.0342305.ref061","article-title":"Proximal policy optimization algorithms","author":"J Schulman","year":"2017","journal-title":"arXiv preprint"},{"issue":"36","key":"pone.0342305.ref062","doi-asserted-by":"crossref","first-page":"1169","DOI":"10.21105\/joss.01169","article-title":"scikit-posthocs: pairwise multiple comparison tests in Python","volume":"4","author":"M Terpilowski","year":"2019","journal-title":"JOSS."},{"issue":"3","key":"pone.0342305.ref063","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: fundamental algorithms for scientific computing in Python","volume":"17","author":"P Virtanen","year":"2020","journal-title":"Nat Methods."},{"issue":"3","key":"pone.0342305.ref064","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/MCSE.2007.55","article-title":"Matplotlib: a 2D graphics environment","volume":"9","author":"JD Hunter","year":"2007","journal-title":"Comput Sci Eng."},{"key":"pone.0342305.ref065","volume-title":"Evaluating commercial game design decisions via the scientification of games: asymmetrical task switching costs predict self-reported fun in Ghost Blitz","author":"B Dyson","year":"2024"},{"issue":"901","key":"pone.0342305.ref066","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1098\/rspa.1937.0109","article-title":"Properties of sufficiency and statistical tests","volume":"160","year":"1937","journal-title":"Proc R Soc Lond A."},{"issue":"4","key":"pone.0342305.ref067","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1027\/1614-2241\/a000016","article-title":"Is it really robust?","volume":"6","author":"E Schmider","year":"2010","journal-title":"Methodology."},{"key":"pone.0342305.ref068","first-page":"65","volume-title":"Scandinavian Journal of Statistics","author":"S Holm","year":"1979"},{"key":"pone.0342305.ref069","first-page":"421","volume-title":"Attention and performance","author":"A Allport","year":"1994"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0342305","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,6]],"date-time":"2026-02-06T18:38:07Z","timestamp":1770403087000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0342305"}},"subtitle":[],"editor":[{"given":"Tomislav","family":"Jagu\u0161t","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,2,6]]},"references-count":69,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2,6]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0342305","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,6]]}}}