{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T15:36:42Z","timestamp":1783438602635,"version":"3.54.6"},"reference-count":121,"publisher":"Emerald","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,8,26]]},"abstract":"<jats:p>An agent that accumulates knowledge to develop increasingly sophisticated skills over a long lifetime could advance the frontier of artificial intelligence capabilities. The design of such agents, which remains a long-standing challenge, is addressed by the subject of continual learning. This monograph clarifies and formalizes concepts of continual learning, introducing a framework and tools to stimulate further research. We also present a range of empirical case studies to illustrate the roles of forgetting, relearning, exploration, and auxiliary learning.<\/jats:p>\n                  <jats:p>Metrics presented in previous literature for evaluating continual learning agents tend to focus on particular behaviors that are deemed desirable, such as avoiding catastrophic forgetting, retaining plasticity, relearning quickly, and maintaining low memory or compute footprints. In order to systematically reason about design choices and compare agents, a coherent, holistic objective that encompasses all such requirements would be helpful. To provide such an objective, we cast continual learning as reinforcement learning with limited compute resources. In particular, we pose the continual learning objective to be the maximization of infinite-horizon average reward subject to a computational constraint. Continual supervised learning, for example, is a special case of our general formulation where the reward is taken to be negative log-loss or accuracy. Among the implications of maximizing average reward are that remembering all information from the past is unnecessary, forgetting nonrecurring information is not \u201ccatastrophic,\u201d and learning about how an environment changes over time is useful.<\/jats:p>\n                  <jats:p>Computational constraints give rise to informational constraints in the sense that they limit the amount of information used to make decisions. A consequence is that, unlike in more common framings of machine learning in which per-timestep regret vanishes as an agent accumulates information, the regret experienced in continual learning typically persists. Related to this is that even in stationary environments, informational constraints can incentivize perpetual adaptation. Informational constraints also give rise to the familiar stability-plasticity dilemma, which we formalize in information-theoretic terms.<\/jats:p>","DOI":"10.1561\/2200000116","type":"journal-article","created":{"date-parts":[[2025,8,20]],"date-time":"2025-08-20T08:18:37Z","timestamp":1755677917000},"page":"913-1053","source":"Crossref","is-referenced-by-count":2,"title":["Continual Learning as Computationally Constrained Reinforcement Learning"],"prefix":"10.1108","volume":"18","author":[{"given":"Saurabh","family":"Kumar","sequence":"first","affiliation":[{"name":"Stanford University Department of Computer Science, ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Henrik","family":"Marklund","sequence":"additional","affiliation":[{"name":"Stanford University Department of Computer Science, ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ashish","family":"Rao","sequence":"additional","affiliation":[{"name":"Stanford University Department of Computer Science, ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yifan","family":"Zhu","sequence":"additional","affiliation":[{"name":"Stanford University Department of Electrical Engineering, ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hong Jun","family":"Jeon","sequence":"additional","affiliation":[{"name":"Stanford University Department of Computer Science, ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liu","family":"Yueyang","sequence":"additional","affiliation":[{"name":"Jones Graduate School of Business, Rice University ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin","family":"Van Roy","sequence":"additional","affiliation":[{"name":"Stanford University Department of Electrical Engineering, ,","place":["USA"]},{"name":"Stanford University Department of Management Science and Engineering, ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"140","published-online":{"date-parts":[[2025,8,26]]},"reference":[{"key":"2026041607013517100_ref001","doi-asserted-by":"crossref","DOI":"10.52202\/075280-2192","volume-title":"A Definition of Continual Reinforcement Learning","author":"Abel","year":"2023"},{"key":"2026041607013517100_ref002","first-page":"10","article-title":"Reinforcement learning: Theory and algorithms","volume-title":"CS Dept., UW Seattle, Seattle, WA, USA, Tech. Rep:","author":"Agarwal","year":"2019"},{"key":"2026041607013517100_ref003","first-page":"32","article-title":"Gradient based sample selection for online continual learning","volume-title":"Advances in neural information processing systems","author":"Aljundi","year":"2019"},{"key":"2026041607013517100_ref004","first-page":"373","volume-title":"Deciding what to learn: A rate-distortion approach","author":"Arumugam","year":"2021"},{"key":"2026041607013517100_ref005","first-page":"9816","article-title":"The value of information when deciding what to learn","volume":"34","author":"Arumugam","year":"2021","journal-title":"Advances in neural information processing systems"},{"key":"2026041607013517100_ref006","volume-title":"Does the Adam Optimizer Exacerbate Catastrophic Forgetting?","author":"Ashley","year":"2021"},{"key":"2026041607013517100_ref007","first-page":"138","volume-title":"Adaptively tracking the best bandit arm with an unknown number of distribution changes","author":"Auer","year":"2019"},{"key":"2026041607013517100_ref008","first-page":"36","article-title":"Online label shift: Optimal dynamic regret meets practical algorithms","volume-title":"Advances in Neural Information Processing Systems","author":"Baby","year":"2024"},{"key":"2026041607013517100_ref009","volume-title":"Beyond Supervised Continual Learning: a Review","author":"Bagus","year":"2022"},{"key":"2026041607013517100_ref010","volume-title":"Neuro-dynamic programming","author":"Bertsekas","year":"1996"},{"key":"2026041607013517100_ref011","first-page":"27","article-title":"Stochastic multi-armed-bandit problem with non-stationary rewards","volume-title":"Advances in neural information processing systems","author":"Besbes","year":"2014"},{"issue":"5","key":"2026041607013517100_ref012","doi-asserted-by":"crossref","first-page":"1227","DOI":"10.1287\/opre.2015.1408","article-title":"Non-stationary stochastic optimization","volume":"63","author":"Besbes","year":"2015","journal-title":"Operations research"},{"issue":"4","key":"2026041607013517100_ref013","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1287\/stsy.2019.0033","article-title":"Optimal Exploration-Exploitation in a Multi-Armed-Bandit Problem with Non-Stationary Rewards","volume":"9","author":"Besbes","year":"2019","journal-title":"Stochastic Systems"},{"key":"2026041607013517100_ref014","first-page":"35","article-title":"The generalized likelihood ratio test meets KLUCB: an improved algorithm for piece-wise non-stationary bandits","volume":"1","author":"Besson","year":"2019","journal-title":"Proceedings of Machine Learning Research vol XX"},{"key":"2026041607013517100_ref015","first-page":"314","volume-title":"Artificial Intelligence and Statistics","author":"Bogunovic","year":"2016"},{"issue":"3","key":"2026041607013517100_ref016","doi-asserted-by":"crossref","first-page":"809","DOI":"10.1214\/aoms\/1177706899","article-title":"The individual ergodic theorem of information theory","volume":"28","author":"Breiman","year":"1957","journal-title":"The Annals of Mathematical Statistics"},{"key":"2026041607013517100_ref017","first-page":"1877","volume-title":"Advances in Neural Information Processing Systems","author":"Brown","year":"2020"},{"key":"2026041607013517100_ref018","first-page":"15920","article-title":"Dark experience for general continual learning: a strong, simple baseline","volume":"33","author":"Buzzega","year":"2020","journal-title":"Advances in neural information processing systems"},{"key":"2026041607013517100_ref019","first-page":"8281","volume-title":"Online continual learning with natural distribution shifts: An empirical study with visual data","author":"Cai","year":"2021"},{"key":"2026041607013517100_ref020","volume-title":"Continual learning with tiny episodic memories","author":"Chaudhry","year":"2019"},{"key":"2026041607013517100_ref021","doi-asserted-by":"crossref","first-page":"14784","DOI":"10.52202\/068431-1075","article-title":"You Only Live Once: Single-Life Reinforcement Learning","volume":"35","author":"Chen","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026041607013517100_ref022","first-page":"7895","article-title":"Non-stationary bandits with auto-regressive temporal dependency","volume":"36","author":"Chen","year":"2023","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026041607013517100_ref023","first-page":"696","volume-title":"A new algorithm for non-stationary contextual bandits: Efficient, optimal and parameter-free","author":"Chen","year":"2019"},{"key":"2026041607013517100_ref024","first-page":"1079","article-title":"Learning to optimize under non-stationarity","author":"Cheung","year":"2019"},{"key":"2026041607013517100_ref025","first-page":"1952","volume-title":"Online continual learning from imbalanced data","author":"Chrysakis","year":"2020"},{"key":"2026041607013517100_ref026","volume-title":"Elements of Information Theory","author":"Cover","year":"2006","edition":"2nd"},{"key":"2026041607013517100_ref027","volume-title":"Elements of Information Theory","author":"Cover","year":"2012"},{"key":"2026041607013517100_ref028","first-page":"1405","article-title":"Strongly adaptive online learning","author":"Daniely","year":"2015"},{"key":"2026041607013517100_ref029","first-page":"213","volume-title":"Q-learning for history-based reinforcement learning","author":"Daswani","year":"2013"},{"key":"2026041607013517100_ref030","article-title":"Feature reinforcement learning: state of the art","author":"Daswani","year":"2014"},{"issue":"3","key":"2026041607013517100_ref031","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1287\/moor.13.3.395","article-title":"Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards","volume":"13","author":"Dekker","year":"1988","journal-title":"Mathematics of Operations Research"},{"key":"2026041607013517100_ref032","volume-title":"Average-reward model-free reinforcement learning: a systematic review and literature mapping","author":"Dewanto","year":"2020"},{"key":"2026041607013517100_ref033","first-page":"115","article-title":"de Finetti\u2019s theorem for Markov chains","volume-title":"The Annals of Probability","author":"Diaconis","year":"1980"},{"key":"2026041607013517100_ref034","volume-title":"Continual backprop: Stochastic gradient descent with persistent randomness","author":"Dohare","year":"2021"},{"key":"2026041607013517100_ref035","first-page":"3538","volume-title":"A kernel-based approach to non-stationary reinforcement learning in metric spaces","author":"Domingues","year":"2021"},{"issue":"255","key":"2026041607013517100_ref036","first-page":"1","article-title":"Simple agent, complex environment: Efficient reinforcement learning with agent states","volume":"23","author":"Dong","year":"2022","journal-title":"Journal of Machine Learning Research"},{"key":"2026041607013517100_ref037","volume-title":"Rl2: Fast reinforcement learning via slow reinforcement learning","author":"Duan","year":"2016"},{"key":"2026041607013517100_ref038","first-page":"6743","article-title":"Dynamic regret of policy optimization in non-stationary environments","volume":"33","author":"Fei","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026041607013517100_ref039","first-page":"179","article-title":"Funzione caratteristica di un fenomeno aleatorio","volume-title":"Atti del Congresso Internazionale dei Matematici: Bologna del 3 al 10 de settembre di","author":"de Finetti","year":"1929"},{"key":"2026041607013517100_ref040","first-page":"720","volume-title":"Online continual learning under extreme memory constraints","author":"Fini","year":"2020"},{"key":"2026041607013517100_ref041","volume-title":"Bootstrapped meta-learning","author":"Flennerhag","year":"2021"},{"key":"2026041607013517100_ref042","first-page":"2995","volume-title":"On the problem of on-line learning with log-loss","author":"Fogel","year":"2017"},{"key":"2026041607013517100_ref043","volume-title":"Retrieval-augmented generation for large language models: A survey","author":"Gao","year":"2023"},{"key":"2026041607013517100_ref044","volume-title":"On upper-confidence bound policies for non-stationary bandit problems","author":"Garivier","year":"2008"},{"issue":"10","key":"2026041607013517100_ref045","doi-asserted-by":"publisher","first-page":"1670","DOI":"10.1109\/TC.2020.3022634","article-title":"A change-detection-based Thompson sampling framework for non-Stationary bandits","volume":"70","author":"Ghatak","year":"2021","journal-title":"IEEE Transactions on Computers"},{"key":"2026041607013517100_ref046","volume-title":"Real-time evaluation in online continual learning: A new paradigm","author":"Ghunaim","year":"2023"},{"key":"2026041607013517100_ref047","volume-title":"An empirical investigation of catastrophic forgetting in gradient-based neural networks","author":"Goodfellow","year":"2013"},{"key":"2026041607013517100_ref048","first-page":"484","volume-title":"Thompson sampling for dynamic multi-armed bandits","author":"Gupta","year":"2011"},{"issue":"12","key":"2026041607013517100_ref049","doi-asserted-by":"crossref","first-page":"1028","DOI":"10.1016\/j.tics.2020.09.004","article-title":"Embracing Change: Continual Learning in Deep Neural Networks","volume":"24","author":"Hadsell","year":"2020","journal-title":"Trends in Cognitive Sciences"},{"key":"2026041607013517100_ref050","volume-title":"Rapid Adaptation in Online Continual Learning: Are We Evaluating It Right?","author":"Hammoud","year":"2023"},{"key":"2026041607013517100_ref051","volume-title":"Multi-armed bandit, dynamic environments and meta-bandits","author":"Hartland","year":"2006"},{"key":"2026041607013517100_ref052","volume-title":"Distilling the Knowledge in a Neural Network","author":"Hinton","year":"2015"},{"key":"2026041607013517100_ref053","volume-title":"Training Compute-Optimal Large Language Models","author":"Hoffmann","year":"2022"},{"key":"2026041607013517100_ref054","doi-asserted-by":"crossref","DOI":"10.1109\/TPAMI.2022.3218265","article-title":"Drinking from a firehose: Continual learning with web-scale natural language","volume-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence","author":"Hu","year":"2022"},{"key":"2026041607013517100_ref055","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1007\/978-3-540-68677-4_8","volume-title":"Artificial General Intelligence","author":"Hutter","year":"2007"},{"key":"2026041607013517100_ref056","volume-title":"Information-theoretic foundations for machine learning","author":"Jeon","year":"2024"},{"key":"2026041607013517100_ref057","volume-title":"An Information-Theoretic Framework for Supervised Learning","author":"Jeon","year":"2023"},{"key":"2026041607013517100_ref058","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A New Approach to Linear Filtering and Prediction Problems","volume":"82","author":"Kalman","year":"1960","journal-title":"Transactions of the ASME\u2013Journal of Basic Engineering"},{"key":"2026041607013517100_ref059","doi-asserted-by":"crossref","first-page":"1401","DOI":"10.1613\/jair.1.13673","article-title":"Towards continual reinforcement learning: A review and perspectives","volume":"75","author":"Khetarpal","year":"2022","journal-title":"Journal of Artificial Intelligence Research"},{"issue":"13","key":"2026041607013517100_ref060","doi-asserted-by":"crossref","first-page":"3521","DOI":"10.1073\/pnas.1611835114","article-title":"Overcoming catastrophic forgetting in neural networks","volume":"114","author":"Kirkpatrick","year":"2017","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"2026041607013517100_ref061","first-page":"51","volume-title":"Discounted UCB","author":"Kocsis","year":"2006"},{"key":"2026041607013517100_ref062","volume-title":"Online Boundary-Free Continual Learning by Scheduled Data Prior","author":"Koh","year":"2023"},{"key":"2026041607013517100_ref063","volume-title":"Foundations of the theory of probability: Second English Edition","author":"Kolmogorov","year":"2018"},{"issue":"4","key":"2026041607013517100_ref064","doi-asserted-by":"publisher","DOI":"10.4108\/icst.valuetools.2014.258207","article-title":"Exploration vs Exploitation with Partially Observable Gaussian Autoregressive Arms","volume":"1","author":"Kuhn","year":"2015","journal-title":"EAI Endorsed Transactions on Self-Adaptive Systems"},{"key":"2026041607013517100_ref065","article-title":"Wireless channel selection with reward-observing restless multi-armed bandits","volume-title":"Markov Decision Processes in Practice","author":"Kuhn","year":"2015"},{"key":"2026041607013517100_ref066","doi-asserted-by":"crossref","DOI":"10.1017\/9781108571401","volume-title":"Bandit algorithms","author":"Lattimore","year":"2020"},{"key":"2026041607013517100_ref067","volume-title":"Continual learning: Tackling catastrophic forgetting in deep neural networks with replay processes","author":"Lesort","year":"2020"},{"key":"2026041607013517100_ref068","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive nlp tasks","volume":"33","author":"Lewis","year":"2020","journal-title":"Advances in neural information processing systems"},{"key":"2026041607013517100_ref069","volume-title":"The clear benchmark: Continual learning on real-world imagery","author":"Lin","year":"2021"},{"key":"2026041607013517100_ref070","volume-title":"A Definition of Non-Stationary Bandits","author":"Liu","year":"2023"},{"key":"2026041607013517100_ref071","first-page":"6215","volume-title":"Nonstationary bandit learning via predictive sampling","author":"Liu","year":"2023"},{"issue":"6","key":"2026041607013517100_ref072","doi-asserted-by":"crossref","first-page":"733","DOI":"10.1561\/2200000097","article-title":"Reinforcement Learning, Bit by Bit","volume":"16","author":"Lu","year":"2023","journal-title":"Foundations and Trends\u00ae in Machine Learning"},{"key":"2026041607013517100_ref073","first-page":"1739","volume-title":"Efficient contextual bandits in non-stationary worlds","author":"Luo","year":"2018"},{"key":"2026041607013517100_ref074","volume-title":"Understanding plasticity in neural networks","author":"Lyle","year":"2023"},{"key":"2026041607013517100_ref075","first-page":"387","volume-title":"Instance-based utile distinctions for reinforcement learning with hidden state","author":"McCallum","year":"1995"},{"key":"2026041607013517100_ref076","first-page":"109","volume-title":"Psychology of Learning and Motivation","author":"McCloskey","year":"1989"},{"key":"2026041607013517100_ref077","first-page":"196","article-title":"The basic theorems of information theory","volume-title":"The Annals of mathematical statistics:","author":"McMillan","year":"1953"},{"key":"2026041607013517100_ref078","first-page":"442","volume-title":"Thompson Sampling in Switching Environments with Bayesian Online Change Detection","author":"Mellor","year":"2013"},{"key":"2026041607013517100_ref079","doi-asserted-by":"crossref","DOI":"10.1017\/9781009051873","volume-title":"Control systems and reinforcement learning","author":"Meyn","year":"2022"},{"key":"2026041607013517100_ref080","first-page":"24831","volume-title":"An information-theoretic analysis of nonstationary bandit learning","author":"Min","year":"2023"},{"key":"2026041607013517100_ref081","first-page":"15699","volume-title":"Wide neural networks forget less catastrophically","author":"Mirzadeh","year":"2022"},{"issue":"1","key":"2026041607013517100_ref082","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1016\/j.patcog.2011.06.019","article-title":"A unifying view on dataset shift in classification","volume":"45","author":"Moreno-Torres","year":"2012","journal-title":"Pattern recognition"},{"key":"2026041607013517100_ref083","volume-title":"Deep Reinforcement Learning with Plasticity Injection","author":"Nikishin","year":"2023"},{"issue":"6","key":"2026041607013517100_ref084","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3459991","article-title":"A survey of reinforcement learning algorithms for dynamically varying environments","volume":"54","author":"Padakandla","year":"2021","journal-title":"ACM Computing Surveys (CSUR)"},{"key":"2026041607013517100_ref085","volume-title":"Online continual learning without the storage constraint","author":"Prabhu","year":"2023"},{"key":"2026041607013517100_ref086","volume-title":"Computationally Budgeted Continual Learning: What Does Matter?","author":"Prabhu","year":"2023"},{"key":"2026041607013517100_ref087","volume-title":"Scaling Language Models: Methods, Analysis & Insights from Training Gopher","author":"Rae","year":"2022"},{"key":"2026041607013517100_ref088","volume-title":"Taming non-stationary bandits: A Bayesian approach","author":"Raj","year":"2017"},{"issue":"2","key":"2026041607013517100_ref089","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1037\/0033-295X.97.2.285","article-title":"Connectionist models of recognition memory: constraints imposed by learning and forgetting functions","volume":"97","author":"Ratcliff","year":"1990","journal-title":"Psychological review"},{"key":"2026041607013517100_ref090","volume-title":"Toward a formal framework for continual learning","author":"Ring","year":"2005"},{"key":"2026041607013517100_ref091","volume-title":"Continual Learning in Reinforcement Environments","author":"Ring","year":"1994"},{"key":"2026041607013517100_ref092","volume-title":"Artificial intelligence: a modern approach","author":"Russell","year":"2016"},{"issue":"1","key":"2026041607013517100_ref093","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000070","article-title":"A tutorial on Thompson sampling","volume":"11","author":"Russo","year":"2018","journal-title":"Foundations and Trends\u00ae in Machine Learning"},{"issue":"3","key":"2026041607013517100_ref094","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"The Bell system technical journal"},{"key":"2026041607013517100_ref095","volume-title":"Autonomous reinforcement learning: Formalism and benchmarking","author":"Sharma","year":"2021"},{"key":"2026041607013517100_ref096","volume-title":"Outrageously large neural networks: The sparsely-gated mixture-of-experts layer","author":"Shazeer","year":"2017"},{"key":"2026041607013517100_ref097","first-page":"753","volume-title":"Algorithmic Learning Theory","author":"Shkel","year":"2018"},{"key":"2026041607013517100_ref098","first-page":"343","article-title":"Adapting to a Changing Environment: the Brownian Restless Bandits","volume-title":"COLT","author":"Slivkins","year":"2008"},{"key":"2026041607013517100_ref099","volume-title":"Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model","author":"Smith","year":"2022"},{"key":"2026041607013517100_ref100","first-page":"69","article-title":"VC dimension of neural networks","volume":"168","author":"Sontag","year":"1998","journal-title":"NATO ASI Series F Computer and Systems Sciences"},{"key":"2026041607013517100_ref101","first-page":"216","volume-title":"Integrated architectures for learning, planning, and reacting based on approximating dynamic programming","author":"Sutton","year":"1990"},{"key":"2026041607013517100_ref102","volume-title":"Reinforcement learning: An introduction","author":"Sutton","year":"2018"},{"key":"2026041607013517100_ref103","first-page":"871","volume-title":"On the role of tracking in stationary environments","author":"Sutton","year":"2007"},{"key":"2026041607013517100_ref104","first-page":"171","volume-title":"Adapting Bias by Gradient Descent: An Incremental Version of Delta-Bar-Delta","author":"Sutton","year":"1992"},{"key":"2026041607013517100_ref105","volume-title":"The Bitter Lesson","author":"Sutton","year":"2019"},{"issue":"1","key":"2026041607013517100_ref106","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/978-3-031-01551-9","article-title":"Algorithms for reinforcement learning","volume":"4","author":"Szepesv\u00e1ri","year":"2010","journal-title":"Synthesis lectures on artificial intelligence and machine learning"},{"issue":"10","key":"2026041607013517100_ref107","doi-asserted-by":"crossref","first-page":"364","DOI":"10.3390\/e18100364","article-title":"Entropy rate estimates for natural language\u2014A new extrapolation of compressed large-scale corpora","volume":"18","author":"Takahira","year":"2016","journal-title":"Entropy"},{"issue":"3-4","key":"2026041607013517100_ref108","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1093\/biomet\/25.3-4.285","article-title":"On the likelihood that one unknown probability exceeds another in view of the evidence of two samples","volume":"25","author":"Thompson","year":"1933","journal-title":"Biometrika"},{"key":"2026041607013517100_ref109","volume-title":"LaMDA: Language Models for Dialog Applications","author":"Thoppilan","year":"2022"},{"key":"2026041607013517100_ref110","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-1-4615-5529-2_1","article-title":"Learning to learn: Introduction and overview","volume-title":"Learning to learn:","author":"Thrun","year":"1998"},{"key":"2026041607013517100_ref111","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1613\/jair.1.11407","article-title":"Sliding-window Thompson sampling for non-stationary settings","volume":"68","author":"Trovo","year":"2020","journal-title":"Journal of Artificial Intelligence Research"},{"key":"2026041607013517100_ref112","doi-asserted-by":"crossref","DOI":"10.1117\/12.2624855","article-title":"Lifelong learning for robust AI systems","volume-title":"Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications IV","author":"Vallabha","year":"2022"},{"key":"2026041607013517100_ref113","first-page":"29","article-title":"Metagrad: Multiple learning rates in online learning","volume-title":"Advances in Neural Information Processing Systems","author":"Van Erven","year":"2016"},{"key":"2026041607013517100_ref114","first-page":"399","volume-title":"Thompson sampling for Bayesian bandits with resets","author":"Viappiani","year":"2013"},{"issue":"1","key":"2026041607013517100_ref115","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1145\/3147.3165","article-title":"Random sampling with a reservoir","volume":"11","author":"Vitter","year":"1985","journal-title":"ACM Transactions on Mathematical Software (TOMS)"},{"key":"2026041607013517100_ref116","volume-title":"A comprehensive survey of continual learning: Theory, method and application","author":"Wang","year":"2023"},{"key":"2026041607013517100_ref117","volume-title":"Learning from delayed rewards","author":"Watkins","year":"1989"},{"key":"2026041607013517100_ref118","first-page":"96","article-title":"Adaptive switching circuits","volume":"4","author":"Widrow","year":"1960","journal-title":"IRE WESCON Convention Record"},{"key":"2026041607013517100_ref119","first-page":"1","volume-title":"Revealing the real-world applicable setting of online continual learning","author":"Xu","year":"2022"},{"key":"2026041607013517100_ref120","volume-title":"Online coreset selection for rehearsal-based continual learning","author":"Yoon","year":"2021"},{"key":"2026041607013517100_ref121","first-page":"31","article-title":"Adaptive online learning in dynamic environments","volume-title":"Advances in neural information processing systems","author":"Zhang","year":"2018"}],"container-title":["Foundations and Trends\u00ae in Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/ftmal\/article-pdf\/18\/5\/913\/11521539\/2200000116en.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/www.emerald.com\/ftmal\/article-pdf\/18\/5\/913\/11521539\/2200000116en.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T18:10:47Z","timestamp":1777486247000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.emerald.com\/ftmal\/article\/18\/5\/913\/1332834\/Continual-Learning-as-Computationally-Constrained"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,26]]},"references-count":121,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,8,26]]}},"URL":"https:\/\/doi.org\/10.1561\/2200000116","relation":{},"ISSN":["1935-8237","1935-8245"],"issn-type":[{"value":"1935-8237","type":"print"},{"value":"1935-8245","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,26]]}}}