Selected Papers

This page contains a subset of the papers that were mentioned or discussed during the workshop.

Thank you to Esra'a Saleh for compiling the list.

Monday

• Mark Ring's Continual Learning Workshop Introduction

• Scaling Goal-based Exploration via Pruning Proto-goals

• The Bitter Lesson

• Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration

• Continual Auxiliary Task Learning

• The Lifecycle of Software Objects

• The Dynamics of Active Categorical Perception in an Evolved Model Agent

• Towards Continual Reinforcement Learning: A Review and Perspectives

• The Difficulty of Passive Learning in Deep Reinforcement Learning

• Agent57

• Human-level Atari 200x faster

• On the Role of Tracking in Stationary Environments

• Finite and Infinite Games

• Online Real-Time Recurrent Learning Using Sparse Connections and Selective Learning

Tuesday

• The Implicit Bias of Gradient Descent on Separable Data

• Neural Collapse: A Review on Modelling Principles and Generalization

• Representation Search through Generate and Test

• The Phenomenon of Policy Churn

• When to use parametric models in reinforcement learning?

• The Effect of Planning Shape on Dyna-style Planning in High-dimensional State Spaces

• Selective Dyna-style Planning Under Limited Model Capacity

• MuZero

• Value-Aware Loss Function for Model-based Reinforcement Learning

• Proper Value Equivalence

• Muesli

• Learning How to Infer Partial MDPs for In-Context Adaptation and Exploration

• Should Models Be Accurate?

• Dreamer V3

• On the role of planning in model-based deep reinforcement learning

• Meta-Gradient Reinforcement Learning

• A Self-Tuning Actor-Critic Algorithm

• Meta-Gradient Reinforcement Learning with an Objective Discovered Online

• Discovering Reinforcement Learning Algorithms

• Incremental Delta-Bar-Delta

• Bootstrapped Meta-Learning

• Meta-gradient in Non-stationary Environments

• Optimistic Meta-Gradients

• Learning to Prioritize Planning Updates in Model-based Reinforcement Learning

• Natural Value Approximators: Learning when to Trust Past Estimates

• Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability

• Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability

• Rapid Training of Deep Neural Networks without Skip Connections or Normalization Layers Using Deep Kernel Shaping

• On Warm-Starting Neural Network Training

• Mean Field Residual Networks: On the Edge of Chaos

• The Large Learning Rate Phase of Deep Learning: The Catapult Mechanism

• Reward is enough for convex MDPs

• Successor Features for Transfer in Reinforcement Learning

• Fast Reinforcement Learning with Generalized Policy Updates

• The LoCA Regret: A Consistent Metric to Evaluate Model-Based Behavior in Reinforcement Learning

• Towards Evaluating Adaptivity of Model-Based Reinforcement Learning Methods

• Replay Buffer With Local Forgetting for Adaptive Deep Model-Based Reinforcement Learning

• Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimality

• ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPs

• Expected Eligibility Traces

• Interference and Generalization in Temporal Difference Learning

Wednesday

• Natural Neural Networks

• Training Uncertainty-Aware Classifiers with Conformalized Deep Learning

• Learning Values Across Many Orders of Magnitude

• Rich's Gaps in the Foundations of Planning with Approximation€ talk - what he wanted to say about MBRL

• The Quest for a Common Model of the Intelligent Decision Maker

• Mark's "CHILD" 1997 paper - CHILD: A First Step Towards Continual Learning

• Understanding Human Intelligence through Human Limitations

• Ray Interference: a Source of Plateaus in Deep Reinforcement Learning

• Deep Double Descent

• Interlude: A Mechanistic Interpretability Analysis of Grokking — Neel Nanda

• Gradient-based Optimization is Not Necessary for Generalization in Neural Networks

• Transient Non-Stationarity and Generalisation in Deep Reinforcement Learning

• A Deep Neural Network's Loss Surface Contains Every Low-dimensional Pattern

• Memory-based Parameter Adaptation

• Conjugate Markov Decision Processes

• Policy Gradient Coagent Networks

• On Catastrophic Interference in Atari 2600 Games

• The Primacy Bias in Deep Reinforcement Learning

• Sample-Efficient Reinforcement Learning by Breaking the Replay Ratio Barrier

Thursday

• Discovering Reinforcement Learning Algorithms

• Blog post on XLand

• Human-Timescale Adaptation in an Open-Ended Task Space

• A Survey of Meta-Reinforcement Learning

• Luisa's thesis about fast adaptation in RL

• Continual Learning In Environments With Polynomial Mixing Times

• Markov Chains

• Steady State Analysis of Episodic Reinforcement Learning

• Passive setting: Optimizing for the Future in Non-Stationary MDPs

• Active setting: Off-Policy Evaluation for Action-Dependent Non-stationary Environments

• Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer

• Weight utility reference - Skeletonization : A Technique for Trimming the Fat from a Network Via Relevance Assessment

• Utility-based Perturbed Gradient Descent: An Optimizer for Continual Learning

• References for Shapley values. [Chapter in the Interpretable ML book] and [The Many Shapley Values for Model Explanation]