Pith. sign in

REVIEW 3 major objections 6 minor 16 references

Capacity-Constrained Continual Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read Under a capacity constraint that limits the mutual information between an agent's internal state and its observation history to $B$ bits, the optimal LQG predictor is a linear Gaussian shrinkage of the Kalman estimate, $F_t \bar{\theta}_t…

desk verdict A clean, correct solution to the relaxed capacity-constrained LQG prediction problem, but the continual-learning claim is only conditional and the paper is honest about it. read the letter →

arxiv 2507.21479 v1 pith:2IZ33IPT submitted 2025-07-29 cs.LG cs.AIcs.ITcs.SYeess.SYmath.ITstat.ML

classification cs.LGcs.AIcs.ITcs.SYeess.SYmath.ITstat.ML MSC 93E1194A17
keywords continuallearningcapacity-constrainedagentlinearquadraticGaussiansequentialpredictioncapacityallocationmutualinformationKalmanfilterrate-distortion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Any agent that can actually be built has finite memory and compute, and the paper's question is how such an agent should spend its limited capacity. It formalizes the limit as a mutual-information budget: at every time, the agent's internal state may carry at most $B$ bits of information about the observation history, $I(S_t;H_t)\leq B$. For the capacity-constrained linear-quadratic-Gaussian sequential prediction problem, the paper derives an exact optimal solution: the best predictor is a linearly shrunk Kalman estimate plus controlled Gaussian noise, with the per-direction shrinkage and noise set by the Gaussian distortion-rate function. It then gives sufficient conditions for this predictor to be implemented by incremental updates, obtains the steady-state cost, and shows how to allocate a total bit budget optimally across independent subsystems. The payoff is a concrete, closed-form description of what an optimal continual learner should forget.

What carries the argument

The engine of the argument is the Gaussian distortion-rate function $D(B,\Sigma)$, the minimum mean-square distortion at which a Gaussian vector with covariance $\Sigma$ can be described in $B$ bits. At each time $t$ it is applied to $\mathrm{Cov}(\bar{\theta}_t)=\Sigma-M_t$: with eigenvalues $\lambda_{t1},\ldots,\lambda_{td}$, the water-filling equation $\sum_i [\log(2\lambda_{ti}/\eta_t)]_+ = 2B$ fixes per-direction bit budgets $B_{ti}=\frac{1}{2}[\log(2\lambda_{ti}/\eta_t)]_+$, and from those budgets the optimal predictor is built as $F_t=U_t\,\mathrm{diag}(1-\exp(-2B_{ti}))\,U_t^\top$ and $\Psi_t=U_t\,\mathrm{diag}([1-\exp(-2B_{ti})]\exp(-2B_{ti})\lambda_{ti})\,U_t^\top$. Kalman filtering supplies the posterior estimate $\bar{\theta}_t$ and its covariance recursion, and the paper's sufficient conditions are exactly the inequalities that let the next optimal prediction $\hat{\theta}_{t+1}=F_{t+1}\bar{\theta}_{t+1}+\varepsilon_{t+1}$ be re-expressed as a linear function of $\hat{\theta}_t$ and $Y_{t+1}$ plus fresh independent Gaussian noise.

What would settle it

Simulate the scalar LQG system with a hard memory limit---an internal state allowed only $2^B$ distinct values---for a small capacity such as $B=1$, and compare the minimal average quadratic prediction error found by exhaustive search over quantized state updates with the paper's mutual-information cost $M+\exp(-2B)(\Sigma-M)$ from Corollary 9; a gap at small $B$ would show the mutual-information surrogate is not tight.

Watch

Extended reading notes

Core claim

Under the capacity constraint $I(S_t;H_t)\leq B$, the paper's central claim is that one optimal predictor for the LQG problem is the linear Gaussian rule $\hat{\theta}_t = F_t \bar{\theta}_t + \varepsilon_t$, where $\bar{\theta}_t$ is the Kalman-filter estimate $E[\theta_t|H_t]$, $F_t$ shrinks each eigen-direction of $\mathrm{Cov}(\bar{\theta}_t)$ by the factor $1-\exp(-2B_{ti})$, and $\varepsilon_t$ is independent Gaussian noise with covariance $\Psi_t$; both $F_t$ and $\Psi_t$ are computed from the Gaussian distortion-rate function via the water-filling equation $\sum_{i=1}^d [\log(2\lambda_{ti}/\eta_t)]_+ = 2B$. For the continual-learning version that requires incremental state updates, Theorem 3 supplies sufficient conditions---$F_t$ invertible and the matrix inequality (14)---under which the same relaxed-problem solution can be realized by a linear Gaussian agent with no increase in total cost, and Theorem 6 gives analogous steady-state conditions. The steady-state optimal asymptotic cost is $\mathrm{tr}(M)+\sum_{i=1}^d \exp(-2B_i)\lambda_i$, which reduces to the Kalman-filter cost $\mathrm{tr}(M)$ at $B=\infty$ and to the no-information cost $\mathrm{tr}(\Sigma)$ at $B=0$. For systems that decompose into independent subsystems, the paper reduces optimal capacity allocation to a strictly convex water-filling problem and illustrates the resulting bit allocations numerically.

Load-bearing premise

The load-bearing premise is that capping the mutual information between an agent's state and its observation history at $B$ bits accurately captures what it means for the agent to have a $B$-bit memory; if that information-theoretic surrogate is not faithful to a real memory limit, the derived optimal predictors solve a relaxation rather than the motivating problem.

Editorial extensions

If this is right

  • At infinite capacity the steady-state formula reduces to the classical Kalman-filter cost $\mathrm{tr}(M)$, and at zero capacity to $\mathrm{tr}(\Sigma)$, so the capacity-constrained solution interpolates continuously between the two classical extremes.
  • For the scalar case, the sufficient conditions of Theorem 3 hold for every $B>0$, so the optimal continual-learning agent is provably incremental.
  • In diagonal systems, the optimal allocation of total capacity is the unique minimizer of a strictly convex problem, so bits should be distributed until the marginal costs $\exp(-2B_i)(\Sigma_i-M_i)$ are equal across active subsystems.
  • When the Theorem 3 conditions hold, the relaxed prediction problem and the incremental continual-learning problem have the same optimal total cost, meaning the incremental memory structure can be imposed for free.
  • In the block-diagonal numerical example, coupled two-dimensional subsystems receive more than twice the per-dimension capacity of independent one-dimensional subsystems with the same dynamics, and the ratio grows as total capacity shrinks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because only second-order statistics enter the LQG solution, the same water-filling allocation is a natural candidate for non-Gaussian autoregressive processes with matching covariance; testing it there would show whether the distortion-rate structure, not Gaussianity, is what drives the result.
  • The mutual-information constraint averages over histories, so a worst-case or hard $B$-bit state representation could be strictly more restrictive; a quantized-state version of the scalar LQG problem would reveal how large the gap is at practically relevant capacities.
  • The paper leaves open whether the Theorem 3 conditions are necessary; a numerical search over nonlinear stochastic state-update rules for examples where inequality (14) fails could settle whether the cost gap appears exactly at that boundary.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies an agent that makes sequential linear-Gaussian predictions under an information-capacity constraint I(S_t;H_t) ≤ B. It first solves a relaxed problem, C2P-LQG, in which the incremental-update constraint is dropped, and obtains an optimal predictor θ̂_t = F_t θ̄_t + ε_t via Gaussian rate-distortion theory (Theorem 2). It then gives sufficient conditions under which this predictor can be realized by an incremental (continual) agent, so that the continual learning problem C3L-LQG has the same optimal cost (Theorems 3 and 6); these conditions are shown to hold for scalar and diagonal systems. In steady state the optimal asymptotic cost is tr(M) + Σ_i exp(-2B_i) λ_i (Theorem 4). Section 7 applies this formula to capacity allocation across subsystems and presents small experiments, including a block-diagonal case where the sufficient conditions fail for small capacity.

Significance. If the results hold, this is a useful first theoretical step: it gives a closed-form optimal solution to capacity-constrained LQG prediction and identifies a class of systems for which the incremental-update constraint does not increase the optimal cost. The derivations in Theorem 2 and Theorem 4 are clean, and the reduction to the Gaussian distortion-rate function is transparent; Theorem 4's formula is a parameter-free prediction that could be tested in simulation. The main weakness is that the paper's headline result about the continual learning problem is conditional on restrictive sufficient conditions that fail in generic small-capacity regimes, and no results are given for those regimes. The paper is honest about these limitations, but the contribution is narrower than the title and abstract suggest.

major comments (3)
  1. [Theorems 3 and 6 (Sections 5 and 6.2)] The central claim that the capacity-constrained continual learning problem C3L-LQG is solved is conditional and is not established for the problem as formulated in (6). In Theorem 3, the proof requires F_t to be invertible for all t and the matrix inequality (14); since F_t = U_t diag(1 - exp(-2B_{ti})) U_t^T, invertibility fails whenever the reverse water-filling solution assigns zero rate to any eigenvalue, which is generic for small B in high-dimensional systems. In Theorem 6, the proof requires conditions (A) and (B), and Section 7.3 Case 4 explicitly reports that these conditions fail for small total capacity, leading to the grey band in Figure 7. For those cases, the paper provides no optimal incremental agent and no upper bound on the C3L-LQG cost beyond the trivial constant-predictor bound, so the equality of C3L-LQG and C2P-LQG costs remains an open problem rather than a theorem. A revision should either prove necessity of the conditions, extend the analysis to the failing regime, or reframe the contribution as solving the relaxed problem plus a restricted family of systems.
  2. [Section 7 (Sections 7.2 and 7.3)] The abstract's claim that the paper shows how to optimally allocate capacity across sub-problems in the steady state is broader than what is proven. Section 7.2 derives the allocation only for diagonal systems, relying on Theorem 6. The block-diagonal example in Section 7.3 (Case 4) is a decomposable problem for which the paper cannot determine the optimal allocation for small total capacity; the grey band in Figure 7 is an explicit admission of this. The allocation result therefore holds for a restricted class of decomposable systems, and the manuscript should either generalize the result to block-diagonal (or other coupled) systems or explicitly state the restriction in the abstract and introduction.
  3. [Equation (1) (Section 3.2)] The capacity constraint (1) is an average mutual-information bound on a real-valued agent state, not a bound on the number of memory bits, and the paper provides no validation that this surrogate is faithful for the finite-B regime where its predictions are most distinctive (for example, when zero-rate dimensions appear). The paper cites Kumar et al. (2023) for a large-scale approximation, but no finite-B evidence is given. A concrete test would be to compare the predicted capacity allocation against a finite-state or quantized agent with B bits for small B. Without such a check, the practical interpretation of the results as statements about memory/compute limits remains a modeling assumption rather than a proven conclusion.
minor comments (6)
  1. [Section 6.2] The definition of U_2 as 'the last m-d columns' contains a dimension typo; U_2 should be d x (d-m), and when m=0 the convention for an empty matrix should be stated.
  2. [Equation (26)] The last diagonal entry of D is written with λ_1 instead of λ_m; the definition should be diag(exp(-2B_1)λ_1/(1-exp(-2B_1)), ..., exp(-2B_m)λ_m/(1-exp(-2B_m))).
  3. [Section 3.2] The capacity constraint is listed as holding for t=0,...,T-1 in item 3 but the optimization in (6) states it for t=1,...,T; the two ranges should be made consistent.
  4. [Section 5.1, Equation (19)] The justification of steps (a) and (e) refers to a denominator 'Σ2ν'; the expression should be Σν/(P_t C^2 + Σν) ≤ 1.
  5. [Section 7.3, Case 1] The text contains a typo: 'I_100 is an identify matrix' should read 'identity matrix'; similar typos appear in the figure captions.
  6. [Proof of Theorem 3] The proof introduces the additional noise ε'_{t+1} without defining it; a sentence stating that ε'_{t+1} is independent of the history and of the other noises would improve readability.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the central results are a forward application of the external Gaussian distortion-rate theorem, and the only self-citation is motivational rather than load-bearing.

full rationale

The paper's derivation chain is: (i) formulate the relaxed capacity-constrained prediction problem C2P-LQG; (ii) solve it in Theorem 2 by reducing each one-step problem to the Gaussian distortion-rate function, an external result proved in Appendix B; (iii) show in Theorems 3 and 6 that, under explicitly stated sufficient conditions, this C2P-LQG solution admits an incremental update and is therefore feasible for the more constrained C3L-LQG problem; and (iv) derive steady-state costs and optimal capacity allocation by direct minimization of the resulting closed-form expressions. None of these steps defines the target quantity in terms of itself or fits free parameters to the claimed prediction. The sufficient conditions in Theorem 3 and Theorem 6 are verified exactly (e.g., condition (14) is checked as a covariance inequality), and the paper openly notes when they fail, as in Case 4, where it states that the optimal allocation is unknown. The only self-citation is to Kumar et al. (2023) for the motivational statement that the mutual-information constraint approximates a hard memory limit at large scale; this does not enter the theorem proofs and is not load-bearing for the mathematical results. Thus there is no significant circularity; the score of 1 reflects only the minor presence of a non-load-bearing self-citation.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new free parameters or invented entities. The water-filling levels η and the allocated bits B_i are determined by the capacity constraint and model parameters, not fitted to data. The main assumptions are modeling choices: the mutual-information surrogate for capacity, the known-model setting, and the unverified sufficient conditions for the incremental-update theorems.

assumptions (5)
  • domain assumption The mutual information constraint I(S_t; H_t) ≤ B is an adequate relaxation of a real memory/compute limit (equation 1).
    Section 1 states this is a relaxation of a hard memory constraint and cites Kumar et al. (2023) and Cover and Thomas (2006). All results are for this surrogate; the paper provides no direct evidence it approximates practical memory limits at small B.
  • domain assumption The agent knows the true process parameters A, C, Σ0, Σω, Σν.
    Section 3.1 states this 'as is traditional in control theory'. This removes system identification from the problem, so the 'learning' in the title refers to tracking/compression, not parameter estimation.
  • domain assumption Linear-Gaussian dynamics and observation model (equations 2-3).
    The Gaussian assumption is required for the closed-form distortion-rate solution and Kalman filtering.
  • domain assumption The sufficient conditions in Theorem 3 (F_t invertible and inequality (14)) and Theorem 6 (range and matrix inequality) hold for the systems for which the continual learning solution is claimed.
    Theorem 3 states these conditions are sufficient; they are proven for scalar (Section 5.1) and diagonal (Section 6.3) systems. For general systems they remain unverified, and Case 4 shows they can fail for small capacity (grey band). The C3L-LQG solution is only guaranteed under these conditions.
  • standard math Standard Gaussian distortion-rate theory (Theorem 11, Appendix B) and Kalman filtering (Appendix A).
    Used as external results; they are classical.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Capacity-Constrained Continual Learning." pith.science (2026). https://pith.science/paper/2IZ33IPT

@misc{pith2026250721479,
  author       = {Pith},
  title        = {Pith review of: Capacity-Constrained Continual Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2IZ33IPT}},
  note         = {Machine review of arXiv:2507.21479}
}
read the original abstract

Any agents we can possibly build are subject to capacity constraints, as memory and compute resources are inherently finite. However, comparatively little attention has been dedicated to understanding how agents with limited capacity should allocate their resources for optimal performance. The goal of this paper is to shed some light on this question by studying a simple yet relevant continual learning problem: the capacity-constrained linear-quadratic-Gaussian (LQG) sequential prediction problem. We derive a solution to this problem under appropriate technical conditions. Moreover, for problems that can be decomposed into a set of sub-problems, we also demonstrate how to optimally allocate capacity across these sub-problems in the steady state. We view the results of this paper as a first step in the systematic theoretical study of learning under capacity constraints.

Figures

Figures reproduced from arXiv: 2507.21479 by the authors.

Figure 1
Figure 1. Optimal capacity allocation for Case 1 16 [PITH_FULL_IMAGE:figures/full_fig_p016_1.png] view at source ↗
Figure 2
Figure 2. Optimal asymptotic loss for Case 1 Case 2: Different system noise magnitudes We then consider a setting where the subsystems differ in the magnitudes of the system noises, measured in the variance. Specifically, we choose A = 0.95I300, C = I300, Σω = diag([10I100, 3I100, I100]), and Σν = I300. Note that all scalar subsystems have the same ai = 0.95, ci = 1, σ 2 ν,i = 1, but have different σ 2 ω,i’s. Note that larger… view at source ↗
Figure 3
Figure 3. Optimal capacity allocation for Case 2 0 200 400 600 800 1000 total capacity (bits) 0 2000 4000 6000 8000 10000 12000 14000 asymptotic cost optimal asymptotic cost 2 , i = 1 2 , i = 3 2 , i = 10 [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Optimal asymptotic loss for Case 2 We also choose C = I300, Σω = I300, and Σν = I300. Note that A is a block-diagonal matrix with 100 2 × 2 diagonal blocks and another 100 1 × 1 diagonal blocks. Since C, Σω, and 18 [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Optimal capacity allocation for Case 3 0 200 400 600 800 1000 total capacity (bits) 0 500 1000 1500 2000 2500 3000 asymptotic cost optimal asymptotic cost ci = 0.25, 2 , i = 10 ci = 1, 2 , i = 10 ci = 1, 2 , i = 0.1 [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Optimal asymptotic loss for Case 3 Σν are also block-diagonal, this case is referred to as a block-diagonal case. Moreover, it is straightforward to show that P, K, and L are also block-diagonal matrices. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Optimal capacity allocation for Case 4 [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Optimal capacity allocation ratio for Case 4 [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 13 canonical work pages

  1. [1]

    Predictability, complexity, and learning

    William Bialek, Ilya Nemenman, and Naftali Tishby. Predictability, complexity, and learning. Neural computation, 13 0 (11): 0 2409--2463, 2001

  2. [2]

    Optimal manifold representation of data: an information theoretic approach

    Denis Chigirev and William Bialek. Optimal manifold representation of data: an information theoretic approach. Advances in Neural Information Processing Systems, 16, 2003

  3. [3]

    Kalman filtering

    Charles K Chui, Guanrong Chen, et al. Kalman filtering. Springer, 2017

  4. [4]

    Cover and Joy A

    Thomas M. Cover and Joy A. Thomas. Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience, USA, 2006. ISBN 0471241954

  5. [5]

    Robust finite-state controllers for uncertain pomdps

    Murat Cubuktepe, Nils Jansen, Sebastian Junges, Ahmadreza Marandi, Marnix Suilen, and Ufuk Topcu. Robust finite-state controllers for uncertain pomdps. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11792--11800, 2021

  6. [6]

    Understanding human intelligence through human limitations

    Thomas L Griffiths. Understanding human intelligence through human limitations. Trends in Cognitive Sciences, 24 0 (11): 0 873--883, 2020

  7. [7]

    Efficient estimation of ooms

    Herbert Jaeger, Mingjie Zhao, and Andreas Kolling. Efficient estimation of ooms. Advances in Neural Information Processing Systems, 18, 2005

  8. [8]

    A new approach to linear filtering and prediction problems

    Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 1960

Show all 16 references
  1. [9]

    Continual learning as computationally constrained reinforcement learning

    Saurabh Kumar, Henrik Marklund, Ashish Rao, Yifan Zhu, Hong Jun Jeon, Yueyang Liu, and Benjamin Van Roy. Continual learning as computationally constrained reinforcement learning. arXiv preprint arXiv:2307.04345, 2023

  2. [10]

    Predictive representations of state

    Michael Littman and Richard S Sutton. Predictive representations of state. Advances in neural information processing systems, 14, 2001

  3. [11]

    Periodic finite state controllers for efficient pomdp and dec-pomdp planning

    Joni Pajarinen and Jaakko Peltonen. Periodic finite state controllers for efficient pomdp and dec-pomdp planning. Advances in neural information processing systems, 24, 2011

  4. [12]

    Provably bounded-optimal agents

    Stuart J Russell and Devika Subramanian. Provably bounded-optimal agents. Journal of Artificial Intelligence Research, 2: 0 575--609, 1994

  5. [13]

    Herbert A. Simon. Bounded Rationality, pages 15--18. Palgrave Macmillan UK, London, 1990. ISBN 978-1-349-20568-4. doi:10.1007/978-1-349-20568-4_5. URL https://doi.org/10.1007/978-1-349-20568-4_5

  6. [14]

    Predictive state representations: A new theory for modeling dynamical systems

    Satinder Singh, Michael James, and Matthew Rudary. Predictive state representations: A new theory for modeling dynamical systems. arXiv preprint arXiv:1207.4167, 2012

  7. [15]

    Reinforcement learning: An introduction

    Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018

  8. [16]

    On the role of tracking in stationary environments

    Richard S Sutton, Anna Koop, and David Silver. On the role of tracking in stationary environments. In Proceedings of the 24th international conference on Machine learning, pages 871--878, 2007

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.