arxiv: 2605.08529 · v1 · submitted 2026-05-08 · 💻 cs.LG

Recognition: 2 theorem links

· Lean Theorem

The Propagation Field: A Geometric Substrate Theory of Deep Learning

Xingrui Gu

Authors on Pith no claims yet

Pith reviewed 2026-05-12 01:33 UTC · model grok-4.3

classification 💻 cs.LG

keywords deep learningpropagation fieldinternal geometryJacobian structurecontinual learningout-of-distribution robustnessgeneralizationfield-aware objectives

0 comments

The pith

Deep learning models are propagation fields whose internal geometry is underdetermined by endpoint losses alone.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes that deep neural networks are best understood as propagation fields consisting of hidden state trajectories and Jacobians through their layers rather than simply as endpoint functions. Standard training only constrains the input and output boundaries, which leaves the internal geometry free to vary even when endpoint performance is identical. By measuring field properties like path sensitivity and using objectives that preserve desirable field structures, the authors demonstrate gains in generalization to new paths, robustness to out-of-distribution data, and performance in continual learning scenarios such as Split CIFAR-100. This matters because it suggests a way to train models that maintain consistent internal computation rather than just matching final answers.

Core claim

We define a neural propagation field as the collection of hidden-state trajectories and local Jacobian operators across depth. Endpoint losses constrain only the boundary behavior of this field, leaving its interior geometry underdetermined. Endpoint-equivalent models can differ by orders of magnitude in trajectory and Jacobian structure. In controlled teacher-flow and PDE systems, endpoint fitting fails to recover the underlying propagation law. In real multi-path tasks, field-aware objectives improve unseen-path generalization, OOD robustness, and calibration when aligned with the observation structure. In continual learning, field-preservation regularization complements replay and distil

What carries the argument

The neural propagation field: the collection of hidden-state trajectories and local Jacobian operators across network depth that describes the internal geometry of computation.

If this is right

Endpoint fitting alone fails to recover the true propagation law in teacher-flow and PDE systems.
Field-aware objectives improve unseen-path generalization, OOD robustness, and calibration when aligned with the observation structure.
On Split CIFAR-100, DER++ combined with field preservation improves average accuracy, backward transfer, and field-retention metrics.
Over-constraining the field can cause performance collapse in multi-path tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Architectures could be redesigned to explicitly support preservation of specific trajectory or Jacobian properties during training.
Field metrics might serve as diagnostics to predict which models will fail on novel inputs even when they match on standard benchmarks.
The same geometric perspective could apply to sequential models like RNNs or transformers to analyze attention or recurrence dynamics.

Load-bearing premise

That the proposed field metrics capture causally relevant properties of internal computation not already implicitly optimized by standard endpoint losses, and that observed improvements stem from field alignment rather than incidental regularization effects.

What would settle it

A controlled comparison on Split CIFAR-100 or multi-path tasks where models trained with field-preservation objectives show no gains in accuracy, backward transfer, or OOD metrics over endpoint-only baselines when total regularization strength is matched.

read the original abstract

Modern deep learning treats neural networks primarily as endpoint functions from inputs to outputs. Inspired by the shift from force to geometry in physics, we ask whether a network should instead be understood through the geometry of its internal propagation. We define a neural propagation field as the collection of hidden-state trajectories and local Jacobian operators across depth. Endpoint losses constrain only the boundary behavior of this field, leaving its interior geometry underdetermined. We show that endpoint-equivalent models can differ by orders of magnitude in trajectory and Jacobian structure, and introduce observable field metrics such as path sensitivity, solver consistency, and trajectory/Jacobian retention. In controlled teacher-flow and PDE systems, endpoint fitting fails to recover the underlying propagation law. In real multi-path tasks, field-aware objectives improve unseen-path generalization, OOD robustness, and calibration when aligned with the observation structure, but can collapse when over-constrained. In continual learning, field-preservation regularization complements replay and distillation: on Split CIFAR-100, DER++ with field preservation improves average accuracy, backward transfer, and field-retention metrics. These results identify propagation-field quality as a measurable and trainable property of neural networks beyond endpoint performance.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Endpoint losses leave internal propagation underdetermined, and the paper shows this cleanly in synthetic systems while reporting gains from field preservation on Split CIFAR-100, but those gains are not isolated from generic regularization.

read the letter

The main point is that networks with the same endpoints can have very different internal trajectories and Jacobians, and the paper treats this propagation field as something you can measure and regularize directly. It defines concrete metrics like path sensitivity, solver consistency, and retention of trajectories or Jacobians, then shows in teacher-flow and PDE setups that endpoint matching alone fails to recover the true propagation rules. On Split CIFAR-100, adding field preservation to DER++ improves average accuracy, backward transfer, and the field metrics themselves. Those controlled experiments are the strongest part; they make the under-determination claim without the usual real-data noise. The idea of field-aware objectives for multi-path generalization and OOD calibration is also laid out clearly. The weakness is that the continual-learning gains are not shown to come from the geometric alignment rather than from any extra interior penalty. No ablation holds total loss complexity fixed while replacing the field terms with a non-geometric regularizer of matched strength, so the specific substrate interpretation rests on weaker ground than the synthetic results. Details on baseline strength, exclusion rules, and statistical controls are also thin in the reported summary. This is worth a reading group for people working on continual learning or internal dynamics, since the hypothesis is coherent and the synthetic evidence is direct. It deserves peer review because it has a testable framing and some positive results, though referees will need to press on the isolation of the field effect.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes a neural propagation field as the collection of hidden-state trajectories and local Jacobian operators across network depth. It claims that endpoint losses underdetermine this interior geometry, that endpoint-equivalent models can differ by orders of magnitude in trajectory and Jacobian structure, and that field metrics (path sensitivity, solver consistency, trajectory/Jacobian retention) can be used to diagnose this and to construct field-aware objectives. Experiments on teacher-flow/PDE systems show endpoint fitting fails to recover propagation laws; on multi-path tasks and Split CIFAR-100 continual learning with DER++, field-preservation regularization improves unseen-path generalization, OOD robustness, calibration, average accuracy, backward transfer, and field-retention metrics when aligned with observation structure.

Significance. If the central claim holds after isolating the geometric contribution, the work could provide a measurable geometric substrate for neural networks beyond endpoint optimization, with potential implications for robustness and continual learning. Strengths include the controlled synthetic systems demonstrating underdetermination and the empirical gains reported on Split CIFAR-100. However, significance is limited by the absence of controls separating field alignment from general regularization effects.

major comments (3)

[Continual learning experiments] In the Split CIFAR-100 continual learning experiments, the reported gains for DER++ with field preservation lack ablations that hold total loss complexity fixed while randomizing or replacing the auxiliary field term with a non-geometric regularizer of matched strength; without this, it remains unclear whether improvements arise from propagation-field alignment or incidental interior constraints.
[Definition of field metrics and objectives] The field metrics (path sensitivity, solver consistency, trajectory/Jacobian retention) are defined directly from the proposed geometric construction and then used both to diagnose endpoint underdetermination and to define the improved objective; while teacher-flow and PDE controls provide external grounding, the claim that field quality is independently trainable requires explicit tests that these quantities capture causally relevant interior properties not already implicitly optimized by endpoint losses.
[Experimental results] The abstract and experimental sections report positive results on controlled systems and Split CIFAR-100 but omit details on baseline strength, statistical controls, data exclusion rules, and whether gains survive ablation of the new field terms; this weakens assessment of whether the improvements are robust or attributable to the geometric substrate.

minor comments (2)

[Abstract] The abstract states that field-aware objectives 'can collapse when over-constrained' but does not specify the conditions or point to the relevant figure or section.
[Introduction/Methods] Notation for the propagation field (trajectories and Jacobians) would benefit from an explicit early mathematical definition to aid readability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive review and for identifying areas where additional controls and details would strengthen the presentation. We address each major comment below, indicating where revisions will be made to incorporate the suggestions while preserving the core claims supported by the existing experiments.

read point-by-point responses

Referee: In the Split CIFAR-100 continual learning experiments, the reported gains for DER++ with field preservation lack ablations that hold total loss complexity fixed while randomizing or replacing the auxiliary field term with a non-geometric regularizer of matched strength; without this, it remains unclear whether improvements arise from propagation-field alignment or incidental interior constraints.

Authors: We agree that isolating the geometric contribution from general regularization effects requires further controls. In the revised manuscript we will add ablations on Split CIFAR-100 that replace the field-preservation term with (i) a random auxiliary loss of matched magnitude and (ii) a standard non-geometric regularizer (e.g., increased weight decay) while keeping the total loss complexity and hyper-parameter budget fixed. These results will be reported alongside the existing DER++ comparisons to clarify the source of the observed gains in accuracy, backward transfer, and field-retention metrics. revision: yes
Referee: The field metrics (path sensitivity, solver consistency, trajectory/Jacobian retention) are defined directly from the proposed geometric construction and then used both to diagnose endpoint underdetermination and to define the improved objective; while teacher-flow and PDE controls provide external grounding, the claim that field quality is independently trainable requires explicit tests that these quantities capture causally relevant interior properties not already implicitly optimized by endpoint losses.

Authors: The teacher-flow and PDE experiments already demonstrate that endpoint-equivalent networks can differ by orders of magnitude in trajectory and Jacobian structure, showing that standard losses do not implicitly optimize the reported field metrics. To strengthen the causal claim, the revision will include additional controlled experiments in which we directly optimize or penalize the field metrics (path sensitivity and retention) while holding endpoint loss fixed, then measure downstream effects on unseen-path generalization and OOD robustness. These tests will be presented as explicit evidence that the metrics capture trainable interior properties beyond endpoint optimization. revision: partial
Referee: The abstract and experimental sections report positive results on controlled systems and Split CIFAR-100 but omit details on baseline strength, statistical controls, data exclusion rules, and whether gains survive ablation of the new field terms; this weakens assessment of whether the improvements are robust or attributable to the geometric substrate.

Authors: We acknowledge the need for greater transparency. The revised experimental section will (i) compare against stronger baselines with matched computational budgets, (ii) report means and standard deviations over multiple random seeds with statistical significance tests, (iii) explicitly state any data exclusion or preprocessing rules, and (iv) include ablations that remove the field terms while retaining all other components to verify that reported gains depend on field preservation. These additions will be placed in the main text and supplementary material. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation introduces new observables and tests them empirically

full rationale

The paper defines the propagation field and associated metrics (path sensitivity, trajectory/Jacobian retention) from first principles as collections of hidden-state trajectories and Jacobians, then empirically demonstrates that endpoint losses leave these underdetermined and that field-preserving regularization yields measurable gains on accuracy, transfer, and OOD metrics in controlled PDE/teacher systems and Split CIFAR-100. No equation reduces a claimed prediction to a fitted input by construction, no load-bearing uniqueness theorem is imported via self-citation, and the central claims rest on independent experimental outcomes rather than definitional equivalence. The framework is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the domain assumption that internal propagation geometry is a primary substrate for generalization and forgetting, plus the invented entity of the propagation field itself. No explicit free parameters are stated in the abstract.

axioms (1)

domain assumption Endpoint losses constrain only boundary behavior, leaving interior propagation geometry underdetermined and independently optimizable.
Invoked to justify why standard training is insufficient and why field-aware objectives are needed.

invented entities (1)

Neural propagation field no independent evidence
purpose: Collection of hidden-state trajectories and local Jacobian operators across depth that serves as the geometric substrate.
New conceptual object introduced to reframe network internals; no independent evidence outside the paper's definitions and experiments is provided.

pith-pipeline@v0.9.0 · 5493 in / 1325 out tokens · 57684 ms · 2026-05-12T01:33:26.006347+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear
We define a neural propagation field as the collection of hidden-state trajectories and local Jacobian operators across depth... field-aware objectives... Lfield = λr Lreveal + λs Lsolver + λJ Ljac
IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear
endpoint-equivalent models can differ by orders of magnitude in trajectory and Jacobian structure

Reference graph

Works this paper leans on

17 extracted references · 17 canonical work pages · 7 internal anchors

[1]

nature , volume=

Deep learning , author=. nature , volume=. 2015 , publisher=

work page 2015
[2]

nature , volume=

Highly accurate protein structure prediction with AlphaFold , author=. nature , volume=. 2021 , publisher=

work page 2021
[3]

Understanding deep learning requires rethinking generalization

Understanding deep learning requires rethinking generalization , author=. arXiv preprint arXiv:1611.03530 , year=

work page internal anchor Pith review arXiv
[4]

nature , volume=

Learning representations by back-propagating errors , author=. nature , volume=. 1986 , publisher=

work page 1986
[5]

Intriguing properties of neural networks

Intriguing properties of neural networks , author=. arXiv preprint arXiv:1312.6199 , year=

work page internal anchor Pith review Pith/arXiv arXiv
[6]

Explaining and Harnessing Adversarial Examples

Explaining and harnessing adversarial examples , author=. arXiv preprint arXiv:1412.6572 , year=

work page internal anchor Pith review Pith/arXiv arXiv
[7]

Emergent Abilities of Large Language Models

Emergent abilities of large language models , author=. arXiv preprint arXiv:2206.07682 , year=

work page internal anchor Pith review Pith/arXiv arXiv
[8]

In-context Learning and Induction Heads

In-context learning and induction heads , author=. arXiv preprint arXiv:2209.11895 , year=

work page internal anchor Pith review Pith/arXiv arXiv
[9]

Proceedings of the national academy of sciences , volume=

Overcoming catastrophic forgetting in neural networks , author=. Proceedings of the national academy of sciences , volume=. 2017 , publisher=

work page 2017
[10]

Scaling Laws for Neural Language Models

Scaling laws for neural language models , author=. arXiv preprint arXiv:2001.08361 , year=

work page internal anchor Pith review Pith/arXiv arXiv 2001
[11]

Training Compute-Optimal Large Language Models

Training compute-optimal large language models , author=. arXiv preprint arXiv:2203.15556 , volume=

work page internal anchor Pith review Pith/arXiv arXiv
[12]

Annalen der Physik , volume=

The foundation of the general theory of relativity , author=. Annalen der Physik , volume=

work page
[13]

2018 , publisher=

Geometry, topology and physics , author=. 2018 , publisher=

work page 2018
[14]

Advances in neural information processing systems , volume=

Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

work page
[15]

Pareto optimality, game theory and equilibria , pages=

Pareto optimality , author=. Pareto optimality, game theory and equilibria , pages=. 2008 , publisher=

work page 2008
[16]

Conservation of Isotopic Spin and Isotopic Gauge Invariance , author =. Phys. Rev. , volume =. 1954 , month =. doi:10.1103/PhysRev.96.191 , url =

work page doi:10.1103/physrev.96.191 1954
[17]

Journal of the American Mathematical Society , volume=

Sticky Kakeya sets and the sticky Kakeya conjecture , author=. Journal of the American Mathematical Society , volume=

work page