arxiv: 2604.11011 · v1 · submitted 2026-04-13 · 💻 cs.LG · cs.CL· cs.NE

Recognition: 2 theorem links

· Lean Theorem

K-Way Energy Probes for Metacognition Reduce to Softmax in Discriminative Predictive Coding Networks

Jon-Paul Cacioli

Authors on Pith no claims yet

Pith reviewed 2026-05-10 15:44 UTC · model grok-4.3

classification 💻 cs.LG cs.CLcs.NE

keywords predictive codingenergy probesmetacognitionsoftmaxdiscriminative networksnegative resultCIFAR-10

0 comments

The pith

In standard discriminative predictive coding networks, K-way energy probes for metacognition reduce to a monotone function of the log-softmax margin.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that under target-clamped cross-entropy energy training and effectively feedforward latent dynamics, the settled energies obtained by clamping each class hypothesis in turn decompose into a part that tracks the log-softmax margin plus a residual term that training does not align with correctness. This explains why the apparently richer per-hypothesis energy signal fails to outperform softmax on metacognitive tasks such as distinguishing correct from incorrect predictions. A sympathetic reader cares because the result accounts for the observed gap between probe and softmax across multiple training regimes on CIFAR-10 and identifies the precise conditions under which the reduction holds.

Core claim

With target-clamped CE-energy training and effectively-feedforward latent dynamics, the K-way energy margin decomposes into a monotone function of the log-softmax margin plus a residual that is not trained to correlate with correctness. The decomposition predicts that the structural probe tracks softmax from below, and experiments across six conditions confirm that the probe remains below softmax with a stable gap.

What carries the argument

The approximate reduction of the K-way energy margin to a monotone function of the log-softmax margin under target-clamped CE-energy training and feedforward latent dynamics.

If this is right

The K-way energy probe cannot exceed softmax performance on metacognitive metrics within the tested discriminative PC regime.
The performance gap between probe and softmax remains stable across extended deterministic training, Langevin temperature sweeps, and trajectory-integrated MCPC training.
The reduction does not apply in bidirectional PC, prospective configuration, generative PC, or non-CE energy formulations.
Final-state and trajectory-integrated training produce probes whose AUROC_2 values differ by less than 10^-3 at deterministic evaluation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same decomposition may explain why energy-based metacognitive readouts have not yet shown consistent advantages over softmax in other feedforward-trained architectures.
Productive structural probing would require relaxing the target-clamping or CE-energy assumptions to allow the residual term to be trained.
The result suggests testing whether prospective or bidirectional predictive coding can make the per-hypothesis energy carry additional correctness information beyond the softmax margin.

Load-bearing premise

The network follows the standard Pinchetti-style discriminative predictive coding formulation that uses target-clamped cross-entropy energy and produces effectively feedforward latent dynamics during inference.

What would settle it

A direct measurement on CIFAR-10 showing that the K-way energy margin achieves higher AUROC_2 than the log-softmax margin for distinguishing correct from incorrect predictions under target-clamped CE training would falsify the claimed decomposition.

Figures

Figures reproduced from arXiv: 2604.11011 by Jon-Paul Cacioli.

**Figure 1.** Figure 1: Structural probe AUROC2 versus softmax AUROC2 over 25 epochs of standard discriminative PC training (TinyConvPCN, CIFAR-10, single seed). The shaded region indicates the probe-vs-softmax gap. The gap narrows between epochs 5 and 10 as both probes improve from under-trained baselines, then re-widens through epoch 25 as softmax continues to improve while the structural probe plateaus. 15 [PITH_FULL_IMAGE:f… view at source ↗

**Figure 2.** Figure 2: K-way energy probe AUROC2 on a Langevin-trained network as a function of eval-time noise 𝜎 (log scale, with 𝜎=0 shown at the left edge). The softmax baseline (blue dashed) is invariant to inference noise. The structural probe (red) sits below the softmax baseline at 𝜎 ≤ 10−2 and collapses to near-chance accuracy at 𝜎 ≥ 10−1 . The movement statistic is the maximum across layers of the per-layer mean per-ele… view at source ↗

**Figure 3.** Figure 3: Probe-vs-softmax gap (probe AUROC2 minus softmax AUROC2 ) across the six tested conditions, colour-coded by training family. The BP+decoder condition (gap −0.009) is the smallest, consistent with Prediction 4: under post-hoc reconstruction training of the decoder, the residual term is forced to be approximately 𝑘-invariant, and the K-way margin tracks the log-softmax margin from the BP encoder. The four … view at source ↗

read the original abstract

We present this as a negative result with an explanatory mechanism, not as a formal upper bound. Predictive coding networks (PCNs) admit a K-way energy probe in which each candidate class is fixed as a target, inference is run to settling, and the per-hypothesis settled energies are compared. The probe appears to read a richer signal source than softmax, since the per-hypothesis energy depends on the entire generative chain. We argue this appearance is misleading under the standard Pinchetti-style discriminative PC formulation. We present an approximate reduction showing that with target-clamped CE-energy training and effectively-feedforward latent dynamics, the K-way energy margin decomposes into a monotone function of the log-softmax margin plus a residual that is not trained to correlate with correctness. The decomposition predicts that the structural probe should track softmax from below. We test this across six conditions on CIFAR-10: extended deterministic training, direct measurement of latent movement during inference, a post-hoc decoder fairness control on a backpropagation network, a matched-budget PC vs BP comparison, a five-point Langevin temperature sweep, and trajectory-integrated MCPC training. In every condition the probe sat below softmax. The gap was stable across training procedures within the discriminative PC family. Final-state and trajectory-integrated training produced probes whose AUROC_2 values differed by less than 10^-3 at deterministic evaluation. The empirical regime is small: single seed, 2.1M-parameter network, 1280 test images. We frame the result as a preprint inviting replication. We discuss conditions under which the decomposition does not apply (bidirectional PC, prospective configuration, generative PC, non-CE energy formulations) and directions for productive structural probing the analysis does not foreclose.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

In standard discriminative PCNs the K-way energy probe reduces to a monotone function of log-softmax plus an untrained residual, and the experiments show it tracking below softmax with a stable gap.

read the letter

The core result is that under target-clamped CE training and effectively feedforward latent dynamics, the apparent extra signal from the K-way energy probe is mostly just a softened version of the usual softmax margin. The paper derives this decomposition from the PCN equations and tests it across six conditions on CIFAR-10, all of which show the probe sitting below softmax with a consistent gap. That negative finding with an explicit mechanism is the useful part here, and they are careful to delimit it to the Pinchetti-style regime while noting where the reduction breaks (bidirectional dynamics, generative PC, non-CE energies). The empirical consistency, including direct latent-movement checks and trajectory-integrated training, gives the claim some weight even at small scale. They also run a matched-budget comparison against backprop and a temperature sweep, which helps rule out obvious confounds. The main limitation is the narrow empirical footprint: single seed, 2.1 M parameters, 1280 test images. That is enough to illustrate the point and invite replication, but it leaves room for doubt about how far the gap holds under larger models or different random seeds. The derivation itself is approximate rather than exact, though the residual is explicitly left untrained and uncorrelated with correctness, which avoids circularity. Readers working on predictive coding or energy-based metacognition probes will find this directly relevant as a boundary condition on what structural signals can deliver in this model family. The work shows clear engagement with the equations and prior literature without overclaiming, so it is worth a referee's time. I would send it to review rather than desk-reject, expecting requests for scale-up or additional controls but not fundamental problems with the central argument.

Referee Report

0 major / 1 minor

Summary. The paper claims that in the standard Pinchetti-style discriminative predictive coding formulation with target-clamped cross-entropy energy training and effectively feedforward latent dynamics, the K-way energy margin decomposes approximately into a monotone function of the log-softmax margin plus an untrained residual that does not correlate with correctness by construction. This predicts that the structural probe tracks softmax from below, which is supported by the derivation and by consistent empirical results across six conditions on CIFAR-10 (deterministic training, latent movement measurement, decoder control, matched-budget comparison, Langevin sweep, and trajectory-integrated MCPC) where the probe remains below softmax with a stable gap.

Significance. If the result holds, it is significant as a clarifying negative result that explains why energy-based metacognitive probes may not exceed softmax in this delimited PCN regime. Credit is given for the parameter-free approximate derivation under the stated assumptions, the explicit delimitation of applicability (noting failure in bidirectional/generative PC or non-CE energies), and the reproducible empirical consistency across training procedures with AUROC_2 differences below 10^-3 between final-state and trajectory-integrated variants.

minor comments (1)

The empirical regime is acknowledged as small (single seed, 2.1M parameters, 1280 images); consider adding a brief note in the discussion on how the stable gap might be verified at larger scale to strengthen the invitation for replication.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their careful and accurate summary of the manuscript. We appreciate the recognition of the negative result, the parameter-free derivation, the explicit delimitation of the regime, and the consistency of the empirical findings.

read point-by-point responses

Referee: The paper claims that in the standard Pinchetti-style discriminative predictive coding formulation with target-clamped cross-entropy energy training and effectively feedforward latent dynamics, the K-way energy margin decomposes approximately into a monotone function of the log-softmax margin plus an untrained residual that does not correlate with correctness by construction. This predicts that the structural probe tracks softmax from below, which is supported by the derivation and by consistent empirical results across six conditions on CIFAR-10 (deterministic training, latent movement measurement, decoder control, matched-budget comparison, Langevin sweep, and trajectory-integrated MCPC) where the probe remains below softmax with a stable gap.

Authors: We agree with this description of the central claim and supporting evidence. The approximate decomposition and the six-condition empirical protocol are presented exactly in this form in the manuscript. revision: no
Referee: If the result holds, it is significant as a clarifying negative result that explains why energy-based metacognitive probes may not exceed softmax in this delimited PCN regime. Credit is given for the parameter-free approximate derivation under the stated assumptions, the explicit delimitation of applicability (noting failure in bidirectional/generative PC or non-CE energies), and the reproducible empirical consistency across training procedures with AUROC_2 differences below 10^-3 between final-state and trajectory-integrated variants.

Authors: We are pleased that the referee finds the negative result and its scope to be clearly delimited. The manuscript already emphasizes that the reduction applies only under the stated assumptions (target-clamped CE energy, effectively feedforward dynamics) and explicitly lists the regimes where it does not hold. revision: no

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper derives its central approximate reduction directly from the Pinchetti-style discriminative PCN equations under the explicit assumptions of target-clamped CE-energy training and effectively-feedforward latent dynamics. The K-way energy margin is shown to decompose into a monotone function of the log-softmax margin plus a residual explicitly noted as untrained and uncorrelated with correctness by construction of the training regime. This is framed as a negative result with clear delimitations on applicability (e.g., fails for bidirectional/generative PC or non-CE energies), and is tested empirically across six independent conditions without any parameter fitting that would force the tracking behavior. No load-bearing steps reduce to self-definition, fitted inputs renamed as predictions, or self-citation chains; the derivation is self-contained against the stated model equations and assumptions.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

The central claim rests on domain assumptions of the standard discriminative PC formulation rather than new free parameters or invented entities.

axioms (3)

domain assumption Pinchetti-style discriminative PC formulation
Invoked as the setting in which the reduction holds; stated in abstract as standard.
domain assumption target-clamped CE-energy training
Required for the decomposition to apply; part of the training regime assumed.
domain assumption effectively-feedforward latent dynamics
Assumed to enable the reduction; contrasted with bidirectional cases where it may not hold.

pith-pipeline@v0.9.0 · 5624 in / 1387 out tokens · 27911 ms · 2026-05-10T15:44:04.579401+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

the K-way energy margin decomposes into a monotone function of the log-softmax margin plus a residual that is not trained to correlate with correctness
IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

assumptions A1-A5 (discriminative PC with CE energy, target clamping, effectively feedforward latent dynamics)

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Cross-Entropy Is Load-Bearing: A Pre-Registered Scope Test of the K-Way Energy Probe on Bidirectional Predictive Coding
cs.CL 2026-04 conditional novelty 5.0

Cross-entropy loss is empirically load-bearing for the K-way energy probe outperforming softmax margins in predictive coding on CIFAR-10, with roughly two-thirds of the gap due to logit scale.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages · cited by 1 Pith paper

[1]

Bogacz, R. (2017). A tutorial on the free-energy framework for modelling perception and learning. Journal of Mathematical Psychology , 76, 198–211. Cacioli, J. (2026a). Do LLMs know what they know? Measuring metacognitive eﬀiciency with signal detection theory. arXiv:2603.25112. Cacioli, J. (2026b). LLMs as signal detectors: sensitivity, bias, and the tem...

work page arXiv 2017
[2]

32 Innocenti, F., Kinghorn, P., Yun-Farmbrough, W., De Llanza Varona, M., Singh, R., & Buckley, C. L. (2024). JPC: Flexible inference for predictive coding networks in JAX. arXiv:2412.03676. Maniscalco, B., & Lau, H. (2012). A signal detection theoretic approach for estimating metacogni- tive sensitivity from confidence ratings. Consciousness and Cognitio...

work page arXiv 2024