{"id":"84b99a25-28dc-465a-a04f-7eb3a7037795","arxiv_id":"2412.01596","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper shows that free energy OOD detection fails when feature differences lie in the last layer's null space, and mitigates this with dimension reduction and least singular value regularization.","lead":"FEVER-OOD identifies blind spots in free energy based out-of-distribution detection caused by the null space and the least singular value of a classifier's last linear layer, and adds a dimension reducing layer plus singular value regularizers to shrink those blind spots. On ImageNet-100 it lowers the OOD false positive rate of the Dream-OOD baseline from 39.98% to 35.93%.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The LSVV result (Eq. 12) solves the wrong objective: the true minimal free-energy change over δ⊥Null(W⊤) is zero, not db·σmin(W), so the claimed vulnerability and its regularizer are not theoretically justified.","rationale":"The reader's weakest assumption correctly points to the unvalidated proxy in Eq. (11) and the possible distortion by the log-sum-exp nonlinearity. The stress-test goes further: the free energy is not merely non-monotone in the logit norm, it has exact zero-energy-change directions orthogonal to the null space for any full-rank W with K≥2. This makes Eq. (12) mathematically false as a statement about the minimal free-energy change, and it invalidates the LSVR/CNR 'guarantee'. The NSR mitigation is also questionable because reducing the penultimate dimension does not reduce the null space of the composite feature-to-logit map, but the LSVV flaw is the more direct and decisive theoretical failure. The empirical results are not disputed, but the central contribution — a theoretical analysis of inherent vulnerabilities and a principled mitigation — is not sound as presented. The paper would need substantial revision (e.g., reframing LSVV as a proxy for logit sensitivity, removing the guarantee claim, and empirically characterizing iso-energy directions) before acceptance; as is, the central claim is unsupported, warranting rejection of the preprint in its current form.","tokens_in":21709,"tokens_out":10855,"duration_ms":97867,"concrete_test":"For a two-class linear classifier with W = diag(10,1), f(x)=(0,0), and db=0.5, choose δ = (t/10, log(2-e^t)) with t=-5. Then W⊤δ = (-5, 0.689), ||δ|| ≈ 0.85 ≥ db, δ is not in Null(W⊤), yet exp(0)+exp(0) = exp(-5)+exp(0.689) ≈ 2, so F(x+δ) = F(x) exactly. This is a direct counterexample to Eq. (12), which predicts a minimum of db·σmin = 0.5. A broader check: for random W and random f(x), numerically optimize |F(x+δ)-F(x)| subject to ||δ||≥db and δ⊥Null(W⊤); if the minimum reliably approaches 0 while the LSV direction gives nonzero change, the LSVV theorem is false and the regularizer's guarantee is unsupported.","verdict_should_be":"REJECT","load_bearing_attack":"Equation (10) defines the true minimal free-energy difference, while Eq. (11) replaces it with the norm of the logit change. The two are not equivalent: the free energy F(x) = -log Σ_k exp(f_k(x)) is constant on a codimension-1 hypersurface in logit space, so for K≥2 there exist displacements δ with W⊤δ ≠ 0 and F(x+δ) = F(x) exactly. Concretely, for f(x)=(0,0), any y with e^{y1}+e^{y2}=2 gives zero free-energy change; because W⊤ is surjective for full-rank W, there are corresponding δ of arbitrarily large norm, and one can choose δ orthogonal to Null(W⊤). Hence the infimum in Eq. (10) is 0 for any db and any W with K≥2, not db·σmin(W). Eq. (12) only bounds the logit-norm change, which is sufficient but not necessary for small free-energy change. Consequently, the LSVV vulnerability is misidentified, and the LSVR/CNR regularizers cannot 'guarantee detectable energy variations' as claimed, since zero-energy-change directions persist regardless of σmin. This undermines the central theoretical contribution and the stated mechanism of the proposed mitigations, although the empirical improvements may still be real for other reasons.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies out-of-distribution (OOD) detection based on the free energy score. It claims two inherent vulnerabilities of free-energy scoring for classifiers with a linear last layer: a null-space vulnerability (NSV), where feature-space displacements in the null space of the last-layer weight matrix leave the free energy exactly unchanged, and a least-singular-value vulnerability (LSVV), where the minimal free-energy change for a displacement of norm at least d_b is claimed to be d_b \\sigma_{\\min}(W_{\\text{cls}}). To mitigate these, it proposes r-null-space reduction (r-NSR), a least-singular-value regularizer (LSVR), and a condition-number regularizer (CNR). The method is evaluated on VOS, FFS, and Dream-OOD baselines for CIFAR-10, CIFAR-100, ImageNet-100, and PASCAL VOC object detection, reporting improved FPR95 and AUROC, with a headline ImageNet-100 result of 35.93% FPR95 versus 39.98% for the Dream-OOD baseline.","tokens_in":22021,"tokens_out":10315,"duration_ms":92434,"significance":"If the theoretical claims were correct, the paper would identify a previously unexplored blind spot of free-energy OOD detection and offer a principled fix. The NSV observation itself is correct and is a useful, simple insight: for a linear last layer with d' > K, any feature-space displacement in the null space of W_{\\text{cls}}^T indeed leaves the free energy unchanged. The paper also provides a broad empirical study across several baselines, datasets, and tasks, with code, ablations, and deterministic training. However, the central LSVV derivation is mathematically incorrect: Eq. (12) solves a proxy problem about logit changes, not the stated free-energy minimization, and the two objectives are not equivalent. In addition, hyperparameters are selected post hoc on the same OOD test sets and results are reported from single runs without error bars, so the empirical claims are weaker than stated. The paper overclaims by calling the result state-of-the-art without comparison to recent OOD methods beyond the three chosen baselines.","major_comments":[{"comment":"Equation (12) does not solve the minimization problem stated in Eq. (10). The free energy F(x) = -log \\sum_k \\exp(f_k(x)) is unchanged whenever \\sum_k \\exp(f_k + r_k) = \\sum_k \\exp(f_k). For K \\ge 2, there exist nonzero vectors r of arbitrarily large norm satisfying this identity; for example, when f(x)=0 one may take r_1 = t and r_2 = \\log(2 - e^t) with t < \\log 2, so the sum of exponentials remains 2. Since W_{\\text{cls}}^T restricted to Null(W_{\\text{cls}}^T)^\\perp is a bijection onto \\mathbb{R}^K whenever W_{\\text{cls}} has rank K, any such logit perturbation r is realized by some δ in the feasible set of Eq. (10), and by taking t sufficiently negative one obtains \\|δ\\| \\ge d_b. Hence the minimum in Eq. (10) is 0, not d_b \\sigma_{\\min}(W_{\\text{cls}}), for any full-rank W_{\\text{cls}} with K \\ge 2. The quantity in Eq. (12) bounds the smallest possible change in the logits, which is neither necessary nor sufficient for a small free-energy change. Consequently, the claim in §4.3 that maximizing \\sigma_{\\min} “guarantee[s] detectable energy variations” is unsupported. The LSVV section must be rewritten, either by downgrading the claim to a statement about logit-norm change or by deriving a genuine bound on the free-energy difference; in its current form the central theoretical contribution is not valid.","section":"§4.2, Eqs. (10)–(12)"},{"comment":"The reported gains are based on post hoc selection of hyperparameters on the very OOD test sets used for evaluation. Section 6 states that “The reported values for λLSV and λCN are for the best obtained models,” and Section 5.3 evaluates a grid of values and selects configurations. No held-out OOD validation split is used, and all results come from a single deterministic run, so no uncertainty or significance information is available. The headline improvement of 35.93% versus 39.98% FPR95 is therefore a selected maximum over the grid rather than a pre-registered or cross-validated comparison. The authors should either report results on an OOD validation set separated from the test set, or provide multiple seeds with error bars and a clear statement of the selection procedure.","section":"§5.3 and §6, Tables 1–3"},{"comment":"The empirical evidence that the LSVR/CNR regularizers are the source of improvement is weak and at times contradicted by the paper’s own tables. In Table 3, Dream-OOD-100-NSR without any regularizer attains an average FPR95 of 35.60%, while the claimed best model with LSVR attains 35.93% (AUROC improves from 92.64 to 93.12 but FPR95 worsens). In Table 2, adding LSVR to Dream-OOD-100-NSR degrades FPR95 from 42.77% to 66.15%. The paper’s explanation that pretraining is responsible is not tested. Given that the theoretical justification for the regularizers is in question, the contribution of LSVR/CNR relative to r-NSR alone needs to be established with controlled comparisons; as it stands, the results largely support the conclusion that r-NSR is the main driver of the reported gains.","section":"§6, ImageNet-100 and CIFAR-100 paragraphs"}],"minor_comments":[{"comment":"The abstract reports an average OOD false positive rate of 35.83%, while Table 3 and the conclusion report 35.93%; the numbers should be reconciled.","section":"Abstract and Table 3"},{"comment":"The OOD dataset heading “SHVN” is a typo for “SVHN”.","section":"Table 1"},{"comment":"The text states that for ImageNet-100 the authors use λLSV = 0.001 and λCN = 0.01 based on CIFAR-100 experiments, but Table 3 reports λLSV = 0.01 and λCN = 0.001 for the best configurations; this mismatch should be corrected.","section":"§5.3 and Table 3"},{"comment":"The “state-of-the-art” claim is not supported by a comparison with recent OOD detection methods beyond VOS, FFS, and Dream-OOD; the claim should be qualified or the comparison expanded.","section":"§6 and Conclusion"},{"comment":"The supplementary material contains several typos, including “regualrizing”, “failrs”, and “fos”; these should be fixed in a revision.","section":"Supplementary Material"},{"comment":"The label “Safe Direction” in Figure 2 is not defined in the text or caption; the figure should clarify what makes a direction safe.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The central LSVV theorem is mathematically incorrect and the guarantee claim cannot stand in its current form. I recommend major revision rather than rejection because the NSV observation is valid and the empirical study may be salvageable, but only if the authors replace the false theoretical claim with a correct statement or explicitly reframe LSVR/CNR as heuristics, and if they address the post hoc hyperparameter selection. If the theoretical section cannot be repaired, the paper should not be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nI read FEVER-OOD. The one thing to know: the null-space vulnerability (NSV) is real and useful, but the least-singular-value vulnerability (LSVV) as derived does not hold up. The paper solves the wrong minimization. The infimum of the free-energy change over displacements of size at least d_b orthogonal to the null space is zero, not d_b * sigma_min(W), because for K >= 2 there are logit displacements y != 0 with sum(exp(f_k + y_k)) = sum(exp(f_k)), and these can be realized by feature displacements of arbitrarily large norm orthogonal to the null space. So Eq. (12) is a bound on the logit-norm change, not on the free-energy change. The claimed guarantee that LSVR/CNR ensure detectable energy variations is unjustified.\n\nThat said, the paper does real things. The NSV observation—that any feature displacement in the null space of the last linear layer leaves the free energy unchanged—is exact, simple, and not in the cited prior literature. The r-NSR fix (collapse the feature dimension to reduce nullity) is a natural, cheap remedy, and the experiments consistently show it helps. The empirical work is careful: classification and detection, multiple baselines, ablations over regularizer weights, and they report in-distribution accuracy so you can see the cost. The ImageNet-100 improvement (39.98 -> 35.93 FPR95) is modest but consistent.\n\nSoft spots beyond the LSVV issue: hyperparameters (lambda_LSV, lambda_CN, r) are chosen post hoc on the same OOD test sets, and all results are single runs with fixed seeds, no error bars. The \"state-of-the-art\" claim is benchmarked against only three baselines. The condition-number regularizer is unstable at some settings (see the 100% FPR95 rows in Tables 2 and 3). The limitations section is honest about the residual null space and the lack of a priori guidance for the regularizer weights.\n\nBottom line: this deserves a serious referee. The empirical contribution is real and the NSV is a genuine insight, but the LSVV theory needs to be reworked or explicitly reframed as a heuristic. I would send it to review with the expectation of major revision.","headline":"Null-space vulnerability is real and worth remembering; the least-singular-value derivation solves the wrong objective, but the empirical study is solid enough to warrant serious review.","tokens_in":22554,"tokens_out":3868,"would_cite":true,"duration_ms":33325,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Free-energy out-of-distribution detection has a provable blind spot, and this paper shows how to shrink it.","keywords":["out-of-distribution detection","free energy score","null space vulnerability","least singular value","feature space regularization","energy-based OOD","ImageNet-100","object-level OOD detection"],"falsifier":"Take a trained classifier with a linear last layer, extract real feature vectors, fix a displacement norm d_b, and compare the actual free energy change along the least singular vector with the change along the next few singular vectors; if the least singular vector no longer gives the smallest free energy change, the proxy argument behind Eq. (12) fails. More decisively, run a feature-space search for a displacement with norm at least d_b whose free energy change is smaller than d_b σ_min(W_cls); finding even one such displacement in a FEVER-OOD-trained network would break the claimed guarantee.","tokens_in":21513,"feed_emoji":"🛡️","tokens_out":6145,"duration_ms":43307,"temperature":0.7,"pith_summary":"The paper argues that the free energy score, a widely used uncertainty measure for out-of-distribution (OOD) detection, contains an inherent blind spot: when a classifier's last layer is linear, certain feature-space differences between in-distribution and OOD samples are invisible to the score, and other differences produce only a minimal energy change. The root cause is algebraic, not empirical: any displacement lying in the null space of the last-layer weight matrix leaves the logits and hence the free energy exactly unchanged, and the smallest non-zero change is controlled by the least singular value of that matrix. To counter this, the paper inserts a dimension-reducing layer before the classifier head and adds regularizers that raise the least singular value and balance the condition number. Empirically, these interventions improve OOD detection across several energy-based baselines and achieve a new reported best on ImageNet-100, where the false positive rate at 95% true positive rate drops from 39.98% to 35.93%. The significance is practical: models deployed in open-set settings can now be made less prone to silently accepting outlier inputs.","feed_headline":"Fix for free-energy OOD detection cuts false positives on ImageNet-100","feed_subtitle":"A dimension-reducing layer and least-singular-value regularizer lift Dream-OOD to 35.93% FPR95 from 39.98%.","key_machinery":"The central object is the linear last layer W_cls ∈ $R^{{d'×K}}$ of the classifier, because the free energy score depends on features only through the logits W_cls^T h(x). The rank-nullity theorem supplies the blind subspace: if d' > K, then Null(W_cls^T) is non-trivial, and any displacement δ in that subspace gives W_cls^T δ = 0 and hence an identical free energy. The singular value decomposition supplies the weakest detection direction: over all unit displacements orthogonal to the null space, the minimum logit change is d_b σ_min(W_cls), so outliers near the least singular vector are the hardest to separate. The paper's countermeasure combines three mechanisms: r-NSR, a learned linear map g': $R^{{d'}}$ → R^r with r < d' that shrinks the null space before the head; the least singular value regularizer, which adds λ_LSV σ_min(W_cls)^{-1} to the training loss; and the condition number regularizer, which adds λ_CN κ(W_cls) to balance energy changes over all directions.","core_discovery":"The paper establishes that, for a classifier whose final layer is linear, f(x) = W_cls^T h(x), the free energy F(x) depends on the feature vector h(x) only through the logits. Consequently, any OOD feature displacement δ with h(v) = h(x) + δ that lies in the null space of W_cls^T leaves the logits, and therefore the free energy, exactly unchanged, so distinct feature representations can be OOD-invisible; this is the null space vulnerability (NSV). For displacements orthogonal to the null space, the smallest possible change in the logits, and by the paper's proxy argument the smallest change in free energy, is d_b σ_min(W_cls), attained along the direction of the least singular vector; this is the least singular value vulnerability (LSVV). The paper's proposed remedy, FEVER-OOD, inserts an extra linear layer that projects features down to dimension r < d' before the head (r-NSR), adds a least singular value regularizer that maximizes σ_min(W_cls), and optionally a condition number regularizer that equalizes energy changes across directions. Across CIFAR-10, CIFAR-100, ImageNet-100, and object-detection benchmarks built on VOS, FFS, and Dream-OOD baselines, the combined method consistently improves FPR95 and AUROC, with the headline ImageNet-100 result reaching 35.93% average FPR95 versus the 39.98% baseline.","pith_inferences":["Beyond the paper's direct experiments, the null-space argument transfers to any confidence score that depends on features through a linear head, so the dimension-reduction and singular-value regularizers could plausibly strengthen softmax-based and logit-based OOD detectors as well.","The least singular value analysis also describes a potential attack surface: an adversary who can manipulate feature representations could place OOD samples along the least singular vector to make them look in-distribution, and the LSV regularizer effectively raises the cost of that attack.","A testable extension would be to freeze the feature extractor and apply only the null-space-reducing layer and least-singular-value regularizer at the head, isolating how much of the benefit comes from the head geometry rather than from joint retraining.","The ablation results suggest the method's benefit is largest when the reduction target r is chosen just below the feature dimension; extremely small r (for example r = 10 on CIFAR-10) degrades in-distribution accuracy, indicating that the null-space cure can be overdosed."],"forward_implications":["Any energy-based OOD detector built on a classifier whose feature dimension exceeds its class count inherits a non-trivial null space and therefore a set of OOD displacements that are invisible to the free energy score.","Reducing the feature dimension before the final linear layer shrinks that blind subspace and improves OOD detection without degrading in-distribution accuracy, provided the reduction is not extreme.","Raising the least singular value of the last layer increases the free energy change along the previously weakest direction, so OOD samples near that direction become detectable.","On ImageNet-100, applying FEVER-OOD to the Dream-OOD baseline yields 35.93% average FPR95 versus 39.98% and 93.12 AUROC versus 91.64, which the paper reports as state of the art for that benchmark.","The same null-space and least-singular-value reasoning applies to object-level OOD detection, where the paper reports improved FPR95 and AUROC on MS-COCO and OpenImages with PASCAL VOC as in-distribution data."],"supporting_citations":[{"why":"Supplies the free energy scoring function that the paper analyzes and whose vulnerabilities it identifies.","marker":"[26]"},{"why":"Provides the VOS baseline, an energy-based OOD method with virtual outlier synthesis, to which FEVER-OOD is applied in classification and detection experiments.","marker":"[6]"},{"why":"Provides the Dream-OOD baseline, including the ImageNet-100 setup and the baseline numbers that FEVER-OOD improves to the paper's reported state of the art.","marker":"[7]"},{"why":"Provides the FFS baseline, a normalizing-flow outlier synthesis method used as a second energy-based OOD baseline for FEVER-OOD.","marker":"[19]"},{"why":"Inspires the minimum-change analysis by supplying the adversarial perturbation framework that the paper adapts from maximizing to minimizing response change over bounded displacements.","marker":"[34]"}],"fun_headline_variants":["Free-energy OOD detection gets a null-space fix","FEVER-OOD squashes false positives on ImageNet-100","Lower-dim layers plug free-energy OOD holes","Regularizer beats OOD false positives at 95% TPR","Null-space vulnerability patched in free-energy OOD"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on the free energy's log-sum-exp nonlinearity being monotone enough that the direction of minimal logit change is also the direction of minimal free energy change, and on the last layer being full rank; either failing could leave an OOD direction that is still nearly invisible.","fun_headline_variants_meta":{"raw":{"variants":["Free-energy OOD detection gets a null-space fix","FEVER-OOD squashes false positives on ImageNet-100","Lower-dim layers plug free-energy OOD holes","Regularizer beats OOD false positives at 95% TPR","Null-space vulnerability patched in free-energy OOD"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000246,"raw_usage":{"total_tokens":1631,"prompt_tokens":1125,"completion_tokens":506,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":741,"completion_tokens_details":{"reasoning_tokens":425}},"tokens_in":741,"tokens_out":506,"duration_ms":4893,"temperature":1.0,"reasoning_tokens":425,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:16:59.736229+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained classifier with a linear last layer, extract real feature vectors, fix a displacement norm d_b, and compare the actual free energy change along the least singular vector with the change along the next few singular vectors; if the least singular vector no longer gives the smallest free energy change, the proxy argument behind Eq. (12) fails. More decisively, run a feature-space search for a displacement with norm at least d_b whose free energy change is smaller than d_b σ_min(W_cls); finding even one such displacement in a FEVER-OOD-trained network would break the claimed guarantee.","supporting_citations":[{"cited_title":"Energy-based out-of-distribution detection","cited_arxiv_id":null,"evidence_quote":"Supplies the free energy scoring function that the paper analyzes and whose vulnerabilities it identifies."},{"cited_title":"V os: Learning what you don’t know by virtual outlier synthesis","cited_arxiv_id":null,"evidence_quote":"Provides the VOS baseline, an energy-based OOD method with virtual outlier synthesis, to which FEVER-OOD is applied in classification and detection experiments."},{"cited_title":"Dream the impossible: Outlier imagination with diffusion mod- els","cited_arxiv_id":null,"evidence_quote":"Provides the Dream-OOD baseline, including the ImageNet-100 setup and the baseline numbers that FEVER-OOD improves to the paper's reported state of the art."},{"cited_title":"Normalizing flow based feature synthe- sis for outlier-aware object detection","cited_arxiv_id":null,"evidence_quote":"Provides the FFS baseline, a normalizing-flow outlier synthesis method used as a second energy-based OOD baseline for FEVER-OOD."},{"cited_title":"First-order ad- versarial vulnerability of neural networks and input dimen- sion","cited_arxiv_id":null,"evidence_quote":"Inspires the minimum-change analysis by supplying the adversarial perturbation framework that the paper adapts from maximizing to minimizing response change over bounded displacements."}],"review_version":1}