{"id":"58f9f5ee-edbe-42c2-aaa3-f368aeb4ac41","arxiv_id":"2502.00168","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Supervised Quadratic Feature Analysis learns linear features by maximizing Fisher-Rao distances between Gaussian class conditionals, achieving competitive classification accuracy and best results with a Hellinger-distance variant.","lead":"This paper introduces SQFA, a supervised method that learns low-dimensional linear features by maximizing the Fisher-Rao distance, an information-geometric measure, between Gaussian class distributions. On three real-world datasets, SQFA features match or exceed standard dimensionality reduction methods, with a Hellinger-distance variant performing best.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Calvo-Oller bound is validated only on original data-space class pairs, not on the low-dimensional feature-space distributions SQFA actually optimizes; a loose bound can make SQFA maximize a surrogate rather than the Fisher-Rao distance.","rationale":"The reader's weakest assumption is the tightness of the Calvo-Oller bound and its generalization to new datasets. I agree this is the right area but want to sharpen it: the empirical validation in Appendix B is done at the raw-data statistics, whereas the optimization lives in projected feature space. A lower bound can be loose at exactly the points the optimizer visits; tightness at raw data does not imply tightness along the optimization path. This is internal to the paper's own framing: Section 2.2 motivates the objective as accumulated local discriminability, and Section 2.3 substitutes a bound for lack of a closed-form expression. The experimental claim that SQFA variants are competitive does not depend only on this—the method could still work as a heuristic—but the paper's central conceptual claim that SQFA maximizes Fisher-Rao distances does depend on the surrogate. I do not recommend rejection because the method is clearly specified, computationally efficient, and empirically competitive, and because the concern is directly testable. The reader's CONDITIONAL verdict stands; adding the feature-space tightness check would either resolve or confirm the condition.","tokens_in":24169,"tokens_out":8970,"duration_ms":94797,"concrete_test":"On MNIST (the dataset with nonzero class means), compute for the filters F learned by SQFA the true Fisher-Rao distances between all pairs of projected Gaussians N(F^T gamma_i, F^T Phi_i F) using pyBregMan's numerical method, and compare them to the Calvo-Oller values used during training; report maximum and median relative slack. Then, for a small version (e.g., m=2 filters and a subset of classes), optimize Equation 1 with the numerical true distance substituted for the Calvo-Oller bound, and compare achieved QDA accuracy and Fisher-Rao value against SQFA's solution. If the bound-optimal filters are substantially worse on the true-distance objective, or if the ranking of candidate filters by the bound disagrees with the ranking by true distance, the surrogate is unfaithful and the central information-geometric claim is weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.3 defines SQFA's general-Gaussian objective as maximizing d_AI(Omega_i, Omega_j)/sqrt(2), the Calvo-Oller lower bound on the Fisher-Rao distance. Appendix B (Figure 10) validates tightness by comparing the bound to the numerically computed true distance for class-conditional Gaussians in the original data space. But SQFA's optimization (Equation 1) operates on the projected Gaussians N(F^T gamma_i, F^T Phi_i F), whose means and covariances are not the ones validated. The bound is exact when means are equal and can be loose otherwise (Figure 10 shows MNIST deviations). Maximizing a lower bound need not maximize the true distance: an optimizer can select projections where the bound-to-true-distance ratio is especially favorable. Thus the paper's central claim—that SQFA learns features maximizing Fisher-Rao distances for general Gaussians—is not established. The geometric motivation in Section 2.2 is load-bearing only if the optimized surrogate tracks the true geodesic distance at the feature-space optima, not merely at the raw-data statistics.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Supervised Quadratic Feature Analysis (SQFA), a linear supervised dimensionality reduction method that learns filters by maximizing Fisher-Rao distances between class-conditional Gaussian distributions in the low-dimensional feature space. For zero-mean Gaussians it uses the exact affine-invariant distance (smSQFA), and for general Gaussians it uses the Calvo-Oller lower bound as a closed-form surrogate. The authors compare SQFA and its Bhattacharyya and Hellinger variants against PCA, LDA, SPCA, and LMNN on SVHN, MNIST, and a naturalistic speed-estimation task, reporting QDA accuracies in which SQFA variants are competitive and SQFA-H is often best. The paper includes several appendices on the distance proxies, computational complexity, invariance properties, and robustness to non-Gaussianity, and it provides a Python implementation.","tokens_in":24387,"tokens_out":4407,"duration_ms":46190,"significance":"If the central claims hold, SQFA is a computationally efficient linear feature learning method that gives the Fisher-Rao geometry a practical role in supervised dimensionality reduction, and the paper provides evidence that the rarely used Hellinger distance is a strong multiclass objective. The manuscript is careful in several respects: the method is precisely specified, the optimization cost is analyzed (Appendix D), the Calvo-Oller bound is numerically compared with the true Fisher-Rao distance (Appendix B), robustness of the evaluation to non-Gaussianity is checked with Gaussian-simulated data and a kNN decoder (Appendix G), and a public implementation is provided. The main unresolved issue is that the validity of the general-Gaussian SQFA objective is established only on raw-data class pairs, not on the feature-space distributions actually optimized, so the geometric interpretation is not yet fully supported at the points where it matters.","major_comments":[{"comment":"The claim that SQFA maximizes Fisher-Rao distances for general Gaussians rests on the Calvo-Oller bound being a faithful surrogate. Appendix B validates the bound by comparing it with the numerically computed true Fisher-Rao distance for class-conditional Gaussians in the original data space. However, the optimization in Eq. (1) operates on the projected feature-space parameters (µ_i = F^T γ_i, Σ_i = F^T Φ_i F), whose means and covariances are different from the raw-data pairs validated in Figure 10. Maximizing a lower bound need not maximize the true distance if the tightness of the bound varies across projections, and Figure 10 shows nontrivial deviations for MNIST. The geometric motivation in Section 2.2 is therefore load-bearing only if the surrogate tracks the true geodesic distance at the feature-space optima. Please add a validation of the bound at the learned feature-space distributions for each dataset, or provide a theoretical guarantee on the bound's tightness under projection.","section":"Section 2.3, Appendix B (Figure 10)"},{"comment":"The empirical comparison is based on point estimates. The SQFA variants are reported with medians and interquartile ranges over 20 initializations, but the baseline methods have no error bars and no significance testing is reported. The abstract's claim that SQFA-H achieves the best classification accuracy is not supported without uncertainty quantification on the differences. Please report variability over initializations and data resamples, or perform paired significance tests, for all methods, and state explicitly how many repetitions were used for each baseline.","section":"Sections 5.2-5.4, Figures 4-6"},{"comment":"The regularization parameter σ² is selected by grid search on a validation split for the digit datasets and fixed to 0.001 for the speed-estimation dataset, but no sensitivity analysis is reported. Since regularization breaks the invariance properties that motivate the objective and can bias filters toward directions with larger second-moment matrices, the comparison could depend on this choice. Please report the sensitivity of the main results to σ² and to the shrinkage parameter used for LDA, or otherwise justify that the chosen values do not drive the conclusions.","section":"Section 4, Appendix F.2"}],"minor_comments":[{"comment":"The text in Appendix B.1 refers to \"Figure 8\" when presenting the real-world comparison of the Calvo-Oller bound with the true Fisher-Rao distance, but the actual figure is Figure 10; please correct the cross-references throughout the supplement.","section":"Appendix B"},{"comment":"In the first paragraph of Appendix G, the phrase \"which that the data is Gaussian\" is ungrammatical and should read \"which assumes that the data is Gaussian.\"","section":"Appendix G"},{"comment":"The phrase \"alternative expression\" should be \"alternative expression\" (spelling), and the sentences introducing Equations (11) and (12) could be merged for readability.","section":"Appendix B.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope and the central methodological idea is promising. The main issue is the unvalidated surrogate objective at feature-space optima; this is addressable with additional numerical experiments and does not require a rejection. The empirical claims would also benefit from standard error reporting. The citation pattern appears appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, useful paper. It introduces a new objective for supervised linear dimensionality reduction—maximize Fisher-Rao distance between class-conditional Gaussians—and shows that on three real datasets (SVHN, MNIST, speed estimation) the resulting features support QDA accuracy competitive with or better than LDA, PCA, SPCA, LMNN, and Bhattacharyya-based objectives. The practical highlight is that the Hellinger variant (SQFA-H) consistently beats the Bhattacharyya variant in the multiclass case, which is new and likely useful to practitioners. The Calvo-Oller lower bound makes the Fisher-Rao objective tractable, the optimization is fast (seconds on a laptop), and the authors ship a clean Python implementation. They also check robustness to non-Gaussianity, both by simulating Gaussian data and by using a kNN decoder. I'd send this to review.\n\nThe main soft spot is exactly the one your stress-test flags. The tightness of the Calvo-Oller bound is validated in Appendix B on the raw data-space class pairs, not on the low-dimensional feature-space distributions that SQFA actually optimizes. The bound is exact for equal means and can be loose otherwise, and Figure 10 shows noticeable deviations on MNIST even at the raw-data level. So maximizing the bound does not necessarily maximize the true Fisher-Rao distance; the geometric interpretation is carried by a surrogate whose tightness at the optima is unverified. This does not sink the empirical results—the method still learns good features—but it weakens the central 'Fisher-Rao' claim. The fix is straightforward: for a few trained models, compute the true Fisher-Rao distance numerically on the feature-space distributions at the learned filters (using, say, pyBregMan, which they already cite) and show the bound tracks it. That would close the gap.\n\nOther issues are minor. The accuracy comparisons lack significance testing, and error bars are only shown for the SQFA variants, so 'SQFA-H best' is plausible but not statistically established. LMNN was trained on a reduced dataset (100 PCs, 10% of samples), which is a slightly unfair baseline. Regularization is grid-selected, which is fine but under-discussed. None of these change the main conclusion.\n\nWho is this for? Researchers in linear dimensionality reduction and information geometry, and anyone wanting a fast, interpretable, supervised feature extractor. It is not a field-reorganizer, but it is a clean contribution with reproducible code. Give it a serious referee, with the request to address the feature-space bound validation and add significance testing.","headline":"Clean new objective and a useful Hellinger finding, but the Fisher-Rao interpretation rests on a lower bound whose tightness at the optima is unverified; worth reviewing with that fix.","tokens_in":24890,"tokens_out":3999,"would_cite":true,"duration_ms":37874,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62H30","62B10","68T10"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that linear filters learned by maximizing Fisher-Rao distances between class-conditional Gaussians, with the Calvo-Oller lower bound as a closed-form surrogate, produce features whose QDA accuracy matches or beats LDA…","keywords":["Fisher-Rao distance","information geometry","supervised dimensionality reduction","quadratic discriminant analysis","Hellinger distance","Bhattacharyya distance","Calvo-Oller bound","linear feature learning"],"falsifier":"Compute the true Fisher-Rao distance for every class pair of a new dataset together with the Calvo-Oller bound, and check both the bound's tightness and whether SQFA's filters are the ones maximizing the bound. If on a dataset with strong mean-covariance interactions the bound is loose by a wide margin and SQFA's QDA accuracy falls below SQFA-H or LMNN, the claim that the surrogate faithfully represents the Fisher-Rao objective is undermined; a cleaner version would train filters for small $m$ by directly optimizing the numerical Fisher-Rao distance and compare their filters and accuracy with the bound-based SQFA.","tokens_in":23973,"feed_emoji":"📐","tokens_out":14033,"duration_ms":121078,"temperature":0.7,"pith_summary":"Supervised Quadratic Feature Analysis (SQFA) learns $m$ linear filters by maximizing pairwise Fisher-Rao distances between class-conditional Gaussian distributions in the projected space. Because the Fisher-Rao distance integrates local discriminability along the geodesic between distributions, the authors treat it as a geometrically motivated proxy for class separability. For general Gaussians the true distance has no closed form, so SQFA maximizes the Calvo-Oller lower bound, and a zero-mean variant (smSQFA) uses the exact affine-invariant distance on the manifold of covariance matrices. On SVHN, MNIST, and a naturalistic speed-estimation task, SQFA features support QDA accuracy competitive with or better than LDA, PCA, SPCA, LMNN, and Bhattacharyya-based objectives; the Hellinger variant SQFA-H is consistently the best. The paper's claim is that information geometry offers a practical, efficient objective for supervised linear dimensionality reduction, not just a theoretical framing.","feed_headline":"Hellinger variant of SQFA beats LDA, PCA, SPCA, and LMNN","feed_subtitle":"Maximizing Fisher-Rao distances between class distributions matches or beats standard linear methods on real data","key_machinery":"The central object is the Fisher-Rao distance on the manifold of Gaussian distributions, made computable through two surrogates. For zero-mean Gaussians the manifold is $\\mathrm{SPD}(m)$, and the distance is the affine-invariant distance $d_{AI}(A,B)=\\sqrt{\\sum_{k=1}^m \\log^2\\lambda_k}$, with $\\lambda_k$ the generalized eigenvalues of the pair $(A,B)$; this is the exact objective of smSQFA. For arbitrary Gaussians, the Calvo-Oller embedding $\\Omega_i=\\begin{pmatrix} \\Sigma_i+\\mu_i\\mu_i^T & \\mu_i \\\\ \\mu_i^T & 1 \\end{pmatrix}$ maps each class into $\\mathrm{SPD}(m+1)$, and the affine-invariant distance in that embedding is a lower bound that is exact when means are equal and, the paper argues, close to the true distance for its datasets. SQFA maximizes the sum of pairwise distances over the linear filter matrix $F\\in\\mathbb{R}^{n\\times m}$, with each gradient step costing $O(c^2m^3+cmn^2)$, which keeps training on a consumer laptop to seconds. The same optimization machinery with the Bhattacharyya and Hellinger distances defines SQFA-B and SQFA-H, isolating the effect of the objective function from the effect of the optimization procedure.","core_discovery":"SQFA's central claim is that maximizing Fisher-Rao distances between class-conditional Gaussians is a workable and competitive objective for supervised linear dimensionality reduction. In the zero-mean case the objective is exact: with class second-moment matrices $\\Psi_i$ and $\\Psi_j$, the Fisher-Rao distance equals the affine-invariant distance $d_{AI}(\\Psi_i,\\Psi_j)=\\sqrt{\\sum_{k=1}^m \\log^2 \\lambda_k}$, where $\\lambda_k$ are the generalized eigenvalues of the matrix pair. For general Gaussians, SQFA maximizes the Calvo-Oller bound $d_{FR}(\\theta_i,\\theta_j)\\ge d_{AI}(\\Omega_i,\\Omega_j)/\\sqrt{2}$ obtained by embedding each Gaussian into an $(m+1)\\times(m+1)$ symmetric positive definite matrix via its mean and covariance; the paper verifies empirically that the bound closely tracks the numerically computed Fisher-Rao distance for the class pairs in its three datasets. Using QDA accuracy as the evaluation criterion, SQFA features are competitive with features that maximize Bhattacharyya distance and with LDA, PCA, SPCA, and LMNN, and on the speed-estimation task they come close to AMA-Gauss, a method that directly optimizes quadratic decodability. A second, more specific claim is that the Hellinger variant, SQFA-H, consistently outperforms the Bhattacharyya variant, SQFA-B, in multiclass problems even though the two objectives are monotone-equivalent in the two-class case.","pith_inferences":["Editorial extension: the paper does not prove that the Hellinger advantage comes from its boundedness, but if it does, replacing other unbounded pairwise objectives with bounded variants could improve multiclass dimensionality reduction methods beyond SQFA.","Editorial extension: the tightness of the Calvo-Oller bound is validated only on the three datasets studied; a direct test would optimize the numerically computed Fisher-Rao distance for small $m$ and compare the resulting filters and QDA accuracy with those produced by the bound-based SQFA.","Editorial extension: since the same machinery extends to elliptical distributions, a natural next experiment is to repeat the comparisons on heavy-tailed class-conditional data, which would separate the Gaussian assumption from the information-geometry objective itself.","Editorial extension: in neuroscience, SQFA-style objectives could identify axes of neural population activity where condition-dependent covariance differences are largest; the paper frames this as a potential application but does not test it."],"forward_implications":["SQFA makes the Fisher-Rao distance usable in practice: a closed-form, laptop-scale optimization replaces the numerically expensive true distance, giving a linear method whose QDA accuracy matches or exceeds standard supervised and unsupervised baselines on image and naturalistic-video tasks.","The Hellinger distance, rarely used for dimensionality reduction, emerges as a consistently strong multiclass objective, outperforming the more common Bhattacharyya distance when classes are unequally separated.","Because the Calvo-Oller bound extends to other elliptical distributions (Student-t and Cauchy), the Gaussian assumption can be relaxed without abandoning the information-geometric objective.","On the speed-estimation task, SQFA features resemble motion-sensitive receptive fields and perform close to AMA-Gauss, which directly optimizes the quadratic decoder, suggesting that Fisher-Rao maximization is a good proxy for optimal quadratic decoding.","LDA is the special case of this framework when class covariances are equal: maximizing pairwise squared Mahalanobis distances is equivalent to the LDA criterion, so SQFA generalizes LDA to heteroscedastic classes."],"supporting_citations":[{"why":"provides the embedding into SPD(m+1) and the lower bound that SQFA maximizes as a surrogate for Fisher-Rao distance in the general Gaussian case.","marker":"(Calvo and Oller, 1990)"},{"why":"gives the affine-invariant Rao distance on SPD(m) that equals the Fisher-Rao distance for zero-mean Gaussians and is the exact objective of smSQFA.","marker":"(Atkinson and Mitchell, 1981)"},{"why":"supplies the statement that no closed-form Fisher-Rao distance exists for arbitrary Gaussians and the numerical method used to validate the Calvo-Oller bound.","marker":"(Nielsen, 2023)"},{"why":"frames dissimilarity maximization as a proxy for discriminability and connects generalized eigenvalues to quadratic discriminability.","marker":"(Fukunaga, 1990)"},{"why":"supplies the naturalistic speed-estimation dataset, its synthesis procedure, and the biologically motivated regularization value used in the speed experiment.","marker":"(Burge and Geisler, 2015)"},{"why":"provides AMA-Gauss, the benchmark method that directly optimizes QDA-style decodability and that SQFA approaches in the speed task.","marker":"(Jaini and Burge, 2017)"},{"why":"is the Bhattacharyya-distance feature extraction criterion used to define the SQFA-B comparison variant.","marker":"(Choi and Lee, 2003)"},{"why":"supports the point that pairwise dissimilarities do not determine multiclass error, motivating the empirical comparison and the interpretation of the Hellinger result.","marker":"(Loog et al., 2001)"},{"why":"provides the LMNN baseline used as a stand-in for Wasserstein discriminant analysis.","marker":"(Weinberger and Saul, 2009)"},{"why":"provides the supervised PCA (SPCA) baseline compared on all three datasets.","marker":"(Barshan et al., 2011)"}],"fun_headline_variants":["SQFA: Fisher-Rao distances for supervised dimensionality reduction","Hellinger variant of SQFA rivals top linear methods","Information geometry meets linear feature learning: SQFA","Fisher-Rao distance as a supervised learning objective","SQFA-H: Hellinger distance for best classification accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Calvo-Oller lower bound stays numerically close to the true Fisher-Rao distance for the class-conditional Gaussians of the data being processed; the paper verifies this on its three datasets but offers no guarantee for a new one.","fun_headline_variants_meta":{"raw":{"variants":["SQFA: Fisher-Rao distances for supervised dimensionality reduction","Hellinger variant of SQFA rivals top linear methods","Information geometry meets linear feature learning: SQFA","Fisher-Rao distance as a supervised learning objective","SQFA-H: Hellinger distance for best classification accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001341,"raw_usage":{"total_tokens":5536,"prompt_tokens":1118,"completion_tokens":4418,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":734,"completion_tokens_details":{"reasoning_tokens":4340}},"tokens_in":734,"tokens_out":4418,"duration_ms":33177,"temperature":1.0,"reasoning_tokens":4340,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T19:56:58.399923+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the true Fisher-Rao distance for every class pair of a new dataset together with the Calvo-Oller bound, and check both the bound's tightness and whether SQFA's filters are the ones maximizing the bound. If on a dataset with strong mean-covariance interactions the bound is loose by a wide margin and SQFA's QDA accuracy falls below SQFA-H or LMNN, the claim that the surrogate faithfully represents the Fisher-Rao objective is undermined; a cleaner version would train filters for small $m$ by directly optimizing the numerical Fisher-Rao distance and compare their filters and accuracy with the bound-based SQFA.","supporting_citations":[{"cited_title":"and Oller, J","cited_arxiv_id":null,"evidence_quote":"provides the embedding into SPD(m+1) and the lower bound that SQFA maximizes as a surrogate for Fisher-Rao distance in the general Gaussian case."},{"cited_title":"and Mitchell, A","cited_arxiv_id":null,"evidence_quote":"gives the affine-invariant Rao distance on SPD(m) that equals the Fisher-Rao distance for zero-mean Gaussians and is the exact objective of smSQFA."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the statement that no closed-form Fisher-Rao distance exists for arbitrary Gaussians and the numerical method used to validate the Calvo-Oller bound."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"frames dissimilarity maximization as a proxy for discriminability and connects generalized eigenvalues to quadratic discriminability."},{"cited_title":"and Geisler, W","cited_arxiv_id":null,"evidence_quote":"supplies the naturalistic speed-estimation dataset, its synthesis procedure, and the biologically motivated regularization value used in the speed experiment."},{"cited_title":"and Burge, J","cited_arxiv_id":null,"evidence_quote":"provides AMA-Gauss, the benchmark method that directly optimizes QDA-style decodability and that SQFA approaches in the speed task."},{"cited_title":"and Lee, C","cited_arxiv_id":null,"evidence_quote":"is the Bhattacharyya-distance feature extraction criterion used to define the SQFA-B comparison variant."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supports the point that pairwise dissimilarities do not determine multiclass error, motivating the empirical comparison and the interpretation of the Hellinger result."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the LMNN baseline used as a stand-in for Wasserstein discriminant analysis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the supervised PCA (SPCA) baseline compared on all three datasets."}],"review_version":1}