{"id":"ec46b170-5277-41c4-ab89-43df513c1389","arxiv_id":"2507.09091","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An implicit neural network framework learns PCA or ICA decompositions directly from irregularly sampled continuous signals.","lead":"This paper describes a way to perform PCA and ICA style decomposition when data points are scattered irregularly instead of arranged in a grid. It trains neural networks to represent both the patterns being found and how strongly each pattern appears, using loss terms that enforce decorrelation or independence.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Assumption 1 cannot justify replacing ensemble expectations with time averages in Eqs. (4), (6), and (7); ergodicity is required and not stated, and the CQT experiment uses nonstationary note activations, so the PCA/ICA statistical claim is not established.","rationale":"The reader's weakest assumption and my load-bearing concern coincide: the statistical interpretation of the framework rests on substituting time averages for ensemble expectations. I agree that this is the point where the argument is least secure. In the synthetic sinusoid example the replacement can accidentally work because deterministic periodic components are close to ergodic, but the paper makes a general claim and the CQT demonstration violates stationarity. The paper also lacks a quantitative baseline; however this is a validation issue rather than a flaw in the formulation. My check directly tests whether the optimized loss estimates the ensemble quantities required by Eq. (1). If the test passes, the concern is resolved; if it fails, the verdict should remain CONDITIONAL or move toward REJECT. Because the concern is the same one that motivates the reader's conditional verdict, I recommend no change.","tokens_in":8989,"tokens_out":11480,"duration_ms":159497,"concrete_test":"Construct a stationary non-ergodic source process, e.g., S_t^(i)(omega) = A_i(omega) cos(2 pi f_i t + Theta_i(omega)) with Theta_i uniform and A_i drawn once per realization from a distribution chosen so that the realized A_i^2/2 differs substantially from E[A_i^2]/2; mix it with a known T, train the proposed loss on the single observed path, and compare the covariance actually optimized, the time average over t of the learned H, with the true ensemble covariance E[S S^T]. If they differ substantially, the loss is not implementing PCA/ICA as defined in Eq. (1), settling the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that minimizing the full loss (5) trains the network to recover the source process S and mixing operator T in the PCA/ICA sense. All three loss terms, L_reconstr in Eq. (4), phi_PCA in Eq. (6), and phi_ICA in Eq. (7), are written as ensemble expectations E[.]. The dataset is a single sample path D = {X_{t_1}(omega), ..., X_{t_N}(omega)}, so each X_t is observed once. To estimate the expectations, the paper averages over the observed time points t_i, invoking Assumption 1 (stationarity). Stationarity is not sufficient for this replacement: equality of time averages and ensemble averages is an ergodicity property. For any non-ergodic stationary source, for example a process whose amplitude or phase is drawn once per realization, the time average over one path does not approach E[S^(i) S^(j)], so the contrast functions (6) and (7) are not enforcing decorrelation or independence of the random variables S^(i), but only an empirical property of the particular observed path. The audio experiment aggravates this: the CQT activations in Fig. 3 are sparse, switching note envelopes, which are nonstationary over the observed interval, so the premise of Assumption 1 visibly fails. In addition, the reported covariance in Fig. 2 is close to the exact quantity minimized by Eq. (6), so the success metric is partly circular, and footnote 1 concedes that the basis orthogonality needed for classical PCA is not imposed, leaving an extra orthogonal-rotation ambiguity in the recovered components.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a continuous-time generalization of PCA and ICA using implicit neural representations. It models observed data as a single sample path X_t = T·S_t of a stationary stochastic process, with unknown source process S_t and unknown linear operator T. The method learns neural approximations of the source functions and basis functions by minimizing a reconstruction loss plus a contrast function that enforces decorrelation (PCA) or nonlinear decorrelation (ICA). The authors argue this framework applies to irregularly sampled data where standard matrix-based PCA/ICA fail, and demonstrate it on an eigenface experiment and a constant-Q transform audio separation experiment. The central claim is that minimizing the proposed loss (5) recovers the desired source signals and mixing operator in the PCA/ICA sense.","tokens_in":9356,"tokens_out":4859,"duration_ms":58660,"significance":"If the central claim were fully supported, the paper would provide a useful unification of PCA and ICA in a continuous-time, irregularly-sampling setting, with potential applications to audio and point-cloud processing. The idea of using implicit neural representations with contrast losses is natural and the paper is clearly written. However, the statistical justification is incomplete (stationarity vs. ergodicity), the experiments are thin and partly circular, and the method's identifiability is limited by the intentionally omitted basis-orthogonality constraint. The paper would be a contribution to an MLSP workshop, but as a journal paper it needs substantial revision to establish the statistical and empirical claims.","major_comments":[{"comment":"The paper replaces ensemble expectations E[·] with averages over the observed time points t_i in D, invoking Assumption 1 (stationarity). Stationarity alone does not justify this replacement; ergodicity is required. For a stationary but non-ergodic process, e.g., a mixture of sinusoids whose phases and amplitudes are drawn once per realization, the time average over a single sample path does not converge to the ensemble expectation. The contrast functions (6) and (7) therefore do not in general enforce decorrelation/independence of the random variables S^(i); they only enforce empirical properties of the particular observed path. This is load-bearing for the central claim that the trained network recovers a PCA/ICA decomposition. The authors should either add an explicit ergodicity assumption and discuss its applicability, or reframe the method as a deterministic low-rank decomposition of the observed function with no statistical guarantee.","section":"Section 1.2 and Eqs. (4), (6), (7)"},{"comment":"The CQT experiment uses sparse, switching note activations (notes played in isolation, then overlapping), which are visibly nonstationary over the observation interval. This violates the paper's own Assumption 1, so the time-averaged contrast loss used here has no clear relation to statistical independence of the source process. Moreover, the reported 'minimal leakage' is exactly the quantity that phi_ICA in Eq. (7) drives toward zero, making the success metric partly circular. To support the ICA claim, the authors need a nonstationarity-aware formulation or a quantitative evaluation on synthetic mixtures with known ground truth and performance metrics such as signal-to-interference ratio, together with repeated runs.","section":"Section 3.2, Figure 3"},{"comment":"The reported off-diagonal covariance values of about 10^-3 are the quantity directly minimized by the PCA contrast function phi_PCA in Eq. (6); reporting this as evidence of decorrelation is circular. The non-circular evidence (79% variance explained vs. 85% for traditional PCA and the visual similarity to eigenfaces) is encouraging but is not enough on its own. No hyperparameters, network architecture details, learning rates, number of sampled points per image, or repeated trials are provided, so the robustness of the result cannot be assessed. Please include the full training setup and report variability across runs.","section":"Section 3.1, Figure 2"},{"comment":"The paper explicitly does not impose mutual orthogonality of the basis functions, so the learned decomposition under phi_PCA is a decorrelating transform but not PCA in the classical sense. Any orthogonal rotation of the basis functions preserves decorrelation, leaving an identifiability ambiguity. The claim in Section 1.1 that the method 'extract[s] the same information as traditional matrix PCA' is therefore not fully supported. The authors should either add an orthogonality constraint to the loss and verify it does not degrade reconstruction, or explicitly discuss the rotational ambiguity and its implications for recovering the mixing operator.","section":"Section 2, footnote 1"},{"comment":"Algorithm 1 updates network parameters after each individual data point, but the loss terms in Eqs. (4), (6), and (7) are defined as expectations over the process, approximated by averaging over time points. The batching scheme is unclear: if the contrast loss is computed on a single time point per update, it is not an estimator of the ensemble covariance or independence criterion. The paper should specify the minibatch size and the exact empirical estimator used for the contrast term, and explain how the stationary/ergodic average is approximated during stochastic gradient descent.","section":"Section 2, Algorithm 1"}],"minor_comments":[{"comment":"The matrix norm ||·|| is not defined; specify the Frobenius norm or another norm.","section":"Eqs. (6) and (7)"},{"comment":"The extension of a discrete-time process to continuous time via 'Remark 4.2 [7]' is too terse; elaborate the construction and its statistical equivalence explicitly.","section":"Section 1.2"},{"comment":"The labels 'Eigenvector value' and 'Eigenvalue' in the top row are confusing because the figure shows basis functions, not eigenvectors; consider renaming them.","section":"Figure 1"},{"comment":"The number of randomly sampled points per image and the total number of training tuples are not reported; add these details to allow reproducibility.","section":"Section 3.1"},{"comment":"The loop index N is used both for the dataset size and the number of data points; also specify the optimizer and learning-rate schedule used in the experiments.","section":"Algorithm 1"},{"comment":"In Eq. (4), X and \\hat{X} are written as random variables, but the dataset is a single realization; this notational conflation should be clarified in the text.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is better suited to a workshop than a top journal in its current form. The core idea is simple and plausible, but the statistical argument needs an ergodicity assumption or a substantial reframing, and the experiments need to be considerably expanded with non-circular metrics and reproducibility details. If the authors can address these, the paper could become a solid contribution; otherwise the load-bearing claims remain unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a legitimate idea—training implicit neural bases and activations with a contrast loss to do PCA or ICA on continuously indexed, irregularly sampled signals. It is not a huge theoretical leap, since functional PCA already covers the principle, but the paper's specific contribution is a practical, model-agnostic neural machinery that unifies PCA and ICA and handles point-cloud-style data. That is worth something.\n\nWhat it does well: the problem statement is clear, the connection to fPCA and Karhunen-Loeve is honestly drawn, and the authors ship code. The eigenface demo is suggestive: even with random pixel sampling, the learned bases look like eigenfaces, and the 79% vs 85% variance capture gives a semi-independent sanity check. The CQT example shows the intended use case for irregular time-frequency data.\n\nSoft spots: the biggest one is the ergodicity gap. Assumption 1 is stationarity, but the loss functions in Eqs. (4), (6), and (7) are ensemble expectations. The paper replaces these with time averages over one observed sample path. Stationarity alone does not justify that; you need ergodicity. For the CQT example the activations are sparse, switching notes, which is visibly nonstationary, so the statistical interpretation is not established. That does not kill the method as a heuristic—it can still learn a useful decomposition—but the title and framing overclaim.\n\nSecond, the success metrics are partly circular. The near-zero off-diagonal covariance in Fig. 2 is exactly what phi_PCA minimizes, and the ICA 'minimal leakage' is what phi_ICA enforces. So those figures do not validate the approach. What would validate it: comparison against fPCA or against vanilla PCA on a grid-interpolated version of the same data, plus quantitative separation metrics for the audio example.\n\nThird, there is no baseline, no hyperparameter table, no repeated-run variance. For a workshop paper that is common, but for a claim of generalizing PCA/ICA it is thin. Also footnote 1 concedes that basis orthogonality is not enforced for PCA, so the recovered components carry an orthogonal rotation ambiguity; that should be stated in the main text, not only in a footnote.\n\nOverall, I think the central idea is sound as a neural heuristic and may well be useful for missing-data and irregular-sampling problems. The statistical framing needs reworking, and the validation needs external anchors. It deserves a serious referee and a request for major revision rather than a desk reject. I would not cite it in its present form, but I would keep an eye on a revised version.\n\nFor your reading group: maybe—good for a discussion about when ergodicity assumptions are silently invoked.","headline":"A useful-looking implicit neural framework for irregularly sampled PCA/ICA, but the statistical grounding is thinner than the paper claims and the experiments are mostly suggestive.","tokens_in":9863,"tokens_out":2515,"would_cite":false,"duration_ms":29969,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes that PCA and ICA can be solved on continuous-time signals by modeling data as a sample path of a stochastic process and training implicit neural networks with a contrast-function loss, recovering the latent sources and…","keywords":["continuous-time stochastic processes","implicit neural representations","principal component analysis","independent component analysis","irregular sampling","contrast function","low-rank decomposition","blind source separation"],"falsifier":"Synthesize a known stationary mixture $X_t=T S_t$ with independent, unit-variance source processes, sample it at random times and random function coordinates, train the implicit network, and compare the recovered sources and mixing operator with the ground truth up to permutation and scaling. If the reconstruction error is small but the recovered activations' sample covariance has off-diagonal entries far from zero (for PCA) or the recovered sources remain measurably dependent (for ICA), then the claim that the contrast loss enforces the target statistics on irregular samples is false.","tokens_in":8781,"feed_emoji":"🎛️","tokens_out":9272,"duration_ms":100150,"temperature":0.7,"pith_summary":"Principal component analysis and independent component analysis are usually defined on finite data matrices, which rules out signals that are sampled irregularly or exist on continuous domains. This paper treats an observed dataset as discrete samples of a single sample path of a continuous-time stochastic process, and represents both the latent source signals and the mixing operator as implicit neural networks. A reconstruction loss combined with a contrast function—decorrelation for PCA, nonlinear decorrelation for ICA—yields a rank-$k$ decomposition whose components are continuous functions rather than fixed-size vectors. The authors demonstrate the same machinery on irregularly sampled face images and on a non-uniform constant-Q spectrogram, recovering eigenfaces and note spectra where matrix PCA and ICA cannot be applied. If the claim holds, dimensionality reduction and blind source separation become available for point clouds, missing-data settings, and any signal with no natural matrix organization.","feed_headline":"Neural PCA and ICA now work on irregularly sampled signals","feed_subtitle":"One loss function unifies PCA and ICA for irregularly sampled, continuous-time signals.","key_machinery":"The carrying mechanism is the neural parametrization of the infinite-dimensional decomposition. Each basis function $\\hat f_n$ and each activation function $\\hat H_n$ is a small feedforward network with Fourier positional encodings applied to its one-dimensional input; together they define $\\hat T(\\xi)=[\\hat f_1(\\xi),\\dots,\\hat f_k(\\xi)]$ and $\\hat S_t=[\\hat H_1(t),\\dots,\\hat H_k(t)]^\\top$. Inserting these into the rank-$k$ reconstruction turns the PCA and ICA problems into an unconstrained optimization over network weights. The contrast function $\\phi$ carries the statistical content and is the only part that changes between the two problems: Eq. (6) enforces decorrelation and Eq. (7) enforces nonlinear decorrelation toward independence. The stationarity assumption on the source process lets the expectations in the losses be estimated by averaging over observed time points, which is what makes a single sample path statistically usable.","core_discovery":"The central claim is that PCA and ICA are not inherently discrete operations: both can be obtained as the minimizer of a single continuous loss. Given observations $D=\\{X_{t_1}(\\omega),\\dots,X_{t_N}(\\omega)\\}$ drawn from one sample path of a continuous-time process, the observed signal is approximated by $\\hat X_t(\\xi)=\\sum_{n=1}^k \\hat H_n(t)\\hat f_n(\\xi)$, where the neural networks $\\hat H_n$ and $\\hat f_n$ represent the source processes and the basis functions that form the mixing operator. Training the networks to minimize $\\mathcal L=\\mathcal L_{\\text{reconstr}}+\\beta\\phi(\\hat S)$ recovers both the latent sources and the operator. For PCA the contrast function penalizes the deviation of the covariance of $\\hat S$ from a diagonal matrix; for ICA it penalizes correlation between $\\varphi(S)$ and $S$ for a nonlinear $\\varphi$, a criterion the paper adopts as a general independence measure. Because each term in the loss can be evaluated at any $(t,\\xi)$ pair, no regular sampling grid is required, and the learned decomposition returns continuous functions that can be evaluated anywhere.","pith_inferences":["The paper leaves implicit that the same loss template could enforce constraints beyond decorrelation and independence; choosing a different contrast function (for instance one penalizing non-negativity or sparsity) would yield other neural decompositions with no change to the architecture.","For non-stationary sources, including the sparse, switching note activations in the audio experiment, the learned separation is better read as a structural factorization than as a statistical ICA solution, since the stationarity assumption that licenses time-averaged expectations is violated.","Because the bases are implicit functions evaluable anywhere, a natural untested extension is missing-data imputation and reconstruction at resolutions finer than the training samples, which the paper mentions only as future application."],"forward_implications":["Irregularly sampled data, point clouds, and non-uniform transforms become valid inputs for PCA and ICA, because the losses are evaluated pointwise at whatever $(t,\\xi)$ coordinates are observed.","The recovered decomposition is continuous: the same learned source signals and bases can be evaluated at any time or coordinate, including locations not present during training.","A single algorithm solves both PCA and ICA; the only change is the contrast function in the loss.","Standard discrete PCA and ICA are contained as a special case: a discrete dataset can be embedded as a piecewise-constant continuous process, and sampling the learned solution at the original indices recovers a classical matrix decomposition.","The construction extends to data whose points are themselves multidimensional functions, such as images, by letting the basis networks accept multiple coordinate inputs."],"supporting_citations":[{"why":"Supplies the standard PCA problem statement and decorrelation objective that the continuous formulation generalizes.","marker":"[1]"},{"why":"Provides the ICA formulation, the contrast-function concept, and the nonlinear decorrelation criterion used in Eq. (7).","marker":"[3]"},{"why":"Supports the reduction of discrete PCA/ICA to the continuous setup by filling gaps with constant values to extend a discrete process.","marker":"[7]"},{"why":"Provides the functional PCA theory for vector random functions that this method generalizes beyond discretely indexed observations.","marker":"[8]"},{"why":"Supplies the Fourier positional encodings that allow the small networks to fit high-frequency functions.","marker":"[13]"},{"why":"Shows the implicit-neural-representation approach applied to non-negative matrix factorization on irregularly sampled audio data, the direct precedent.","marker":"[16]"},{"why":"Supports the claim that the nonlinear decorrelation criterion in Eq. (7) is as general as possible among ICA contrast criteria.","marker":"[19]"},{"why":"Provides the face image dataset used in the eigenface experiment.","marker":"[20]"},{"why":"Defines the constant-Q spectral transform whose non-uniform sampling motivates the spectrogram experiment.","marker":"[21]"},{"why":"Provides the nonstationary Gabor-frame variant of the constant-Q transform with non-uniform time and frequency sampling used in the audio experiment.","marker":"[22]"}],"fun_headline_variants":["One loss unifies PCA and ICA for continuous-time signals","Neural nets generalize PCA and ICA to any sampling pattern","Continuous-time PCA and ICA via a single contrast function","Implicit neural nets decompose irregularly sampled signals","Unified PCA/ICA for point clouds and irregular time series"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the unobserved source signals have statistics that do not change over time, so that the single observed sample path can stand in for many independent draws of the same process; if this premise fails, the components learned by the contrast loss are not PCA or ICA components in any statistical sense.","fun_headline_variants_meta":{"raw":{"variants":["One loss unifies PCA and ICA for continuous-time signals","Neural nets generalize PCA and ICA to any sampling pattern","Continuous-time PCA and ICA via a single contrast function","Implicit neural nets decompose irregularly sampled signals","Unified PCA/ICA for point clouds and irregular time series"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000156,"raw_usage":{"total_tokens":1188,"prompt_tokens":887,"completion_tokens":301,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":222}},"tokens_in":503,"tokens_out":301,"duration_ms":3890,"temperature":1.0,"reasoning_tokens":222,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:05:08.846596+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Synthesize a known stationary mixture $X_t=T S_t$ with independent, unit-variance source processes, sample it at random times and random function coordinates, train the implicit network, and compare the recovered sources and mixing operator with the ground truth up to permutation and scaling. If the reconstruction error is small but the recovered activations' sample covariance has off-diagonal entries far from zero (for PCA) or the recovered sources remain measurably dependent (for ICA), then the claim that the contrast loss enforces the target statistics on irregular samples is false.","supporting_citations":[{"cited_title":"Continuous-Time Signal Decomposition: An Implicit Neural Generalization of PCA and ICA","cited_arxiv_id":"2507.09091","evidence_quote":"Supplies the standard PCA problem statement and decorrelation objective that the continuous formulation generalizes."},{"cited_title":"First, we examine the case of extracting face features via PCA (eigenfaces), but from irregularly sampled images","cited_arxiv_id":null,"evidence_quote":"Provides the ICA formulation, the contrast-function concept, and the nonlinear decorrelation criterion used in Eq. (7)."},{"cited_title":"Wiley, New York, 2001","cited_arxiv_id":null,"evidence_quote":"Supports the reduction of discrete PCA/ICA to the continuous setup by filling gaps with constant values to extend a discrete process."},{"cited_title":"A Unifying Review of Linear Gaussian Models,","cited_arxiv_id":null,"evidence_quote":"Provides the functional PCA theory for vector random functions that this method generalizes beyond discretely indexed observations."},{"cited_title":"Func- tional data analysis for sparse longitudinal data,","cited_arxiv_id":null,"evidence_quote":"Supplies the Fourier positional encodings that allow the small networks to fit high-frequency functions."},{"cited_title":"Levy, Karhunen Loeve Expansion of Gaussian Pro- cesses, pp","cited_arxiv_id":null,"evidence_quote":"Shows the implicit-neural-representation approach applied to non-negative matrix factorization on irregularly sampled audio data, the direct precedent."},{"cited_title":"Nerf: representing scenes as neural radiance fields for view synthe- sis,","cited_arxiv_id":null,"evidence_quote":"Supports the claim that the nonlinear decorrelation criterion in Eq. (7) is as general as possible among ICA contrast criteria."},{"cited_title":"Rethinking Non-Negative Matrix Factorization with Implicit Neural Representations","cited_arxiv_id":"2404.04439","evidence_quote":"Provides the face image dataset used in the eigenface experiment."},{"cited_title":"Point Cloud Audio Processing,","cited_arxiv_id":null,"evidence_quote":"Defines the constant-Q spectral transform whose non-uniform sampling motivates the spectrogram experiment."},{"cited_title":"Robust learn- ing algorithm for blind separation of signals,","cited_arxiv_id":null,"evidence_quote":"Provides the nonstationary Gabor-frame variant of the constant-Q transform with non-uniform time and frequency sampling used in the audio experiment."}],"review_version":1}