{"id":"cd8268f3-4781-4322-9894-b946b72eaae2","arxiv_id":"2411.09827","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A dissertation synthesizing the author's papers on continuous kernel convolutions and symmetry-preserving architectures, claiming these inductive biases improve deep learning efficiency.","lead":"This PhD thesis compiles ten previously published papers arguing that inductive biases, continuous modeling and symmetry preservation, make deep learning more efficient across compute, data, parameter, and design axes. It is a synthesis of the author's own results, with an explicit efficiency taxonomy and a critical limitations discussion.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central efficiency claim lacks a quantitative, unified protocol; 'design efficiency' is marked in Table 1.1 but never measured, leaving abstract's 'substantial benefits' non-falsifiable.","rationale":"The reader's weakest assumption is that the efficiency gains reported in the author's own papers, measured on small benchmarks and with favorable baselines, are representative of real-world efficiency. I agree: the central claim is a meta-claim about efficiency across four axes, and the evidence is self-reported with no unified benchmark or quantitative definition of design efficiency. My stress-test sharpens the same concern by pointing to a concrete missing measurement: Table 1.1 marks 'design efficiency' for several chapters, yet no chapter measures design cost. The Sec. 2.5 one-epoch neural-ODE baseline is another concrete instance of the evaluation-protocol problem, though as an equal-compute comparison it is not itself invalid; the issue is that it is reported without accuracy-vs-compute curves, which prevents the reader from distinguishing model capability from training budget. The thesis is a compilation of peer-reviewed work and the individual contributions are plausible, so a conditional verdict remains appropriate. The paper should be accepted only if the author either supplies an independent replication or clearly scopes the claim to the reported benchmarks. This does not change the reader's CONDITIONAL verdict; it reinforces it. No ad hominem is intended: the critique is about the evaluation protocol, not the author's integrity.","tokens_in":51895,"tokens_out":7782,"duration_ms":90100,"concrete_test":"Define a concrete proxy for design efficiency and re-run a head-to-head comparison on one new dataset, e.g., a NAS-Bench-360 task: record total GPU-hours and the number of human-specified architecture/kernel-size choices needed for DNArch (Ch. 6) and FlexConv (Ch. 5) versus a discrete NAS baseline and a manually tuned ResNet/Transformer to reach a fixed target accuracy. If the continuous-modeling pipeline does not require fewer resources under this proxy, Table 1.1's 'design efficiency' entries for Chs. 5-6 are unsupported. As a secondary check, replot the Sec. 2.5 Speech Commands raw comparison as accuracy vs. total compute (FLOPs or GPU-hours) for CKCNN and each neural-ODE baseline; if a baseline overtakes CKCNN at larger budgets, the compute-efficiency claim is equivocal.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the central claim, the thesis must show that the efficiency gains reported in the constituent papers are real and comparable. The load-bearing gap is the absence of a quantitative, unified efficiency protocol. Section 1.2.1 defines design efficiency qualitatively as 'resources in terms of compute, human hours, memory, experimentation, etc. needed to design a high-performing architecture,' and Table 1.1 assigns 'design efficiency' checkmarks to Chapters 2, 3, 5, and 6, but no chapter reports a measurement of such resources. For example, Ch. 5 claims FlexConv relieves users from pre-specifying kernel sizes, and Ch. 6 claims DNArch learns architectures by backpropagation, but neither measures the human or compute cost of achieving a target accuracy on a new dataset compared with a manual design or a discrete NAS baseline. Without this, the 'substantial benefits for design efficiency' stated in the abstract cannot be evaluated. Similar issues affect compute efficiency: each chapter uses different baselines, and speed comparisons are not accompanied by matched-budget accuracy-vs-compute curves. The one-epoch neural-ODE baselines in Sec. 2.5 are the most visible instance: the table reports their SC-raw accuracy as ~10.0, but the text reveals this is after a single training epoch, so the reader cannot separate model capacity from training budget. The thesis's own Ch. 12.1 limitations are qualitative and do not repair this missing metric. The central claim may be true, but the synthesis currently rests on the author's own categorization rather than on falsifiable measurement.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript is a PhD dissertation, posted on arXiv, which argues that two inductive biases—continuous modeling and symmetry preservation—can substantially improve the efficiency of deep learning along compute, data, parameter, and design axes. The first part (Chapters 2–6) presents continuous-kernel and continuous-architecture methods (CKConv, CCNN, gridification, FlexConv, DNArch), while the second part (Chapters 7–11) presents equivariant and partially equivariant architectures. The dissertation contributes an efficiency taxonomy (Sec. 1.2.1), a summary table of per-chapter efficiency contributions (Table 1.1), and a concluding chapter with explicit limitations. Because each chapter is based on previously published papers by the author, the central claim is a synthesis-level claim about efficiency rather than a report of new experiments. The review copy provided to me is truncated after Sec. 5.4.1, so Part II is assessed here only through its abstract, chapter summaries, and Table 1.1.","tokens_in":52241,"tokens_out":9581,"duration_ms":96515,"significance":"If the synthesis claim were adequately supported, the dissertation would provide a useful organizing perspective on how continuous modeling and symmetry preservation translate into resource savings. Several individual contributions have already survived peer review (ICLR, ICML, NeurIPS, TMLR), and the thesis contains genuinely useful analytic results, including the resolution-change formula in Eq. (2.5) and the alias-frequency bound in Eqs. (D.12)–(D.16). The thesis is also unusually candid: Ch. 12.1 explicitly acknowledges the computational costs of global convolutions, the input-dependence of long convolutional models, and the need for symmetry pre-specification. What is missing is a quantitative, unified efficiency protocol: the central claim is stated in the abstract and Table 1.1, but no chapter measures the resources it claims to save, and no independent verification of the efficiency attributions is provided. The value of the dissertation as a structured compendium of the author's contributions is clear; the value of its efficiency framing as a falsifiable research claim is not yet established.","major_comments":[{"comment":"Design efficiency is defined in Sec. 1.2.1 as 'the resources in terms of compute, human hours, memory, experimentation, etc. needed to design a high-performing architecture,' and Table 1.1 assigns design-efficiency checkmarks to Chapters 2, 3, 5, 6, and 11. However, no chapter reports a measurement of any of these quantities. For example, Ch. 5 claims that FlexConv relieves users from pre-specifying kernel sizes, and Ch. 6 claims that DNArch learns kernel sizes, widths, depths, and downsampling positions by backpropagation, but neither measures the human effort, number of configurations explored, or compute required to reach a target accuracy relative to manual design or a discrete NAS baseline. The abstract's claim of 'substantial benefits' for design efficiency is therefore non-falsifiable from the evidence in this manuscript. The dissertation's own limitation statement in Ch. 12.1 is qualitative and does not repair this gap, and the acknowledged sensitivity of CKConv to the hyperparameter ω0 (Sec. 2.6) is an example of a design cost that is never included in the accounting.","section":"Sec. 1.2.1, Table 1.1, Chs. 2, 5, 6"},{"comment":"The compute-efficiency claim for Chapter 2 relies in part on comparisons that do not separate model capacity from training budget. In Table 2.3, the neural-ODE baselines on SC raw are reported with accuracy ≈10.0, and Sec. 2.5 states that this result is obtained after a single training epoch under a computational budget matched to the CKCNN. The text further states that 'CKCNNs trained on SC raw are able to outperform several Neural ODE models trained on the preprocessed data (SC).' This second comparison uses different input representations, and the first comparison gives the baselines no opportunity to show standard accuracy-vs-epoch behavior. To support the chapter's compute-efficiency checkmark, the thesis should provide accuracy-versus-compute curves or matched-budget comparisons in which the baselines are also trained to convergence or clearly shown to be unable to reach it within a much larger budget.","section":"Sec. 2.5, Table 2.3"},{"comment":"The synthesis-level efficiency claim is supported almost entirely by the author's own previously published papers. Each chapter in Sec. 1.4 is based on a paper authored or co-authored by the dissertation author, and the efficiency attributions in Table 1.1 are categorical checks rather than quantitative results. This is not a logical circularity in any derivation, but it does mean that the thesis's central claim is a summary of the author's published claims rather than an independent evaluation of them. If the dissertation is intended as a research synthesis rather than a compendium, it needs a dedicated quantitative meta-analysis over the constituent papers: a common set of efficiency metrics, matched training budgets, and effect sizes that would let a reader see whether 'substantial benefits' is supported across chapters and modalities.","section":"Sec. 1.4, Table 1.1"}],"minor_comments":[{"comment":"The sentence 'Architectures that required lower lower overall inversions are more financially efficient' contains a duplicated 'lower' and the likely intended word is 'investments' rather than 'inversions'.","section":"Sec. 1.2.1"},{"comment":"In the table's footnote, 'financial efficiently' should be 'financial efficiency'.","section":"Table 1.1 footnote"},{"comment":"The concluding paragraph says gridification allows 'performing neural operations in there'; this should be 'in the grid' or 'on the grid' for clarity.","section":"Sec. 4.6"},{"comment":"The admission that CKCNNs are very susceptible to the selection of ω0, and that finding a good value induces an important cost in hyperparameter search, is honest and useful; this cost should be reported quantitatively because it bears directly on the design-efficiency claims made for the method.","section":"Sec. 2.6"}],"recommendation":"major_revision","confidential_remarks":"This manuscript is a PhD dissertation rather than a typical journal article: it contains no new experiments beyond the author's already-published papers, and the review copy provided to me is truncated after Sec. 5.4.1, so Part II was assessed only from its summaries and Table 1.1. The main risk is scope: if the journal expects a self-contained research contribution, the lack of a new efficiency evaluation is a serious concern. I frame the recommendation as major_revision rather than reject because the load-bearing gaps—design-efficiency measurement and a matched-budget efficiency protocol—are in principle fixable by adding a dedicated analysis. The heavy self-citation pattern is structural for a dissertation and I do not treat it as misconduct, but it does increase the need for independent verification if the efficiency claim is to be published as a standalone research assertion."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a PhD thesis posted to arXiv, not a fresh research paper. The genuinely new material is the six-axis efficiency taxonomy in Sec 1.2.1 and the 'design efficiency' framing; the chapters are the author's own previously published, peer-reviewed papers. That is not a defect in itself — it is a dissertation — but it changes the bar. The thesis should be judged as a synthesis, and as a synthesis it is honest and well organized.\n\nWhat is good: the taxonomy separates compute, data, parameter, design, environmental, and financial efficiency, which is a useful vocabulary for the efficient-DL discussion. The assembled papers (CKConv, CCNN, FlexConv, DNArch, gridification, and the equivariant attention/convolution work) are real contributions with experiments, ablations, appendices, and usually code. Chapter 12 openly says where the methods hurt: global convolutions cost memory, group convolutions are expensive, symmetry must often be pre-specified. That candor is real credit.\n\nThe soft spot is exactly where the stress-test note lands. Table 1.1 assigns design-efficiency checkmarks to Chapters 2, 3, 5, 6, and 11, but no chapter reports even one measurement of design resources — human hours, search compute, experiments to reach a target accuracy. Design efficiency is defined qualitatively in Sec 1.2.1 and never operationalized. So the abstract's 'substantial benefits for design efficiency' is not falsifiable from this document. The same is true, to a lesser degree, for compute efficiency: baselines differ chapter to chapter, and there are no matched-budget accuracy-vs-compute curves. The one-epoch neural-ODE baselines in Sec 2.5 are the most visible example. The text does disclose the budget, which is fair, but putting ~10% in the table invites the reader to compare it as a normal result. That is a presentational slip, not a hidden deception.\n\nThe self-referentiality is normal for a thesis: the evidence is the author's own papers because that is the point. But it means the synthesis adds framing and narrative, not independent verification. If the thesis wants to be read as a scientific claim about efficiency, it needs either a unified evaluation protocol or clear pointer to independent replications.\n\nWho this helps: a reader new to continuous kernels and equivariant architectures gets a coherent tour with a candid limitations chapter; someone working on efficiency metrics can borrow the taxonomy. A serious referee would not desk-reject this if it is submitted as a position/monograph; as a new-results paper, the bar is different. My recommendation: engage with the underlying papers, cite the taxonomy if you use it, and push the authors to supply a real measurement of design efficiency before the synthesis is quoted as a result.","headline":"A candid, well-organized dissertation of peer-reviewed work; the six-axis taxonomy is useful, but the 'design efficiency' claim is asserted, not measured.","tokens_in":52743,"tokens_out":3751,"would_cite":true,"duration_ms":39647,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This dissertation argues that two inductive biases—continuous modeling and symmetry preservation—improve deep learning efficiency across compute, data, parameters, and design.","keywords":["inductive biases","continuous modeling","symmetry preservation","continuous kernel convolutions","group equivariance","partial equivariance","deep learning efficiency"],"falsifier":"Run the proposed continuous and equivariant models on a shared, larger-scale benchmark suite, for instance ImageNet-scale image classification and long-context language modeling, with matched parameter, compute, and data budgets alongside the thesis's main baselines; if the claimed compute, data, parameter, or design-efficiency advantages disappear or reverse, the central claim is falsified.","tokens_in":51707,"feed_emoji":"⚙️","tokens_out":6491,"duration_ms":64470,"temperature":0.7,"pith_summary":"This dissertation tries to establish that two design principles—modeling neural operations in a continuous space and building operations that respect the symmetries of data—make deep learning substantially more efficient, not just on one axis but across compute, data, parameters, and the human effort of architecture design. It argues that continuous kernels let a fixed parameter budget cover arbitrarily long contexts, survive resolution changes, handle irregular sampling, and open architecture search to gradient descent. It argues that symmetry-preserving layers improve data and parameter efficiency through weight sharing, at the cost of extra computation. The dissertation bundles a series of the author's contributed methods—continuous kernel convolutions, a general-purpose CNN, learned gridification, learnable kernel sizes, differentiable architecture search, and equivariant attention and convolution—and reads them as collective evidence for the two biases. The contribution is the synthesis: framing efficiency broadly and attributing gains to particular inductive biases.","feed_headline":"Two inductive biases cut deep learning's costs across four axes","feed_subtitle":"A dissertation argues that continuous kernels and symmetry-preserving layers shrink compute, data, parameter, and design costs.","key_machinery":"The central objects are the continuous kernel parameterization—an MLP or multiplicative filter network, such as a SIREN (a multilayer perceptron with sine activations) or a MAGNet (a multiplicative filter network built from anisotropic Gabor functions), that maps relative coordinates to kernel values—and the Gaussian mask mechanism in FlexConv that makes kernel size differentiable. On the symmetry side, the central objects are group convolutions, group-equivariant attention, and partial group convolutions, which encode translation, rotation, and scale symmetries through weight sharing. These mechanisms carry the efficiency claims: the continuous-kernel family removes the dependence of parameter count on context length, supports resolution transfer and irregular data, and turns architecture search into gradient-based optimization; the symmetry-preserving family reduces the data and parameters needed while preserving prediction consistency under input symmetries.","core_discovery":"The dissertation's central claim is that two inductive biases—continuous modeling and symmetry preservation—are broadly efficiency-improving design principles for deep learning. Continuous modeling parameterizes operations, notably convolutional kernels, as functions of continuous coordinates, so a fixed parameter budget yields arbitrarily large, resolution-agnostic kernels and makes architectural components learnable by gradient descent. Symmetry preservation builds transformations that respect data symmetries, improving data and parameter efficiency through weight sharing, though it can raise computational cost. The thesis evaluates these claims through a series of contributed methods and concludes that the biases yield gains in compute, data, parameter, and design efficiency, with acknowledged trade-offs.","pith_inferences":["A direct testable extension would combine continuous kernel parameterization with symmetry preservation, for instance equivariant continuous kernels for point clouds, and measure whether the two efficiency gains compound or partly cancel.","The thesis does not define a quantitative metric for design efficiency; a reader who wants to verify that claim would need to operationalize it, for example as human hours or compute needed to reach a target accuracy on a new dataset.","The small-benchmark evidence suggests a high-risk prediction: if scaled to large models and datasets, the fixed-parameter continuous kernel advantage may shrink relative to learned sparse or recurrent alternatives, because storing full-resolution kernel responses grows with input length.","The symmetry part implies that in domains where symmetries are only approximate, fully equivariant models may underperform learnable partial equivariance; this suggests a broader principle of treating inductive bias strength as a tunable hyperparameter rather than a binary choice."],"forward_implications":["One network architecture can be trained at low resolution and deployed at higher resolutions, or on irregularly sampled data, without redesigning the model.","Convolutional layers can model entire sequences with global kernels under a fixed parameter budget, matching or beating recurrent and attention models on benchmark sequential tasks.","Kernel sizes, layer widths, downsampling locations, and network depth become learnable by backpropagation, removing part of the manual architecture-design burden.","Symmetry-preserving models achieve higher accuracy per training example and per parameter, but pay a computational overhead; partial equivariance can soften this trade-off by letting the model decide how much symmetry to enforce.","Point-cloud pipelines can map irregular data to compact grids and then use standard grid convolutions, improving scalability while preserving competitive accuracy."],"supporting_citations":[{"why":"Introduces continuous kernel convolution (CKConv), the parameterization that grounds most of Part I's continuous-modeling claims.","marker":"[321]"},{"why":"Introduces FlexConv and MAGNets, enabling learnable kernel sizes and alias-free resolution transfer used in chapters 3 and 5.","marker":"[319]"},{"why":"Defines the general-purpose CCNN and provides multi-dataset evidence that a single continuous architecture works across 1D, 2D, and 3D data.","marker":"[191]"},{"why":"Proposes learnable gridification, the point-cloud efficiency method evaluated and attributed to compute and parameter gains in chapter 4.","marker":"[394]"},{"why":"Supplies the SIREN parameterization whose high-frequency fidelity makes long continuous kernels practical.","marker":"[358]"},{"why":"Supplies multiplicative filter networks, the basis for MAGNets and for analytic control of kernel frequencies.","marker":"[104]"},{"why":"Establishes the spectral-bias and random-Fourier-feature framing used to motivate high-frequency-capable kernel and positional-embedding networks.","marker":"[377]"},{"why":"Provides the state-space S4 model used as a comparison point for long-range dependency and efficiency claims.","marker":"[129]"},{"why":"Provides neural ODE and neural CDE baselines for irregularly sampled data, against which CKConv claims data and compute efficiency.","marker":"[183]"},{"why":"Offers the Perceiver as the main alternative general-purpose architecture that continuous CNNs are compared against.","marker":"[168]"}],"fun_headline_variants":["Two inductive biases cut DL costs on four fronts","Continuous kernels and symmetry shrink DL compute and data costs","Efficiency via inductive biases: continuous and symmetric models","Design principles for efficient DL: two inductive biases"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the efficiency gains reported on small benchmarks and against the specific baselines chosen in the author's own papers are representative of how these methods would behave in real-world, large-scale use, and that design efficiency is a well-defined quantity even though the thesis gives no metric for it.","fun_headline_variants_meta":{"raw":{"variants":["Two inductive biases cut DL costs on four fronts","Continuous kernels and symmetry shrink DL compute and data costs","Efficiency via inductive biases: continuous and symmetric models","Design principles for efficient DL: two inductive biases"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001279,"raw_usage":{"total_tokens":5229,"prompt_tokens":944,"completion_tokens":4285,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":4224}},"tokens_in":560,"tokens_out":4285,"duration_ms":35978,"temperature":1.0,"reasoning_tokens":4224,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:16:20.143866+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the proposed continuous and equivariant models on a shared, larger-scale benchmark suite, for instance ImageNet-scale image classification and long-context language modeling, with matched parameter, compute, and data budgets alongside the thesis's main baselines; if the claimed compute, data, parameter, or design-efficiency advantages disappear or reverse, the central claim is falsified.","supporting_citations":[],"review_version":1}